Why does my family try to push my virgo brother to talk?

We was at a birthday party, and it was kinda a family reunion, he's extremely shy...So they kinda put him on the spot, and he stormed outta the room pissed off. He told me they make him feel like he's not part of the family. I think he kinda singles himself out tho....Anyways we ended up sitting in the backroom the whole party with my Scorpio guy cousin..he was the only silent person there. The whole party was Geminis & Aquarius people...(the whole rest of the family) i'm Virgo too...So i guess my question in all this is, Why do i not relate to my family? Me my cousin and my bro seem to be the 'misfits' Do you think this is because are signs are so different?