Should we care?

How often have you heard the phrase don't worry about what people think?

I believed it for a long time. But then I started doubting this. After all, why is it that we care so much what others think? Isn't it because we're afraid they may be right? (Yes, I know there are supplementary reasons, but that's not the point).

Does any of us really have that much better judgement than our peers? Can we function entirely independently of anyone else? Should we?

Is it not arrogant to think that our opinion is all that matters?

Obviously, we should be able to face ridicule. Sometimes we are righter. Often it is necessary to obey Elohim (God) and not man. But other humans have brains too.

There are certain people (like atheists and people from nudist colonies and radical Muslims) whose opinion of us should not be valued highly. But our friends and family, who understand us and our problems, may have a point. It may seem like I'm saying you can only trust people who agree with you, but sharks don't know what it's like for manta rays. They don't have much in common.