I know this is probably a key difference between INFJ and INTJ and is completely unavoidable but from my perspective this line of thinking always felt very wrong to me, and I always thought of it as a crutch (as it is easier to completly block out an entire part of you vs. not having a clear boundary).
In my eyes, what other people think is very important and makes you a more approachable, interesting and dynamic person over all. And more importantly I feel like it gives you the gift of having an open mind (and heart) which is key when gaining knowledge and new ideas with others (and allows you to give your knowledge). This can create powerful and interesting dynamics.
The problem is, are the people who are out to actively hurt you are are toxic to what keeps you going and your values. People who clearly are not out there to try and help out and are only in it for maximum personal gain in the way that they will actively be the ones to screw you over for themselves. Basically people who just are a bad influence to be around - those are the people who you should ignore and blow off.
There's a key difference though between the above and a friend/stranger who is giving you legitimate constructive critisisim because he/she wants to be helpful to you.
I just feel like the "who cares what other people think" attitude as a whole is the #1 source of why friend/relationship/colleague issues happen, because people often apply that line of thinking as a way of life instead of a case-by-case basis "as it should be".