It’s describing a psychological barrier, like you have a limited amount of mental space for a really ‘knowing’ people. People are complex and varying and it takes a lot of space and effort to learn their differences (and that’s important, some people can be dangerous or whatever). Once you run out of space (so to speak) their existence is mentally more abstract and less likely to come to your mind when you’re making your decisions. That negligence is the source of conflict, not you literally not thinking of them as people and harming them intentionally.