I think it might help to differentiate “I disagree” and “I don’t like this” downvotes from “this thing is a waste of resources” or “I don’t care what this person has to say, they are an asshole that I don’t want to interact with” ones. Comment filtering should use the latter rather than the former.

Allowing controversial comments to still be a part of the conversation without a “this isn’t popular!” target painted on them might avoid the echo chamber effect while still giving users a way to passively disagree.