It looks like this was a larger decision from the lemmy development community in an attempt to eliminate karma farming. They say it’s psychologically damaging, and as someone who looks at them a lot, they may be right.
Here’s a GitHub thread discussing it where our Voyager dev weighs in:
https://github.com/LemmyNet/lemmy/issues/3393#issuecomment-1779400639
I really think you are grossly oversimplifying the problem.
When I click on any user’s profile, I’m seeing content from a lot of posts that my app instance hasn’t seen before. Each user follows different communities and accesses the app at different times, so each user’s instance will only have data on the posts that they have clicked through when they are browsing. The score calculation that you are suggesting would vary wildly depending on how much overlap you’ve had with that user previously. 5 or 6 posts out of hundreds or thousands would not be enough to consistently see a valid trend.