It’s basically their anatomy. If they have seeds in them, they are berries. Fruits with a kernel like peaches are drupes. Multiple drupes fused together are polydrupes like a raspberry.

Then there are weird stuff like apples and strawberries whose flesh is not part of the fruit because it doesn’t come from the plant’s ovary.