It’s not all about the impact. It’s also about the chance or impact. If you’re going at that speed on pedestrian zones, you’ll eventually hit someone, or be very close to doing so. Pedestrians go in any direction, and can change at any time in an instant. If you go fast, no matter how fast your reflexes are, they won’t be fast enough to brake in time on pedestrian zones.

The roads not being safe for non-cars is not the problem for pedestrians. Use the bike lane in those cases.