Two things: first, there’s probably a few biases at play between conformation bias and the fact that the taller a guy gets a similar sized woman would look shorter and shorter and vice -versa.

Also you say (apparently) the taller a guy gets the shorter the women are; so why do taller men prefer shorter and shorter women and not: why do shorter women like taller and taller guys?