As an American, I was very confused. I:

• have never heard of the show
• never knew there’s a difference between “pants” and “trousers” (trousers here are old-timey pants)
• am not fancy at all; I’m lucky if I wear long pants, much less dress pants

I’d appreciate any more context you’ve got though. Everything I found was from weird male fashion sites and channels.