Gold fell more than 4% in the four days since Election Day, when the broad U.S. stock market climbed nearly 4%.