We often think of that today, but historically speaking, no. Abrahamic religions (Judaism, Islam, and Christianity) were the most aggressively successful incarnations of this tendency, but many pre-modern religions were pluralistic in that they often did not deny the validity of other faiths.

One I talk about a lot is Roman paganism, wherein they believed that all faiths were essentially worshipping the same gods, and thus continuation of provincial ‘barbarian’ practices was not only harmless, but could be regarded positively, as the provincials were following the ways of their ancestors - something that was regarded as just and honorable, so long as it did not otherwise violate Roman norms, or be foisted on Romans.

However, even polytheistic faiths without this sense of universality often held that other gods existed, they were just not ‘our’ gods. When the Egyptians smote the Hittites, or vice-versa, it was because their gods were powerful or blessed them; while the enemy gods were WEAK or punishing the enemy for their great impiety! Thus, truth was not what the religion laid exclusive claim to, only loyalty.

You also see this with syncretism in non-Abrahamic religions, wherein faiths that do not have quite such a strict sense of orthodoxy, outside of particular fanatics, absorb and share features with other religions in the region. Buddhism is particularly famous for absorbing regional religious practices like this, just subsuming them into the greater context of the teachings of the Buddhas. Thus Buddhism does not claim, in most cases, to be the only religion that speaks of true metaphysical realities, just the one that puts all true religions into the correct context needed for enlightenment.