I am using Firefox ESR on Linux. Does it make any sense to disable hardware acceleration for security reasons?

Whereas Firefox is open source, many hardware drivers, and most computer hardware are not. Enabling hardware acceleration in Firefox means using these non-open-source components, which could be a security risk. My impression is that bugs in drivers and hardware are able to cause far more damage than ones in userland software. Does this reasoning make sense?