In case someone is like me who just assumed it’s not possible and never tried to look it up before.

When playing a DS game (or GBA apparently), press Home, then hold Start and use up and down on the D-Pad to change the brightness.

No need to exit the game. Neat.