I think in many cases a bigger issue is going to be how the game is designed, whether open or closed source.

If your open source game is a mess of poorly-documented, barely-decipherable spaghetti code, held together by a bunch of tacky bullshit, that’s going to be a nightmare to mod.

On the other-hand, a closed source game may have absolutely phenomenal documentation and tools available specifically to enable the modding community.

Similarly, you can have well-written, open-sourced code with built-in mod support with proper documentation, and you can have ridiculous bullshit closed-source code that no one is quite sure how or why it even works.

One thing you get with open-source games is families of variants, where the code for a game has been forked and developed in different directions by different developers.

OG roguelikes such as Nethack and Angband are an example of this. Rather than mods for a single game, there are variants — code forks that add new features, quests, monsters, skill systems, user interface elements, and so on. There are several dozen Angband variants. Some variants draw upon multiple other ones, combining patches to make a particular developer’s preferred form of the game.

I think open source eliminates one potential obstacle to modding, but having good interfaces is the main thing. If some open source code is unreadable spaghetti, then it’s gonna be hard to mod. Likewise if a closed-source game has well-documented interfaces available, it could be easier than a poorly-interfaced open source project.

I haven’t written video game mods but I’ve written a lot of code. Even in my own projects, ie where I have access and authority on all code, I want to be ignoring 99% of the code at any given time.

With good interfaces, you don’t need to look behind them. In that way, if the interfaces are good enough, it ceases to matter whether you can see the code behind them.

Modding relies on stability AFAIK.

There are FOSS games with a single branch for development, which doesn’t scale well for stability, but works fine with slow development.

Others, have stable and experimental branches with backporting and so on. Probably works better, but too much delay between stable and experimental makes people play on experimental. Stable releases are expensive (redundant bugfixing, backporting, feature freezing/completing etc.), so not every projects has or wants them. Fast development might enable modding to turn into internal (optional) features though.

Pillars of Eternity was moddable only because it was written in C# (Unity iirc), making it possible (though not straightforward) to more or less modify the code however you want. So a mod was essentially a bunch of overwritten classes.

A game that you could literally just fork on GitHub and change things would certainly be moddable in a sense… Handling multiple mods without conflicts, however, is another thing. For that you need a proper modding interface, like the original Baldur’s Gate or Skyrim.

OpenTTD has a ton of mods

Depends, was the game originally designed for mods?

It’s much easier to make changes to the game if you have the code for the game.

Probably significantly harder. Open source tends to have a lot of churn and breaking changes. That’s more work to keep mods working. If the interface isn’t there for a mod, it could be added, but that’s more work for the modder.