I’m struggling with overly confident players. Obviously, killing a PC is one way to make them think twice about the next time they pick a fight, but I don’t want to resort to that if I don’t have to. Is there a long term status condition or something I can subject them to that’s good at making players averse to picking fights, even when they think they can win?
Another approach you can take is simply making it so a violent resolution does not lead the players to accomplish their goals as well.
Trying to get information about a big nasty with a cult, and the players decide to just murder all the cult members? Well, the players might be able to beat the cult in a fight, but not fast enough to prevent the cult from burning their sacred texts, and now you have to piece info together out of the ashes.
This is a difficult line to walk: you have to plausibly present that the outcome would have been better if they had negotiated or infiltrated, versus just “well the DM was never going to give us the text anyway”. You also have to make sure you don’t just lock off the plot because they fought.
You need a clear backup plan that’s just annoying enough to make it clear putting a little more thought into your first approach could have saved a lot of time., and maybe a slight downgrade of the end result of the plot (time is classic here, maybe a couple people the party was expecting to save got sacrificed while the party was messing around).