Had a backup script running via cron for months. Worked fine until it didn’t — turns out the disk filled up three weeks ago and the job started failing silently. Nobody noticed until we actually needed a restore.
The obvious answer is “check your logs” but let’s be honest, nobody’s reading cron logs daily for 15 different scheduled tasks across 4 servers.
What’s your setup for making sure crons are actually completing? Do you just grep logs periodically, or do you have something more structured? Curious how others handle this without turning it into a whole project.
After completing a task, my periodic scripts report success to a monitoring system (I use a foss, self-hosted on called gatus) which in turn will alert me if it doesn’t receive a heartbeat every number of hours.