Ah that’s interesting. If you can swap the devices from one pi to another, try powering it all up on machine A, then swap the devices to machine B and power that on. Might tell you if the issue is with on the pi side or with the devices.
Is latency higher on the first boot than on subsequent ones? I’d be looking into race conditions if you’re seeing a bit of lag cascade out into bigger problems. Race conditions are the worst, especially when the race most often goes the right way and just occasionally goes the wrong way. Though you can force the wrong way by adding delays in your code, if you have an idea of where the race is happening.
We have 3 theoretically identical systems here and this same issue occurs on 2 of them. The 3rd one… has bigger issues right now. That would be interesting to see what happens if I swap the Pis around but I’d give it >95% chance the same thing happens.
The important bit is to power one on first before the swap, then you’ll have one setup where the pi was recently powered on and another setup where the connected devices were recently powered on. You might see the issue on only one of the devices, at which point you can say if it’s the pi being off for a while or the devices that triggers the issue.
Good point. I disabled the internet on both systems so when I come in on Monday hopefully I can confirm whether or not the date/time aspect is a problem. I’ll try this as well.
It was the date/time thing. Minimal latency on any of the systems that weren’t connected to the internet. I found a linux command that will prevent the Pis from syncing the time via the internet.
Awesome, glad you got it figured out! Thanks for replying with that info, too, as I was curious if my suggestions helped at all. Helping debug a problem based on a short description makes me feel like a bit less of an imposter.
One suggestion is to occasionally enable the time sync via internet. It doesn’t need to be a daily thing (or however often systems want to do it), but those crystal clocks are usually inaccurate enough to get noticeably out of sync over a long enough time period, plus you can let localization settings take care of things like daylight savings time if we don’t get rid of it soon (or the nightmare of some regions getting rid of it while others don’t). Assuming your use case even cares about the actual time. If you just need to track intervals, syncing doesn’t really affect the accuracy of those.
Ah that’s interesting. If you can swap the devices from one pi to another, try powering it all up on machine A, then swap the devices to machine B and power that on. Might tell you if the issue is with on the pi side or with the devices.
Is latency higher on the first boot than on subsequent ones? I’d be looking into race conditions if you’re seeing a bit of lag cascade out into bigger problems. Race conditions are the worst, especially when the race most often goes the right way and just occasionally goes the wrong way. Though you can force the wrong way by adding delays in your code, if you have an idea of where the race is happening.
We have 3 theoretically identical systems here and this same issue occurs on 2 of them. The 3rd one… has bigger issues right now. That would be interesting to see what happens if I swap the Pis around but I’d give it >95% chance the same thing happens.
The important bit is to power one on first before the swap, then you’ll have one setup where the pi was recently powered on and another setup where the connected devices were recently powered on. You might see the issue on only one of the devices, at which point you can say if it’s the pi being off for a while or the devices that triggers the issue.
Good point. I disabled the internet on both systems so when I come in on Monday hopefully I can confirm whether or not the date/time aspect is a problem. I’ll try this as well.
It was the date/time thing. Minimal latency on any of the systems that weren’t connected to the internet. I found a linux command that will prevent the Pis from syncing the time via the internet.
Awesome, glad you got it figured out! Thanks for replying with that info, too, as I was curious if my suggestions helped at all. Helping debug a problem based on a short description makes me feel like a bit less of an imposter.
One suggestion is to occasionally enable the time sync via internet. It doesn’t need to be a daily thing (or however often systems want to do it), but those crystal clocks are usually inaccurate enough to get noticeably out of sync over a long enough time period, plus you can let localization settings take care of things like daylight savings time if we don’t get rid of it soon (or the nightmare of some regions getting rid of it while others don’t). Assuming your use case even cares about the actual time. If you just need to track intervals, syncing doesn’t really affect the accuracy of those.