Matter not sunlit in space really is cold (and sunlit matter is baking hot, btw). It’s just there is so little matter in space (vacuum and all) that getting rid of heat via conduction is virtually impossible, meaning it’s insulated. So technically space is cold, also hot, and also damn well insulated.

Water not exposed to sunlight could get as low as - 260C (close to absolute zero) by radiating away all it’s heat… eventually. Meanwhile water exposed to our sun in Earth orbit would be at around boiling temps, like 100-140C. Just check the Moon surface temperature readings.