After needing to find a small delimiter for my data format I started wondering if I could use 0x1E-0x1F?
They are part of the control codes so I thought they might do something weird?
https://en.wikipedia.org/wiki/C0_and_C1_control_codes#Field_separators
Generally a bad idea to use in-band signalling like that. They won’t do anything weird but consider what happens if the actual data contains them.
Then you’d escape them by using another character in front. But if their data format is ASCII text or is guaranteed not to have characters below ASCII 32 then using ASCII delimiters is fine.
But who escapes the escape characters?
It’s escape characters all the way down
You can use Unicode pictures: ␜ ␝ ␞ ␟
https://en.wikipedia.org/wiki/Control_Pictures
Use emoji as escape characters
Then you can just use a conmon delimiter like comma or semicolon or something. It’s better even as you’re less likely to have something that seems to work until your exotic delimiter pops up in the data.
Better yet, use a commonly used data format like csv or json and don’t build your own
Indeed. Escape characters add a lot of additional complexity, footguns, and performance penalties.