Two files of a million digits, statistically identical, and yet one compresses to three lines and the other never will. Chasing that gap splits the idea of 'compressible' into two very different things — and leads to a strange wall: you can always prove a thing can be compressed, but never that it can't.
Allow me to take a different, simpler perspective on this subject of randomness…
I just tried a simple experiment, simply flipping a coin 8 times. The results should give me a random byte, right?
Not particularly. My coin flips gave me ‘Heads, Tails, Heads, Tails, Tails, Tails, Heads, Tails’
That gives me the binary sequence of ‘10100010’
Now, it would seem this should be fairly random, but one byte of information should have an equal randomness of the numbers from 0 trough 255. Should…
But it almost certainly will not. It’ll produce something closer, if not basically identical, to the Bell Curve in statistics. You’re almost guaranteed the lowest chances of generating ‘00000000’ or ‘11111111’ using such a simple binary coin flip for each bit. Not to say it won’t happen, it’s just less likely in the long run experiment.
As I conclude this comment, I flipped it once more. Heads.
I mean, yeah two d6 are random but adding them doesn’t give you an even 1-12. It’s a bit more obvious with the d6’s though.
I’m gonna need you to explain your math here. More specifically, I want to know which bytes are going to appear more often when you are generating them in sequence with an equal probability of each digit being zero and one.
Let’s take the other track and instead assume that byte generation is uniformly distributed. In fact, let’s go stronger and assume that any binary number generated by coin flips is uniformly distributed. The base case is a single flip. We have two possibilities, head or tails, each with 50% probability. This means our resulting numbers, 0 and 1, occur with equal probability. This is the uniform distribution. Now assume a binary number of length k - 1 generated by coin flips is uniformly distributed. A binary number of length k is composed of a number of length k - 1 and a single flip. The first k - 1 parts create a range of [0, 2 ^ (k - 1) - 1] and are uniformly distributed. Let’s put our new bit at the front. A 0 gives gives the current range, [0, 2 ^ (k - 1) - 1], and a 1 gives us [2 ^ (k - 1), 2 ^ k - 1]. Note we have an equal probability of falling into either space and they are the same size. In other words, a binary number of length k generated by coin flips is uniformly distributed.
Entropy. Noise in the system. Chaos.
Grouping 8 bits (or other grouping of bits) out of an endless sequence of truly random 50/50 bits will show a bias towards chaos, not solid patterns like 00000000 or 11111111.
I mean yeah it’ll happen occasionally, but I think it’s way less likely than pure noise, where half the bits are zero and half the bits are one…
If you are saying the probability of sampling something with half ones and zeroes is greater than the probability of getting 0000 0000 or 1111 1111, that is a correct statement because you’re sampling the output, not the generation. The probability of generating 1010 1010 is the same as the probability of generating 0101 0101 which is the same probability as generating 0000 0000 or 1111 1111 or 1111 0000 or 0000 1111. You’re looking at the difference between (8 choose 4) vs (8 choose 8).
However, in a truly random system, 0000 0000 will appear as often as 1010 1010. The distinction of “solid patterns” is meaningless at scale. You’re the one differentiating between the two. Now as we analyze the randomness, we would expect more samples that have four ones than have eight ones, but we would also expect an equalish number of each pattern to appear (eg 0011 1100 appears the same amount as 1100 0011 which appears the same amount as 0000 0000).
You’ve conflated the number of patterns with the number of ones or zeroes that appear in the patterns.