I was mostly thinking about hand-written tests and manual test procedures, but yeah, fuzzing can help you catch issues as well and you don’t necessarily consciously know about the test cases you put into the system in that case.
Then again, you have to design the fuzzing input consciously so I guess that’s kind of a “what you can think about”-limitation.
You can write shit tests. Finding new bugs doesn’t surprise me. Putting that much effort in does, but 600:1? That’s some serious red flags there. There are only so many variables in a single line of code. How many unhappy paths can there be for a single line?
Holy mother of god
And they still find new bugs
A limitation of testing is that you can only write tests for cases that you can think of, and cases you can think of ways to write tests for.
It’s still valuable despite this limitation, of course.
That’s not entirely true, e.g. you can do fuzz testing or constrained random testing. Maybe you aren’t including those in “testing”?
I was mostly thinking about hand-written tests and manual test procedures, but yeah, fuzzing can help you catch issues as well and you don’t necessarily consciously know about the test cases you put into the system in that case.
Then again, you have to design the fuzzing input consciously so I guess that’s kind of a “what you can think about”-limitation.
Good point regardless, thanks
You can write shit tests. Finding new bugs doesn’t surprise me. Putting that much effort in does, but 600:1? That’s some serious red flags there. There are only so many variables in a single line of code. How many unhappy paths can there be for a single line?
SQLite is one of the best tested codebases in existence. Having only so many variables per line means nothing