I have seen a lot of buzz lately about defects introduced to working code by AI. Some benchmarks show AI creating 1.7x more issues than humans. This is the worst AI code generation, and I expect this gap to close. Even if AI is much better than a human, any change to a codebase has the possibility of side effects.
When looking at the cost of any code change to a system, the highest cost was often the human cost to develop the change. We would weigh this cost and decide if something was worth doing based on effort. The challenge is that this method is broken with AI code generation. AI continues to drive the cost to develop a change down. So what should we look at to gauge the cost of a change as the effort approaches zero?