It seems most people are on board with the idea that AI will change the world. While I agree it having some impact, I also think it is overinflated by marketing. Operating an AI takes huge computing power, which costs heaps of money and energy. So how are people suggesting that exponential improvement is feasible? I do not get it.
Further, aren’t we supposed to reduce energy usage? Why are we trying to overspend what little is left? I hate how this is taking priority over the environment.
Creating this post mainly to rant, I thought OpenAI firing Sam Altman was a signal for a reality check. It seems they are wrapping it up and trying to rehire him though… What a drama.
It’s unsustainable right now because hardware and software are not aligned (yet). Software is currently out pacing hardware, but there are loads of companies working on specialist chips that will deal with the computing power problem and the energy consumption problem just by the shear factor of optimization benefits.
Plus, software optimizations are also well under way and models are always being fine tuned to run better/train better with less
I doubt how good results that could achieve. I agree that 10~100 times improvement is feasible by optimizing the hardware. But the hardware in general need to be improved, yet the impenetrable barrier light speed is blocking on the way.
And more complete AI systems should require hundreds of thousands times the computation power. Really, this has the same issue as bitcoin.
I think the specialized hardware for this task will be better than you expect. It’s like using a sledgehammer to carve something. Pretty soon a chisel will be given to the computer and it will be able to do its job much easier.
I doubt it since GPU was already not a bad tool for this job. The generality of GPGPU helped a lot here.