The “each answer evaporates a glass of water” claim originated in a paper that never got passed into peer review, and is about calculating the costs of building, powering (as in, running a power plant) and running a data center with an open water cooling system (rather than recycling the water).
It also only measured the training of an LLM (the most processor hungry part) - then assumes that rate of water use as the base line for all subsequent questions being asked (eg. If it takes 6 seconds to respond to a question it was really re-building, re-powering, re-running the datacenter whilst re-training the model for 6-seconds), rather than just traversing a pre-trained branch and regurgitating the info - which is what actually happens.
IRL the work for each model is only trained once per new release. This is why you can download and run models on your home computer… And obviously asking a locally run AI doesn’t evaporate a glass of water each time (you’d feel the heat from your computer if that were true).
So the ‘water per question’ claim is a myth.
…and this is not to say that I think LLMs are a good technology, or free from environmental concerns (the training, chip manufacturing and data center construction, let alone the ponzi scheme of investment, and wealth disparity… And easily corrupted governments are all concerns)… Just that this one claim about water is bogus once the model has been trained.
Most water coolers compress and recycle water (using refrigerant chemicals). Personally I’m fine with water in our atmosphere, it’s the other chemicals (methane, carbons) that I’m concerned about, as they’re bad for climate change.
The “each answer evaporates a glass of water” claim originated in a paper that never got passed into peer review, and is about calculating the costs of building, powering (as in, running a power plant) and running a data center with an open water cooling system (rather than recycling the water).
It also only measured the training of an LLM (the most processor hungry part) - then assumes that rate of water use as the base line for all subsequent questions being asked (eg. If it takes 6 seconds to respond to a question it was really re-building, re-powering, re-running the datacenter whilst re-training the model for 6-seconds), rather than just traversing a pre-trained branch and regurgitating the info - which is what actually happens.
IRL the work for each model is only trained once per new release. This is why you can download and run models on your home computer… And obviously asking a locally run AI doesn’t evaporate a glass of water each time (you’d feel the heat from your computer if that were true).
So the ‘water per question’ claim is a myth.
…and this is not to say that I think LLMs are a good technology, or free from environmental concerns (the training, chip manufacturing and data center construction, let alone the ponzi scheme of investment, and wealth disparity… And easily corrupted governments are all concerns)… Just that this one claim about water is bogus once the model has been trained.
Most water coolers compress and recycle water (using refrigerant chemicals). Personally I’m fine with water in our atmosphere, it’s the other chemicals (methane, carbons) that I’m concerned about, as they’re bad for climate change.