I can kinda answer that.

There’s a term used in tech called “empire building”. It’s where managers and execs promote their little slice of the company to persevere and grow their own career. At a certain level, it leads to someone that leads a division like AI having enough influence that they can say “let’s put AI into search”.

The sad thing about tech is that at a certain level, an executive rises above the customer in dictating what is best for a product. Data and stats can tell you whatever story you want to promote, so at Google HQ they’re probably worried about the negative press, but they’re looking at “successful” numbers of questions answered by AI and are patting themselves on the back. Both search and AI execs look good because they delivered something, and they’ll likely get a nice bump from their bosses in terms of rep.

The thing with empires is that they fall. Not overnight, and maybe not with the same emperor, but they do fall.

Quarterly finances kinda answer that. Jumping onto the AI bubble brings investors, makes your company highly valued and gives managers fat quarterly and annual bonuses. It doesn’t matter if the company or whole industry goes under in the future, because those bonuses have already been collected.

Man even you’re missing the bright side.

At least an insanely tiny group of people got absurdly wealthy! Who cares the world is going to complete shit, Nvidia stock!! Nvidia stock!! It’s all that matters!

I’m all for eliminating marketing jobs, but not if we still have marketing.

is your post missing a /s

does anyone know how to short utils

it’s just a little sad that we, humanity, will never succeed in our summoning ritual to pull you into our plane of existence. I mean, we won’t before boiling away our oceans and igniting the atmosphere. Of course, we should be trying harder! The ice caps aren’t even melted yet!

I joined the Microsoft climate solutions team, I think doing so has crushed all my hopes. That team is doing small things like “let’s all go plastic free for July!” Top 20 company in the world and the best we can do is go plastic free for a month? Meanwhile MSFT is not on course to meet their own climate goal by 2030

“It only uses 5x as much energy as a regular search! Think of how much energy YOU’RE using with searches!” Okay, so you’re just using 5x as much energy for worse results? And also probably doing it more often than people who just use a normal search engine, because they don’t expect the search engine to talk to them. I’ve never understood how that was supposed to be an exoneration for it, even without taking into account that nobody ever seems to know whether or not that figure includes energy spent on training.

there’s this type of reply guy on fedi lately who does the “well actually querying LLMs only happens in bursts and training is much more efficient than you’d think and nvidia says their gpus are energy-efficient” thing whenever the topic comes up

This kind of person (also happened a lot with cryptocurrencies) always goes ‘that isn’t how it works, this isn’t a problem’ then doesn’t explain what the mistake is you are supposed to have made, and then a few weeks/months/days/search later it is revealed that it was how it works and it is a huge problem. And it is so annoyingly common im very happy with the moderation here.

I live in Ireland and these companies keep trying to build data centres here even though they are using way too much of our energy as is :)) From an Irish environmental org:

A few years ago when researching new gas power stations in Ireland, we noticed that many of these developments were being proposed alongside another type of infrastructure: data centres. […] As of June 2023, there are 82 operational data centres in Ireland, with another 14 under construction. Additionally, 40 data centres have had planning permission approved with another 12 awaiting a decision. […] Data centres account for 18% of all electricity use in Ireland […] This is putting unprecedented strain on the electricity grid, with grid operator Eirgrid estimating that data centres may account for up to 27% of Ireland’s electricity demand by 2028. The Commission for Regulation of Utilities (CRU) in Ireland has noted a risk of rolling electricity blackouts…

I always thought data centers ran clean and dirty loops of cooling (as far as computers are concerned).
The clean loop has all the chemicals and whatnot to keep cooling blocks and tubing “safe”. The dirty side is just plain old water. And a big heat exchanger transfers the heat from the clean (hot) loop to the “dirty” (cold) side.
Is there really that much pollution in that? Can’t be worse than rain going through storm drains or whatever.

But AI does use a phenomenal amount of power.
And, IMO, it’s a problem compared to the lack of value people are getting from AI.
The new Blackwell B200 consumes 1.2kw of power, and will produce 1.2kw of heat.
A cooling system with a COP of 5 needs to consume 240w to dissipate this.
The backplane for the B200 holds 8 of these GPUs in a 10 RU space, and with overheads will peak 14.3kw (cooling would be 3kw consumption).
So, a 42u data center rack with 3 of these, supporting hardware and UPS efficiencies (80%) is going to be 52kw (+10kw cooling). 62kw total, which is like 4 homes drawing their full load all the time.

I hope they finally find an application for AI, instead of just constantly chasing the dragon with more training, more parameters, more performance etc.

The last part is absolutely false. The Nvidia H100 TDP is like 700W, though ostensibly configurable. The B200 is 1000W. The AMD MI300X is 750W.

They also skimp on RAM with many SKUs so you have to buy the higher clocked ones.

They run in insane power bands just to eek out a tiny bit more performance. If they ran at like a third of their power, I bet they would be at least twice as power efficient, and power use scales over nonlinearly with voltage/clock speed.

But no, just pedal to the metal. Run the silicon as hard as it can, and screw power consumption.

Other AI companies like Cerebras are much better, running at quite sane voltages. Ironically (or perhaps smartly), the Saudis invested in them.

On one hand, yes let’s get annoyed at companies for using alot of energy. On the other hand, we have known our energy needs will be exponential for decades yet we haven’t tackled the issue. We still burn fossil fuels, despite having the ability to not do that.

Being angry at companies for building a new technology yet not being angry at the government and NIMBY fuckwads is a little… Backwards imo.

There is currently a barrage of AI hate getting spewed out by most media (not manufactured, don’t worry, it’s all organic and you aren’t being manipulated) but this has to be the stupidest one. The grid should be able to take whatever energy we throw at it and shouldn’t be running this dirty in this day and age. Our problem is corrupt politicians and the oil and coal industry, this is nothing but a scape goat.

this type of reply guy on fedi lately

I guess that’s an easy way to ignore all dissenting opinion. I doubt you want to listen but here goes anyways:

It is more energy efficient to run it on big cards than have individuals use their own GPU (even though I personally like the open models running on my own computer). That’s what the reply guy was actually trying to tell you.

GPUs do consume a lot of power, they are also used in many things. Would you ever think for a second to post something similar about gaming or rendering? Of course not.

Cooling systems cycle the water, they are closed loop systems.

Meta posted all the energy ii took to train llama, it’s a lot but not that much when you actually look at the amount of people using it. A lot of industries do this, I suggest you look up the data on your favorite gaming companies so you can compare in case your opinion is overblown. I did some napkin math the other day and the energy usage to train llama (used by millions currently) was about the energy usage of 1000 people taking a flight to and from Europe.

Building power plants with renewables to lessen their impact on the grid and make sure they aren’t burning coal to run their GPUs should be encouraged.

This is why philosophy should be mandatory in college (and possibly high school). Die Frage nach der Technik by Heidegger discusses this misconception that technology can solve all of our problems. He was thinking about this issue in 1954.

Music and art are also important to study. In “Faith Alone” by Bad Religion, the lyrics include these lines:

Watched the scientists throw up their hands conceding, “Progress will resolve it all”

Saw the manufacturers of earth’s debris ignore another Green Peace call

Greg Graffin was discussing this in 1990.

When they fail, it won’t be their fault of course, it’ll be AI’s fault.

deleted by creator

I’m sorry you have such narrow boundaries of things you want to know about. or such lax requirements on correctness.

but I’m not sorry about how much shit you’ll eat somewhere down the line.

i’m amazed that you folks come here from your own will and just type these inane sentences expecting… what? praise?

@StaySquared @dgerard this has strong “I own a Tesla” energy