Post New

I thought this was a great post on the Macroeconomics board on the massive power consumption used for training AI LLMs. There will be need for massive new investments in power generation and distribution. Are the regulators ready to provide the right incentives for it to happen?

https://www.shrewdm.com/MB?pid=673996981

For comparison, the largest single power plant complex in the United States is Grand Coulee Dam which is rated at a capacity of 7,097 megawatts or 7,097,000 kilowatts. If I haven't scrambled a units conversion somewhere, the power consumed by the GPT-4 training could have consumed all of the power from Grand Coulee Dam for 52 minutes.

Yes, the amount of money and land and resources (electricity, water for cooling etc) is staggering.

FWIW, my prediction is that these huge data centers in ten years will be like shopping malls are now, i.e. white elephants (dead ones) for which no one quite knows how to deal with the corpses.

A lot of the power is consumed by matrix multiply of the large neural nets. But there are advances in matrix multiply algorithms (type "efficient matrix multiply for AI?" into perplexity.ai, also try "approximate"), the new approaches can be up to 100x faster.

But that's just tweaking the implementation of current neural networks. The "neurons" used in current AI neural nets are very simple, so one needs a lot of them, resulting in a lot of connections, and hence lots of matrix multiplies. There's work on using more complicated (more neuromorphic) neurons and fewer connections (again, perplexity.ai is your friend).

Lest one think that there's been a revolution in understanding human thought processes that led to the current explosion in AI, Hinton himself has remarked (sorry, I don't have the reference to hand) that although there's been some very very nice technical and algorithmic advances, neural networks today aren't that much different than the old backprop days. What's made a bigger contribution to what is happening now is the advances in compute power, and the available online training datasets.

There's work on growing brain cells in a dish with implanted electrodes (brain organoids), Front. Sci., 27 February 2023 Volume 1 - 2023 | https://doi.org/10.3389/fsci.2023.1017235
One can envision some hybrid of "dish brains" with current, simulated, neural networks. Then there's Musk's "NeuralLink" and similar efforts to interface with the fully developed human brain as opposed to "brain organoids in a dish".

Here's a number to ponder when thinking of current AI power requirements: the human brain runs on roughly 20 watts.

"Yes, the amount of money and land and resources (electricity, water for cooling etc) is staggering."

Imagine there was a structural engineer who lived around 1900 just when cars were starting to become a thing. Most roads were little more than dusty cart paths. Sure there were some bricked in roads in the fancy parts of cities, but most everything else was just mud and dust. Then someone then teleported you to 2024 and you saw the road infrastructure then. All of the concrete and paved roads. Expressways with huge cloverleaf interchanges where they intersected with other highways. Some with interchanges off the ground where lanes were 100s of feet in the air. Not only the roads themselves, but the other infrastructure that went with it. Sewers and drainage systems to insure rain and runoff would keep the roads clear. Electricity to power all of the stoplights and streetlights to keep drivers safe. All of the signage, the glow in the dark lane markers, the guardrails, sensors in the roadway, etc.

It would all be staggering to such a person. He would go back and calculate how much steel, concrete, and asphalt it would take to build the future and the amounts would be incomprehensible to such a person. The energy required to make the steel, concrete, and asphalt would be incalculable. He would take the biggest steel plant at the time and realize it would take orders of magnitude more of such steel plants to make all of the steel. Where would all of the ore come from? Same with the manpower required. Then look at the land required. and so on and so on.

The amount of money, land, manpower, and resources would all just be absurdly huge, incomprehensible numbers to such a person.

The reason is that the person is trying to calculate how much it will take to get to the future point in an instant. The thing is, it isn't going to happen in an instant. It is going to happen in small steps. Right now, people are building computers that are sucking up ridiculous amounts of electricity and throwing off tons of heat. Electricity is relatively cheap so that is the way to go. Eventually if electricity is in so much demand the price of it will rise and some of the projects going won't be worth the cost. Then we (humankind) will figure out how to get cheaper electricity. We will figure out new ways to do the computations more efficiently. Humankind will encounter obstacles (such as a lack of enough electricity) and they will overcome those obstacles. That is how humankind progresses.

As an aside, people look at the allure of fusion energy because they see unlimited (i.e. cheap) energy and think of free electric bills for their house and car. Thinking they can transport stuff super cheaply. All of that is true, but to me the real allure of the unlimited energy from fusion power will be near unlimited computing power at mankind's fingertips. With near unlimited energy, mankind will be able to build huge computer centers that are many orders of magnitude bigger and more powerful than we have today.

And we wouldn't have to hear any "no way it'll ever work because it'll break the elecric grid forever" whining.

I do wonder how many people were writing about "there'll never be enough gas" and "all the horse tenders will be out of a job" during the 20-30 years it took for cars to really become common, compared to the above about EVs and fusion.

One key difference in comparing the transition from horses to cars (and the necessary car/truck infrastructure e.g. roads to support it), to the development of the new "AI" (and the computing/datacenter infrastructure to support it), is that cars/trucks were more efficient than horses, to the tune of hundreds of horsepower packed into the space of a few horses. Also, as the efficiency of internal combustion engines increased, the need for road infrastructure still remained.
Current neural net "AI's" implementations, with their huge computing and data center power needs, are horribly inefficient compared to the human brain operating at 20 watts.

IMO, for whatever that's worth, theoretical advances in neural networks over the next ten years will obviate the need for huge computing and data centers. In contrast, efficiency developments in internal combustion engines didn't obviate the need for road infrastructure. Also, as mentioned, there are other routes to "AI" such as brain organoids in a dish, that are being developed.

The key number to remember when considering what power is needed to compute like a human is 20 watts.

Fusion power is an interesting thought. Perhaps it might lead to fueling super human computing power.

Anyway, it's just my opinion: I think these huge computing data centers will become like dead shopping malls within ten years, i.e. huge white dead elephants that no one knows what to do with their (expensive) corpses.

I think these huge computing data centers will become like dead shopping malls within ten years, i.e. huge white dead elephants that no one knows what to do with their (expensive) corpses.


I agree that these data centres and, hopefully, the tremendous amounts of electricity they consume, may be unneeded in 10-20 years. However, the major long-term investment involved is, as far as I can tell, the production of all that electricity, and we will have no trouble finding something else to do with that.

And the malls that we no longer need, that get converted to data centres, will just have to be converted to something else again, or just demolished. The servers themselves are unlikely to have a life of more than about 5-10 years (is that right?), so we won't have to get rid of a bunch of brand new servers we just built, we'll be able to phase them out gradually, as they wear out, if they are no longer needed.

We will have lots of these challenges - tearing out all the gas pumps and even most of the gasoline refuelling stations, as we go to 100% electric vehicles. But the changes are likely to be slow enough that we can just not replace old fuel reservoirs and pumps, as they age out of the system, and occasionally close a marginally profitable refuelling station.


cars/trucks were more efficient than horses, to the tune of hundreds of horsepower packed into the space of a few horses. Also, as the efficiency of internal combustion engines increased, the need for road infrastructure still remained. Current neural net "AI's" implementations, with their huge computing and data center power needs, are horribly inefficient compared to the human brain operating at 20 watts.


Horses were much more efficient than the cars that replaced them, in the same way that a human brain is much more efficient than an AI data centre, if you are just looking at energy/functional unit. We have replaced horses, let's say they use 1 horsepower, with cars that typically use 200 times more power. It's a lot harder to say how much energy is needed per human brain, if we use a lot of AI, but it is surely orders of magnitude above 20W. So we need a lot more energy to get around than when we used horses, and will need a lot more to do some of our thinking with AI, but fossil fuels and now, cheap renewable power, make it possible for us to upgrade to the less efficient but more effective solution.

dtb

Anyway, it's just my opinion: I think these huge computing data centers will become like dead shopping malls within ten years, i.e. huge white dead elephants that no one knows what to do with their (expensive) corpses.

It's quite possible that this will be the case someday. It all depends on how quantum computing develops, if it moves as quickly as regular computing did, then in 50 years or less, all major computing power will be quantum. But nothing that big ever happens in 10 years.

What worries me is that a larger and larger share of physical resources is being spent on mental universe. There is no physical return.

Bitcoin (blockchain technology) is a prime example. So much oil and natural gas wasted to maintain an incredibly inefficient database which doesn't live up to its hype - it's not trustless, it's not decentralized, it's not a monerary unit and it's not a measure of value. It's like spending trillions of watt-hours to print a few million baseball cards.
AI is another drain. Without an underlying truth source like Wikipedia (yes) or NYT or human trainers, token generation by LLMs is nothing intelligent. When all human-produced data has been mined, AI will stagnate as surely as an exhausted mine or an exhausted oilfield.

Current neural net "AI's" implementations, with their huge computing and data center power needs, are horribly inefficient compared to the human brain operating at 20 watts.

It is not each operating AI out in the field that needs those gigantic multi-gigawatt computing centers. That is for TRAINING the AI. Once the AI is trained, the result of the training can be used in MILLIONS (or BILLIONS or TRILLIONS) of copies, for many OOMs (google it) of operations. So TESLA's big training center will at some point train a driving AI that finally does what is needed. That training AI will be copied on to many MILLIONS of cars, with each car spending something like 100 Watts to run its self-driving computer for that car, and that is with CURRENT technology. To put 100W into context, you could run the AI for about 8 cars on the power needed to run one blow-dryer.

As to the other applications of AI. the chatGPT on your laptop or phone runs the equivalent of a few minutes at 20W to answer a query. In my experience, the answers it provides would have taken me 10 to 15 minutes to produce. So I think chatGPT is already more efficient than human intelligence.

And we are just barely at the beginning.

R:

Post New

Falling Knives FAQ

---

Contact Shrewd'm
Contact the developer of these message boards.