Notice: You have been identified as a bot, so no internal UID will be assigned to you. If you are a real person messing with your useragent, you should change it back to something normal.
Anonymous B joined in and replied with this , 15 minutes later[^][v]#1,468,389
Hyperscalers have huge debt on balance sheets and are building speculative supply. Data centers account for up to 2% of GDP. The railroad buildout was 2%-4.5% of GDP through the 1873 bust. Thing is, steel tracks lasted 50 years and data centers need a reboot in 3%-5%, and live in essential but abstracted data processing. Railroads moved physical items. I wonder if we will see a GPU buy up on marked down assets.
Anonymous C joined in and replied with this , 9 minutes later, 25 minutes after the original post[^][v]#1,468,392
@previous (B) > data centers need a reboot in 3%-5%
This implies that computation is getting much more efficient very fast, and that is a good counter to people saying the electrical use of AI isn't worth it.
Companies are paying market price for tokens today, but the trend is for performance to increase as price decreases.
Anonymous D joined in and replied with this , 2 minutes later, 27 minutes after the original post[^][v]#1,468,393
@1,468,389 (B)
That’s one of the things I’ve been wondering about. There’s an argument that most technological developments led to over-investment in infrastructure buildout. But the thing that’s different about computers is they don’t last that long. From what I found on google, generally the chips themselves can last 5-10 years before they stop working in a data center environment, but for frontier AI models they only remain relevant for 2-4 years before they become outdated. Even if they get the economics of it to work out, it will be an incredibly wasteful industry. On the bright side, maybe gamers will finally be able to get their low cost GPUs at the landfill someday. Lol
Anonymous D replied with this , 3 minutes later, 35 minutes after the original post[^][v]#1,468,396
@previous (C)
I think the issue is more that in order to make a model more powerful you have to make the model larger, and to run a larger model, you have to use more powerful GPUs. Anthropic and OpenAI have to keep spending money on more hardware so they can keep pushing the frontier, because if they ever stop then other companies will catch up. They’re all essentially selling the same product, the only differentiating factors are the price and how powerful the model is.
Anonymous C replied with this , 2 minutes later, 40 minutes after the original post[^][v]#1,468,398
@1,468,396 (D) @previous (D)
That's more to do with VRAM. Quantization and other tricks help too.
But even if the models stayed the same size they'd want to upgrade the chips every few year because it's worth it when the data center is on 24/7.
There's also a trend for small optimized models. Companies use general purpose when that's all that's available and then try to optimize because that saves them a lot of money.
Anonymous D replied with this , 24 seconds later, 40 minutes after the original post[^][v]#1,468,399
The issue isn’t whether AI will exist or not, the issue is whether you can really exponentially scale the size of AI models forever to keep having this frontier advantage. Because there is a physical limit to how small you can make a transistor. Moores law is pretty much dead. So what they’re doing is they’re using more parallel processing, so they’re making bigger chips.
Anonymous D double-posted this , 1 minute later, 42 minutes after the original post[^][v]#1,468,400
@1,468,398 (C)
More space efficient models are dumber though. It makes sense if you’re running an open weight model, but for a frontier AI model it would defeat the purpose.
Anonymous D triple-posted this , 20 minutes later, 1 hour after the original post[^][v]#1,468,402
The number of transistors on a chip has been increasing, but the rate at which clock speeds have been increasing has slowed down substantially over time because that gets faster when transistors get smaller, but they haven’t been getting smaller as fast recently.
It’s hard to find a good graph to demonstrate it because it seems like they’re going with whatever the record clock speed was that year rather than the average consumer clocks speed. But you can see that clock speeds stopped increasing but core counts increased.
Anonymous D quadruple-posted this , 4 minutes later, 1 hour after the original post[^][v]#1,468,403
So we felt like our computers were getting faster over time because of increased parallelism and some efficiency tricks like better branch prediction, out of order execution, jit compilers, etc. We can keep making more powerful computers, but the catch is they’re going to have to be physically larger with higher power demands.
Anonymous D quintuple-posted this , 10 minutes later, 1 hour after the original post[^][v]#1,468,404
Although, this problem is not impossible to fix. The performance per watt of the human brain is orders of magnitude more efficient than computers, so it is physically possible, we just don’t have the ability to manufacture it or the scientific understanding of how the human brain is that much more power efficient. And we’re not close.
I asked AI to put numbers on that:
The human brain consumes about 20 watts of power while handling roughly 1 quadrillion (10¹⁵) synaptic operations per second, yielding an extreme energy efficiency of about 50 trillion operations per watt (or ~5 teraflops per watt of equivalent biological processing).
By comparison, a modern data-center GPU like the Nvidia H100 performs roughly 200 trillion operations per second (200 TOPS/FLOPS) while drawing around 700 watts of power, translating to roughly 285 billion operations per watt.
Anonymous D sextuple-posted this , 7 minutes later, 1 hour after the original post[^][v]#1,468,405
Part of why is even though neurons are larger than transistors, the brain is a 3-dimensional object vs a 2D chip, so that extra dimension allows for a lot more connectivity. That’s not completely the reason, but that’s part of it.