“The other assumption — those so-called scaling laws — has been that by simply building bigger data centers with more GPUs (the expensive, power-hungry graphics processing units used to both run and train these models) and throwing as much training data at them as possible, they’d simply start sprouting new capabilities, despite there being little proof that they’d do so. Microsoft, Meta, Amazon, and Google have all burned billions on the assumption that doing so would create something — be it a human-level “artificial general intelligence” or, I dunno, a product that would justify the costs — and it’s become painfully obvious that it isn’t going to work.”