Why have language models become so impressive? Many people say that it's the size of the models. The large in 'large language models' has been thought to be key to the models' success: as the number ...

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models. Two years ago, Yuri Burda and Harri ...