Just watched the latest pod from Cerebras for those of you who are interested.

So Cerebras finally released a Gem with 850B parameters inside, that is to my knowledge the largest gem ever released. But not only that they also included a 100B cached model that runs 3 times faster than the 850B model but takes way less compute and memory. <br><br>The biggest thing is that if you have a 100B or above AI (think 2 years or so at the speed we are seeing) you can cache that model and get twice the speed of the large and about twice the speed of the cached smaller model (10 times less compute required). <br><br>Also, Cerebras can support up to 100s of TB for a single model if they decide to do so.