In a rare four-hour talk, the reclusive founder reveals an almost Daoist philosophy of AI—AGI as a tide no company can own, he argues China's only real gap with America is compute.
Exactly. At work, we’re already looking at getting our own compute for an open model for some automatons we built. Google’s models are alright, but they are retiring them too fast. So we want to get off that treadmill. Plus a local model gives us opportunities to experiment with LoRAs and techniques like what cactus hybrid did with a small head predicting certainty (https://news.ycombinator.com/item?id=49010782). Plus, the big players are removing sampling options thinking they know what’s best when it is obvious they don’t.
Exactly. At work, we’re already looking at getting our own compute for an open model for some automatons we built. Google’s models are alright, but they are retiring them too fast. So we want to get off that treadmill. Plus a local model gives us opportunities to experiment with LoRAs and techniques like what cactus hybrid did with a small head predicting certainty (https://news.ycombinator.com/item?id=49010782). Plus, the big players are removing sampling options thinking they know what’s best when it is obvious they don’t.