Well, that’s the rub, right? Garbage in, garbage out. For an LLM, the value is predicting the next token, but we’ve seen how racist current datasets can be. If you filter it, there’s not as much lot of high quality data left.
So yes, we have a remarkable amount of (often wrong) information to pull from.
Mhm, I wonder when we’ll have the resources to build one that can tell the truth from other lies. I suppose you have to learn to crawl before you learn to walk, but these things still having trouble rolling over.
Well, that’s the rub, right? Garbage in, garbage out. For an LLM, the value is predicting the next token, but we’ve seen how racist current datasets can be. If you filter it, there’s not as much lot of high quality data left.
So yes, we have a remarkable amount of (often wrong) information to pull from.
Mhm, I wonder when we’ll have the resources to build one that can tell the truth from other lies. I suppose you have to learn to crawl before you learn to walk, but these things still having trouble rolling over.