This might be controversial, but you and I both have the means to mass collect data, or find illicit datasets already collected. The kind of data collection that we don’t have access to (the kind that’s taken from your phone without your consent) isn’t really helpful for training LLM’s. But, again, if you have the means to replicate their methodology to begin with then you likely already have all of the material. You’re not going to recreate their model on consumer hardware anyway.
They’re just not advertising where that data is (and neither should anyone here)
if you have to break the law to be able to compile it yourself, its not foss.
Not if you consider apps like jellyfin or plex to be FOSS, but even that comparison is apples and oranges because training a model that big isn’t something you can do on your own hardware. Just because they haven’t given you the data to alter the model doesn’t mean they haven’t given you everything you need to use it with your own data and your own hardware. I get that people inherently distrust AI companies (and Chinese companies especially, but I won’t get into that here), but I think it’s misplaced here.
unless i can compile it myself, its not foss
They aren’t going to break the law for you. If you want to train your own LLM you’ll have to source your own copyrighted dataset for the task.
Jellyfin doesn’t come with a media library, you have to ‘rip’ your own dvd’s and home videos. Same deal.
thats precisely my point. if you have to break the law to be able to compile it yourself, its not foss.
even if regular joes like you or me had the means to mass collect the data they did.
This might be controversial, but you and I both have the means to mass collect data, or find illicit datasets already collected. The kind of data collection that we don’t have access to (the kind that’s taken from your phone without your consent) isn’t really helpful for training LLM’s. But, again, if you have the means to replicate their methodology to begin with then you likely already have all of the material. You’re not going to recreate their model on consumer hardware anyway.
They’re just not advertising where that data is (and neither should anyone here)
Not if you consider apps like jellyfin or plex to be FOSS, but even that comparison is apples and oranges because training a model that big isn’t something you can do on your own hardware. Just because they haven’t given you the data to alter the model doesn’t mean they haven’t given you everything you need to use it with your own data and your own hardware. I get that people inherently distrust AI companies (and Chinese companies especially, but I won’t get into that here), but I think it’s misplaced here.