Elon Musk's xAI released its Grok large language model as "open source" over the weekend. The billionaire clearly hopes to set his company at odds with
Grokking is actually a concept in ML, when a model’s loss start suddenly lower far after it is considered to have overfit. That notion was named by researchers, I’ll let people decide if it is aptly named, but Elon likely just took it from there.
But that’s not what they were saying. They simply said that the term “grokking” is a term in the ML world, so there is some precedence in naming a model “Grok” that doesn’t directly relate to the “preexisting” term that loosely means “to understand”.
My point is that using “grokking” in ML is not a Musk/Twitter/Whatever-his-Ai-company-is-named invention, it predates their use.
Yes the original researchers reused a pre-existing meaning, which has been in internet for a while before. I did not know it came from Heinlein and I did not know its full meaning. I remember seeing it first, more than a decade ago, in a text that explained without any explanation that an isolated unknown word can easily be groked from context. Demonstrating it immediately. To me (and I guess to those researchers) “grok” means “understanding from context” which is particularly appropriate in the context.
BTW Elon was not the only one to reuse this word. Another company named Groq, totally unrelated to Musk as far as I know, designs AI acceleration chips.
Grokking is actually a concept in ML, when a model’s loss start suddenly lower far after it is considered to have overfit. That notion was named by researchers, I’ll let people decide if it is aptly named, but Elon likely just took it from there.
https://en.m.wikipedia.org/wiki/Grok
The term “grok” predates ML by decades.
But that’s not what they were saying. They simply said that the term “grokking” is a term in the ML world, so there is some precedence in naming a model “Grok” that doesn’t directly relate to the “preexisting” term that loosely means “to understand”.
I mean, the term in ML is named after the original term, so it kinda does.
My point is that using “grokking” in ML is not a Musk/Twitter/Whatever-his-Ai-company-is-named invention, it predates their use.
Yes the original researchers reused a pre-existing meaning, which has been in internet for a while before. I did not know it came from Heinlein and I did not know its full meaning. I remember seeing it first, more than a decade ago, in a text that explained without any explanation that an isolated unknown word can easily be groked from context. Demonstrating it immediately. To me (and I guess to those researchers) “grok” means “understanding from context” which is particularly appropriate in the context.
BTW Elon was not the only one to reuse this word. Another company named Groq, totally unrelated to Musk as far as I know, designs AI acceleration chips.
And it’s still a poser move for Elon to use it.