• Kühlschrank
    link
    fedilink
    English
    arrow-up
    7
    ·
    23 hours ago

    Yeah 1 trillion params has to be pretty much all of human written creation

    • Echo Dot@feddit.uk
      link
      fedilink
      English
      arrow-up
      5
      ·
      6 hours ago

      Isn’t parameters just the number of virtual dials that can be changed, essentially the number of neurons, I don’t think it’s got anything to do with the actual amount of data.

      After all distilling works (somehow), by taking all the knowledge of the large AI with a lot of parameters and putting it into a smaller AI with fewer parameters. The amount of data hasn’t changed only the number of parameters. Presumably the inverse is also true, you can have a lot of parameters but not a lot of data.