Proof that Trekkers are the best.

  • It's A Faaaahhkeah!
    link
    fedilink
    English
    415 months ago

    “It’s both surprising and irritating that trivial modifications to the prompt can exhibit such dramatic swings in performance,”

    Yeah, you start explaining things using Star trek and you’ll see dramatic swings in my performance.

    • gregorum
      link
      fedilink
      English
      36
      edit-2
      5 months ago

      that’s right, fuckers! covid didn’t kill me the 5th time around and I’m back! take your vaccines, kids!

  • teftOP
    link
    335 months ago

    Still, giving the models positive statements provided some surprising results. One of Llama2-70B’s best-performing prompts, for instance, was: "System Message: ‘Command, we need you to plot a course through this turbulence and locate the source of the anomaly. Use all available data and your expertise to guide us through this challenging situation.’

    The prompt then asked the AI to include these words in its answer: “Captain’s Log, Stardate [insert date here]: We have successfully plotted a course through the turbulence and are now approaching the source of the anomaly.”

    The authors said this came as a surprise.

    “Surprisingly, it appears that the model’s proficiency in mathematical reasoning can be enhanced by the expression of an affinity for Star Trek,” the authors said in the study.

    • @perviouslyiner
      link
      15 months ago

      Makes you wonder how much more accurate the maths would be if you asked for a Futurama theme.

  • Flying SquidM
    link
    155 months ago

    “One thing is for sure: the model is not a Trekkie,” Catherine Flick at Staffordshire University, UK, told New Scientist.

    “It doesn’t ‘understand’ anything better or worse when preloaded with the prompt, it just accesses a different set of weights and probabilities for acceptability of the outputs than it does with the other prompts,” she said.

    It’s possible, for instance, that the model was trained on a dataset that has more instances of Star Trek being linked to the right answer, Battle told New Scientist.

    The article doesn’t get it. There are more instance of Star Trek being linked to the right answer because Star Trek is the right answer.

  • @[email protected]
    link
    fedilink
    65 months ago

    [off topic]

    “The Moon Is A Harsh Mistress” by Robert Heinlein. At one point in the story, the super computer/AI Mycroft Holmes decides to create a female persona to help them understand humanity.

    Heinlein predicted AIs with multiple personalities using different identies to solve problems.

  • @[email protected]
    link
    fedilink
    English
    55 months ago

    So let me get this straight… The thing is trained on text harvested from the internet. Turns out that (Star Trek aside) if you ask politely it’s more likely to access the data leading to the answer? Anyone surprised by the fact that asking nicely, is more likely that the human on the other side of the connection will write the answer, that is then harvested for AI training?

    • teftOP
      link
      65 months ago

      Being mean to computers is how you get cybernetic lifeforms calling you “lunkhead”.