So the people running the systems that underlie our lives are incompetent, and don’t know how to keep them running properly. I’m supposed to be reassured by that?

  • historicaldocuments
    link
    fedilink
    English
    arrow-up
    2
    ·
    2 days ago

    Back when I worked at Microsoft, they were converting all of the testers into telemetry and data engineers because everyone can test their own code and we can test shit in prod with AB testing and telemetry.

    As in one set of Windows users got the “A” updates and one set got the “B” updates?

    • xtr0n@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 days ago

      Yeah. It’s a common term in software development. Basically for any software that gets updated over the network you can setup a system to give the update to some percentage of users and then gather data on their usage and if it looks good, roll out the changes to more users. It is a very useful tool but should not be a replacement for common sense local testing.

      https://en.wikipedia.org/wiki/A/B_testing

      • historicaldocuments
        link
        fedilink
        English
        arrow-up
        2
        ·
        2 days ago

        Cool cool cool cool cool.

        So if there are two windows computers hooked up to, I don’t know, some commercial freezers, and the only requirement is that they have the latest updates installed, one computer might have the “A” updates and one has the “B”?

        Doesn’t have to be freezers in the example; it could be industrial sawmills, or computers running Labview.

        • hitmyspot@aussie.zone
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Generally, a/b testing is for marketing or feature updates, not security updates. However, even security updates are not rolled out to everyone at once. They get rolled out to a few, monitor for system borking problems, expand the the pool for updates, repeat.

        • xtr0n@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          3
          ·
          2 days ago

          Yeah. In practice with Windows the machines would all get the same update but Windows Update might say it’s availalable for each machine at a different time, depending on the roll out schedule. I think most of the experimentation happens on machines where the user opts into the Windows Insider program. But I haven’t paid close attention to Windows updates in years so take Windows specific details with a huge grain of salt.

          In general though, for some random software they might release an A version witha ad at the top of the screen and a B version with the ad at the bottom and then use telemetry to figure out which version is the most effective.