• jj4211
    link
    fedilink
    English
    arrow-up
    15
    ·
    5 hours ago

    While frequently true, the recent events seem to be more the opposite, deploying improvements to systems that weren’t broken (yet).

    The FAA? Upgrading to a new piece of network equipment. An act of finally working toward refreshing long neglected infrastructure.

    Microsoft? Some developer pushed a windows os change in July, and circumstances fired off now to expose it as flawed.

    Banking? Again, some developer at one bank pushing some change.

    As to why all at once, could be coincidence, but if it isn’t the likely connecting factor is AI. You need to create the complex configuration for a piece of network gear? What a pain, just feed Claude the current config and ask to port it to new equipment. Want some nebulous software improvement but don’t have any ideas? Have Claude look for, provide a solution, and also have it “audit” and “test” its own solution. I have seen all of the above happen and go horribly wrong when I am actually in a position to intercept, but it is super popular.

    • AEsheron
      link
      fedilink
      English
      arrow-up
      16
      ·
      5 hours ago

      Read a comment elsewhere that the banks aren’t quite comfortable enough with AI to let it make the code. But, what they are doing, is using AI to hunt for vulnerabilities, and get people to patch them. And the AI are finding tons of holes, so there is pressure to push tons of updates to fix them, because if their AI can find them, then hackers’ AI can find them. And so the workload has exploded with tasks labeled Priority 1, Critical! non-stop, resulting in sloppy updates.

    • Katana314
      link
      fedilink
      English
      arrow-up
      2
      ·
      5 hours ago

      It is very possible they became pressured to perform an upgrade because something on the old system was compromised and out of support.

      Legacy stacks quite often maintain old tech, and then somewhere in there will be a library that has a critical exploit and has been totally replaced for future major versions.

      • korazail@eviltoast.org
        link
        fedilink
        English
        arrow-up
        1
        ·
        5 hours ago

        Unsure if you’re defending or objecting to the comment. Flawed and outdated code needs to be replaced and that pressure should be maintained. The ‘Go Fast And Break Things’ model works OK in an internal corporate environment, but not when we’re talking about the FAA or banking. Outlook can fuck right off.

        Outdated hardware needs to be replaced, but I’m with JJ about the likely root cause. Some boss decided that the cheapest way to fulfill their job/contract was to install or deploy whatever ChatGPT told them would work.

        • PuddleOfKittens@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          2
          ·
          4 hours ago

          Outdated hardware needs to be replaced,

          No it doesn’t. Nuclear launch systems were running on floppy disks in the 2010s, IIRC, if it ain’t broke then “fixing it” is at best a nice to have.

          Now, if it’s insecure then we should replace it. Unless you’re replacing it with something even less secure (like AI, perhaps).

          • korazail@eviltoast.org
            link
            fedilink
            English
            arrow-up
            1
            ·
            2 hours ago

            fair - not broken (yet) doesn’t mean shouldn’t be maintained, though.

            Most solid state things are pretty much static, by definition, but there’s still stuff like plastic dry rot or metal corrosion. We’re talking decades, but we’re also talking decades. If that critical router failed outright the result might have been worse while a replacement was sourced, with a follow-on failure when the code bombed too.

            I’m not defending the outage – there should have been better testing – but I’ve been on the receiving end of a minimal but crucial server’s power supply capacitor failing (the post-mortem was wild to figure out what broke). That server was ancient and should have been replaced before it failed and the fire drill to recover it (even just taking a backup and restoring to a VM) was unpleasant.

      • jj4211
        link
        fedilink
        English
        arrow-up
        1
        ·
        5 hours ago

        On the bank, a possibility, on Microsoft, based on what they detailed sounded like a go at performance optimization.

        For security, I get it, there is little time to lose in this scenario, usually. But huge volume of changes for change’s sake are also in play.