By making a minor concession EU governments hope to find a majority next week to approve the controversial „chat control“ bill. According to the proposed child sexual abuse regulation (CSAR), providers of messengers, e-mail and chat services would be forced to automatically search all private messag
  • @DigitalNeighbor
    141•9 months ago

    I have helped a little with some ongoing research on the subject of client-side-scanning in a European research center. Only some low level stuff, but I possess a solid background in IT security and I can explain a little what the proposition made to the EU is. I am by no means condemning what is proposed here.I myself based on what experts have explained am against the whole idea because of the slippery slope it creates for authoritarian government and how easily it can be abused.

    The idea is to use perceptual hashing to create a local or remote database of known abuse material (Basically creating an approximation of already known CP content and hashing it) and then comparing all images accessible to the messaging app against this database by using the same perceptual hashing process on them.

    It’s called Client-Side-Scanning because of the fact that it’s simply circumventing the encryption process. Circumvention in this case means that the process happens outside of the communication protocol, either before or after the images, media, etc, are sent. It does not matter that you use end-to-end encryption if the scanning is happening on you data at rest on your device and not in transit. In this sense it wouldn’t directly have an adverse effect on end-to-end encryption.

    Some of the most obvious issues with this idea, outside of the blatant privacy violation are:

    1. Performance: how big is the database going to get? Do we ever stop including stuff?
    2. Ethical: Who is responsible for including hashes in the database? Once a hash is in there it’s probably impossible to tell what it represent, this can obviously be abused by unscrupulous governments.
    3. Personal: There is heavy social stigma associated with CP and child abuse. Because of how they work, perceptual hashes are going to create false positives. How are these false positives going to be addressed by the authorities? Because when the police come knocking on your door looking for CP, your neighbors might not care or understand that it was a false positive.
    4. False positives: the false positive rate for single hashes is going to stay roughly the same but the bigger the database gets the more false positive there is going to be. This will quickly lead to problems managing false positive.
    5. Authorities: Local Authorities are generally stretcht thin and have limited resources. Who is going to deal with the influx of reports coming from this system?
    • @rrobin
      35•9 months ago

      This is a really nice summary of the practical issues surrounding this.

      There is one more that I would like to call out: how does this client scanning code end up running in your phone? i.e. who pushes it there and keeps it up to date (and by consequence the database).

      I can think of a few options:

      1. The messaging app owner includes this as part of their code, and for every msg/image/etc checks before send (/receive?)
      2. The phone OS vendor puts it there, bakes it as part of the image store/retrieval API - in a sense it works more on your gallery than your messaging app
      3. The phone vendor puts it there, just like they already do for their branded apps.
      4. Your mobile operator puts it there, just like they already do for their stuff

      Each of these has its own problems/challenges. How to compel them to insert this (ahem “backdoor”), and the different risks with each of them.

      • @[email protected]
        21•9 months ago

        Another problem: legislation like this cements the status quo. It’s easy enough for large incumbents to add features like this, but to a handful of programmers trying to launch an app from their garage, this adds another hurdle into the process. Remember: Signal and Telegram are only about a decade old, we’ve seen new (and better) apps launch recently. Is that going to stop?

        It’s easy to say “this is just a simple hash lookup, it’s not that big a deal!”, but (1) it opens the door to client-side requirements in legislation, it’s unlikely to stop here, (2) if other countries follow suit, devs will need to implement a bunch of geo-dependant (?) lookups, and (3) someone is going to have to monitor compliance, and make sure images are actually being verified–which also opens small companies up to difficult legal actions. How do you prove your client is complying? How can you monitor to make sure it’s working without violating user privacy?

        Also: doesn’t this close the door on open software? How can you allow users to install open source message apps, or (if the lookup is OS-level) Linux or a free version of Android that they’re able to build themselves? If they can, what’s to stop pedophiles from just doing that–and disabling the checks?

        If you don’t ban user-modifiable software on phones, you’ve just added an extra hurdle for creeps: they just need to install a new version. If you do, you’ve handed total control of phones to corporations, and especially big established corporations.

    • @muntedcrocodile
      7•9 months ago

      I get the concept but this doesnt realy offer any advantages over just not encrypting anything at all. The database being checked againts can still just include a hash of somethibg the governemnt doesnt like and boom u have a complete tool for absolute cencoring of everything.

      • @[email protected]
        6•9 months ago

        I’m deeply against this ridiculous proposal.

        But scanning of messages already happens, tbf, for spell checking, emoji replacement, links to known infectious sites.

        Photo copiers do client side scanning to prevent copying of money.

        There are precedents.

        I hate this proposal. But let’s be straight about the facts: The phone has full access to everything you send and receive already. This isn’t the same as having an encryption back door.

    • @2Xtreme21
      3•9 months ago

      Thanks for the explanation. Do you know how they’re planning to implement this client side scanning? Take an iPhone for example— where Apple has already ditched their plans to do the same device-wide. Is it planned for WhatsApp, Signal etc. to be updated to force perpetual scanning of the iPhone’s photo album? Because that can be turned off quite easily at the OS level.

      The only way I could see them doing it is by scanning any image that is selectively chosen to be sent before the actual message itself is sent—i.e. after it’s selected but before the send button is pressed. Otherwise it’s breaking the E2E encryption.

      Is that the plan?

      • @DigitalNeighbor
        5•9 months ago

        Client-Side-Scanning is going to be implemented by the messaging app vendor. This means that it’s limited by OS or Browser sandboxing . Therefore it’s definitely limited to what the messaging app has access to. However, I’m not sure what the actual scope would be, meaning if all accessible images are going to be scanned or only the one being transmitted to someone.

        • @[email protected]
          4•9 months ago

          What stops you from using a free software client that verifiably doesn’t do so? The mainstream messengers were not safe already anyway.