Through the efforts of @[email protected], the lemmy-safety tool can now run via docker, which should help you run it without having to mess with python on your end.
For those who don’t remember, lemmy-safety is a script you can read to clean up your pict-rs images from potential CSAM.
Thanks for the shout out!
Full disclosure, I use podman, not docker. If anyone has any issues with this using docker, let me know and I’ll get it fixed ASAP. I’m not 100% sure the --device option works the same way with docker.
I added instructions on how to add the nvidia-container-toolkit repo on the two distros I have (one rpm based, one deb based). If adding the repo is different on your distro, please consider adding it to the instructions. The instructions on the nvidia-container-toolkit web page are… subpar in my opinion.
for docker the syntax is
--gpus allhttps://docs.docker.com/config/containers/resource_constraints/#expose-gpus-for-use
bonus: syntax to expose the gpu in a docker compose
deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu]Would it be too much for you to ask to test that out and update the documentation? I don’t have docker, and installing it would mess up my podman-docker setup, which would impact some things I have running. podman-docker simulates docker with podman so I can use docker-compose with podman.
Thank you.
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.I was getting this error with docker-desktop installed, but it worked after purging and installing docker-ce instead, and running with the
--gpus allcommand.Interesting. This might be a different between podman and docker. I was using podman in my setup. Unfortunately we might need two different sets of instructions for podman and docker.
Thank you for this great service to the community.
Does this run some ML wizardry on the images?
Yes, it’s uisng the clip image2text model
By the looks of it, pretty much. Not in the sense of building an AI model, but more like traditional image recognition. Seems to process everything locally too, which is a plus; no sending data off to unknown servers.
Applying AI-voodoo to a non-existing problem with unknown side effects? Sign me up!
This was created in response to mass posting of CSAM on lemmy recently. It might be one of the reasons its a “non-existing problem”.
