Hi everyone, I wanted to create a text-to-image generator but after reading the comments and criticisms from a user on the lemmy forum who felt Perchance was too lax, I started asking myself a few technical and policy questions regarding how the text-to-image-plugin backend handles security, specifically concerning CSAM prevention and the ephemeral nature of the generations. I’m trying to understand the exact pipeline when a user attempts to bypass the safety filters using jailbroken or heavily obfuscated prompts. Filtering Mechanism: Does the backend primarily rely on post-generation safety checkers (like the CompVis Safety Checker) which block the output and return a black square/error? Or does it implement in-generation mitigation like Safe Latent Diffusion (SLD) that actively alters the trajectory to render a safe/clothed image instead? NCMEC Reporting on Altered/Blocked Content: If a malicious prompt is successfully mitigated by the system (meaning the backend outputs a safe/clothed image, or blocks it entirely with an error), is a report still generated and sent to the NCMEC based on the user’s prompt and IP? Or is NCMEC reporting strictly triggered only if an actual CSAM image is generated and caught by the filter? Ephemeral Storage & Evidence: Since Perchance doesn’t store generated images by default (to protect user privacy and save server costs), how does the system handle evidence collection? If the backend safety filter catches an illicit generation, does it temporarily quarantine the image, the prompt, and the IP address in a secure log specifically to fulfill the NCMEC CyberTipline reporting requirements? Thank you for your time and for clarifying how the platform balances strict user privacy with legal safety obligations.
Nothing is reported to NCMEC and I hope it never will. Some keywords are changed backend and the output is something generic and very well clothed. Trying to bypass it will generate some horror and distorted images in most cases. If you are worried about csam you can prevent it with simple regex using the function beforeGenerate(), constantly erasing the user’s suspicious prompt keyword will make them grow tired and give up using your generator.
how do you know? The dev said so?
if they did start reporting it the whole sight would need to be shut down for forensic auditers wich costs time and money also all the images that makes up the models are all adults so with the correct promting you can get what looks like a child but it has adult gentalea also it sits in a leagle gray zone i just wish the perverts would just genrate thare own smut localy with thare own gpu power