Luu Tuyen to TechnologyEnglish • 13 hours agoTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comexternal-linkmessage-square63arrow-up1350arrow-down14cross-posted to: [email protected]
arrow-up1346arrow-down1external-linkTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comLuu Tuyen to TechnologyEnglish • 13 hours agomessage-square63cross-posted to: [email protected]
minus-square@[email protected]linkfedilinkEnglish6•4 hours agoI think a common nginx config is to just redirect malicious bots to some well-cached terrabyte file. I think hetzner hosts one iirc
minus-squareSomething Burger 🍔linkfedilinkEnglish6•3 hours agohttps://github.com/iamtraction/ZOD 42kB ZIP file which decompresses into 4.5 PB.
I think a common nginx config is to just redirect malicious bots to some well-cached terrabyte file. I think hetzner hosts one iirc
https://github.com/iamtraction/ZOD
42kB ZIP file which decompresses into 4.5 PB.