Luu Tuyen to TechnologyEnglish • 13 hours agoTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comexternal-linkmessage-square63arrow-up1350arrow-down14cross-posted to: [email protected]
arrow-up1346arrow-down1external-linkTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comLuu Tuyen to TechnologyEnglish • 13 hours agomessage-square63cross-posted to: [email protected]
minus-square@[email protected]linkfedilinkEnglish20•6 hours agoBullshit. This bot doesn’t identify itself as a bot and doesn’t rate limit itself to anything that would be an appropriate amount. We were seeing more traffic from this thing that all other crawlers combined.
minus-square@[email protected]linkfedilinkEnglish4•4 hours agoNot rate limiting is bad. Hate them because of that, not because they’re a not. Some bots are nice
Bullshit. This bot doesn’t identify itself as a bot and doesn’t rate limit itself to anything that would be an appropriate amount. We were seeing more traffic from this thing that all other crawlers combined.
Not rate limiting is bad. Hate them because of that, not because they’re a not.
Some bots are nice