@[email protected] to RedditEnglish • 3 months agoReddit shares plunge almost 25% in two days, finish the week below first day closewww.cnbc.comexternal-linkmessage-square172fedilinkarrow-up1860arrow-down115cross-posted to: [email protected][email protected][email protected]
arrow-up1845arrow-down1external-linkReddit shares plunge almost 25% in two days, finish the week below first day closewww.cnbc.com@[email protected] to RedditEnglish • 3 months agomessage-square172fedilinkcross-posted to: [email protected][email protected][email protected]
minus-square@SacrificedBeanslink12•3 months agoRobots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.
Wasn’t there robots.txt at least?
Robots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.
There was, and still is