Furbland to [email protected] • 6 months agorulebots.txtimagemessage-square33arrow-up1322arrow-down12
minus-square@TriflingToadlink3•6 months agoreally? ddg will show me reddit links, did they have to make a webscraper or something
minus-square@[email protected]linkfedilink4•6 months agoThere’s a cutoff date, anything indexed before the robots.txt was changed stays in the index
really? ddg will show me reddit links, did they have to make a webscraper or something
There’s a cutoff date, anything indexed before the robots.txt was changed stays in the index