@[email protected] to Lemmy Shitpost • 1 year agoChad scrapersh.itjust.worksimagemessage-square97fedilinkarrow-up11.06Karrow-down129
arrow-up11.04Karrow-down1imageChad scrapersh.itjust.works@[email protected] to Lemmy Shitpost • 1 year agomessage-square97fedilink
minus-square@ViscloReaderlink73•1 year agoJust a heads up for companies thinking it’s wrong to scrap: if you don’t want info to be scraped, don’t put it on the internet.
Just a heads up for companies thinking it’s wrong to scrap: if you don’t want info to be scraped, don’t put it on the internet.
But, but, norobots.txt!
Chad doesn’t care!
uses curl manually instead