Hacker News new | past | comments | ask | show | jobs | submit login

It's good that there are businesses like this testing the legality of scraping. The notion that scraping should be illegal is absurd IMO.



Forbidding scrapping should be allowed and respected, especially scrapping by businesses.

Scraping by itself shouldn't be illegal per se but not respecting policies and using glorifed bot nets won't help anyone. Scraping costs the hosting person/ company money and/ or resources. For example, the advertised use case of scrapping a job board to create an aggregator for job postings is maybe good from the point of someone searching for a job but a hefty punch to the hosted of the job board that has potentially hefty costs (running the board, moderating, ...) without any gain.

Scrapbee is not helping anyone but themselves and is only challenging the legality of commercial botnetes IMO.

Edit: changed 'public botnet' to 'commercial botnet' in the last sentence


It's a tug of war between "acceptable behavior" and "bad behavior", but in the interest of an open internet, this cat and mouse needs to exist. Service providers negatively impacted by scraping can price in their mitigations.


They don't respect robots.txt or have a way for websites to opt out. These might not be illegal but mighty rude.


If I'm making a $1m a year in revenue and the absolute worst you can call me is "rude" - I'm down.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: