Before, there was a very strong if not existential incentive to get indexed by scrapers, in particular search engines. Now the situation is flipped where there's a strong incentive to put up ever higher walls, in particular for LLM scrapers.
The difference is obvious: LLMs don't give back. They just rob your life's work and monetize it locally. Why would a platform or its users open up to that?
Before, there was a very strong if not existential incentive to get indexed by scrapers, in particular search engines. Now the situation is flipped where there's a strong incentive to put up ever higher walls, in particular for LLM scrapers.
The difference is obvious: LLMs don't give back. They just rob your life's work and monetize it locally. Why would a platform or its users open up to that?