The more interesting question is does a site's individual policies apply? Does scraping a site constitute acceptance of the site's terms if no compliance is sought before it's use?
Surely Google would be violating millions of such policies.
In this case it sounds like a login was required to access the data, in which case the account holder has probably had to agree to such terms.
As someone starting a business which plans to use web-crawling to a large extent I find the publicly-accessible side of this very interesting.
Unfortunately it's still a very gray area - nobody seems to have an answer for me. The advice seems to be "try it, but you may have to defend it", which means a lot more money than most startups have.
In this case it sounds like a login was required to access the data, in which case the account holder has probably had to agree to such terms.
As someone starting a business which plans to use web-crawling to a large extent I find the publicly-accessible side of this very interesting. Unfortunately it's still a very gray area - nobody seems to have an answer for me. The advice seems to be "try it, but you may have to defend it", which means a lot more money than most startups have.