Petal is one of the bots I block from accessing my sites. They were DoSed multiple times as Petal's spiders ignored the "429 Too Many Requests" response header. They just don't care and perform several requests per second anyway. I won't use a search engine that does not even support basic rate limitation measures.
I've blocked them in the past too for this, but it seems they are better behaved now. I don't have any longer term data, but over the past week they averaged 1.5 request per minute and that is divided among ~630 unique websites.
Infact, the UAs I blocked are no longer in use it seems so they are no longer limited.
EDIT: btw, not endorsing their search or anything, just mentioning thye may have learned and improved ;)
Which major engines or sites allow you to reject ALL cookies? Google is just planning doing that, but pretty much all services require accepting at least essential cookies. You are picking up on them for not being holier than Pope.
An intriguing thing about Huawei is that even after being cut off from Google, cut off from TSMC, cut off from any 5G phone SOC from any supplier, they still keep on developing and marketing towards the European consumer market.
This must be extremely expensive, and there is no way they are making money here, at least for the foreseeable future.
they produce lot of wearables (I think that's nowadays their most popular product in Europe - bands and watches), routers, scales and other devices (now they are trying with large smart displays "TVs" (not technically TV since it doesnt have DVBT AFAIK), so it makes sense for them to keep their ecosystem alive
plus despite being legally different company from Honor, which is now extremely growing in China and already overtaken again Xiaomi (and I guess they will be also agressive in Europe now with Google services) I am still pretty sure they share knowledge/development among companies
What we definitely need is an independent, not for profit, decentralized (ideally p2p) search engine. Because of conflicts of interest, web search can not be left for companies.
Decentralized search is pretty tricky to get working well. Crawling and indexing is trivial to parallelize, but to get a performant search engine you need a good ranking algorithm, and that pretty much requires a central node with access to the entire link graph.
Sometime ago, I thought about some form of simple ranking that could give good results. I thought how about giving points for sites that are linked by a small set of "sources" like hacker news, wikipedia, stack overflow, reddit, news sites...
I always wondered how such results could be. Never had the knowledge to test it.
Those points are then given site-wide no matter what a certain page content would rank? For sintance on dev.to you have a sizeable amount of garbage/spam articles which would benefit of this...
That's probably doable, but the problem is this information needs to be available upstream.
Say you search for "seafood" and there are 50,000,000 results from 100,000 domains because everyone just loves making websites about clams and sushi; it's not really feasible to send you all of those results, your query would take 15 minutes and that's not a nice experience. The tricky part in search is finding the needles in the haystack.
So the crux is each of the index shards needs to know the ranking to be able to send you just the most relevant results.
Petal pounds sites. Ignores 429, hits the same pages repeatedly
Easy block for us.
Behavior is so similar to bad actors, I was surprised to find that it had documentation.
What are your daily driver tech stacks? Desktop OS, apps, Phone, ...
How are you able to entirely avoid anything that ever asks for your location (and not just deny the request and keep using)?
You do know that any site can infer your location within a reasonably accurate ballpark (e.g. metropolitan area or district) from just your IP address, without asking.
> You do know that any site can infer your location within a reasonably accurate ballpark (e.g. metropolitan area or district) from just your IP address, without asking.
Which is more than enough for a search engine and given that it'll be accurate in 80% of cases, no need to bombard the user with intrusive permission prompts right from the start.
Yes that is a better user experience. Suppose we want to design a site which is as nice as possible to the users who dislike the whole location business. So we make it opt-in. We can (A) use the location stuff in the browser or (B) just go by IP address.
Under both A and B we have our own opt-in preference. Under A, when the user enables the op-in preference, they are additionally bugged by the browser to allow that. Under B, they are not bugged by the browser: we just go ahead and start making use of their IP address.
By the way, I do not see anywhere in the Firefox settings a place where you can set what your location is. If users could specify their location, then that would be a good reason to prefer that mechanism over IP-based geolocation where the only way to change your apparent location is to use a VPN.
You mean, like, the search engine should be regarded as a news story or article whose title is "Petal Search", that being the piece of text from the HTML <title> tag that goes into the tab?
Tried it. Firefox asked me "Allow petalsearch to access your location?". Closed tab.
Better luck next time.
EDIT: I know I can deny it, I know other search services do the same, I know it can be used to search "close to me". My point is that trying to get this information by default is a bad sign.
> My point is that trying to get this information by default is a bad sign.
Suppose we want to create a search site that can use location info (for some entirely legit purpose), but we don't ask by default. That implies it must be an opt-in user preference.
To maintain a user preference, we have to create a tracking profile for the user. That's "evil" too, oops, and there will be a cookie pop-up.
Say we are already doing that; we have various preferences. Why would we complicate them by adding a location preference, if that preference is already handled in the user's browser? Firefox indicates right in the address bar that you denied location access to the site.
I always click deny on these requests, but if I had to code it, I probably wouldn't do it differently.
It shouldn't ask for location by default, but offer a toggle in the UI to enable local searches, which when enabled would ask for it. This way, it won't annoy the user for their location every time they search in a private window.
pretty much all search engines do this to provide location based results, nobody stops you from Denying such request, but it seems quite stupid to pick up on one service when many other services do the same thing
edit: my bad, seems major search engines don't ask for location, but I remember seeing this request way too often on many sites to consider it something unusual to complain about specifically at this site
Neither Google, Bing, DDG or Yandex does it. They probably get rough location from GeoIP. If you use private browsing, this is extremely annoying, every session you have to click "block".
> If you use private browsing, this is extremely annoying, every session you have to click "block".
I just verified that if, in Firefox, you checkbox "[x] Block new requests asking to access your location" in the location settings, then a private window inherit this; and the popup no longer appears.
For some reason, the per-site blocking is not inherited by the private windows. Maybe it's considered an unacceptable info leak or something. Or possibly it might be this: the individual per-site location setting can be positive or negative (accept or deny). If the settings as such were inherited by a private window, that would be bad because if any accepts are present, they conflict with the concept of a private window. Maybe if the private windows just inherited the deny entries, that would be reasonable; could be nobody has bothered coding that up, thinking that users who are bothered about location can just use the global switch.
It sounds like you should have clicked on "learn more", and then uninstalled Firefox.
Because, there you learn that:
> If you allow location access, Firefox gathers information about nearby wireless access points and your computer’s IP address, and sends this information to the default geolocation service provider, Google Location Services, to get an estimate of your location. That location estimate is then shared with the requesting website.
If you disagree with location access, such that you won't use an application or site that asks for it, why would you use a browser that goes out of its way to make that possible in the first place? If asking for it is morally wrong, then adding the code to the browser which figures out that location and gives it to Google first, and then to the site must be a capital crime.
Also from "learn more":
> How do I stop all sites from asking to access my location?
> If a site indicates to Firefox that it wants to access your location, Firefox asks, by default, whether you want to grant permission. You can set Firefox to automatically deny new permission requests to all websites, without asking.
I feel like asking for location right away is a bit intrusive, especially as the API isn’t granular enough to specify whether they just want my country, city or exact location. The former might’ve been OK, the latter is way too much.
Edit: OK, not only realted, this is built by Huawei: https://en.wikipedia.org/wiki/Petal_Search