Basically, search engines are used to find things publicly available on the internet, policing should be done on the actual hosts of the content.

Same goes for ISPs, they are here to provide access to publicly available computers, without interfering with the content or accessibility.

Second, if I search for “the Pirate Bay website” I’m pretty sure there’s an objective result that should be there at the top.

1 / ∞ = 0

The title tag of the site is as follows: "Download music, movies, games, software! The Pirate Bay - The galaxy's most resilient BitTorrent site"

And it's the number one search result for that phrase on DDG. Because search engines are actually programmed well enough to figure out that's the right answer. I'm not sure why you think it's the wrong answer to show that site as the first result, but I'm pretty sure everybody else in the world disagrees with you, and would prefer that they see the website they are searching for, instead of news articles about the site they are searching for.

Now, if only you could find some spaghetti to go with all the other stuff...

I don't think there's sombody manually removing each result from search queries by hand. That wouldnt meet latency constraints

I think it's easy to take an absolutist approach when it's somebody else who faces the consequences. In some ways it's a rehash of the arguments when the Navalny app was delisted.

In a vacuum, "should someone follow politically motivated requests to take down content?" is an easy question to answer. But when you have to worry about consequences like giving up your freedom, I can't say that I'd have the courage to follow through. The Dutch prisons might be nicer than Russia's, but asking someone else to give up their freedom for ideals is still a tall order.

For example, leave a gap in the search results where the missing one would have been, perhaps with some details of the result which do not fall under the removal request (for example, the title of the page, even if the URL must be removed).

This used to be the case. As the internet grew in size search providers felt it was in best interest in general user to cull these results and present their definition of best at the top. They wanted to solve the needle in the hay-stack problem by taking all the noise out and funneling the best result to the top.

Best is a subjective term and I don't think I agree with it. However, your definition is not the same as Googles. Search providers can define what their search engine does the same way a restaurant gets to decide what's on its menu even though all restaurants serve food.

I mean, zero moderation is some kind of internet libertarian ideal, but there's plenty of examples out there what that looks like - 4chan, 'old' reddit, parler & co (which ironically censor a lot of stuff), all of the dark web, etc.

You state "policing should be done on the actual hosts of the content", but what if the host says "lol no"? That's what's happening here; BREIN demanded (and has done so for at least the past 10 years, probably longer) that since the hosts are untouchable and not legally required to comply (these things get complicated once you go abroad), they went for the ISP's and search engines instead.

If something is deemed illegal, and the source is untouchable, you have to go for whatever passes it on. I mean hard drugs are illegal, they are shipped across the world, and only when they arrive at a port is there something that can be done to stop it.

Should Google de-list Pirate Bay? In my opinion no.

Should Google de-list some things? No doubt yes. That is how society works - your rights end where somebody else's start.

As the down-voting of parent shows, HN gets defensive when this is pointed out.

you say that and then I see this:

>Should Google de-list Pirate Bay? In my opinion no.

Looks like pretty black in white about Pirate Bay here.

Also doesn't look like you are too much against censorship. You do not even discuss whether it should be or not. I sense it is pretty white there for you too.

It reminds me how russia controlled media spread idea that not everything is black and white but when they decided to attack Ukraine using military and took Crimea and parts of other two regions with force it suddenly became black and white for the moment of attack and then again coming back to 'oh it's not black and white we didn't took anything'

I personally love how "everything is not black and white" concept is used to justify censorship and promote everything that serves totalitarian dreams.

There are clear laws and court decisions which Google must obey. The title says "voluntarily", which is the actual problem in this case.

You could post whatever you wanted to Usenet. You could email whatever you wanted to anyone, and they would see it in their inbox. Heck you could “finger” to see who was online in remote networks, and “talk” to open a live chat with anyone, totally unmediated by any commercial product. You could log in to open FTPs and trade files.

We’ve been to that particular heaven, and most people didn’t like it.

Why? While trafficking in CSM was an important and awful consequence, the negative that dominated most people’s experience was spam. That’s why web forums beat Usenet; that’s why centralized webmail beat a forest of naked email servers. Etc.

People don’t actually want unmoderated search; it would be choked with spam.

What people actually want is whatever they want, as easily and cheaply as they can get it. The Pirate Bay and other file sharing services are popular not because they represent some sort of libertarian ideal, but simply because they shovel a lot of great content to people for free.

Of course they’re popular! A person handing out $10 bills on the street corner will be popular too. But it kind of matters where he or she got those $10 bills. There are societal side effects we might want to manage.

Many of us prefer that heaven to the hell of centralized control, censorship and commercialism that we have now!

There is a couple of problem with this, first I see no problems at all with 4chan, old reddit, or parler. So using them as an example of something bad that should be banned is ridiculous, parler in peculiar was gaslite into a false narrative around 1/6 protests when in reality most of the communication, as stated by the FBI, where done via Facebook and other larger platforms not parler. Parler was the scape goat. Parler should fail for any number of reasons but not because of censorship, it is a terrible platform technoligically, it is terrible security posture, and various other usablity issue.

Then there is the unintended blowback this type of censorships leads to. Such as increased levels of echo chambers and extremism. Take for example the fall out from Backpage removal. Did it end trafficking, and prostitution. No, not by a long shot, it just make the criminals harder to catch, the victims harder to find, and made things more dangerous for legal age sex workers.. Good Job Government.

I like the Freenet's author take on this. He says that porn, terrorists, drug dealers and all kinds of "abhorrent" content are a price you pay for the lives of whistleblowers and people fleeing from their dictatorship states, sects, and terrorists; and for preservation of valuable, but controversial, content.

Well, it makes sense in case of Freenet, which offers full anonymity and resilient storage (you can upload content, and as long as there are people viewing it, it will propagate itself along the node connections path, and it's impossible to take the content down). Torrents, Reddit, and 4chan are different, so maybe here the trade-off will be different too.

It's a myth that there was zero moderation. What the libertarian ideal was privacy and anonymity. Where a user could say something like sharing a link to the pirate bay and not worry about it too much when it got moderated. Websites would respect a users privacy. Moderation was about cleaning the site, not sharing a users personal data with authorities. It wasn't about free speech so much as a kind of place for users on the internet with no real world consequences for sharing, for example, a link to the pirate bay.

They all moderated.

If a host doesnt comply, they get denied by the firewall.

4chan has rules that are enforced, especially since the split between 4chan and 4channel. What might give the impression that it's "some kind of internet libertarian ideal" is that people often not report posts. But when they do, the janitors usually do their jobs. These days even troll posts are deleted!

That is unlikely. Google breaks many privacy-related laws across the globe and deals with billions of euros of fines regularly. They have an unlimited wallet and couldn't care less about spending a few billions to extend their dominance over the Internet.

Is default but optional moderation a bad thing?

So this raises so many questions. Has _no party_ requested to delist TPB under DMCA?!

That sounds incredible if true. But the only explanation I have for Google feeling like they need to go this far, delisting on their own initiative! It's like a scenario that should never have to happen, and something they should never do either.

So what's going on here?

Update: Ugh, editorialization... Apparently court has been involved here and while Google may not be individually targetted and forced YET, so "voluntary", it's easy to see how Google see the writing on the wall and choose to comply. No point in fighting this with such a clear cut DMCA violation. But in this case I'm still surprised it took this long!

Google willingly interferes, in a heavy-handed way, with both search suggestions and search results, with both Google Search and YouTube. Just try to look up any topic on YouTube, and you'll have to get past the hundreds of mainstream media channels covering the event, before you actually find the original video. They artificially promote "authoritative sources" which are anything but, since they may be second-hand coverage of original videos that get buried in search results.

Google's results are not only manipulated by clever SEO people[0], but by employees[1], in direct contradiction to Sundar Pichai's sworn testimony. Some of that is very defendable. But there's zero transparency. Given their search monopoly, and that most people aren't aware that search results are manipulated, Google Search is the web. Websites that get delisted are presumed to have ceased to exist. Focusing on browsers (Firefox) is good, but no longer enough.

That's why Google Search itself is now a threat to the open web. Switch to other engines. DuckDuckGo, Brave Search are the ones I trust, and there's tons more.

[0]: how many times have you tried looking for a machine's user manual, mistyped the model number, and somehow "found" a webpage with the manual for a product that didn't exist? and then modified the model number further and found more results from the same website, with relevant keywords and yet another incorrect model number? My understanding of SEO isn't good enough to know how they do that. I don't believe websites can dynamically alter their index to show up for so many typoed search queries.

[1]: https://medium.com/@mikewacker/googles-manual-interventions-...

Indeed, they co-opt every search now. Anything related to local businesses or directions, flights and hotels, etc. And everything I try to find about pop culture or new tech or video games seems to be a page full of YouTube links. I want to grab the appropriate people at Google by the ear, and scream, "I DO NOT WANT TO SIT THROUGH A VIDEO ABOUT THIS." I guess I'm just old or something, but video seems like SUCH an inefficient way to impart information that could just be typed out, and then read. It's not even like it would be more work for these video producers. They're typing out the script to read over some generic video loops. JUST PUT THE COPY ON A PAGE! GAH!

For reference: https://huggingface.co/transformers/preprocessing.html

If it is video, I move on to the next contender. That's simply a hard no.

Certainly, there is still a lot of text in the world, but the reason things are moving towards videos is exactly because C-f doesn't work. You'll click a video hoping it will help, it won't, but the advertiser will still pay the content creator. If you visit a text-based website, search for what you're looking for, and don't find it, you'll be gone before the ads even load. And that was in a world where advertisers paid for text ads, which isn't the current world.

I'd argue that Google these days fall in line more of a publisher than a search engine. More often than not I'm feeling that the search engine is trying to sell me something rather than do the work requested and I don't mean ads (which I think are good monetization balance).

Same goes for algorithm social networks.

Fwiw, my teenage son prefers video content over written content. Even if you hate it, there are other people that love it.

Learn to search (hint: searchlores), it is an immensely useful skill in this age of information; you don't need to be irked by bad tools any more.

Though that's in very large part an inevitable and widely-foreseen consequence of Google's other arm: advertising.

The monetisation of content is the root of great evil:

Writing for money and reservation of copyright are, at bottom, the ruin of literature. No one writes anything that is worth writing, unless he writes entirely for the sake of his subject. What an inestimable boon it would be, if in every branch of literature there were only a few books, but those excellent! This can never happen, as long as money is to be made by writing. It seems as though the money lay under a curse; for every author degenerates as soon as he begins to put pen to paper in any way for the sake of gain. The best works of the greatest men all come from the time when they had to write for nothing or for very little. And here, too, that Spanish proverb holds good, which declares that honor and money are not to be found in the same purse—honora y provecho no caben en un saco.

-- Arthur Schopenhauer, "On Authorship"

https://en.wikisource.org/wiki/On_Authorship

It is because people badly want to and enjoy writing that we have vast quantities of books. If people looked at writing as a commercial endeavour, most of us would never, ever consider writing a novel - it's a crazily oversaturated market, to the point that writing a novel to make money is much like playing the lottery: You invest far more time that you could have spent on other things, and the potential payout for the vast majority of authors is below minimum wage most places.

We might lose some great works from that tiny proportion of writers who earn well enough that their income might cause them to make more commercial choices in their writing instead of writing the best they could. Maybe. I'm not convinced.

That said, outside of writing novels, things are different. You can probably earn more writing for a sketchy content farm than most of us will ever earn from writing novels, so the point is not entirely invalid, but it is not a good fit for literature.

EDIT: Changed second to last paragraph for clarity, see response for mangled original.

Works, in Schopenhauer's time, and today, are commissioned, written, edited, and promoted for their commercial potential in ways that are directly addressed by Schopenhauer's rant.

The works which aren't (excepting those written or promoted for propagandistic value, itself a major share of promoted works) rather prove his point.

The essence of Schopenhauer's concern is that information and entertainment should be intrinsically motivated, not extrinsically motivated. Trying to please the public, or tap into the revenue stream (typically advertising), or feed the algorithm, etc., rapidly leads to corruption and devaluation of content, which is what I was addressing.

Even where good works are promoted this often happens without benefitting the original author or creator. Sometimes in literature (Mark Twain struggled financially his entire life, F. Scott Fitzgerald's Great Gatsby only entered the American canon during WWII, as a cheap paperback shipped overseas to soldiers, long after Fitzgerald's own pickled death). The music world (both classical and popular) and art world are similarly rife with examples of this as well.

Old Art was a very sour puss, but with good reason.

Sorry, but I can't agree. He explicitly argues for fewer but better works, and laments that this can't be achieved as long as you can profit from writing, but the fact is *this can't be achieved*, as we can se from the fact that most writing is not profitable today, and never has been.

Thinking that if only the commercialisation stopped, literature would suddenly be in a better place is utter nonsense that is based on a fantasy world where writing well is far easier than it is.

It's not commercialisation of writing that is stopping us from getting better literature, but the difficulty.

While there certainly is a lot of commercial writing one can easily dismiss, the notion that a profit motive prevents writers from writing from extrinsic motivation ignores that most literature is written by people who can never in a lifetime hope to life of their writing.

There has never been more books written with extrinsic motivations.

As I pointed out, there may well be a point here in that some types of commercialised content often is pure dreck. But only a fraction of a percent of literature is "commissioned, written, edited and promoted for their commercial potential".

A lot of it for good reason, because a lot of that content is also pure dreck. But even fantastic authors often struggle to get published and only a fraction of them make a living of it.

The argument that Schopenhauer is espousing a fantasy probably does have merits, and we've the benefit of another century and a quarter of publishing (the novel really only emerged into mass culture during the 19th century). There's the challenge of recognising greatness as it first appears (it almost always takes at least some time for that awareness to dawn), and the sheer arbitraryness of assessments as well.

That said ...

We're still left with the fact that what financial compensation promotes is at the very least appeal to a minimum common standard. That lesson has emerged again and again in the history of mass-media, beginning with street carnivals and players, the penny press (both newspapers and "penny-dreadful" fiction), the mass media of radio and television (Murrow's "Wires and Lights in a Box" and Minow's "Vast Wasteland"), and the Web, mobile media, Reddit, and Facebook (once Literally Harvard, now ... not so much).

In the case of online content and services, it seems that true gems are virtually always underfunded. (The constant gripes aimed at Wikipedia's apparent mostly sufficient endowment are a rare exception.) Sites such as LWN eke out an existance, Linux Journal ultimately folded --- for all that adtech supposedly pays for the Internet, it certainly failed there.

Looking at collections of great books, what strikes me is how many of them predate not only recent history (say, the past 50 years), but all of modernity. How much of this is measurement bias and a varianty of the Lindy Effect, and how much is a well-placed assessment on whatever truth there may be in merits is of course very hard to say. But it's quite persistent.

As I read through works (fiction and non), what I'm struck by is how little of what is recent is truly novel. The refrain from Ecclesiastes, that there is nothing new under the sun, isn't entirely accurate, but it's far more so than it has any right to be. I suspect it's a combination of pressures to publish and an ignorance (often cultivated through deliberate presentism and deprecation) of earlier literature that leads to this.

On your "difficulty* point: part of the cause is also the haste and rush to publish leading to just plain sloppy work. That's not entirely new, and the practice can even be an art form (Kerouak's On the Road). But far too many leading works --- bestsellers and the like --- are riddled with poor editing, rambling structure, typings and misspellings, and poorly-checked facts. There are of course exceptions, but again it seems that the pressure to publish and transact leads to poor results.

(Self-published works can of course exhibit this to a far greater extent, but they're also produced under profit pressures, and with far fewer available resources than traditionally-published works, for the most part.)

I suspect that underlying this discussion are two questions that haven't been asked yet, so I'll ask them:

1. What makes a work "great"?

2. What are the circumstances in which such works emerge?

If you could provide any examples of "recent" (I'll give you anything published since Schopenhauer wrote, so 1891, though more recent would be more compelling) books meeting both the "great" and "not commercially motivated" criteria, either fiction or not, I'd be interested in seeing what you come up with.

(Others can contribute as well.)

... I actually think that would be more interesting than continuing the debate above. I think your argument has some merits though I'm not fully convinced.

Though I'd argue almost no published novels other than possibly subsequent works by bestselling authors are generally commercially motivated.

As someone who have published two novels: If you write to get rich, you're an idiot. It can happen, but it's so fundamentally unlikely that it's grossly irrational to write with that as motivation unless you've already been signed to a publisher. Even then it's a dubious gamble.

I've had the experience of trying to keep a friend well-stocked in audiobooks, and have made something of a practice of seeking out "best of" lists (best short-stories of the year, best books of the year, etc.), and ... find that there's not a whole lot that shows up in any decade that's especially good. Their own tastes tend to mid-century, relatively classical, and literary, and tends to discount themes increasingly prevalent in post-1960s literature (I feel the exposure would do good, but we're talking preferences here). Literary awards, "best books of" anthologies, etc., tend to improve the pickings but remain slim.

And again, financial motivation isn't helping, and by promoting far more low-quality literature, further clouds the field. For books --- big, solid, meaty, information-dense objects that take hours or days or weeks to assimilate, quality assessment itself is difficult. And financial motive, in authoring, publishing (cultivating authors, commissioning works, encouraging production, editing and rewriting assistance, packaging, marketing, and promotion) don't help the process.

Schopenhauer's argument isn't that most authors are financially motivated. It's that financial motivation leads to worse writing.

Again, you're focusing on anecdotes and "most authors" rather than the industry's own revenue focus. I find both uncompelling.

There are of course legions of writers (of books, of music) ... and other creators (art, photography, etc.) who do chase that dollar. Back in the day, Writers' Market was full of all the standard encouragement and secrets-of-the-trade for breaking through. That same advice is now much more scattered, but you'll find it online, much in the form of YouTube videos on storyboarding, either generally, or using writing tools (Scrivener seems popular) specifically oriented for that task. The objective is to quickly create cookie-cutter literature that fits a market's wants and needs, not creation of great literature.

Typical current advice (there are many video results):

Storyboarding generally: https://youtube.com/watch?v=JGeVXafMkwM

Scrivener: https://youtube.com/watch?v=AJyGox2ldHo

In the research world, it's grants-chasing.

The issue is that creative media (print, visual, video, music, etc.) follow power laws and tent-pole effects. There are a few big hits, there are an awful lot of also-rans. Ironically, the more global the market, the fewer winners (rather than numerous top-ten contents, there is only one --- any practice based on cardinality, that is, ranking, is inherently zero-sum. One of the better treatments of this I've found is in Charles Perrow's Complex Organizations (1972, 1979, 1986) (https://www.worldcat.org/title/complex-organizations-a-criti...), in the chapter addressing the music industry. Interestingly, its discussion of hit-making, labels, talent, backing performers, and corruption-dependent distribution systems (radio payola and the like) has eerily strong similarities with the tech sector's VC, founders, tech talent, and overly-credulous tech media (and lately, mobile-device app markets). There's a powerful lesson for HN's audience here.

I don't understand this sentence.

I meant to write that we might lose some great works from that tiny proportion of writers who earn well enough that their income causes them to make more commercial choices in their writing instead of writing the best they could.

I'll edit the comment so it makes more sense.

These provide services to areas which themselves can be generally ranked in terms of wealth, leading to the opportunity for a progressive, unmetered, universal payment mechanism. In this system, various publishers (and notably local news and information sources) would be afforded compensation or revenue, and subscribers would have access to any available information.

Remote-but-locally-focused providers (e.g., the small-town paper across the state, or the country, or in another continent) would be compensated principally through their local service providers (with some balancing within states having widely-varying economic distributions, e.g., NYC / upstate New York, Chicago / downstate Illinois, coastal / inland California, etc.

National / general-interest content (national or international news, e.g.) might receive a share of local/regional revenues, say, for national TV and radio broadcast news organisations, nationally-distributed newspapers (WSJ, NYT, WaPo in the US as examples), and book and magazine publishers. (Other countries and regions could make similar allocations.)

Much of this would offset online advertising, which costs a typical household of four on the order of $1,800/year within North America and EU ($455 billion projected 2021 spend, ~1 billion population). This is what people *are already paying for "free" online content. Direct content spend is on the order of $100/person, or 1/4 the cost of advertising. Distributed across all households (pro-rated by wealth as noted), on an all-you-can-eat basis, would be a remarkable game-changer.

Earlier:

https://news.ycombinator.com/item?id=27803591

https://news.ycombinator.com/item?id=10077674

https://news.ycombinator.com/item?id=20545446

https://news.ycombinator.com/item?id=15683719

> we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

[0] http://infolab.stanford.edu/~backrub/google.html

Then we can turn it into a verb. "Go Duck <term>"

They could even capitalize on the famous autocorrect substitutions: "Need an answer? Duck it!"

Weird names can be overcome (see "google"), but weird + clunky + needs to disrupt massive established players... what are they thinking?

At least lose the "Go".. DuckDuck we can work with.

AskJeeves would also be abbreviated in conversation to just "Jeeves", which is definitely as the original creators intended. "DuckDuckGo" doesn't have these features going for it.

Bing.

AltaVista.

https://en.wikipedia.org/wiki/Veronica_(search_engine)

Maybe I'm wrong and someone has a list of somewhat successful general search engine with a somewhat serious names besides Alta Vista (and I never thought about that before) and Fast?

DuckDuckGo has the disadvantage of not only sounding silly, but also being composed of silly-sounding real words. And not having a secret geeky meaning that people find out about and then feel like they're part of an elite club.

It was, simply put, the biggest named number anyone knew and was frequently used as such in arguments... e.g. I watched that movie 1000 times! Oh ya? Well I watched it a google times.

Same here, as someone who was in high school in the late 90s and switched from AskJeeves/Yahoo/Altavista to Google around ~2000, part of the appeal was "Somebody named a search engine after the biggest number meme we know? AND it's fast? AND it gives good results? Lemme check this out..."

// did that get auto-corrected? if so, ironic, as it’s said "Google" was an accidental misspelling of "googol" by Sean Anderson back in ‘97…

http://graphics.stanford.edu/~dk/google_name_origin.html

I'm sure the irony is lost on the Freedom-Democracy-landers.

How about a 3-column search that's Google, DDG (so, Bing), and Yandex, with a single search field for all three? Bonus points for syncing it up so if I click, say, the "Images" tab on one of them, it switches the others over, too.

https://www.allsides.com/unbiased-balanced-news

What are those things that Yandex can't find? It seems to me it shows pretty relevant results for "putin's golden toilet" [1], "putin's blasting houses" with US State Department / Soros funded web-site as one of the top results [2]. A query with "navalny's statement" shows both his twitter and his web-site on the first page.

When comparing to google, "gab" in yandex shows the web-site's link as the first result. Google doesn't show it at all, there is only CNN, wikipedia, some woke dictionaries all telling you gab is not something you should search.

So, who's got totalitarian system with censorship after all?

[1]: https://yandex.com/search/?text=%D0%B7%D0%BE%D0%BB%D0%BE%D1%...

[2]: https://yandex.ru/search/?text=%D0%BF%D1%83%D1%82%D0%B8%D0%B...

[3]: https://yandex.ru/search/?text=%D0%B7%D0%B0%D1%8F%D0%B2%D0%B...

Now, "голосование на пеньках" (voting on stomps) is a mockery of Putin's constitution referendum by opposition. When searching this phrase in Google the front page is again Putin controlled media: tass, rbc, kommersant, rg.

Yandex apart from the aforementioned web-sites has US State Department / Soros funded meduza and svoboda.org, liberal (in American sense) tjournal, Deutsche Welle.

It seems Yandex gives preference to the most relevant sites when Google decides what's good ("credible") for you.

[1]: https://yandex.ru/search/?text=%D0%B3%D0%BE%D0%BB%D0%BE%D1%8...

It'd be weird for Google to bury bad think only for non-American IP addresses but allow it for Americans if it was on some woke crusade for the benefit of the US government. Now, I'm not sure why it's removed from your results, but my point is that what you cited as a piece of evidence for malign intent on Google's part isn't actually supporting evidence.

It's worth noticing that it's pointless to censor requests like "gab.com" anyway (though that happened in my case). A person typing them already knows what they want. Now, try "right wing social media", "free speech media", "social media without censorship", etc. For me it doesn't show anything relevant but the links to the "correct" think tanks. Does this fact prove the malign intent of Google? No, but it's still there and you know it (except they consider this a virtue).

One of the problems is that black hat SEOs (among other people) have huge economic incentives to spread misinformation, including about the election. This is of course, an eternal cat and mouse game. This is not new, I remember when gmail started filtering chain letter scams more than a decade ago.

[0] https://search.brave.com/

I compared it with brave search and the latter remains unfiltered for the time being.

Today I see this:

DDG: https://imgur.com/a/CfDXfoq Brave Search (for comparison): https://imgur.com/a/vvEfaMF

And the fact that the our societies have been rushing towards totalitarianism doesn't make Putin Mr. Nice Guy all of a sudden. It is not that simple.

This feels important to me, and at this point it actually feels like we have more choice than we have had in 15 years and we have a chance to change history.

If you are one of the many who have noticed that Google has declined in recent years you can start by just changing to DDG, or even Bing as default. (Use Google as fallback if you want and see if you realize the same as me: their results are just as broken. If you don't, you can consider yourself lucky for now and go back to Google, but at least keep this in mind as they start heating your frog pot as well ;-).

Do experiment with alternative engines: Kagi seems close to production ready now and search.marginalia.nu is just a delightful tour de force as to what can be done by a determined person!

FWIW, people still often react weirdly when I tell them to 'bing' something. "When was that movie released?" "I dunno, just bing it". Reactions are usually confusion, mild amusement, but sometimes I've had some weird aggressive/anger issues emerge when I've suggested someone use bing. Strange...

Google still gets lots of searches from me, but the better DDG does, the less it'll happen.

There no real alternative to the big players. It's google or bing, basically.

The reason I mention this is that just like with mainstream media and unlike alternative medias and newsletters it seems easier to forget the bias of search engines.

it should work - it's up to society to make this work by passing laws to that effect.

As soon as they introduce their own bias(curating content, recommendations, emergence of so called "walled gardens") it opens the floodgates.

The history of communications and information is rife with channels and sources of all description being controlled.

If anything, "neutral" sources are all the more subject to influence as they simply abrogate the role of censor / amplifier to others. All the more so if they're heavily used and relied upon, as this means that there is an influenceable audience present.

As my friend Woozle puts it:

Because of a high percentage of the population being present, there is now substantial power to be had by influencing the discussions that take place.

https://old.reddit.com/r/dredmorbius/comments/5wg0hp/when_ep...

I don't believe that. I came from a time when Yahoo and Altavista were used. Google took over that market very fast (personal experience, not data ;)). So I'm sure that also now, people could shift very fast when the results are better somewhere else.

Although I can agree on YouTube, because that's a different beast. They actually host almost all the web video content. So there is no 2nd party to easily switch to.

Google could plainly say we blackhole this and that content and ignore these keywords and don’t autocomplete on these others, etc. same for Bing and any other. Just be open and transparent about the purposeful bias then the user can find and use the one that most matches what they’re looking to use.

Do a search for `American inventors`, and see for yourself. Google will show you that the majority of American inventors throughout history have been African-American. You may be skeptical of Google's claim (disguised as search results), but the next generation, working on their school reports, will see the world as Google wants them to see it. Now imagine how much of what we learn about the reality of current events is coming from the same source with the same agenda and policies regarding narrative vs. reality.

But in case you are thinking of contradicting Google's claims, remember: that's what their campaign against "misinformation" is intended to silence. They are indeed serious about misinformation.

A couple of years ago, when they were testifying to Congress that they did not deliberately distort search results. At the same time, someone else in the organization explained why they deliberately bias results. Oops. I can only guess that it was some underling who, in a sea of leftists back in the office (and probably a few non-leftists who know what would happen if they were outed) took for granted that biasing search results for political objectives was something to be proud of. They didn't realize what their execs knew, that if you're going to distort the truth for a higher objective, you don't admit doing so.

On a Google web page that I can no longer find, they publicly explained (it was public, not an internal page) that search results that improved society were sometimes more important than simply reporting the facts. That must have sounded good to an internal audience, but apparently it had to be reworked into externally calling it a fight against misinformation.

(I make no claims about why this is—I haven't a clue—and don't, personally, really care that it's the case, but am just confirming that I do indeed see the described behavior)

I'm not sure that is the best example. For one, this only applies to image results, regular search seem normal. For another, Bing (and the search engines that use Bing) does the same thing.

Given that, I think that there may be some sort of guerilla SEO campaign to distort these results. That campaign could be leftists trying to change perceptions of race or it could be rightists trying to change perceptions of Google/search.

Bing Search, which has an independent web index from Google/Bing, has the same issue. It's interesting how search results can be biased without any malevolence or even any intervention by search engines.

It could just be that there is a lot of activity on the web around African-American inventors which Google is reflecting. Google isn’t an encyclopedia.

Their campaign against misinformation is purportedly intended to reduce the spread of lies and propaganda. There might be some collateral damage where sources of lies and propaganda get generally downgraded (even when telling the truth), but is that really a problem? Most of us do that in our interpersonal relationships. If you are a liar, I’ll tend not to believe you, even if when you tell me something true. (I’ll assume there is some agenda or some part which may be false.)

I don’t feel like you have presented sufficient evidence to support your (apparently politically motivated) case.

Of course it is if Google is one of the sources of propaganda rather than an objective source of truth. In that case "misinformation" is not a neutral question of factual correctness.

When the judge and prosecutor are the same person, the judge considers most of the prosecutor's arguments to be true and the defense to be an unreliable "source of lies" that needs to be kept in line.

It turns out the line between "politically inconvienent" and "misinformation" can be very fuzzy, even for high profile / important topics.

There is plenty of evidence if you look for it (and I am pretty sure I fall on the opposite side of the political divide from the commentor you are responding to.)

Edit: Google's biases don't just hurt the political right. Google's quick bar for the 2016 Democratic presidential primary was happy list Hillary with all her pledged votes in addition to those she'd actually won, in violation of journalistic standards and Google's previous practices.

Whatever cool/hilarious/human reason they have for it (I don't care), I wish they also added another alias that is easy to remember and type.

I think Google was a harder name to remember for non-nerds who weren't familiar with the rather obscure word.

There is a short alias: As someone else mentioned, duck.com redirects to duckduckgo.com

The game is probably more or less universal, but the name is unrecognizable.

Should everyone in your opinion know the production companies of all upcoming films so they can look up their website to view trailers?

Russia spreads lies propaganda and misinformation by all channels possible including RT yandex and anything under their control. If you unaware what russia does you should educate yourself.

If you try to get any information from russia controlled channels not only you'll be heavily mislead and manipulated you'll also support that aggressive regime that actively attacks any light of democracy and freedom in the world. Russia already have used not only propaganda but also military to attack any attempts to build free society respecting democracy especially in neighboring countries.

For instance in Ukraine yandex is simply completely blocked as result of a war that Russia started against Ukraine and it is done for a reason. Do not be less wise then Ukraine and just do not go there.

True, the US typically attacks countries that are farther away, but their devastation of Irak and Afghanistan is much worse than Ukraine or Georgia (another victim of Russian aggression) have suffered in the last 20 years at least. And while Iraq and Afghanistan were by no means democratic regimes, we can see plenty of democratic governments toppled by American power and their allies - In Yemen in 2014 (mostly Saudi led, but with US arms and support), Haiti in 2004, Nicaragua and El Salvador in the late 1970s, Iran in 1953. There are also many other despotic regimes that the US successfully helped against revolutions that may have toppled them (most memorably, in South Vietnam, but also at various times in Iraq and much of South America).

Empires are always evil, and they are always seeking to spread their propaganda. The only way of getting a slightly clearer picture is to use sources from multiple such empires and try to build a more accurate picture than any one of them wants you to.

So yes, please use both Yandex and Google for search, read RT as well as the New York Times, Al Jazeera, AFP, etc.

We, the USA and its allies, have directly killed thousands to tens of thousands of Iraqi and Afghan civilians, and many more "enemy combatants" (it's important to remember that under the rules of the "War on Terror", any adult male is counted as an enemy combatant by default, unless explicitly proven otherwise). The Russians killed as many or more during the Soviet invasion.

The opposition forces are also responsible for a great many civilian deaths (many more than the US led forces, at least in Iraq, according to official numbers). But this is what happens when you invade a country, especially one already divided along sectarian lines: the people who rise up against you start seeing enemies and collaborators everywhere and killing their own as well.

Were there sectarian killings in Iraq and Afghanistan before the war? Absolutely, but a mere fraction of what happened during the war.

There is nothing comparable between Russia and US and it's allies because Russia is not democratic country and as such values and intentions are completely different.

>So yes, please use both Yandex and Google for search, read RT as well as the New York Times, Al Jazeera, AFP, etc.

I am sorry but you cannot recommend deliberate lies machines as the source of information. I mean you can but the value of RT as source of information is zero if one has brain and knows how RT works.

For the source of information one should check the source and how it is reliable. If it's not then it is not the source of information.

The suggestion was to use Yandex as an additional source when American search engines are not providing the expected results, not to rely on it exclusively without questioning the information provided.

No one pretending that some one is perfect but degree of imperfectness for intentional killer of democracy as the main value is very different from the imperfectness of democratic state relaying on functioning democracy to exist.

One of the russian propaganda trends is to present to naive listener that "everybody do lie". Of course to solve that manipulation one should recall about degree and intentions. Russia used military to attack Ukraine and lied about it usage and still does. Russia does it x100 in comparison to democratic world and the intentions are not to have some profits but to kill on a massive scale to destroy democracy with any way possible.

With all respect the level of lies coming from russia is not comparable to democratic countries with all their imperfectness.

>The suggestion was to use Yandex as an additional source when American search engines are not providing the expected results, not to rely on it exclusively without questioning the information provided.

Questioning information is one thing and looking for information in the source of deliberate lies ill-designed to question the truth with intention to destroy the truth and any trust is another. Suggestion to use yandex and RT was wrong.

One way to do it is to have some path through the site that lists relevant data, but also includes links at the bottom to typoed versions of the data. The web crawler will chase those links, and if each page that you retrieve generates more such typo links, you can feed the crawler an infinite stream of typos.

Google will consume quite a large chunk of an infinite stream before it trips any kind of "circuit breaker" to cut off how much crawling they'll do on one site. They have a lot of storage.

I find this problem extremely frustrating. Does anyone have suggestions for how to easily find primary video sources, instead of the massive heap of commentary that a youtube search yields?

Google search seems to be something of a feedback loop at this point.

Which is fine as 1) it's good for searching Google properties and 2) it opens the door for web search engines that search the web.

I understand Google's utter dominance in search, but Google is not "the open web".

It will still show results if you explicitly use `site:bluelight.org`, otherwise it never appears in search results - prior to Google delisted it, it always ranked very highly (top 5).

I can't help but wonder about the harm that decision had made - people may well have literally died because of it.

Which makes it easy to find new Piracy websites [2].

It only mentions the TLD not the URL

https://transparencyreport.google.com/copyright/overview

https://transparencyreport.google.com/copyright/request/1280...

That seems so naive now. It's become painfully obvious that Google's mission is to manipulate access to information for profit.

site:[famous non western tracker].com [insert music here]

zero results.

Same query, on duckduckgo: Everything there.

Another query:

[famous ebook sharing sites] on google.

Non-related results

Same query on duckduckgo: First hit is the correct website.

I'd like for search engines to be highly and independently responsive to CP, spam, SEO gaming, fraud, malware, cyberstalking, cyberbullying, revenge pr0n,and similar threats. Relying on court orders would simply be too slow.

I'd also like for search engines to be responsive to propaganda and disinformation. This calls for a much more nuanced response than the first category, and is inherently political rather than merely criminal.

In the case of general copyright claims, specific targeted removals if executed reasonably and fairly, which quite arguably YouTube's ContentID is not, as an example, should be possible with a fairly minimal degree of legal mechanism (e.g., DMCA 512 takedowns under US law). Though that process should be amended to reduce abusive exploitation of such measures.

Delisting entire domains for copyright infringement absent a specific legal action ... moves into much stickier waters. Ultimately, torrent sites will likely have to provide their own well-known search alternative which is resistant to such threats. I'd like to see that possibility further developed.

(I'm aware that many such sites already claim the legal shield of serving as directories to content, not as hosters of that content themselves, which would inlcude The Pirate Bay itself.)

Widespread global civil disobedience in the face of overwhelmingly asymmetric and self-serving copyright legislation and case law is among the very few avenues the average citizenry have of voicing opposition to such laws. And that fact alone should carry great weight.

I'd like for search engines to be strongly resistant to removals based on overly-broad copyright claims,

If you can't find the torrent site, you Google enough to find a community that will link to it. If you can't, you ask on Twitter. Etc. Etc. It's never going to stop someone determined. Credible threats of jail-time may, but these minor changes by Google and others aren't going to change anything important.

  <anything> filetype:torrent

I wonder if they'd say the same if they were the ones who would be fined/prisoned for breaking those laws.

The fix is to change the law, not asking companies to break them

https://yacy.net

Google is a private company can do what they want, but this is still a pretty poor decision.

Not when they are a monopoly.

The problem is that the indexes (google, bing, yandex) won't let you freely aggregate them anymore. DDG is a client of Microsoft just like startpage is a client of google.

Five years ago I would still google pretty much any error message or problem.

Nowadays I typically go to directly to the project's GitHub and search through issues there, go directly to their documentation, or just browse the code from my editor and figure stuff out myself. I have like 15 tabs just corresponding to the stuff I'm working on right now. It is pathetic Google managed to become so awful they managed to undo a decade of conditioning.

Crippling their search intentionally even more won't help.

yandex.com is seriously awesome for infosec related searches and I guess (it's been a few years since I had to pirate something) that it's equally great for pirated content.

One example is i like chill hop radio. whenever i search on DDG might get results, but google will usually send me directly to their bandcamp page which the the ideal result