Search engines like Google are indispensable, able to find answers to all of your technical questions; but along the way, the fun of web surfing was lost. In the early days of the web, pages were made primarily by hobbyists, academics, and computer savvy people about subjects they were interested in. Later on, the web became saturated with commercial pages that overcrowded everything else. All the personalized websites are hidden among a pile of commercial pages. Google isn't great at finding those gems, its focus is on finding answers to technical questions, and it works well. But finding things you didn't know you wanted to know, which was the real joy of web surfing, no longer happens. In addition, many pages today are created using bloated scripts that add slick cosmetic features in order to mask the lack of content available on them. Those pages contribute to the blandness of today's web.
The wiby search engine is building a web of pages as it was in the earlier days of the internet.
Ah yes, I remember stumbling into this page when looking for cold-case/solved mystery stories. I got through all 13 pages in one afternoon! Excellent story and writing.
I started to work on something like it. With intention of filtering-out ad-serving websites first (thanks to EasyList filtering rules) and heavy-JS second. I'm glad that someone already did something like this.
Is there some more information about it? I found that it is done by https://www.wireservice.ca
Is there a technical info?
I was wondering how big the index is. Because it may be just possible to serve the multi-gigabyte index with torrent. Then the update mechanism would serve only deltas. It could be then used locally. However index would probably have to be without content - so no snippets. I don't know if it would be useful that way.
It could use a comparison site. Side by side with Google for the same search. I would certainly want to use it this way for a while.
I think this is the way for the alternative search engine to somewhat succeed. Because I think it's almost impossible to compete with Google at being Google. Look at how Bing manages.
I agree! I'd never thought of using tor, but I had very similar ideas about an engine that doesn't index a site if it has ads, etc.
I had an idea of using a delayed response in order to lessen the hardware requirements. So a user submits a query, then checks back later for a response. A little bit like a traditional library, haha!
Anyway, i'm really glad to hear others are doing similar stuff, I think there is a market. The info is out there, it's just lost in a sea of noise.
> But finding things you didn't know you wanted to know...
... is exactly the thing that I'm most worried about losing. Especially when I hear about all sorts of "adaptive" crap that learns about what I do and adjusts my daily feeds, search results etc. accordingly. It probably works great for most people, but I'm definitely not one of them and I want to opt out of the adaptiveness of modern Web. Most of the time there is no clear way of doing that.
I really dislike youtube's 'recommended videos' for exactly this reason. It will never show me anything interesting, because it thinks I only want to watch things that I've already watched.
Aand, come Russkies (that's what you meant, right?), Ukraine and the Baltic states would be dead by frontline bombers (look at Syria, American and Russian tactics are not all that different in that respect) and advanced artillery. Look, ma, no tanks!
I tried searching for "cyclocross", a slightly obscure cycling niche, perfect for a search engine like this, you might think? Unfortunately, no one has curated any content around that topic :(
I do remember those websites. For me, the difference is that now the Web is much more useful, but back then it was a lot more fun. True, you could sometimes waste hours following random links hoping to find something good, but the thrill of discovery when you stumbled across a gold mine of information was a huge part of the appeal.
Nowadays, anyone with a basic understanding of search engines can find almost anything they want within seconds. That makes the Web on the whole much more useful, but the thrill of the hunt is gone -- that's what Jakob Nielsen was referring to all those years ago when he talked about "information scent".
Nowadays, anyone with a basic understanding of search engines can find almost anything they want within seconds.
Almost anything popular and rather insipid, yes. Try to find more detailed information on very specific topics, however, and you'll discover that search engines like Google have been optimising more and more for the former content, making it harder to find the latter. I don't think that's a good thing at all.
To add insult to injury, if you do try very hard to seek the latter by carefully repeating similar search queries with slight word tweaks, quotes, and trying to dig through all the pages of results to see if you've found what you're looking for, Google will quickly decide that you're a bot and either give you a CAPTCHA or just ban you for a little while entirely.
Google is not great for finding stuff. I’ve complained at length about how bad it is for doing research, but recently I’ve also found it’s totally useless for finding things like reviews of products (unless it’s something like the new iPhone). It’s like legitimate reviews are a casualty in the war between Google and clickbait SEO crap.
This is true. The only way I can find product buying advice is:
1. To find a community of enthusiasts and see what they have to say about products in that space. This works pretty well for products that actually have enthusiasts.
2. Go to Amazon and read a product's bad reviews. The good reviews are nearly useless. But, with bad reviews, you have to weigh the number of bad reviews against the product's popularity. If something has twelve negative reviews, that means very different things for a product that sells 10 units a month versus a product that sells 50K units a month.
3. Read the tiny handful of reliable product recommendation sites. Cool Tools, Wirecutter, annnnnnnd.... I'll let you know when I find another one.
I bought a subscription to Consumer Reports and have been happy with the two purchases I made on their recommendation (headphones in the $100 range, and a new food processor).
Do you consider them compromised? It's actually just about time to renew my subscription and i've been wondering whether I should.
I've also used your strategy of reading the bad reviews on Amazon!
I used to be a subscriber. Theoretically I like CR's approach better than Wirecutters; CR purchases their products anonymous and therefore should be quite impartial and totally independent, receiving no special treatment or assistance from manufacturers.
But I found that they simply didn't update their guides nearly often enough to be useful.
Wirecutter's modern twist on CR's formula -- maintaining a frequently updated leaderboard in popular product categories, plus information on deals -- just winds up being more useful to me.
It also seems like Wirecutter has more relevant experts writing their articles. Maybe that's because they're actually better experts; maybe it just seems that way because WC articles begin with the author/authors stating their credentials and experience.
The appliances I bought using their advice has been hit/miss - ranges are decent but the fridges and dishwasher I bought are noisy and required service barely outside warranty period. These are big name brands too.
I don't necessarily fault CR but I with the hundreds of models released all slightly similar but different (I hear partly to prevent price-matching across stores) it must be as impossible to review for CR as it is for consumers to keep track of.
Also, with bad reviews, you can keep in mind if the bad reviews seem to be legitimate complaints, or whether they're just written by either someone with an axe to grind or someone who doesn't understand the purpose of the product. For example, some negative reviews of textbooks on Amazon seem to be people who are reviewing a course they took using the textbook. And some negative reviews of restaurants on Yelp seem to have wanted the restaurant to serve a different kind of food than they do.
Also also, regarding restaurant reviews: Always consider that a negative review that only calls out bad service is quite possibly a review of the reviewer more than the restaurant. Anecdotally, I've seen my share of notes of bad service for restaurants that I frequent and have never had issues. I suspect that the reviewer had at least some part to play in such issues, but conveniently leave out or drastically underplay their own contributions.
That's also good heuristic for 1. - if you're looking for a community about $obscure_topic, there's a good chance there exists reddit.com/r/$obscure_topic. Personally, I always search for a related subreddit as a first order of business when researching an unfamiliar area.
I have very mixed feelings about that. I have some serious experience there; I moderate a subreddit dedicated to audio hardware in which people often ask for and receive purchasing advice.
So on the plus side, you have some fairly knowledgeable people offering advice.
However, even your average "enthusiast" is pretty lacking. No enthusiast has experience with enough relevant speakers to really give informed recommendations. There are thousands of speakers on the market. I've used... dozens? And owned... half a dozen? I mean, nobody has direct experience with even one percent of what's on the market. We try to supplement that direct experience by staying on top of reviews and things, so we have a sense for what's popular and seems to have garnered a lot of happy owners, but it's still very conjectural.
And speakers are a product category where it is somewhat possible to own multiple products, and there is a semblance of objective measurement possible. So we have it easy.
What if you want to buy a fridge? Or silverware? Or socks? Or blankets? There probably aren't fridge enthusiasts and if there are, it's doubtful they own many fridges each. So, you wind up with users just giving anecdotal stories, tainted with heavy doses of confirmation bias, because of course people want to like the fridge they spent their cash on.
Or, worse, you get the opposite effect: brands that sell lots of volume will appear to have the highest failure rate, even if their failure rate is low. If you knew nothing about cars, or the sales volumes for various brands, combing the internets would give the impression that Corollas fail much more often than Lamborghinis. Because tales of mechanical problems for Corollas probably outnumber those for Lambos by a factor of 1,000:1. (My wife owns a Corolla! It's had a problem or two) Which would seem really damning if you didn't know that Corollas outsell Lambos probably 100,000:1 and in reality are probably an order of magnitude more reliable.
You get this effect with Apple products: any Apple-related forum has countless horror stories of failures from Apple owners. But, Apple products are actually generally reliable compared to the competition. It just so happens that they're the Corolla in this analogy, not the Lambo.
Yeah! It's pretty good IMO, are you a fan? The links are Amazon referral links (same with Wirecutter) so I still try to be heathily skeptical but I've found both to be reliable...
I just googled it. Can't say I'm a fan, I didn't even know about it. It seems very focused on home-maintenance and tools, which I know nothing about, but I'll certainly check it out in greater depth ;)
Not OP, but tbh, I thought that was more or less common practice. Many people I know do that already.
I tend to only look at 1-star reviews and 5-star reviews in passing. The first ones are obviously pissed with the product and that will colour their opinions. The latter are either enthusiasts that will love everything about it or people that heve just received it and/or are still on the honeymoon phase. I think ignoring them outright is as bad as focusing only on them, but I don't pay them too much attention.
I find that focusing on the reviews around the middle (and, of course, the number of those compared to the total number of reviews) usually gives me a better idea of the pros and cons of a product.
Actually, I don't think that is the case. I think people unwilling to pay for reviews killed reviews. Before, you had magazines and newspapers to do reviews. Since those are dying, people tried to do reviews online. Now it's hard to tell who is reviewing for real, who's reviewing for ads, who's reviewing because they're paid, etc.
Oh, this is so true. Just the other day I was thinking how hard it is to find reviews of non-tech products. I was lloking for information on a new fridge and what I came up with was useless marketing crap and what seemed like affiliate-link collections dressed up as review sites.
The Wirecutter fills this role with in-depth reviews, but their selection is limited and I prefer to have multiple sources when comparing products.
User reviews aren't helpful to me. They are very subjective, may value different attributes of a product and disappointed customers seem to have a greater incentive to broadcast their negative experience. The fact that your fridge broke down isn't necessarily representative of the product.
Another task Google is really bad at, is finding results by lyrics. As a music enthusiast and crate digger I often search for obscure, rare music I hear in sets, mixes and bootleg recordings. If there are vocals it's an obvious attempt to search for the lyrics - mostly without results. On the other hand the song I'm looking for is very likely up on Youtube, which has a surprisingly large selection of rare music uploaded by other collectors. Yet it's impossible to find those uploads by searching for the lyrics featured in them. Google should use its powers to auto-transcribe music on YouTube to lyrics. Sure, there will be a lot of mistakes, but it should be enough to give some results.
People don't search the same. For example, if I'm looking for something, I rarely use Google. Always Reddit, which will take me to a specific community with a wiki, and discussion. My Girlfriend will type something into Google, without using an Adblocker, and click on the first thing (which is always an ad), and filled with SEO garbage. I'm assuming most people search this way as well, which is why it's so lucrative.
I’m still surprised the resistance-to/wailing-about Google’s placing ads inline with search results died down as quickly as it did. It’s a total shitbag move on their part. Bottom-feeding scum behavior, right there in your face at the top of search results screens. Maybe the worst, because so effective, dark pattern on the entire web. That anyone from the company gets away with speaking in public without being shouted down by hecklers is a bad sign for tech and for society in general.
It’s the Internet equivalent of guys who go door to door selling old people driveway treatments and crappy brick façades they don’t need, at obscene markup. Preying on the least able to defend themselves. If you have any friends or relatives who aren’t “good at computers” Google tricks them for money, as a core part of their business model.
And yeah, it’s bottom-feeding scum behavior any time you see it, fit for failed used car salesmen and other low-ability scam artists. That’s the company Google workers keep, ethically speaking. Could be worse, but still bad enough you don’t want them near your kids or grandparents.
It's amazing how between reviews, sellers, features, and availability, when I make a google (or DDG) search of some product none of the links are of the kind that I want.
I've gave up googling for products I intend to buy (what has probably the largest revenue potential).
I swear the last five years or more of searching for products lead to the first two pages being fake reviews/ads. Google must make money from this otherwise they would fix it.
Google for any game, and then check which of the articles doesn't just copy its enture text, images, and score from the stuff on the press server.
If you have access to, say, the bethesda press ftp, and compare the material there to any review of the games, you'll see that over 90% just copy-pasted without writing much themselves.
It's all fake.
Oh, and those that don't copy-paste often still review everything positively, because otherwise they won't get games before release anymore.
That explains why every BioShock review I read back in the day seemed like it had been written by the same person. Suddenly, everyone were talking about protagonists and antagonists in their game-reviews.
Consumer reports was first but it's an ad - I should have stated I stopped using Google five years ago because it was so full of spam but now at least #3/#7/#12 are relevant and the rest are still ads and spam sites for me.
It's also useless for finding specific product information.
I was trying to find troubleshooting advice for a particular model of printer and no matter what keywords or modifiers I tried all the results were for low-quality shopping or review sites. Impossible to find actual information over the flood of SEO 'content'.
I agree, totally. I used to wonder why I'd find reviews that feel like they are trying to "sell" me something instead of providing an honest review. It's extremely frustrating and seems to be more and more of an issue (thinking back to the HN thread a month ago - The war to sell you a mattress).
The company became interested in the topic because a competitor was training their affiliates to write fake reviews of the company's product as a bait-and-switch tactic. (They later did a study with methodology that showed their product does a 33X better job at enabling individuals to create high-traffic sites.)
Some of the worst niches, I’ve noticed, are those that have the potential to prey on people due to the nature of needs - health, make money online, pain relief, weight loss. If you can find a fair review when searching on product reviews, it’s pure luck.
I imagine that Google’s AI component of the algorithm should eventually be able to recognize and penalize reviews that actually do the customer a disservice. Until then, some of the suggestions in this thread are excellent.
Google is excellent at giving the appearance of finding everything. The fact that there is an enormous world of which Google remains completely ignorant is a fact of which we seem increasingly unapprised.
Have you never noticed that all the really good sites you 'find' have been because someone posted the link? For example in a comment on HN, or reddit. Or a related link on Wikipedia or the sidebar of a subreddit?
That to an extent, and also the fact that _real_ artificial intelligence - the kind that you could share exactly where you're up to with programming and it could tell you exactly what site to read next - that doesn't exist yet.
I once read that Google indexes 0.5% of everything publicly known.
Once I found very nice forum about linux servers. I found it thanks to my friend saying I have to check it out. After spending some time there this forum started to showing up on first page of google every time I looked some solution for my server problem. Google is closing your in your own bubble. It will present to you websites that you already know or that are related to sites that you know.
> Google is closing you in your own bubble. It will present to you websites that you already know or that are related to sites that you know.
This is a good explanation for why my friends search results are very different from mine. Google would spin this as individualization.
Is there any way to get around this bubble and unearth fresh results each search ??
Have you tried incognito mode? I also recall there was option to turn off customizing search results (or something similar) in google's privacy settings.
EDIT: I did not notice any differences so I don't really know how to test it. I search highly technical stuff for work or very mundane "popular" things outside of it. Seems like I'm bad target for the customization.
I do historical research in the context of legal work, and for that Hein Online is pretty good. It’ll have old copies of speeches given in Congress, contemporaneous articles, and that sort of thing. It also has tons of old articles and research papers in the social sciences. ProQuest is indispensible for more modern stuff. If you’ve got a Library of Congress card, many of these resources are free.
A lot of stuff just isn’t online. For one case I needed to look through a bunch of 19th-early 20th century rail and telegraph tariffs. These were generally published in booklets, but they were commercial materials so the Library of Congress doesn’t have many examples. But EBay did!
As a millennial I fall into the trap of thinking if it’s not Google it doesn’t exist. But the traditional methods your librarian taught you are still highly relevant even today.
I highly, highly recommend Thomas Mann -- The Oxford Guide to Library Research.
There are all kinds of cool reference works out there. And Mann has a striking criticism of the impoverished, dumbed-down search interface that's ubiquitous now: the search box.
> search engines like Google have been optimising more and more for the former content, making it harder to find the latter
To emphasize this point, just the other day my 4-year-old asked me "how do seeds inside tomatoes grow?" The answer to such a simple question cannot be found in either DDG or Google because there are literally thousands of sites in this search that helpfully tell you how to grow tomatoes from seeds - the obvious search engine optimization to look for anything with "tomatoes", "seeds" and "grow" being definitely wrong in this case.
What Google also loves to do is to show you results with a one term striked-out. The striked-out term happens to filter results the most so it's the most important one. Google decides to serve you more popular (and useless) search results.
There is the Verbatim search option for Google text search, which keeps me sane, but frustratingly there is no equivalent for Google Image search.
Quite often I'll search for an image of, say, a specific aircraft registration and be shown lots of other aeroplanes of the same type. Very clever, Google, but useless.
They (Google) have a feature built-in to the search engine so that if you put all the terms you want to absolutely see in quotes, for example: "tomato seeds grow", it won't return results without any of those terms.
Note that it will only return results with pages that have those terms in that order.
You might already be doing a lot of this, but using the special operators makes Google a lot more useful. Using quotes around text forces it to be included exactly as written and removes the "struck out words" issue the other comment on your post mentions. 'OR' logic is helpful as well, as is - (e.g. -Word) to exclude words.
> hormone controlling the seed dormancy is exhausted or runs out, letting the seed grow in the moist environment inside the fruit. This warm, moist environment is perfect for germinating seed to grow. If the tomato was left uncut in the warm conditions, the new plant sprout would eventually poke through skin of the now decomposing tomato.
> the obvious search engine optimization to look for anything with "tomatoes", "seeds" and "grow" being definitely wrong in this case.
I'm going to side with google for once: you should be doing your search in multiple steps, acquiring more specialised vocabulary as you iterate on your searches. Starting from seed, you'll end up learning that they're fertilised eggs, which you'll use in your second query which'll start digging you into domain-specific academia where younll assimilate enough keywords to have a pretty good idea of what you're looking for in domain-appropriate terms
Just ask that exact question literally as if Google was a human being. That's the key to finding answers to more obscure questions. Use natural language.
The other day I saw a marabou bird in a wildlife documentary and was surprised by the hair-like stuff on its gular sack (the pink bag hanging from its neck). Wikipedia doesn't mention it and if you search for "marabou feathers" you only find that the world apparently obsesses about the fuzzy white crest on its neck that inspired fashion some decades ago. "Marabou hair" and "marabou hair feathers" likewise only yield information about fashion accessories. Even if you explicitly include "gular sack" Google unhelpfully tries to lead you towards information about the crest feathers.
This is definitely a case where even if you think you know all the relevant words you're unable to phrase a search query in a way that convinces Google you're not interested in the generic results everybody else is likely looking for.
And this isn't even a case that matters much. I was just trying to find out whether those feathers are actually a different type or just look more hair-like for some reason. It reminded me of a video I once saw about primitive proto-feathers we now know some dinosaurs had (affectionally called "dino fuzz").
In case you didn't find the answer to your question, those feathers are called 'filoplumes' [1]. Relatively obscure anatomical detail like this is more easily discoverable in the scientific literature than on the web. I found the answer by searching Google Scholar for "marabou stork", browsing the results for a paper about the "gular sac" (although it has other names), then reading the most promising looking paper - the answer was in the second paragraph [2].
2. Akester, A. R., Pomeroy, D. E. and Purton, M. D. (1973), Subcutaneous air pouches in the Marabou stork (Leptoptios curmeniferus). Journal of Zoology, 170: 493–499. doi:10.1111/j.1469-7998.1973.tb05063.x https://sci-hub.io/http://onlinelibrary.wiley.com/doi/10.111...
Ah, interesting. Odd that even the article only describes them as "filoplume-like" not simply filoplumes. But based on what I find when I search for "filoplumes" it seems their function and evolutionary origin isn't entirely clear.
>> Try to find more detailed information on very specific topics, however, and you'll discover that search engines like Google have been optimising more and more for the former content, making it harder to find the latter. I don't think that's a good thing at all.
This. A thousand times, this.
I really feel is the unfortunate consequences of our short attention span and everyone marketing towards an insanely short attention span is you get the firehose of information, but zero depth to any of it.
If I want to dig; really dig into a subject, 99% of what Google pukes at me is not helpful at all. A few times lately, I've ran into something I've found through Google, but trying to dig any deeper is futile. I've had to go down to the local library and start digging through news clippings, books and magazines. Actual, real old fashioned leg work.
I've come to feel like the internet has no depth anymore, everything is just enough to satisfy a .001 second attention span. Sad when you really stop and think about it.
As someone who grew up with Google and is now trying to get into niche interests, I have basically no idea how to do the real, old fashioned leg work. I still don't know exactly how much of this is Google's fault, because it remains great for providing information on a lot of more broadly accessible subjects.
in the old times there were _a lot_ of search engines, and they even had directories, you could find lots of interesting websites and communities linked between them, no optimization or ads, you'd also use usenet to talk about topics (like today's reddit and digg)
Is that information even there anymore? I've sort of taken for granted that like good API documentation, whole subjects simply don't exist on the web in any depth. Maybe it is strictly a google problem, I definitely noticed a decline in quality when they started to combat link spamming. I guess at this point gaming google (sorry SEO) means that only people with commercial interests show up in the first 20 pages of any given search result. The result of most searches these days is either some reddit or stackoverflow like discussion board after the obligatory first page of links to pages trying to sell you shit.
Assuming that the information even exists, in a way, I think actual human curated links (aka like early yahoo.com) might be a better way to search if you could hire a few dozen people to start building them. I've even went down the thought process of building my own webcrawlers for the explicit purpose of building a personal search engine so that I could kill-file the few dozen sites that tend to show up as the default answers to 99% of google queries.
Maybe 5 or 10 years ago. It's a search engine that filters out the top N websites.
However I just tried it and it seems to be poorly maintained -- it's crashing on certain queries, and giving me captchas after only a few searches.
Some people noticed this problem many years ago, and here is a solution which didn't quite work. But it might be something to learn from in attacking this problem.
Alta Vista allowed LexisNexis style querying against the web. It was magical. I only reluctantly switched off of that search engine after they quit indexing the web and their database became uselessly stale.
I always hoped that Google would add that functionality. Perhaps as a premium, paid service. But that ended up not being the business model they went with.
This was very powerful and enabled you to do very powerful keyword searches. In some ways a very different way to search. I was using this for some truly deep searches of internet sites with the content being very much from the original web. The original web being what this article is about.
Nowadays if you 'deep stalk' someone or some thing then you will find their social media posts and profile on linked in.
Back then you were far more likely to be finding them on forums or sites they created, even on those 'guest book' pages - remember them?
Alta Vista and the advanced search functionality was useful up until 2003 or so, then they became part of Yahoo! and we all know what happens when that happens. I stopped using it then but also my particular use case had gone, I was doing different work and no longer needed the search results beyond the search results, i.e. where the whole internets is bisected for the match of two or more particular search terms and their negations. Happy times, but you did have to have the skill to search like that, it was more like using the index cards in a library than browsing the shelves, a mini superuser trick we no longer have even in Google 'verbatim' searches.
i remember web rings. i remember a full circle of sites providing 99% of the same content because it was easier to keep a website updated reusing the same popular content. and since the webring is the #1 source of popular content, it becomes a death torus into total homogeneity.
That is true, in any specialised field (apart from, for some reason, programming, which seems particularly great to search for) it does very badly. For example in Physics it can be impossible to find the answer to very reasonable questions. For example, try googling "Fabry-Perot cavity transfer function" which in my field is a very simple and fundamental concept, and you get nothing relevant at all
I found http://lygte-info.dk/ (website about batteries, chargers, and more stuff like that) through a search engine when looking for specific information about a LiIon cell. But then I started clicking through that maze of links and found so much useful information that I would never have thought of searching for. I spent hours "trapped" on that page.
The navigation is crappy (randomly interlinked handwritten HTML-pages) but the content makes up for it. The article by tttthis very much resonated with me. Many modern sites look great but don't have all that much actual content.
Jesus, that website must have taken years to make. The reviews are so in-depth, fantastic. I've been looking for someone that reviews USB chargers, as they go into mains voltage and are dangerous if not shielded properly. Amazing resource, thanks for linking it!
If you're interesting in electronics and software in general, and can read German (or can stand the surprisingly good Google Translate), another site I recommend: https://www-user.tu-chemnitz.de/~heha/
I recently tried to search for certain Nintendo 64 ROMs because I wanted to try out an emulator. The first couple of results on Google all lead to horrible, ad-infested websites that claimed to offer a download but either kept on redirecting me to more ad-filled sites, or tried to get me to download some sort of "download helper", "media codec" or "PC cleaning software". Then there were the sites with dozens of fake "Download" buttons surrounding the real one. I'm still not sure whether any of them actually offer any ROM downloads (I found trustworthy sites on the second page of the results).
It's no wonder that many users struggle to keep their PCs free of adware, toolbars and similar junk, when it is shoveled into your face every step of the way.
They are really good, but they don't have everything (maybe due to copyright complaints, specially for Nintendo games which might have been re-released for new platforms.)
Ah yes, I didn't even think of that. But now that I've checked it out, it seems like they only have ROMs up too the SNES, and none for the Nintendo 64 ("only" videos, soundtracks and manuals)
You wanna search by obscure-to-outsiders collection names. “No intro”, “tosec”. That kind of thing. Pretty sure they have n64, in with the rest of a large multi-system collection.
The only thing that's changed about shady emulation sites in the last 20 years is that they no longer give you ill-conceived legal advice about ROM ownership being legal if you delete the ROM within 24 hours and/or are not affiliated with law enforcement.
>Nowadays, anyone with a basic understanding of search engines can find almost anything they want within seconds.
But now I'm less likely to find stuff that I didn't know existed and don't quite have a name for. This kind of stuff was/is usually discovered by following hyperlinks. But I'm not sure why I have this experience less often.
Browsing vs. searching. We don't seem to do much browsing anymore. We search and selectively pick what we consume.
It's faster but, as you say, I don't think it is as fun. I used to enjoy picking a subject and then clicking all the DMOZ or Yahoo! links. I'd dig through webrings and and link aggregators with wild abandon.
Now, I search a term and click on the first few results, at most. Sure, there are exceptions but they are rare. As you say, it's much more useful.
> We don't seem to do much browsing anymore. We search and selectively pick what we consume. It's faster but, as you say, I don't think it is as fun.
Dunno, how much of this is also just getting older and set in our life routine? When was the last time I explored my own city on a bike like I would as a kid? Our interests and patience for browsing change.
How old are you now vs in this "golden era of the web"?
Most of those stories just sound like nostalgia to me. Like suggesting that games these days aren't as good as they used to be because look how less time you spend gaming these days.
When I first used networked computers, you paid someone to do searches for you and got a reply in 3 to 7 days.
It pretty much sucked, honestly. Translation was obscenely expensive and not much better than Google can do today. You paid by the minute or, in my case, the university did.
It was pretty terrible.
Then, this fancy world wide web came about, only it had been world wide before that, albeit at a different scale. It was pretty great.
Now? It's much better in that it is much more useful. I can still make discoveries of novel content, but I seldom do. I search, rather than browse, when I seek information on a subject.
It's not nostalgia, I don't think. It's much more useful today. It was pretty terrible the first times I used it. In fact, it wouldn't improve for like a decade, maybe longer.
If it helps, I also hated computers back then. They were slow, unintuitive, and I had to learn to program to actually have a functional (well, useful to me) device. They clanged, beeped, and whirled. They made dreadful noises and did exactly what you told them to do, even if what you told them to do was stupid. They didn't have things like a recycle bin. Hell, they didn't even have internal storage.
One computer was so horrible that I had to buy 'memory chips' (RAM) just so I could use lowercase letters. All told, it cost me more than a brand new car. You know what I did with it? I played Zork.
Edit: that translation was done by a human, by the way. It could take a week, or more, to get something translated.
As for games, in the 80s, no one knew what worked or didn't work as far as computer games went, so everybody was pretty much throwing everything against the wall and seeing what stuck. And you ended up with some memorable games [1][2]. But over time, people learn, genres develop, budgets rise and you get less original and more homogeneous games.
It's the same with the web. When it went big in 1994, no one knew what they were doing, so everybody was pretty much throwing everything against the wall and seeing what stuck. But over time, people learn, genres develop, budgets rise and you get less original and more homogeneous pages, only this time with so many ads that it takes 500 requests to download your typical page these days.
What I think people are missing is this original, outrageous and anything goes type of experience. But I suspect the Cambrian explosion of web expression is dead for the time being.
[1] Qix (https://en.wikipedia.org/wiki/Qix). An abstract game that was mainstream and quite popular at the time. Haven't seen much like it sense.
[2] "The Earth Dies Screaming", a game I've never seen but the name alone is so evocative that I still remember it to this day. I think it was in the "alien invasion" genre popular back then.
>> I used to enjoy picking a subject and then clicking all the DMOZ or Yahoo! links. I'd dig through webrings and and link aggregators with wild abandon.
Yep, that's exactly what I used to do. That's really what "surfing the web" meant back in those days -- that you would spend hours just clicking links, seeing where they would take you. And of course, they would quite often take you to completely unexpected but cool things that you never would have found or even thought to look for otherwise.
People don't seem to build personal websites in the same way anymore - back then they weren't doing it to generate clicks, page views, passive income, build their profile etc, and everything wasn't all slickly styled and presented on a modern blogging platform. As a result websites were more amateurish, idiosyncratic, original, and sometimes charming, and I think the essence of the creator's motivation or interests often came across more strongly. The web doesn't have the same magic it used to for me. My god though some things were crap about it back in the day. If only we could have have kept the old web preserved alongside the new.
The thing is, people do build personal websites, and that old web still exists. Do you really believe that, with (according to Quora) ~140,000 new sites being created per day, no one is putting amateur, personal stuff on the web anymore?
It's just that the web has gotten bigger and more mainstream, and most of us now limit our browsing to a few silos and then act as if everything that no longer bubbles up the filter must have ceased to exist.
But so many personal websites are just self promotion of one kind or another, and that's fine, we all have to make a living, but I'd far rather read something where someone just wants to share what they know/love. And everything looks like the same set of off the shelf web design templates. I liked the weirdness of the old web. Kind of like games on the Amiga, every site was different, and that was part of the charm.
But perhaps you are right. Any tips on how to find the new gems?
That's a good question - I don't know. Discovery of anything not SEO optimized is difficult, and a lot of that content was on Geocities and Tripod, so the modern equivalent might just as likely be on Wordpress or a service like Wix, making the design homogeneous even if the content isn't. Ironically, I've found most off the beaten path sites through references in Youtube videos and Reddit. When I was getting my CS degree, the web design class involved building a quick personal site, so URLs connected to colleges might also be a good place to look.
But what's more likely, that no one is writing their own custom HTML sites anymore, or that some people are, but we just can't see them because they exist in their own ecosystems beyond the Googlesphere (and possibly even the Anglosphere?)
...I just realized I'm vaguely arguing for (but not defining the parameters of) a Drake Equation for the web, where N is new personal content. Even if the parameters are low, the scale of the web means N should be significant.
My recollection is that for every interesting personal website with carefully compiled albeit idiosyncratic information, there were 10 sites along the lines of "Mikey's Ultimate Web Site!!!" with animated "under construction" GIFs, marquee tags, a guestbook script, and about ten webrings.
For better or worse, the closest thing today to the feeling of the old personal webpage era is Tumblr. The kids get up to some crazy stuff over there.
It was/is different: folks who are excited enough about something to build a website to share it have more interesting and deep content than you typically find on social media.
These days The same goes for people with personal blogs (or website) as opposed to medium and other one design blogging platforms.
The stuff I submit to HN or comment on that wows people often came from me digging into websites you dont find with a quick Google. Esp academic web sites or just those made by smart folks with odd little organization schemes. Im guessing same was true for some of the more obscure, but valuable, stuff I see others submit or bring up.
Some websites used to have a "webring" where they would link to each other and you could just click on "next" and it would take you to another site on the same topic.
> I used to enjoy picking a subject and then clicking all the DMOZ or Yahoo! links. I'd dig through webrings and and link aggregators with wild abandon.
I sometimes still do that with Wikipedia articles and their sources, it's a fun way to kill a couple of hours and can lead down some really interesting rabbit-holes.
I remember 10+ years ago we designed websites so that important content was "above the fold". Today people are addicted to scrolling their Facebook feed.
Sites that cared about "above the fold" were part of the problem, not part of the solution. The primary reason one would be actively interested in putting content "above the fold" was if one wanted to "capture audience", which is generally not the first thing on the mind of a person posting deep information as a labour of love.
i don't think browsing's that rare, actually, it's just different, having been centralized by facebook, tumblr, wikipedia, reddit, and the like. it's less like going out and exploring the web and more like just living there.
I grew up with 1980s dial-up modems (slow 300 baud!) to dial directly into BBS systems and USENET. I then continued dialup access into the early internet (GNN not AOL) in 1995. I now have Verizon fiber optic.
To me, the modern web today is better in virtually every way compared to 1990s web surfing.
+ Google search engine is better than Alta Vista and Excite.
+ Google Maps with smooth scroll, 3D streetview, and satellite view is better than the clunky 1990s MapQuest without AJAX. Google Maps also uses colors to highlight traffic congestion.
+ Wikipedia is better than the non-existent online encylopedias of 1990s
+ airline websites now bring up a graphical map of the airplane and let you click on the actual seat you want. When I booked etickets from airline websites in 1998, it was just rudimentary ticket purchase.
+ youtube obviously has videos but it also has virtually every song. If some music forum mentions an obscure song, I think to myself that someone else may have already uploaded it to youtube. Sure enough, 99% of the time, it's there. It's even better than Spotify in that respect.
+ discussion websites like reddit/HN are better than Usenet. The better replies are voted and sorted at the top. I learned a lot of C Language from USENET "comp.lang.c" in the 1980s and I'm jealous that today's programmers have StackOverflow which is better. Again, the superior answers are voted to the top.
+ Amazon.com also made the UI better. In 1990s, they didn't have "Look Inside" feature to see sample pages of books. I use that all the time. One thing they did make worse is you now have to manually click "see all 50 reviews" to read all reviews; I assume they did that to prevent web scrapers. In any case, Amazon is good enough that I stopped wasting gas driving to Barnes & Noble.
I'm having a hard time thinking of a 1990s web experience that's superior to now.
I can only assume that a big part of why some think that today's web is worse has to do with Facebook and its walled garden. To those people, the web feels "less open". Well, I don't have a Facebook account and none of my longtime friends do either. Therefore, it doesn't feel like the web is any less open to me.
The web is more useful now in 2017 than 2007. And back in 2007, it was more useful than 1997.
> discussion websites like reddit/HN are better than Usenet
I strongly disagree with this, as Usenet and even mailing lists offer many advantages over web forums. With web forums, I can't natively...
- Store, read, search content offline
- Read only new messages
- View thread context
- Access information redundantly (no SPOF)
- Killfile senders
- Integrate viewing and composing with local tools
While a particular site mitigates its disadvantages, perhaps via a better UI or via information not available elsewhere, the medium as a whole cannot. A website might be a fine view of information, but it should not be the only interface to that information. Doing so prevents all of the above uses and makes it much harder to reuse content in ways unforeseen by the original site. I suspect this is the root of the problem -- such reuse is not desirable for most sites, as it conflcts with their business model.
Reddit covers almost all of those and has as wide a range as Usenet did. In terms of the broader web, there's a tension between wanting customizability and wanting consistency; realistically the only thing most people do with any given content is read it, and so the web focused on that use case and things that helped with that at the expense of offering consistent structure.
>Usenet and even mailing lists offer many advantages over web forums.
It's interesting you didn't list any disadvantages of Usenet. NNTP protocol doesn't have identity authentication. Therefore, you get spam and forged headers. This isn't any fault of NNTP; it was built for a different era.
From a content standpoint and not a technical one, USENET was at its peak, before 1990s, when it was primarily universities and governments. The smallness of the tech community and the inherent collegial atmosphere didn't exploit the vulnerabilities in NNTP. I'll agree with others that it all rapidly declined after AOL opened it up to the masses. To regain the "reading pleasure" I got from USENET in the 1980s, I have to use Reddit/HN/Stackoverflow.
You listed the following technical advantages:
>- Store, read, search content offline
>- Access information redundantly (no SPOF)
>- Integrate viewing and composing with local tools
I'll group those 3 bullet points together since they are related to each other. I agree that those technical advantages are true for USENET. Your comment spurred me to revisit my USENET usage history and attempt to articulate why it's no longer relevant to how I use the web.
I participated in USENET from ~1988 to 2006 -- so about 18 years worth activity. The last years of downloading and reading messages was a tedious chore. With Usenet messages having no voting system, I had to slog through a sea of bad replies to get to the good comments. It was a horrible waste of limited reading time.
Once I got hooked into Reddit in 2006, reading crowdsourced comments became more of a pleasure again. Yes, I 100% agree that voting leads to "groupthink" and "hivemind". But even with those problems, catching up on 500 unread messages ranked by votes instead of 500 messages in pure chronological order is easier. I already suffered through unvoted messages for 18 years. I don't want to go back to that. Experiencing Reddit made me realize that the "terrible voting system" is the unofficial way we combat the "Eternal September"[1]. (Also, the Google search engine was better than anything else because it weighed the "unoffical votes" of links in the PageRank algorithm.)
Forum voting is like that saying about Capitalism -- "it's the worst forum system, except for all the others".
Usenet was like doing homework. Reddit is like a quick dopamine rush.
It's good that both Reddit and Usenet can coexist. A subreddit like "reddit.com/r/cpp/"[2] can coexist with USENET "comp.lang.c++"[3]. Although I have almost 2 decades of training to compose offline messages for "comp.lang.c++", I have no urge to do so. The technical advantages for Usenet you listed doesn't offset its flaws in content. These days, I just go to the C++ subreddit to get my daily fix on what's new because the content is better presented for limited reading time.
Respectfully disagree on more votes == better. Visible score systems often lead to groupthink, and commenters aiming for the lowest common denominator in the form of karma whoring.
While HN has some measures to reduce hivemind voting, it's quite rampant on the default subreddits.
It seems to me like HN still has a long way to go. I find my detailed, careful, well researched comments typically get an order of magnitude less votes than my one sentence inanities.
I doubt it's fixable - it's just humans being humans. If you write a short one-liner, anyone can quickly evaluate it and decide whether they like it or not. When you write a long essay in a comment, a lot of people won't upvote it because they won't even bother to read it, and others may find that they disagree with one of your many points but don't have time for a thorough response, hence they just won't upvote and move on.
Yes. Upvoting and downvoting are the brokenness; users are just interacting with the UI model they're presented with. Better low-effort, high-reward consensus systems are needed.
I've actually changed the topcolor to #222222 in my settings so I don't see what my account's karma is. It completely ruins the ability to see the "new" "threads" etc links without a truly irritating amount of eyestrain but it's worth it - I recently asked genuine question about something I wanted clarification on and the comment ended up being downvoted to -4. Now I can't see my karma score I don't care about this, whereas before when my karma was shown on almost every single page throughout the site I'd obsess over making my comments universally good.
One simple improvement that would apply both here and on reddit is to move the up/downvote arrows to the end of the comment rather than the start. At the moment it's disproportionately easy to upvote a short comment because you're still at the right place for the arrow.
Voting is gamed by corporations and governments -- easy with a few thousand IP addresses.
Comments are manipulated by teams of paid "trolls", including in English, and including Hacker News (when the article is of interest to a particular corp or gov -- like the Tesla mass firing article on Hacker News recently, just to give an example I know of).
I won't deny that the current Web is much more useful, on average, than it was back in the nineties when I started using it.
On the other hand, during that time the technology became immeasurably better - my first dial-up connection was 9.600 kbps if I remember correctly, now I have 4 Mbps; I didn't have a math co-processor back then and now we have multicore monsters of CPUs - and so on.
The question (to me) is not if the current Web is more useful, but if the increase in usefulness is anywhere near the technological progress?
At some point, the technological limits - like no YouTube because both the necessary download speeds and compression algorithms were not there - were replaced by something else. Back then we could believe that, in time, seeing the rate of progress, we'll get everything we ever wanted. The tech advanced and now we could have all these things we dreamed about, but don't - because of political and economic factors.
Yes, the Web improved drastically, despite a couple missteps (like IE6) along the way. But I believe the improvement is much lesser than the current technology would enable were it not for non-technological factors.
On the brighter side, once developed, technology is here to stay, so if we deal with all the current problems we'll be free to advance the Web further. But that's a pretty big "if"...
i agree in total todays web is better, but today its so closed, and we lost a lot as well.
- excite chat was better than anything we have today, irc chat remains awesome.
- on social networks of yore, you could actually meet new people. you could search for new people and expand your circle to people completely outside it. I believe its against facebooks TOS to try and connect to someone you dont know. i know my account was banned for 3 days because someone declined with 'they dont know me'.
- youtube sucks for viewing songs. like really sucks. you have commercials MID SONG. one time I had a playlist play a 15 minute long commercial. FIFTEEN MINUTES. half the songs in my playlist end up getting removed, and It doesnt tell me the song it use to be. so if i find a song i really like, add it to a play list and it gets removed, i completely forget about it. song lost. if you are going to compare songs, youtube losses hands down. napster / audogalaxy / was far superior. youtube is still great for watching a cat jump in a box, after watching a minute long commercial of course.
- google search is great, its superior to alt vista, which is why i switched from alt vista to google when it came out. but google gets stuck, if it cant give you want you want in the first 5 hits, it basically cant do it ever. you have to change search strategies. alt vista was bad, but the content was diverse. you could still find gems 5 'nexts' in. google also gets locked in on what it knows about me. sometimes i have to search in incognito mode to find what i want.
the 1990s web experiences was not just about how well content was delivered to you. you could chat a person you never met with a simple ASL. maybe you continued talking after the first day, maybe you didnt, but it was fun and you met new people all the time. people dont want to meet new people anymore.
people dont email or contact eachother anymore. before people would put effort into staying in touch. i remember checking emails with excitement. now even though you can talk to your network at any day, because you are always connected people dont. and its just a 'hey whats up'. people dont write long responses saying how their day was, and how their cat got locked int a closet, etc.
edit: i forgot to say the original digg was awesome, and in may ways better. also, I think most of us are comparing the web from 1998 - 2006, not the late 90s.
I think the Internet's gotten worse in general (and, specifically, less useful) over the last decade except for one thing:
YouTube is insanely useful for learning how to do basic household stuff, minor car repairs, that kind of thing. Insane.
Mundane? Yeah, but life is like at least 50% handling mundane shit like that. Maybe 90%. Maybe 95%?
Want to know how to repair some brick stairs in front of your house? Sharpen a lawnmower blade? Mulch a garden? Seal a driveway? Change the cabin air filter on a 2008 Mazda 3? Make pancakes using campsite cooking tools? Multiple videos for each. Perhaps even dozens.
Yes. I think Youtube inherited the culture of the old web. Lots of amazing channels about obscure interests. Household repairs but also welding, machining, woodworking, electronics repair, obscure retro tech, distilling, etc.
I guess it forces the presenter's personality into the spotlight a bit more than pure text and images. Probably attracts a slightly different kind of person. But I think the same spirit is there.
YouTube is also incredible for educational stuff. There's very pop science, pop science, hard science, and then things like hundreds of lectures from Leonard Susskind on everything from classical mechanics to the latest happenings in string theory/m-theory.
Phenomenal stuff, but it seems like the things that most places (universities in particular) are still holding out on are things like problem sets. Hard to know how little you know until you have materials challenging your depth of understanding. Watching a lecture without doing associated problems is like trying to learn to play basketball without touching a basketball.
This is kind of meta, but I think we'd be better served if you wouldn't remove the comment even if proven wrong. It serves everyone better to see both sides of the argument. It would be more appropriate to reply conceding that your initial argument was incorrect, or to add a note on the bottom of your original comment indicating the same.
Sorry, "this comment" referred to itself, I did not post the parent comment. The conversation should not be hidden, just this barely useful meta comment.
Hey, I'm putting some thought into a more thorough reply. I have a reply partially written at home but am now at the office lol. I see the comment thread has garnered a lot of replies in the intervening time though, so I'll just post a quick tl;dr:
okay yes the web is more "useful" to most people, and maybe only slightly more useful to me, but that usefulness is significantly offset by some majorly-serious concerns about privacy, performance, and reliability.
Okay, I'll clarify. Is it more useful? Yeah, depending on what you want to do. But it's also a hell of a lot more dangerous. It's especially more dangerous if you're not careful.
+ Google Search is about as good now as it was back in 2007. I like that its interface has (largely) changed.
+ Google Maps with smooth scroll is nice, and its integration to 3d street view is nice too.
+ 1990s MapQuest without AJAX was nice though, in its own way. Even with Google's smooth scrolling, the only real thing Google Maps has going for it is IMO a cleaner display.
+ Wikipedia's generally better today than 10 years ago, yes.
+ Airline websites are a nightmare to use. Try getting an optimal route that doesn't use a single airline; and try getting that optimal route without ended up getting spammed or surcharged.
+ Youtube having virtually every song is nice for some people. It's not so nice for me. The songs on there are usually pretty boring or low quality audio or video or both. I pay for a music service (di.fm) for a reason.
+ Reddit/HN are pretty neat.
Okay, that's about where we agree (along with your comment on Facebook).
Let's look at some of the downsides.
- Personally, I find Reddit's method of linking to articles vs loading comments to be counter intuitive and confusing.
- Ebay is still about as trustworthy as you can spit.
- Amazon.com is among the crappiest web interfaces I've seen, and that's ignoring the company's history of abusing compensation and livelihoods of their employees and contractors. It's slow, extremely noisy (which contributes it to being slow), difficult to navigate, and I definitely never intend to buy from the marketplace specifically because I might as well have bought stuff from ebay.
- Just about every website has tracking up the wazoo. That contributes to a slower web experience both in bandwidth cost as well as hardware performance. Not everyone has a wonderfully new computer to handle all that. And if you're in America, chances are pretty good that your bandwidth is sub-par as well.
- Email. Why do we still have to deal with phishing and tracking? This is something that has been around pretty much since HTML was introduced to email.
- Instant Messaging. Slack and Discord are the newcomers but they're also somehow the best. They're popular because they get it "right"; it's super easy to text, talk, video, share files, share screens, etc. But they also get it wrong. That Electron UI has got to be the one of the biggest memory behemoths I've ever seen. Both are yet more walled gardens.
- The newer versions of Skype are absolutely dysfunctional compared to the older versions, MSN was better, AIM is (was) better, even IRC does file transfer better more reliably (although optional and clunkier).
- Mobile Apps. JFC banks going to mobile apps when the marketplaces are full of fake apps. This is absolutely not user friendly. Build on top of that the fact that phone numbers are super easy to fake or steal and you've got a clusterf!@# personal security nightmare just waiting to happen.
- Phones, operating systems, applications which track absolutely everything you do. It eats up my bandwidth, my system resources, and provides no visible benefit to me whatsoever. What happens when someone figures out how to track what I say to whom and when, and use that to "guide" me to buy things?
I could go on and on about more stuff but I don't care to at the moment.
It'd also be cool to make a search engine that catalogged all of the websites that other search engines politely did not.
Is there any proper enforcement of robots.txt or is it just polite? Because I don't see how one search engine could maintain an advantage if others start catalogging every site.
It's polite, but the teeth behind it is that it's much easier to secure a CFAA conviction for scraping if you can show evidence that you specifically forbade a scraper from accessing your site (via robots.txt, authentication, or IP blocks) and they circumvented that restriction than if you gave them no notice that you dislike what they're doing.
Legit search engines and aggregators have a strong incentive not to kill the golden goose by getting sued, particularly when the vast majority of sites actually want to be indexed.
I remember the days pre-google when Yahoo was king. And most sites were filed under a topic/category. That was the default way to find a group of somethings. It's also a little suggestive and explorative, whereas keyword based search engines relied more on you being more specific. Crowd stoked search results and AI have bridged the divide these days. What the author failed to mention was web rings. Basically authors with like minded material would link to each other. So once you found something you could hop to something else and bypass the search engine. These days you are almost crippled without the search hand.
Do a lot of people actually use it? It seems like you could just as easily replace that site by simple tweets when you actually stumble upon something worthy. Which is what most people do IMO.
Based on some very perfunctory research: they certainly used to have a lot of users, but I get the feeling they have declined over the past ~5 years.
Anecdotally: I used to use StumbleUpon, and found it fun for a while. I'm not sure what caused me to drift away from it, but the reason I have no interest now is primarily that I am drowning in interesting content. (I have enough articles saved to Pocket to keep me going for years.) Getting recommendations tailored to your interests is nice, but unless the personalisation algorithm is really good, I think you can do better by cultivating your own set of sources and intermediaries.
The web has changed: in some ways for the worse, and in others for the better. I remember websites being like a lottery: sometimes you'd hit jackpot but most of them were "Under construction" GIFs over ugly tiled backgrounds.
There is still ton of content and much more than what I would've dreamt on the 90s. Platforms like Reddit allow everyone, whether they know HTML or not, to publish their own content and even comment on others'.
Yes, Facebook and Twitter suck, but that's mostly it. I'm very, very grateful for everything else on modern internet.
I didn't put discussion on the page in order to keep it as code-light as possible. Reddit is a good option for discussion (at the time of writing), especially considering I didn't want to clutter HN up with a discussion page for each article (no problem on a subreddit).
I agree with you on the value of reddit and platforms (and I actually value Facebook-type platforms as well, with the obvious qualifications), but the old html sites I write about here have a different type of construction, material, and value.
> the old html sites I write about here have a different type of construction, material, and value.
These still exist. They are prettier and more content packed than the 90s ones you're reminiscing about, and there are many, many more than there used to. I honestly think you're seeing the early web with rose-tinted nostalgia glasses.
Unless you liked the DIY amateurish hacker feel of it (which I agree has its value on its own) I think they've changed mostly for the better.
Just to be on the same page: what's exactly that type of construction, material and value you miss? What did you like of it?
EDIT: After re-reading your original post I think I get your point. It's just... #RememberWebsites, of course it's nostalgia-fueled! And that was the point of the post, right? Celebrating it (which I completely misunderstood as a celebration of websites themselves).
Mobile changed websites into displaying less content. Are we better off? Scrolling forever vs clicking next page / last page. I think scrolling is worse if you ever want to find that piece of content again.
For developers, information breadth and quality was excellent when I first onlined in '98, and has improved by leaps and bounds year by year.
For other stuff, I agree with that OP article, that was a uniquely rich caleidoscope that got "blandened" (now lost/gone/fully-transitioned) as the types of folks who'd craft such little labour-of-love sites mostly did so for lack of easier / more-convenient options that soon popped up first with prefab forums and prefab Wordpress / Blogger, then Tumblr/Medium/social-media.
most of them were "Under construction" GIFs over ugly tiled backgrounds
Oh you remind me of the beautiful paean to those GIFs in Olia Lialina's "A Vernacular Web", which is a pretty cool discussion of early internet aesthetics.
Besides, there's very little politics in niche subreddits. I'm subscribed to /r/pic (not /r/pics!), /r/flute, /r/stonerrock, /r/chess, /r/boardgames, /foreignmovies, /r/vermiculture and /r/japanlife and the latter is the only one where American politics sometimes creeps up.
This is pretty harsh critique. I first got on the web in 1992 and what we have now is a paradise compared to what we had then. Sure, you may have to look with intent for what you want, but freely coasting around the web has always carried liabilities. It used to be "you'll find lots of junk" and the junk has simply diversified since then.
I also noticed the author doesn't even use a single hyperlink in his own article. Be the change you want to see.
I was just checking out Project Rho. Before that I was building a link page of my own because I'm getting into ham radio. The old web we love is still here and it'll always be around in some form.
> what we have now is a paradise compared to what we had then
Here's how I recall it:
Back in the day, you'd search for subject {$foo} and you would find mostly websites written by some cranky bearded weirdo who is obsessed with {$foo}, who has devoted weeks of his time to personally collating his every thought about {$foo} into one ghastly-looking site.
Nowadays, you search for {$foo} and find mostly beautifully template-designed pages of text written by indifferent fiverr freelancers who had about 20 minutes to stuff in as many keywords into as many column inches wrapped around as many ad slots as possible before moving on to the next subject.
> Nowadays, you search for {$foo} and find mostly beautifully template-designed pages of text written by indifferent fiverr freelancers who had about 20 minutes to stuff in as many keywords into as many column inches wrapped around as many ad slots as possible before moving on to the next subject.
Yeesh, does anyone else have this opinion?
I sure don't. It's never been more trivial to find good, enriching content.
How many of these posts are just HNers getting more crotchety in their old age? Sometimes it just seems like a pissing contest for who can be most cynical. ;)
I will willingly cop to this. But seriously, genuinely yes, the internet is full of a lot more content-free clickbait than it used to be. Because clickbait didn't used to be a thing that existed.
Having worked for a company that helped a myriad of other companies write shitty clickbait content, I agree with the commenter. The majority of content on The web now is written by people who don’t really give a shit about the topic and have nothing new to say on it, the only reason it exists is to drive clicks, SEO.
It used to almost entirely be populated by people who actually cared.
I have to agree with you that I generally don't see those kinds of pages as well.
Sure, they exist, and I do stumble on them from time to time, but most of the time I will find what I'm looking for, often with a wealth of user comments as well.
Let's take a really simple example. I'd like to see information about "rocket engines." Just masses and masses of enthusiast information for/from a technical point of view. Think something like Atomic Rockets [1] but more technical and less sci fi. An example of the 'ethos' I'm talking about would be something analogous to Fredric Parker's videos about KIC 8462852 / Tabby's Star / 'that star with the really bizarre dimming characteristics' [2]. Just an informed, obsessive, educated enthusiast targeting the exact same. I'm certain such sites exist. In the days before 80% of Google's links went to the same 20% of sites, they were really easy to find.
Google is excellent at finding low grade mass media on the latest topics, but I think their ability to find meaningful information has been deteriorating. E.g. getting back to those video's on tabby's star - if I search google for the same, I find no less than 4 hits with aliens in the title and just about as many alluding to have found 'the' explanation. It's complete trash (though Fredric's videos do hit the second page!). Granted, it's closer than the hardcore porn I might have found on the first page searching in the late 90s, but it seems like we've replaced porn with clickbait. If that's my choice, I'd prefer tits.
Actually, it wasn't meant as a critique, as I like most people highly value the modern internet.
I will be putting in more internal hyperlinks as I have more pages on the website to link to. Most of the articles there are still in progress, and this one is a sort of introduction to the HN community (which is my main source of news and analysis these days).
Please link me to your ham radio site when you get it ready for us. I'm also building a list of these types of blogs/sites.
> Some bigger websites at the time were like mazes with some of the words on each page hyperlinked to other pages on the site that were about each of those separate topics, and you had to open up multiple links from each page otherwise you'd lose them, and you wanted to read every word.
Because what actually happens is I click an interesting link, read it, find a couple interesting-looking links, click them, read the first, see another 3 interesting links (on the now 2 levels-deep nested page), click them, notice that one of them links to a category or tag that's relevant to what I'm looking at (which has, on most wikis, dozens of pages)...
In short, sufficiently high branching factor means that neither depth first nor breadth first search can keep up without a lot of memory.
EDIT: Note that, at over 100 tabs per browser session, I am probably an extreme edge case. :)
One notable example from the 90s that I was referring to with this statement was a site that had many, many pages about unusual science/mystery/gnosis/anarchism. I haven't been able to locate it.
But that might not be the one you mean... I recall one with a black background... hmm.
/edit/ I also found this one: http://www.crystalinks.com/ ...I (probably badly) archived a copy of it around 2000/2001, back then many pages had dark graphical backgrounds. It seems that has changed.
I've never been particularly nostalgic for websites.
I'm slightly nostalgic for shared Windows folders on LANs at college dorms. I remember seeing the first South Park short from such a folder as it was going viral.
I'm extremely nostalgic for the original Napster. I don't ever remember searching for a piece of music and coming up short. And I remember it being a very sudden shift-- one month you're making a mental note to search for a CD you misplaced somewhere back home, the next month you're getting on Napster to check if the theme to Ghostbuster's 2 has lyrics that recount the plot of the movie. It does.
A few weeks ago I typed "Battlestar Galactica" into Netflix, and guess what? It showed me lots and lots of results, none of which were Battlestar Galactica. And this isn't your run of the mill entitlement of a fool addicted to his Iphone apps. That is entitlement of a person yearning for modern functionality to match a shitty piece of software that saw its last stable release 15 years ago.
I'm having a hard time finding any numbers for the actual amount of music that was available on the original Napster at the time. Can anyone put some hard data to my rose colored glasses?
Check out Soulseek, it is very similar to how I remember the original Napster. Lots of obscure rarities on there music-wise, I've not tried it for anything else.
The one P2P thing I am nostalgic for is Audio Galaxy - it had an awesome system where it they indexed everyone's content on their website, so you could see every file that had ever been on the network and add it to your "want list", then when that person came online, it would start downloading. To be fair "wish list searches" in Soulseek serve a similar purpose, but I loved that ability to browse every file that had ever been online!
Soulseek is incredible, discovered all sorts of smaller-time musicians that never wouldve been sold at the Best Buy in the suburban wasteland of my youth.
I can’t believe it’s under active maintenance after all these years!
> I'm extremely nostalgic for the original Napster.
I'm even more nostalgic of Direct Connect: Hey this person has this cool and rare thing I'm searching for let's download his file list to see what else they like.
10 years from now, we'll be waxing nostalgia about entrepreneurs who made a living off Instagram, Amazon, WordPress, and other platforms. 'There was a kid who used to "rate dogs" and he was hilarious! And he did it for free without any corporate backing! Made a killing on T-Shirts and stuff as well!'
Last night I ended up, for some weird nostalgic reason, installing a Gopher client, just to see if there was anything still around. Amazingly, there's a bunch of blogs (called phlogs in Gopherspace) that people are updating regularly! Pretty amazing!
Last year I made some minor changes to my blogging engine [1] to support gopher, so now my blog [2] is also available via gopher [3]. The source code to the gopher server is online [4].
Gopher is a great protocol. I never really used it back in its heyday; I'm not sure I knew it existed at the time (I didn't really get "real" Internet access until 1995 or so). I recently discovered it after having similar sentiments as the author of this article concerning the state of the WWW.
After you get your Gopher client set up (lynx in the terminal works great), a good starting point is gopher://sdf.org
There are many active "phlogs" published on SDF.org; there is also an aggregator that tracks some 20-odd phlogs at: gopher://i-logout.cz/1/en/bongusta (though it seems to be down at the time of this posting).
The "Gopherspace" is a refreshing wormhole -- with a surprising amount of present-day activity -- into the Internet of yesteryear. I highly recommend checking it out and perhaps publishing your own content on Gopher if you're tired of the dumpster fire that is the WWW.
After installing the gopher client (apt install gopher) and running it, you can navigate through the menus to Veronica-2, which indexes a ton of them. You can also just browse around. The linking reminds me a lot of the old days of the web.
> Most websites were written with html, so they were all unique.
Every single website on the internet always was, is and will be HTML (with various kinds of XML/SVG markup sometimes). People just gradually realised proper and standardised web design makes the Web better for everyone, by making it more usable and accessible.
I don't see why I should feel nostalgia for an Internet plagued by `<marquee>`s, poorly-laid out flashing gifs and bright yellow text on a white background.
> People just gradually realised proper and standardised web design makes the Web better for everyone, by making it more usable and accessible.
Then gradually forgot about it? These days we have our WingDings (FontAwesome), popups ("please subscribe to our newsletter" modals), best viewed in Netscape Navigator 4 ("you are using an outdated browser"), best viewed in 800x600 ("please rotate your device"), auto-playing General MIDI music (auto-playing videos), slow load times (not due to a 28.8k modems, but bloat) and other ridiculous practices that don't really have any 1998 equivalents (hijacking scrolling, pop-in-pop-out menus, hijacking the browser history or giant banners that follow you around as you scroll (without using frames, so good practice!)). Often for no good reason. Where we had <marquee> we now have pop-in chat bots, subscription reminders, social media sharing links using a third of the screen, nonsensical page transitions etc.
It's all exploiting what could be very useful technology, and I'm not saying it's much better or much worse, but at least in the 90s the regression you'd experience by not running any scripts was that there wouldn't be any asterisk snowflakes falling around christmas, not that the plain text article you were about to read suddenly became unreadable or unnavigable. It seems the lessons consistently learned from the past are the most trivial and are ignorant of the cause of the original criticisms. Don't use tables for layout (instead implement your convoluted page layout using <div> pyramids and CSS hacks!), don't use frames (use "position: fixed;" to waste screen real estate instead, and use JS to break navigation!)
Web people were for a brief moment enthused with the idea of proper human- and machine readable content markup respecting established standards. But we're not there any more if we ever were, developers preferring to serve the most simple content using 500k of JavaScript and 200k of CSS browser workarounds over plain (X)HTML with simple styling.
Sophisticated technology allows bigger mistakes, but the most important difference between 1998 annoyances and 2017 annoyances is malice and submission to external interests becoming standards and replacing, to a large extent, innocent carelessness and ignorance.
For example, a self-hosted banner ad was an abuse of client bandwidth and an issue between webmasters and their audience, while a modern banner ad from Google is an attack on user privacy that a Google serf who wants a little money perpetrates against another Google serf who wants convenient "services" for the benefit of their common master.
I saw a lot of people trying to avoid HTML in their websites in the past.
The flash ones are the obvious example, but I once had the pleasure of an entire site that looked like any other, except it took forever to load and links worked weirdly.
That was because the entire site was an image map over a giant .bmp for each page.
Oh yes, totally forgot about flash and `<map>`s – but one could argue even those were originally designed to solve the problems of 'vanilla' HTML. They of course failed and HTML kind of prevailed, but only by being coupled with a massive set of APIs on the Web Platform. If it wasn't for those JS APIs, we might be stuck with Flash forever.
> Every single website on the internet always was, is and will be HTML (with various kinds of XML/SVG markup sometimes). People just gradually realised proper and standardised web design makes the Web better for everyone, by making it more usable and accessible.
Unstyled HTML (optionally with user style sheets) is more usable and accessible than today's one-size-fits-the-designer design for the kind of content that used to dominate (and is still a large segment of) the web. (We've got better tools for making usable and accessible content, but they are mostly not used for that purpose.)
For interactive applications, its a different story.
Sure, but 100% unstyled HTML was extremely rare even in the 90s. The problem with everyone writing their own crap HTML is that the sites become terrible and unusable, because they lack good design. What platforms like Twitter/FB initially did was just provide a unified, usable interface to a certain kind of information. Compare to MySpace, which collapsed partly because vast amounts of the profile pages were horrible.
I still (I’m in my twenties) remember building buttons and titles from images. Not really useable unless you are using the resolution the page was designed for, and don’t care about responsiveness or using a screen reader.
Websites used to do something that completely disappeared: not using javascript to fuck with the behaviour of the browser. You used to be able to scroll down and the page would scroll down. Right click on a link and see your contextual menu. Copy or paste something and it would actually copy and paste. Half of the sites today break the browser in one way or another.
Really, I remember plenty of sites in the 90s that would change the colour of your scroll bar and override your right click to tell you how everything is copyrighted.
That would be interesting - a search engine that was resistant to most SEO techniques and allowed one to find these types of sites. Would be a great alternative to Google sometimes.
The article on using Lua from D looks very interesting! I've been meaning to get to know D in more depth for a while now, this looks like a good starting point for a small toy project.
See also the analysis (from 2015) of an Iranian blogger who got arrested in 2008 and got free in 2015 - he did not recognize the new web. (Ironically also posted on Medium...) According to him, hyperlinks turned into a social media stream.
After blocking all the JavaScript and hiding the position=fixed garbage, that was an interesting article.
> So I tried to post a link to one of my stories on Facebook. Turns out Facebook didn’t care much. It ended up looking like a boring classified ad. No description. No image. Nothing. It got three likes. Three! That was it.
I write a blog, and publish links to posts on Facebook because doing so costs me nothing. It also gets me next to nothing. People click on those links about as often as Facebook asks me for my credit card to "boost" them.
I have a "Kill Sticky" bookmarklet to do that job, as the first thing on my bookmarks toolbar. I'm clicking that thing all the time, it works great. So long stupid sliding navbars, enter-your-email popups, fixed video autoplayers, etc. I think I got it from here: https://alisdair.mcdiarmid.org/kill-sticky-headers/
I use this bookmarklet:
javascript:(function()%7B(function%20()%20%7Bvar%20i%2C%20elements%20%3D%20document.querySelectorAll('body%20*')%3Bfor%20(i%20%3D%200%3B%20i%20%3C%20elements.length%3B%20i%2B%2B)%20%7Bif%20(getComputedStyle(elements%5Bi%5D).position%20%3D%3D%3D%20'fixed')%20%7Belements%5Bi%5D.parentNode.removeChild(elements%5Bi%5D)%3B%7D%7D%7D)()%7D)()
Another thing is people often just go to Wikipedia, Wikipedia replaced the need for many fan sites. I had things copied from my sites and put on Wikipedia and tired to get a link back or a source mentioned and it was removed each time. So I lost motivation to even update fan sites, since whatever I type is going to get pasted onto Wikipedia with no link back.
Unfortunately, Wikia looks like it's in the process of becoming a cautionary tale, with its recent move to start injecting its own video content into certain high-traffic wikis and a general decline in customizability and independence.
Wikia seems to be gradually being replaced by self hosted wikis, at least where larger fandoms are concerned. The Nintendo Independent Wiki Alliance (Mario Wiki, Zelda Wiki, Bulbapedia, etc) are one example, the Square Enix Wiki Alliance are another, and I'm sure the likes of the Simpsons Wiki have moved away from the service as well.
Even when Wikipedia does link to the original source, that has minimal benefit to the original source. Very few Wikipedia readers follow the links through to the source, so Wikipedia essentially replaces the original.
I have no doubt mobygames is banking on links from Wikipedia, they are like the only commercial site that seems to be allowed to have links on Wikipedia en masse. A link from Wikipedia is very good for seo and they have tons.
"it was largely a collection of websites made by people who were interested in some subject enough to write about it and put it online. "
Oh,you mean like blogs?
Seriously, this sounds like being nostalgic for it's own sake. I fail to see how using dreamweaver and ftp is somehow better then using Wordpress and the cloud, writing your own html as a prerequisite to having a website was never a good idea - now everyone can have their own website.
I really don't miss the "glorious 90's" type of websites - with the thousand of animated images, weird background images and marquee everywhere. Sure it made every site unique - every site was terrible to the eyes in it's own special way.
Also the idea that all sites now look the same is quite preposterous - sure a lot of sites are cookiecutter websites - especially marketing websites, but there are tons of unique designs - especially for blogs and personal websites.
> I fail to see how using dreamweaver and ftp is somehow better then using Wordpress and the cloud, writing your own html as a prerequisite to having a website was never a good idea - now everyone can have their own website.
Everyone being able to make sites is great, but I miss having sites that were deliberately organized rather than purely chronological. I wish the typical wordpress/livejournal had a frontpage similar to, say,
http://www.gwern.net/ . In theory tags provide the same information, but somehow I've never found them useful in the same way.
Yes, but before they all looked the same and were hosted on the same platform.
I miss the days where you would encounter a blog filled with great content and with a unique look and feel. It's as if it almost burned in in my memory. I still remember how the website looked of some great articles that resonated with me.
> Yes, but before they all looked the same and were hosted on the same platform.
"LiveJournal was started on April 15, 1999 by American programmer Brad Fitzpatrick as a way of keeping his high school friends updated on his activities. In January 2005, blogging software company Six Apart purchased Danga Interactive, the company that operated LiveJournal, from Fitzpatrick."
> Does anyone remember when you they stumbled on a new website written by some guy and read his first article, then clicked back to his homepage and saw he had a list of similar articles that looked like they'd be just as interesting.
Or, more likely, you clicked "back" on the "navigation bar" and it would 404 because the author messed up the links, since it was all hand crafted HTML.
Funny stuff aside, there are still loads of "websites" out there. If I understand the criteria here, we are looking for mostly hand crafted pages maintained by individuals (or small groups) that have interesting content. Something, should I say, very "web 1.0"?
Here's a good one. Make sure to check the GUI Gallery section:
We need a good search index of these and all the other sites likes them. Something that deliberately doesn't include pages from bigger sites or sites full of cruft.
"best viewed with" followed by your query (ex: "best viewed with" dragons) gets you lots of great old results in a regular search engine. It will grab any sites that said "best viewed with truetype fonts", "best viewed with Netscape" et cetera from the internet's heyday.
It's clever of the author to use "website" in this more specific sense. Technically a "website" roughly means anything served as HTML over HTTP. But all us fogeys know what his title meant anyway.
But I still think he protests too much. The style subject-oriented websites which the author is referring to evolved slightly and got the new name "blogs". The blogosphere might not be as popular as social networking, but it is still huger than the web of the '90s.
Eh, I don't think blogs are really that equivalent to old school websites. Blog, being short for "web log" implies a temporal-based feed of posts. Old school sites were often much more varied, webs of pages broken up by subject or topic with links scattered everywhere to the rest of the web.
This is why I really love initiatives such as https://neocities.org/ :). I don't have a use for it myself as I have my own servers, but I'm glad this kind of service exists!
> Does anyone remember websites? These might be unfamiliar to anyone unexposed to the internet before 2005 or so [...] it was largely a collection of websites made by people who were interested in some subject enough to write about it and put it online.
Does the author mean web rings? I do remember these :)
I remember web rings. I used to own a couple of sites that were part of a web ring. I remember it was actually hard to get into them back then -- the maintainers only wanted quality sites and you had to submit yours for review and explain to them why you deserved to be included. They were actually a decent source of traffic in the early days, before search engines became a thing.
I understand the sarcasm, but really, there are a lot of "real" websites, directly controlled by individuals who post what they want. I point you to my own website, https://www.dwheeler.com .... it's not the latest in CSS, no Megabytes of JavaScript, and no cross-site tracking either.
I thought about this a few weeks ago. I remember the late 90's actually searching around the web. Now there are so many walled gardens that individual creation is limited to posting a Medium blog or Facebook post. Today, I only visit a handful of sites and developer documentation.
I was a part of the Geocities team in 1998. Part of the reason a large part of the early web no longer exists is that many of these web pages were hosted for free by Geocities. Tragically when they were purchased by Yahoo, Yahoo decided to simply shred the early web, they decided it was not worth it to keep supporting the service and simply hit delete on a huge chunk of the content of the early web.
I do, which is why many years ago I told myself I was going to make my websites as js free as possible. Pure, simple, readable html5+css is my long term goal.
The thing is, this part of the web still exists, it's actually just simply harder to find due to the control on the filter big corps have these days. You just have to search. I am regularly adding hackers blogs to my bookmarks. Sometimes they stopped updating in 2012, but their writing still looks worthwhile. The real democratization of knowledge the internet offers to us is there for the taking if people would break free from their self-forged filter-bubble shackles.
I remember the NCSA Web Site of the Day, circa 1993. It was a page that listed new web sites that had appeared on the web, with a brief summary and link to each one.
Many days there wasn't a new web site. Some days there were two.
Unfortunately, a large part of the reason the web used to be fun is that there didn't used to be anything like it. The only way we can get that again is by inventing the next world-changing communications technology, not by trying to dig up the old web. It pains me as much as anyone else to say this since I got started on the web back in 1994 and experienced its early magic first-hand.
As the operator of a website old enough to still have a "webmaster" email address, it can be hard to remain relevant in the age of Google's search monopoly. Their algorithm has become the de facto gatekeeper to the entire Internet, and it happens to favor newer content over old. Consequently it favors those crappy flip-shop[1] sites where "writers" find quality content, perform a mild thesaurus modification, and re-post it. The original source, where the actual research and writing occurred, is stomped into insignificance.
Yes, perhaps I'm slightly bitter, why do you ask?
Between that and the "Wikipedia wall," boutique websites have been an endangered species for some time.
I'm "old" and I remember websites, and Bulletin Board Systems too, for that matter. But technology changes, and you gotta keep up, man. Don't let the kids bother you--they're just running through your yard to the next house over. It's a cool house, maybe check it out someone.
Now sites seem to be spread out across the behemoth sites like Medium and of course all the social media platforms. A lot of the fun and sense of adventure that was present in the early days of the web is gone. Now it's "just business" - that blows! When did everyone get so obsessed with boosting their "online presence" in order to make a quick buck?
I will also give a big thumbs up to Neocities. It's brilliant. There's a great retro feel to many of the sites there. People are having fun creating sites with HTML and a dash of CSS - for free.
As the former perpetrator of several hand-coded sites and blogs full of text organized by 52-card-pickup principles, going back to '80s BBS's, online services, USENET, and then Gopher and WWW on a University shared network…
Switching to Wordpress (obLink: https://mdhughes.tech/ ) and slowly reposting the stuff I want visible in an organized, searchable format with a consistent style and a nice CMS is the best possible improvement.
My latest change is a "Starred posts" category, so I can surface the longer, more thought-out pieces and still have ephemeral content like semi-daily music links and status updates.
Manton Reece's http://micro.blog/ is putting Twitter-like interaction under a blog framework, hosted on m.b or on your own site.
Discovery of these things is still hard, and Google's ad-searching site unsurprisingly only surfaces ads, but http://duck.go and social media (like HN) can point you at content you want to see. [ed: sp]
Shameless plug but what he describes was partly my inspiration for building https://nogradient.com/. It is very stripped down and minimal. Would love some feedback on it.
Once Capitalism takes over a system, everything is optimized for profit. Some diehard hobbyists might remain but most convert. I'm still keeping 2 niche blogs.. let's see till when.
Early business Web was Decision Support and not ad distraction based. There were cool "calculators" that would match relocation zip codes in Tulsa most like my favorite DC zip code or pick optimum breed/age/weight dog food.
Product Selection Engines (PSE's) were a different value proposition than product or brand promotion. They are fun to write for engineers because they correct purchase errors with IT good deeds like Consumer Reports but with user variables on priorities or constraints.
The Consumer Web threw out a great deal of baby with the .com era bathwater.
I wrote a little program that does the "follow links and see if I find anything interesting" thing. It takes seed links, pulls all links that don't go back to the same domain, and then chooses 50 and repeats. Then there's another program to find the uncommon sites from everything gathered. Check it out at https://github.com/riley-martine/water_skimmer
I have been thinking on this idea lately and trying to build a tool to filter out the crap. What I have found is operators help, but the only things I have found to work are EDU sites, pre-2010 operator, and intext: welcome to my site. Beyond that, nothing seems to work to find complex content. I also thought about maybe making a tool to filter out content from the top million sites or so, but for now I am going to just use operators.
The most ironic part to me is that the page is PHP, not static HTML.
Although this type of style isn't common with the online web these days, it's still visible in offline caches of popular websites (e.g. offline Wikipedia). Pages load so fast, and the total file size is much smaller because of the lack of JS/jQuery/React/etc bloat.
MySpace, Tripod, GeoCities, LiveJournal, and similar free hosted/services had a lot of interesting, weird, etc pages. It was easy and friction free to create a new site. Everyone had a copy of Frontpage/Dreamweaver/GoLive/HomeSite/iWeb/Composer on his PC and uploading worked with a browser form, one file at a time.
From their about page:
Search engines like Google are indispensable, able to find answers to all of your technical questions; but along the way, the fun of web surfing was lost. In the early days of the web, pages were made primarily by hobbyists, academics, and computer savvy people about subjects they were interested in. Later on, the web became saturated with commercial pages that overcrowded everything else. All the personalized websites are hidden among a pile of commercial pages. Google isn't great at finding those gems, its focus is on finding answers to technical questions, and it works well. But finding things you didn't know you wanted to know, which was the real joy of web surfing, no longer happens. In addition, many pages today are created using bloated scripts that add slick cosmetic features in order to mask the lack of content available on them. Those pages contribute to the blandness of today's web.
The wiby search engine is building a web of pages as it was in the earlier days of the internet.