I read this essay in 2003 and it influenced how my business partner and I built our startup. It influenced what possibilities we chased after. But Shirky.com is now off-line:
This is yet another example of the history of the early Web disappearing. Shirky's early essays were fundamental to way we understood the potential of the Web back in 1999-2008.
I actually discovered this while looking for another old Shirky essay which, as far as I can tell, is now entirely gone from the Web.
This whole thread is giving me feels, but to the basics, I'll tell you how it happened.
I'd been writing about what we came to call social media since the early 90s (alt.culture.usenet and alt.folklore.urban ftw), but by the middle of last decade, all anyone wanted to talk to me about was marketing on Facebook, which was the boringest possible topic.
At the same time, my wordpress host had lousy security, and my site was getting frequently disabled because of some malicious javascript uploaded through some hole they hand't patched. I wasn't writing there anymore, so it was pure cost at that point, and cost of my time, not just dollars.
Then I moved to Shanghai for several years, working on other stuff, and fixed the site a couple of times again, and one time, my host was like "We disabled your site!" because of their own security flaws had let it get hacked again, which, the whole thing had entered 'ugh field'territory.
I never decided to let the site lapse, I was just tired of dealing with it, and the political circumstances in both China and the U.S. seemed much more urgent than rescuing some historical essays, so one day at a time of not dealing with it became years.
And here we are, me reading my own eulogy. Which is incredibly flattering and touching, I have to say.
I'm not even sure what of it can be resuscitated -- maybe if I want it back, I'll have to copy it from Wayback (and will say "Thank you Brewster", not for the first time), but if anyone here has advice about competent and secure hosting for an old Wordpress blog, hmu at cshirky@gmail.com, because reading this, it makes me embarassed not to have just fucking fixed this a year or two ago.
I used to work on a lot of US Department of Defense projects, mostly stuff I can't talk about. One very notable project I CAN talk about was an initiative (pushed by utterly clueless, insular, and frankly corrupt academics) to spend billions of dollars in 2008-2010 timeframe on implementing Semantic Web technologies in various military business systems across the DoD.
As an actual technologist who knew how to build things, I was perpetually in the awful position of having to explain to leadership that these highly credentialed academics were selling garbage. I had tried to implement systems according to their design. The graph databases they pushed (they hated Neo4J, for reasons of purity because it didn't actually use RDF/OWL in the database...... i get a headache just talking about this...) were slow piles of dogshit that couldn't scale. No amount of reality could dissuade the academics. They had their theories, and any collision with reality was merely an implementation detail that I and my team were simply too incompetent to overcome in their eyes. Almost none of them had actual technical experience. A smattering of Comp Sci folks, and a ton of "Library Science" idiots.
Your essays on why the SemWeb was utter bullshit were a potent weapon I used with the generals the academics were pushing, and I eventually got the generals funding the project to see the light. Got them cancelled, and sent the idiot egg-heads packing. I still see them on LinkedIn to this day. They desperately continue trying to push that rock up the hill, and only recently warmed to more practical graph database solutions.
They HATED YOU. It was hilarious, watching them try to refute your obvious points and clear writing with jargon and hand-waving. Utterly unconvincing to the generals.
You cannot possibly know how much this delights me!
Most of my writing was about social media, back when the web was young, but "Ontology is Overrated" is actually my favorite thing I ever wrote, and it makes me happy beyond measure to know that it helped someone manage an actual argument over whether to buy into the semantic web!
I have never been talented enough to write production code, but I often thought of myself as trying to provide ammunition to people like you who are, when talking to bosses who didn't understand that the phrase "Now it's just a simple matter of programming!" was a bitter, sardonic joke, not an upbeat assessment of possibility.
Thank you for telling this story! This whole thread has been like hearing my own eulogy, but this in particular is just :chefs_kiss:
Awesome! Yeah man, you nailed it all on the head back then, and yes, saved my ass. Sorry the thread felt like a eulogy. You've got a lot of mileage left in you my friend, and I'm looking forward to your future insights on the next big trends none of us are predicting yet.
The response of the academics to your writings was actually a master class for me in the nature of academic corruption and groupthink. What I learned from that experience was that (contrary to my prior beliefs) high IQ individuals are actually far more susceptible to cognitive dissonance than others are, not less so. They are far more adept at mentally constructing rationalizations and false realities that bolster and protect their existing belief systems from new information than most people are. Add to that the fact that they are extremely economically vulnerable to reputational damage, and you have a really toxic recipe.
Library Science is what librarians learn. It's a real thing for a real job. Like most credentials and like most jobs, some people try to over-fit experience and knowledge in one field to another.
You see the same with CompSci/tech people treating data like there's no bias in its collection.
As I learned from the people on that project with lib sci degrees, the employment prospects are predominantly low-paying, but these ones found a new boondoggle to employ them as "ontologists" where they could get 6 figure salaries to sit around and build models all day in a piece of software called TopBraid Composer. (GUI program built in Eclipse, where users would create diagrams that would then be translated to an XML offshoot called OWL, a W3C standard that's never been successfully used in any meaningful project I've seen) I witnessed these people sit around and create business models and knowledge graphs of arcane Air Force business processes for 3 years (there were literally 9 of them doing this) before the project was cancelled due to its technical impossibility. The ontologies they created were never used once, and when I actually tried to provide them (in PDF form) to a separate project where Air Force personnel were trying to map out business processes, the personnel stated to me (in writing, with a Colonel CC'd) "These are so inaccurate that they are frequently misleading, and cannot be trusted." The Colonel later pulled me into his office and stated (rather comically):
"You mean to tell me I've been paying people to draw cartoons for 3 years? We're not goddamned Disney here."
If it's just a collection of essays, what about a static site? You can set one up on Github Pages or Gitlab Pages with a minimum of coding. There are also virtually no security concerns and maintenance is minimal.
You'd have to take all of the text from the Wordpress blog and format it into Markdown but that shouldn't take a huge amount of time unless there is a lot of weird formatting or different media types.
This is absolutely the easiest solution. We did this for open.media.mit.edu in order to archive the old wordpress site. Used wget to fetch all of the HTML and ran a script over it to remove some obvious clutter. Happy to help out. Your writing really shaped the way I thought about the potential of an open internet.
Hi Clay, Kevin Marks here. I an help you scrape it out of wordpress into a static site if you like, I've done similar for Dan Gilmor and other old bloggers. Ping me - kevinmarks@gmail.com
You could scrape the site, either from Wayback or from WordPress, if you manage to get it working just briefly, and then host the site statically out of an S3 bucket. wget has an option to recursively crawl and save a site, but there are other tools.
> You’ll never maintain a Wordpress site long term securely. Need to convert it to static html one way or another.
I'm in favor of static HTML myself where possible, but it's not hard to maintain a secure Wordpress install. Keep automatic updates enabled and don't install any third party plugins.
It's that second part that most people screw themselves with.
It may not be very hard to maintain, but you still have to maintain it. Whereas if you just have a collection of articles that you want to keep around as an archive, if you convert them to a static site, you can basically forget about them afterward...
> It may not be very hard to maintain, but you still have to maintain it.
When the maintenance is "ensure auto updates are on, and don't do anything that would not get updated automatically" it's not like it requires regular effort.
> Whereas if you just have a collection of articles that you want to keep around as an archive, if you convert them to a static site, you can basically forget about them afterward...
Your web server, your operating system, etc. still require at bare minimum the same level of maintenance.
You can outsource that maintenance to someone else of course, but you can do the same with WP as well.
--
My point is that WP alone doesn't massively increase the maintenance burden, it's what people tend to do with (to?) WP that increases the burden and eventually leads to unmaintained sites.
>When the maintenance is "ensure auto updates are on, and don't do anything that would not get updated automatically" it's not like it requires regular effort.
no dog in the fight here but I felt impelled to point out that ensuring auto updates are on solves almost all security holes except for the security hole it opens up.
> no dog in the fight here but I felt impelled to point out that ensuring auto updates are on solves almost all security holes except for the security hole it opens up.
In almost any computing context, but especially in the context of a personal blog, the vast majority of exploits are against known security holes for which patches have already been released and those with automatic updates enabled are already safe from.
Yes, hypothetically updates can deliver new flaws of their own and even potentially intentional malicious code, but from a practical sense it's not worth worrying about if you're using mainstream software packages on a major OS.
Right. I should have clarified it’s unlikely to happen if you want to be hands off for years at a time. If that’s the goal the ideal state is to convert it to static.
WP security has come a long way. I've had a site up for over a decade, and while I used to be VERY nervous, now with automatic updates and a fair amount of code-hardening, it really hasn't been a problem.
What strikes me is how timeless your writing is. My own stuff is hit-and-miss, the bulk of it has a best before date measured in a few weeks or months but yours is just as relevant today as when you wrote it and that takes some real talent, not just in writing but also in the depth of insight that you have on this things.
I only know of you because of The Shirky Principle, which was very valuable to me in figuring out what's wrong with homeless services and that we need to focus on fixing housing instead. Thank you for your work and for stopping by to respond here.
Try to take a look at wpengine.com; they're not the cheapest, but I heard they have very good support people, and via some business relations I had with them, I know they have a system for auto-updating wordpress plugins.
He explained that an expanding __________ (I can't remember this word. Maybe he said "network"?) meant that each person in the __________ was exposed to a greater variety of choices, however, the total number of choices decreased. His point was that separate niches each maintained some unique offerings, but when everything was combined together into a single market, and attention focused on certain items, there would be a winnowing effect. The paradox was that everyone could legitimately feel they were enjoying an abundance of options, greater than ever before, while the total number of options decreased.
"Now that more than a billion people have access to the Web, there is no longer a trade-off between size and specificity. The basic math is simple: A tiny piece of an immense pie is huge. A decade ago, reaching one-tenth of 1 percent of Web users amounted to 36,000 people, a number that compared favorably with the circulation of, say, the daily newspaper in Bridgewater, New Jersey. Back then, reaching a million users required a decidedly mainstream offering (Amazon.com and MSN come to mind). Now, getting niche can be the path to getting big; one-tenth of 1 percent of today's Web audience is a million people."
I was just thinking about this - I can go to the store and have fifty variations of peanut butter at my fingertips - but I get basically the same fifty variations anywhere in the US, and much of them anywhere in the world.
Whereas in the past, my store might have 1 or even 0 options on peanut butter, but travel 10/50 miles away and there'd be an entirely different option.
We have a little bit left of this with beer, as most places have a "local" beer available.
Someone once described the general difference between European and U.S. grocery stores: They said that in Europe, a store would have every kind of product, but not very many brands of any particular thing, maybe two or three for most non-staple products. But in the U.S., a store would either have like 20 brands of something or have none of them; i.e. not carry that kind of product at all. It seems to me like this points to “choice” being an important scale by which stores are measured in the U.S., but no, or little, negative associations are made with no products of a particular kind being available at all.
There's some of that European approach creeping into US markets thanks to Aldi Nord and Aldi Sud - we now have both Aldi and Trader Joe's which primarily have their own versions of products and carry only that brand (or occasionally a second 'mainstream' brand as special items). Costco is similar as well in typically having only a single brand of any given type of product.
Ever stopped to ponder the laundry detergent aisle at your local grocery store? Or the potato chip aisle? Especially on detergent, what's there is a huge proportion of the shelf space devoted to a couple of brands (which may, in fact, be from the same parent company) with very large containers taking up a lot of space for relatively little actual product, but lots of different choices for those brands. Or, as I like to say, 31 flavors of Tide Pods. Same goes for potato and corn chips. Lays and Ruffles dominate, which oversize bags that are half air, relegating other brands to lower shelves on one end. But consumers can "choose" from half a dozen different flavors of the same brands.
And it's the same all over the US. The only variation tends to be that some flavors Flamin' Hot Dill Pickle or Wasabi Ginger, e.g., appear more often in some markets than others.
Good summary. The other day I was hearing a political example mentioning that every little local detail, now with social-media, it becomes potentially "a national issue" but that never happen before. Surprisingly, the Founding Fathers were aware of this issue back then and tried to solve (actually mitigate) the problem by keeping the central power as small as possible.
The title of that article is now itself a comment on the passage of time and the fading of things. How old do you have to be today to get the "I've fallen and I can't get up!" reference?
My 27yo ("reference Millennial") even has a mental image of the video (although has never seen it on a Tele-Vision Device). It may have entered the meme reservoir on its own at this point.
I wasn't quite sure when he wrote this. The earliest capture in the Internet Archive was January 2000, but it could be from earlier than that maybe? I do remember that it was quite prescient when I first read it, although it's sort of obvious in 2022.
> People who work on social software are closer in spirit to economists and political scientists than they are to people making compilers. They both look like programming, but when you're dealing with groups of people as one of your run-time phenomena, that is an incredibly different practice. In the political realm, we would call these kinds of crises a constitutional crisis. It's what happens when the tension between the individual and the group, and the rights and responsibilities of individuals and groups, gets so serious that something has to be done.
> And the worst crisis is the first crisis, because it's not just "We need to have some rules." It's also "We need to have some rules for making some rules." And this is what we see over and over again in large and long-lived social software systems. Constitutions are a necessary component of large, long-lived, heterogenous groups.
We focused on the "unfairness" and we thought there might be a technical fix, so we focused on new discovery mechanisms. But our solution did not get traction, so we eventually pivoted towards tools for building commerce sites, and there we had more success.
I suppose we can look at this with nostalgic eyes.
I was a big Shirky reader back in the day.
But the truth is that all kinds of things disappear all the time in all aspects of life. The web is no different at all.
Take my dad for instance - a quite profilic and famous mid level artist. It’s coming to the point where not much people remember him. And when my siblings pass on that will be that.
Let’s not get too nostalgic. If someone is interested they should try to preserve the writings and keep them going.
That’s why certain groups have taken it upon themselves to preserve old important films. And as we know they are always fighting for more donations to keep things going.
Because in the end… no one really cares.
50 years from now children will be asking who was Van Gogh.
> Take my dad for instance - a quite profilic and famous mid level artist. It’s coming to the point where not much people remember him. And when my siblings pass on that will be that.
Why not say his name here, to give him a bit more memory?
> 50 years from now children will be asking who was Van Gogh.
This seems a strange cut-off. Van Gogh is an artist who died 130 years ago; why should the next 50 years be the ones that forget him? There are plenty of artists today whom we remember from earlier than 200 years ago.
I don't think Van Gogh is a good example. There are certain artists whose work outlives them by centuries, not many, but we still talk about Michelangelo, da Vinci, Monet, etc. The rest of your post is accurate, though.
We simply haven't had enough time pass. Eventually, some day, people will forget who Julius Caesar was. It may take 50,000 years. It may take 500,000. They'll forget.
Counterpoint: an ancient Babylonian copper merchant is remembered to this day for being a no-good swindler due to complaints against him recorded in stone tablets [1]. Remembering things is only getting easier with better data storage. I guess you could just move the timeline out to the heat death of the universe, though.
The Mesopotamians were more preoccupied with writing down contracts in cuneiform than writing down historical fiction to last through the ages. Maybe because they were one of the first civilizations to thrive and to invent writing at all, they didn't know their oral traditions and history would be lost to the sands of time without writing them down.
We do have more insight into ancient Egyptian pharaohs and architects, however, as these details were more carefully preserved.
That being said, I'm sure 500K years from now, these details will all be buried on some thumb drive in an underground archive and our descendents will lack the drivers to decode them.
> But the truth is that all kinds of things disappear all the time in all aspects of life. The web is no different at all.
Glad we have the Wayback Machine then. But if you don't want your blog mirrored by Wayback you can declare that in your `robots.txt` file. Do this:
User-agent: *
Disallow: /
But that doesn't mean crawlers/bots will honor that request and presume any content you post publicly will be backed up somewhere. If not somewhere on the net, then on someone's hard-drive!
The blog post you are linking is outdated. They are honoring robots.txt files. From the FAQ:
> Some sites are not available because of robots.txt or other exclusions. What does that mean? Such sites may have been excluded from the Wayback Machine due to a robots.txt file on the site or at a site owner’s direct request.
If you exclude them in your robots.txt file they will also absolutely retroactively remove your site from the index.
I would absolutely love an option that meant "archive and make available forever from this point backwards" to protect against domain expirations and re-registration (possibly by domain squatters or content farms).
I hope you're right! The lack of an update on that post, combined with the FAQ saying the opposite thing, makes it even harder for me to know what their policy is. Respecting robots.txt is a civilized thing to do and I hope they do it.
Well, the fact that they will be asking will lead them to discover to who van Gogh was and so he will live on, as will his paintings and the reproductions of those paintings.
Bach has been dead for hundreds of years and his works are being rediscovered by people every day.
Clay Shirky's essay, "The Semantic Web, Syllogism, and Worldview:" [0] is a powerful warning about the pitfalls of global efforts to categorize information.
In the early 2000s I had a manager who evangelized "the semantic web" to our team as being just years from taking over the world. He was convinced that we needed to integrate RDF into every product to remain relevant. Shirky's essay persuasively articulated why this would have been a waste of effort for us.
Years on, these ideas still influence my analysis of elevator pitches, business plans and requested features.
Also fairly hidden these days is Many-to-Many (also sometimes referred to as Many2Many) http://web.archive.org/web/20081229123241/http://many.corant... where Clay Shirky, Liz Lawley, Ross Mayfield, Sébastien Paquet, David Weinberger and danah boyd posted.
This is terrible. But the upside is that most of them have really good books you can still buy on Amazon, so if not the actual historical essays then at least most of their ideas will be preserved indefinitely.
Clearly a lot of them have been looking back at that period with mixed feelings over the last few years (and have said as much), but even still it's shocking that something like Clay Shirky's blog would disappear when he's very much still alive. The fact that we apparently now have to worry about whether or not something like Danah Boyd's MySpace vs Facebook essay could disappear though is ridiculous; even with all the known technical and social problems with the web it's hard to imagine that it's come to this.
> But the upside is that most of them have really good books you can still buy on Amazon
I am wondering, are there any organizations that actively scan and archive books, even if they don't share them because of copyright laws? Amazon is almost a monopoly when it comes to books, and we cannot rely on it to preserve the books for the next 50+ years, and not every purchased copy is guaranteed to be around by then.
In addition to the national libraries, it's extremely common for librarians and archivists to keep things in a personal collection on the DL regardless of copyright.
I have every ROM released for pre-2000 consoles + a ton of old software, OSes, and PC games on hard drives that I keep backed up. I have friends who prefer to specialize in keeping/archiving zines. Someone else does fanmade ROMs. Etc. Most of us have a good understanding of copyright law even where we disagree with it, and it's common to archive 'grey' material off the record and then fight for the right to make it official.
Books are even easier than digital assets since the laws around book archiving and preservation are much kinder to archivists. So yes, there are definitely DRM-free, digital copies of MOST books floating around and will continue to be for quite some time. The main issue is whether or not we'll be prosecuted if we open up our personal archives or distribute them.
> I am wondering, are there any organizations that actively scan and archive books, even if they don't share them because of copyright laws? Amazon is almost a monopoly when it comes to books, and we cannot rely on it to preserve the books for the next 50+ years, and not every purchased copy is guaranteed to be around by then.
The University of Michigan and Google Books have something like this, at least for the UM library: https://publicaffairs.vpcomm.umich.edu/key-issues/google-set... . I can't find much about whether it stopped completely, or just went quiet, after the lawsuits.
Sweden has the Royal Library, US has the Library of Congress. I believe most developed countries have national libraries, archiving books and newspapers as they are published.
One thing I always loved in any old sci-fi that dealt with internet-like spaces was that most of them had a concept of a public internet. They generally sort of felt like the "slums" of the internet filled with trash and spam and stashes for hackers, but the concept of a public space to which anyone could post anything always appealed to me. I'm quite sad that we don't have something like this.
We've had plenty of things that had that general appeal, but they've always been owned, run, and eventually shut down, but companies of some sort. I'm not opposed to companies having websites, but I'd love to see a public space as well. The idea that we could post our content to that space and expect it to live far longer than us would be a huge deal.
> One thing I always loved in any old sci-fi that dealt with internet-like spaces was that most of them had a concept of a public internet. They generally sort of felt like the "slums" of the internet filled with trash and spam and stashes for hackers, but the concept of a public space to which anyone could post anything always appealed to me. I'm quite sad that we don't have something like this.
Usenet still exists, and it is definitely filled with trash and spam. There are various projects working on similar sorts of public (or sometimes private but open) spaces without those downsides, it remains to be seen what will end up filling this niche.
um...we still have, like, the internet? You can make some html file, post it on a computer exposed at a port, connect it to the internet, assign it to a static IP, and tell other people the address?
Unless you are specifically referring to someone providing general purpose hosting that you don't need to think about administering — well that isn't a feature of our present-day landscape simply because it wouldn't be a profitable venture given the risks and liabilities from the messed up things people could stash there, along with the inherent costs of admin and hosting.
But if you are willing to set up your own box and procure sysadmin for it, what you suggest exists.
Yes. I also like subways, buses, indoor plumbing, garbage collection, libraries, streets, sidewalks, police, fire safety, postal service, and all sorts of things that we share as members of a society.
Governments around the world have taken way too long to treat internet as a public good (i agree with you).
Especially in america, this country has been running as fast as societally possible in the opposite direction from providing such services as a public good -- so if anything even those libraries and sidewalks are mortal, and at the whims of a political machine idealists may not get to steer.
It sure would be nice if such a thing could exist in stability ad infinatum, but at the same time, there is a small turdy nugget of truth to the american principles of not trusting government with acting in the interests of the people's freedom indefinitely, as has been demonstrated by nations around the world.
We need more grassroots organizations like archive.org supported and run by the people who truly believe in it.
100% agreed with everything you said (and I contribute to archive.org).
Governments won't necessarily get it right, but I would still love to see public efforts toward the goal. Despite recent adjustments, the US postal service has been an impressive slice of government service for a very long time.
What's interesting about this particular goal is that it can easily cross our physical borders. A couple savvy countries can easily team up and get things rolling (or at least throw significant support behind existing efforts)
Check out Arweave: https://www.arweave.org/. It allows for 200-year storage of data, compatible with IPFS. It uses an endowment-funding model to achieve this: arweave.org/technology#endowment
I don’t really see what stops you from having your own site with static content you want to publish. Get a domain for like 10 years, with auto renewal enabled. Get a box somewhere, serve you static content. You can publish whatever you want for almost no maintenance for decades.
(Of course, that’s unless you’re talking about sharing the type of content that would make interpol want to track you down)
Good luck having a VPS 'box' that has an uptime record of 10 years. I know through personal experience that your VPS instance will go down, no matter how much you try to mitigate that. There will be bots and bad actors either trying to DDOS it, or trying to brute force `/wp-admin`.
You could go for some obscure CMS to try and thwart that, but you run the risk of having vulns in that software because it doesn't have the eyeballs of vanilla Wordpress. You could always go for the shared hosting approach but the caveat being: there is no guarantee the shared provider will provide 100% uptime either. (And it will go down at the worst possible moment, like during a HN hug of death)
Your best bet is to have your content distributed and mirrored across multiple services such that any attempt to take it down is impossible. I would go into details about that, but due to op-sec reasons I won't. Tip: Plaster your content all over the web such that a removal of one piece of content does not affect the others.
You're moving the goalpost way further or have a completely different goal in mind than I do.
The only thing that you need if you want to publish content online is an nginx config and a machine with a public internet access. And that's it. No CMS, no wordpress. Almost nobody has a need for 100% uptime so I don't see why you would mention that. If someone wants to DDOS you you have completely different situation and it is very likely your content will be offline, which is fine, no? It's costing them money and the only downside on your side is to not have your publications available for some amount of time. If you're talking about business services, sure, that's a big deal, but for a person sharing their writing, that's a non-issue. It takes what, something like 30 minutes to get another machine, setup nginx, then upload your content?
A $5/month server with debian + nginx serving static content can survive way more than a HN hug of death.
You can even use a box on your home network (or even your router itself with a little elbow grease...) Many routers support popular DDNS providers, and most ISPs don't block port 80 or 443. It may be against your ISPs TOS because they don't want you hammering their upload capacity, but if you put it behind a free-tier CDN that soaks up the spikes (e.g. Cloudflare) then they're unlikely to care. Setting the whole thing up only takes a couple hours (if you're inept like me) and you're in complete control.
And once I'm dead, broke, or have simply moved on from the tech universe?
The ephemerality of the thing is the issue I'm speaking to. We've lost something here.
The requirement for books to last is physical space, and those shelves and boxes continue to exist far longer than the publishers, authors, illustrators, etc. We don't have that with this medium (except, of course, archive.org which is excellent and not nearly enough). We've built something that's lighter than books and easier to store in smaller spaces, but we've [collectively] given no thought to maintaining a proper archive.
The freedom to publish to the world in an instant is as magical as it is fleeting. On a longer scale of time - and not a very long one - it's practically worthless.
IPFS is one way, I believe. Archive.org has been pretty dependable too so you can technically post on Twitter or a blog post then have Archive take a version of it.
I still see your point, though. Maybe mixing in IPFS with some public solution, like a guest book, might be something?
On the one hand, I like the idea of the internet being organic where things rot and die, truly forgotten.
On the other hand, it's much easier for future historians to look back a hundred years and see the actual content that's posted at any point in time.
In both instances I think we overestimate how important any one piece of content is. If an idea dies when a site goes down, it probably wasn't a very good idea.
So are we talking about accurate attribution of ideas, or the loss of good ideas per se?
Recently I was trying to look for some documentation on TK. I had remembered an older site, the book on TK, all very good information if slightly dated. Took me a decent amount of searching to remember- effbot.org, which also somewhat micro-famously went down recently due to the author's death. Makes me wonder what other very high quality resources will be lost in the next two, three decades as developers age...
RIP shirky.org, your insights then are just as good today
I think that's a good question. It's just a bunch of articles and perhaps there were some videos linked to youtube? I find it hard to believe that Shirky just abandoned it given how media savvy he is, but stranger things have happened.
Yep, it became a spot for all your erectile dysfunction needs. And the constant pattern of "I fix it, spammers break it' wore me out, so, despite media savvy, I just stopped caring (though this thread convinces me that was the wrong answer...)
I can't seem to find the erectile dysfunction solutions on your web site any more, can you link me to a reputable reliable source, please? I so miss your site. Good luck getting it back up.
Vulins found every other day in some obscure package you thought you were one and done with. You get to spend a couple of hours fixing it. Oh new update on the java core you are using few more hours. Oh that update breaks 2-3 things. More time. You float the idea that someone else takes it. But those who step up have 'other ideas' what they want the site to be. Oh and your base OS is 2 releases back better get on that.
Then the actual cost. While you can get a cheap site up and going for not much. If you get even slightly popular you are now looking at a decent amount of money for many people. You may not see a couple hundred a month as 'no big deal' but many people do. You can pay a provider to take some of that patching work out of your hands but you pay for that.
The programming world is very ephemeral. We get bored easy. We move on quickly. Sometimes we are just cheapos. Things that cost time and money get left to rot or turned off.
What I find interesting in my 'internet' life. Is I always seem to find out about the really interesting places as they are being closed out :(
> Then the actual cost. While you can get a cheap site up and going for not much. If you get even slightly popular you are now looking at a decent amount of money for many people. You may not see a couple hundred a month as 'no big deal' but many people do. You can pay a provider to take some of that patching work out of your hands but you pay for that.
Agree with the whole message and tone of your comment but wondering about this bit. We run a bunch of Wordpress sites with decent traffic and a bunch of badly optimised front-end, heaps of old plugins from decades passed: we can hit 20k uniques and a million requests per day, with nightly backups for $30/mo. It could be less if we didn't care about completely surviving every traffic spike and bot crawl.
$30/month indefinitely adds up pretty fast. It's $360/yr or $18,000 over 50 years. It feels like pocket change to someone with stable IT employment, but it shuts out a fair chunk of the population.
Definitely expensive for an individual or small group which is why I emphasised agreement with the message. The question was specifically about the cost estimate they gave.
Edit: Re-reading what I wrote and how much I quoted I realise now that wasn't clear :)
uh... Not sure where you got that I had a cost estimate? My point was time and money. If they do not have time or interest for something any amount of money is probably too much. That was sort of my point that I had time to think about it.
It is just setting it up. Also sometimes people just lose interest in it. Even a couple of bucks a month would be not worth it. Then if you stand it up and 'forget about it'. What happens when your CC expires? It goes away. You do not care anymore so it is probably not something you care to fix.
For me 50-100 bucks a year is not something that is that big of deal. But if I have totally lost interest in it. It would be on the list of expenses to get rid of. It is one of those things a lot of clean up your financial problems people talk about. Look at all of those little charges. They add up to decent money sometimes. Not saying that happened here. But it probably does happen?
Even having to pay pennies a month is something that has to be maintained (do you remember to update your credit card info, are the emails correct, etc).
It would be nice for something like the Internet Archive to offer "perpetual hosting" where you pay upfront for enough to fund hosting "forever". $100 would generate $1 a year in interest which would be enough to host small data.
> Q: Do I get interest on my deposit? A: No[, but...] We periodically reevaluate this situation, because we think a web account that runs forever purely off of its own interest is a pretty cool idea.
NFSNet also has an interesting part in their FAQ in response to the question If I think services you host are currently unavailable due to lack of funds; is there anything I can do?, they outline a process whereby third-parties can fund a hosted service by creating an account themselves, depositing funds into their own account (NFSNet services are prepaid instead of billed after the fact), and then submitting a manual (but free) request to transfer those funds to the original accountholder based on the service's domain name. I've always thought this was interesting because in theory someone could set up a community, disappear, and then the community could step up to keep it funded long enough for the person to get out of the hospital/be rescued at sea/etc, so long as the infrastructure is solid enough to remain operational without being attended to (not vulnerable to exploits, etc.)
They've also got a policy where if the member who operates the service is a willing participant, they can publish their NearlyFreeSpeech.Net account ID and have donors add funds to cover 100% of service costs via automated transfers. <https://www.nearlyfreespeech.net/about/faq#Lifeboat>
Surprisingly, endowedhosting.com was available, so I bought it. If IA or comparable ever decided to offer such a service, I’d happily hand the name over to them.
>Website Lets You Send a Post-Rapture E-Mail to Friends 'Left Behind'
>If millions of Christians suddenly disappear from the face of the Earth as the opening act for Armageddon, Threat Level thinks most nonbelievers will be too busy freaking the hell out to check their e-mail. But if they do log in, now they can be treated to some post-Rapture needling from their missing friends and loved ones, courtesy of web startup YouveBeenLeftBehind.com.
[...]
Good thing the sysadmins are loving trustworthy Christians:
>Users can also upload up to 150 megabytes of documents, which will be protected by an unidentified encryption algorithm until the Rapture, then released to up to 12 nonbelievers of your choice. The site recommends that you use that storage to house sensitive financial information.
>"In the encrypted portion of your account you can give them access to your banking, brokerage, hidden valuables, and powers of attorneys," the site says. "There won't be any bodies, so probate court will take seven years to clear your assets to your next of kin. Seven years, of course, is all the time that will be left. So, basically the Government of the Antichrist gets your stuff, unless you make it available in another way."
There was a pretty good Law and Order episode where one of those sites accidentally triggered, sent an email confessing to somebody's crimes prematurely, which led to an unfortunate chain of events and salty remarks.
>The owner of a Rapture website is killed by a man working to return Soviet Jews to Israel to fulfill Biblical prophecy. However, the killer seeks shelter at the Iranian embassy, leaving the DA's office in an unenviable position.
Clay- I strongly encourage you to find a way to make your writings available once again. Many of the things you wrote are as true today as they ever were and the rest are valuable as historical reminders. FWIW, GitHub Pages might well be the simplest and best solution.
I was one of very early subscribers of his NEC mailing list. Just found this article he had sent out and now on archive.org. It’s another prescient article which now seems obvious
I hadn't read this before, but I think I have a new guidepost for innovation: real tech flattens the power law for everyone, whereas phantom tech amplifies the power law.
I think you’re drawing the exact wrong conclusion from the article? Increased access to choice amplifies the effect of the power law.
From the article:
> Now, thanks to a series of breakthroughs in network theory by researchers like Albert-Laszlo Barabasi, Duncan Watts, and Bernardo Huberman among others, breakthroughs being described in books like Linked, Six Degrees, and The Laws of the Web, we know that power law distributions tend to arise in social systems where many people express their preferences among many options. We also know that as the number of options rise, the curve becomes more extreme. This is a counter-intuitive finding - most of us would expect a rising number of choices to flatten the curve, but in fact, increasing the size of the system increases the gap between the #1 spot and the median spot.
Unless your real tech equates to censorship or control or something, I am not sure how it would help reduce power law effects?
I also noticed recently that Philip Guo’s site is gone, and was taken down on purpose. I remember a few great articles that showed up on HN. My interpretation was that the ratio of effort to reward just started to skew, which is understandable and sad
http://www.shirky.com/
I'm not sure when this happened but I see that Wikipedia has adjusted to this and now links to the archive.org link:
https://en.wikipedia.org/wiki/Clay_Shirky
This is yet another example of the history of the early Web disappearing. Shirky's early essays were fundamental to way we understood the potential of the Web back in 1999-2008.
I actually discovered this while looking for another old Shirky essay which, as far as I can tell, is now entirely gone from the Web.