Hacker News new | past | comments | ask | show | jobs | submit login
Seattle library network outage nears a month (seattletimes.com)
87 points by dangle1 87 days ago | hide | past | favorite | 36 comments



The British Library - the UK's National Library - was devastated in October 2023 by a ransomware attack [0] which had a massive knock-on effect on academic institutions, students and also 20,000 authors who derive income from Public Lending Rights.

One of the reasons that services still haven't been restored is that the Library relied heavily on ancient bespoke software running on old versions of OS. New IT is being installed that is more proof against modern cyber attacks, but the older library software simply doesn't run in modern environments. So they can't simply restore from backup, they have to port / reimplement some of their operational software.

[0] https://en.wikipedia.org/wiki/British_Library_cyberattack


This is why factories run airgapped networks -- tons of mission critical software only runs on really, really old platforms (think warehouse management systems running on a PDP9/11, or assembly machine control software that's been unsupported since Windows NT4).


> This is why factories run airgapped networks

Yes, also Critical National Infrastructure often distinguishes between 'operational' and 'business' software / networks.


It's a shame the same passion for porting old video games doesn't exist for library software. If we can get doom to run on a refrigerator surely we can get a less computationally intensive software to run - if only in a virtual machine on a more modern OS.


Library operations are simple CRUD, a competent engineer could replace most library systems with a webapp in a week. Running webapps on machines immune to ransomware is a solved problem.

These organizations are simply incompetent and are now paying the price for being run by fools who don’t know what they’re doing.


> Library operations are simple CRUD

Not the ones that manage royalties payments to 1000s of authors depending on data collected by a national system of library IT, or the Secure (!) electronic delivery service that makes more than 100 million items available to researchers worldwide. See [1] for a full list of the massive range of non-trivial services operated by the National Library.

> These organizations are simply incompetent and are now paying the price for being run by fools who don’t know what they’re doing

Well Information Management is literally what some of their staff are trained in, but the National Library is publicly funded, so can't update software on its own terms.

[1] https://en.wikipedia.org/wiki/British_Library


I said most. The complex ones could be replaced by a small team and a month or two. It’s not a hard app.

Public funds should never be used to support such brittle and idiotic institutions, regardless of how noble their mission may be.

Running windows at scale off a read/write drive by untrained staff is negligent and has been for at least 5-10 years now. Putting windows in the hands of people who aren’t expert-level at data security without your own medium-sized IT security team to file off all of the rough edges and foot guns is like handing a pistol to a toddler. It’s your fault, not theirs. Of course they got ransomwared.

Give them a webapp and some iPads or Chromebooks. Stop pretending Windows endpoints for non-engineer staff is sane or reasonable. It’s absolutely not unless you have a dozen+ full-time dudes layered on top whose only job is to make it so. What works for Google doesn’t work for you, and what works for JPMChase and Boeing doesn’t work for you.


I've built library systems before - "simple CRUD" is just being naive at the complexity of not just some workflows, but the politics of library staff not wanting to change how they've done things for 20 years.


Well now they get to use Excel, so we know change is possible.


The spreadsheet method only supports checking books out - not returning them.


>older library software simply doesn't run in modern environments

dosbox is a thing. They could be up and running in couple of weeks if there was political will.


̶C̶o̶m̶p̶u̶t̶e̶r̶s̶ ̶d̶o̶n̶'̶t̶ ̶n̶e̶e̶d̶ ̶n̶e̶t̶w̶o̶r̶k̶s̶ ̶t̶o̶ ̶f̶u̶n̶c̶t̶i̶o̶n̶.̶ ̶I̶t̶ ̶i̶s̶ ̶p̶e̶r̶h̶a̶p̶s̶ ̶a̶n̶ ̶i̶n̶d̶i̶c̶a̶t̶i̶o̶n̶ ̶o̶f̶ ̶h̶o̶w̶ ̶t̶h̶e̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶i̶n̶d̶u̶s̶t̶r̶y̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶e̶d̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶e̶n̶f̶o̶r̶c̶e̶m̶e̶n̶t̶ ̶i̶n̶ ̶t̶h̶e̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶w̶o̶r̶l̶d̶ ̶t̶h̶a̶t̶ ̶l̶i̶b̶r̶a̶r̶y̶ ̶n̶e̶e̶d̶e̶d̶ ̶t̶o̶ ̶b̶e̶ ̶c̶o̶n̶n̶e̶c̶t̶e̶d̶ ̶t̶o̶ ̶t̶h̶e̶ ̶n̶e̶t̶w̶o̶r̶k̶ ̶t̶o̶ ̶l̶e̶n̶d̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶c̶o̶p̶i̶e̶s̶ ̶o̶f̶ ̶b̶o̶o̶k̶s̶.̶ ̶ ̶T̶h̶a̶t̶ ̶s̶a̶i̶d̶,̶ ̶I̶ ̶s̶t̶i̶l̶l̶ ̶t̶h̶i̶n̶k̶ ̶(̶i̶f̶ ̶t̶h̶e̶y̶ ̶a̶r̶e̶n̶'̶t̶ ̶a̶l̶r̶e̶a̶d̶y̶ ̶d̶o̶i̶n̶g̶ ̶i̶t̶)̶ ̶t̶h̶e̶y̶ ̶s̶h̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶k̶e̶p̶t̶ ̶a̶ ̶l̶o̶c̶a̶l̶ ̶c̶o̶p̶y̶ ̶o̶f̶ ̶f̶o̶r̶ ̶e̶x̶a̶m̶p̶l̶e̶ ̶P̶r̶o̶j̶e̶c̶t̶ ̶G̶u̶t̶e̶n̶b̶e̶r̶g̶ ̶a̶n̶d̶ ̶o̶t̶h̶e̶r̶ ̶s̶u̶c̶h̶ ̶a̶r̶c̶h̶i̶v̶e̶s̶ ̶o̶f̶ ̶o̶u̶t̶ ̶o̶f̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶c̶o̶n̶t̶e̶n̶t̶.̶ ̶ ̶E̶d̶i̶t̶:̶ ̶s̶o̶ ̶I̶ ̶r̶e̶a̶d̶ ̶t̶h̶e̶ ̶a̶r̶t̶i̶c̶l̶e̶ ̶a̶n̶d̶ ̶t̶h̶e̶ ̶p̶r̶o̶b̶l̶e̶m̶ ̶s̶e̶e̶m̶s̶ ̶t̶o̶ ̶b̶e̶ ̶m̶o̶r̶e̶ ̶a̶f̶f̶e̶c̶t̶i̶n̶g̶ ̶t̶h̶e̶i̶r̶ ̶a̶c̶c̶o̶u̶n̶t̶i̶n̶g̶ ̶a̶n̶d̶ ̶l̶o̶g̶g̶i̶n̶g̶ ̶o̶f̶ ̶b̶o̶o̶k̶s̶ ̶l̶e̶n̶t̶ ̶e̶t̶c̶.̶ ̶T̶h̶e̶r̶e̶ ̶i̶s̶ ̶n̶o̶t̶h̶i̶n̶g̶ ̶s̶a̶i̶d̶ ̶a̶b̶o̶u̶t̶ ̶a̶n̶y̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶l̶i̶b̶r̶a̶r̶y̶ ̶b̶e̶i̶n̶g̶ ̶d̶o̶w̶n̶ ̶e̶x̶p̶l̶i̶c̶i̶t̶l̶y̶ ̶b̶u̶t̶ ̶i̶'̶d̶ ̶s̶a̶y̶ ̶i̶t̶ ̶s̶t̶i̶l̶l̶ ̶a̶p̶p̶l̶i̶e̶s̶,̶ ̶t̶h̶a̶t̶ ̶k̶e̶e̶p̶i̶n̶g̶ ̶l̶o̶c̶a̶l̶ ̶c̶o̶p̶i̶e̶s̶ ̶o̶f̶ ̶s̶t̶u̶f̶f̶ ̶i̶s̶ ̶u̶s̶e̶f̶u̶l̶.̶

So my comment is not relevant, and should be ignored. Managing the metadata of this scale is certainly no easy task without computer systems.


The Seattle Library has 27 locations and a million books, serving a city with 700,000+ people (and a larger metro area - I forget if you need to be a Seattle resident to get a card).

To be able to query materials across the library system, handle reservations and item transfers, and so much more these computers absolutely need to be networked.

Perhaps you meant "don't need to be connected to the Internet", but I would wager a majority of the library's patrons search the library's catalog online rather than coming in person.

In the case of digital loans, moat patrons want to download their digital books onto their personal devices - over the internet, so clearly the library services need to talk to an online service.


I apologize, I had misunderstood the problem. The problem is the metadata not the actual books were managed on the computer systems. Sorry.


> I forget if you need to be a Seattle resident to get a card

Any resident of King County* can get an SPL (Seattle Public Library) card, due to a reciprocal agreement between SPL and KCLS (King County Library System).

* with an exception for a couple of towns within KC which are not part of KCLS, from https://www.spl.org/using-the-library/get-started/get-starte...:

"If you live, work, go to school or own property in King County (even outside the city of Seattle) you qualify for a free Seattle Public Library card through our reciprocal agreement with the King County Library System. (The only exceptions are the towns of Hunts Point and Yarrow Point, which are not part of the King County Library System.)".

And indeed, as already mentioned in a sibling comment, most counties have reciprocal agreements with both SPL and other counties' library systems too.


This is a more recent happening. Some years ago, you did have to be a resident of Seattle to have an SPL card. However you could still use many library services without one.


No - pretty much anyone in the state can get a card via reciprocality.


I don't know what weird unicode tricks you did to. the text of your comment, but please keep in mind that such tricks significantly interfere with screen readers, and their use should perhaps be reconsidered in the future.



Digital lending (Libby/Hoopla) was down too, but they got that back up within a week or two.

We are able to check out physical books, but can't return them. And we can't place new holds/requests for books to be sent to our branch.


One could of course try to keep the library, its 27 branches and its website up to date without connecting anything to the internet.

But I think it's understandable why they chose not to.


Yes I apologize, I had misunderstood. The computers main purpose was being used with the management of the metadata, there is no mention of book content itself on the network. Sorry.


I think they're rightfully suspicious of what remains in their computers even not connected (knowingly) to the Internet.


This happened in Toronto last year and knocked out the system entirely for four months. It seems that libraries are a big target for these actors. I wonder who they are, and what their agenda is?


Their agenda is to make money and/or create chaos.


More than just books, this is internet for people without home internet:

> “A lot of people come here for internet access, so it’s been quite a blow to the community,” a librarian at the Greenwood library said.

> Some individual library branches have put together lists of nearby places that could offer similar services. But they’re not great.

> The downtown library recommends a FedEx, a quarter mile away, but internet access costs 39 cents a minute. There is a public law library in the King County Courthouse, but computers there are intended for legal matters only. The nearest branches of the King County Library System are all about 40 minutes, by bus, from downtown.

What an enormously awful impact this weasel hacker / group has imposed upon innocent people. It's not bloviating to say this is what evil looks like: is causing suffering and further dividing a group of people from those that do have home PC/internet, for no good reason.


Whenever I hear about ransomware attacks, I wonder how anyone could pay for IT services to set up a system where:

1) there are no meaningful, clean backups and 2) regular users can overwrite many / most files 3) and/or administrators use machines that are easily compromisible 4) and/or the system is built on insecure technology

The fact that 1) could negate 2), 3) and 4) means people are doing IT and are taking good money from corporations or taxpayers when they absolutely should not.

So when there are things like this in the news where large numbers of users are affected, I can't help but wonder why we don't see IT companies getting strung up (figuratively, of course) and publicly embarrassed for each and every one of these incidences.


re-lend from home ?

"Book checkouts are being done by spreadsheet. Column A: the library user’s account number. Column B: the book’s bar code number. The low-tech inventory will be integrated with the library’s normal account system at some future, unknown date."

- this is how we did it not too long ago; and before that, we had a signout card, held by library until return.

before that was a handwritten ledger.


Add an automatic timestamp column and you're mostly golden, you could rebuild most of the inventory state from that. Complex systems should be able to be rebuilt from simple logs. I'm not saying event sourcing makes it trivial to rebuild from scratch, but I suspect their currently ransomware-encrypted system was designed such that it's not even possible.


quite a few years ago, there was a punchclock-like device, the librarian punched the card, time stamp ; and patron number punched on a card [ that was kept in the volume and transfered to the borrowed stack on loan]



I would bet good money they some “whiz technology person” is now rabidly trying to figure out how to turn the card catalog into an “offline saas digital lending service” and pitch venture for a $2M pre-seed to pitch libraries on it


The "whiz technology person" after a few months of toying with NextJS will fully embrace the vendor-lock-in with a poorly optimized and rushed implementation using a team of college drop outs who themselves have only a few months (claimed as years) experience with SPA development.

The resulting monstrosity will not be properly reviewed by government purchasers and will become the absolute bane of existence for every end user.

Several years in the future a massive multimillion dollar contract will be issued by the government to overhaul the software. It will arrive years late and achieve exactly the same result.

2 new homes will be purchased in Atherton.


Rabidly is probably a good word to use.

This whiz technology person is also going to discover, too late, that most libraries will refuse to touch it unless it can be proven that a patron's lending history is not recorded unless the patron opts-in.


I would think that instead of offline, they would move to using the blockchain.


Good call, totally missed that opportunity




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: