I immediately thought of the same thing. Personally, I like the idea of the Library of Babel more than, say, "all data is found in Pi". There's an important aspect to the story that really drives home the point, a hopelessness in the search as well as dealing with the nigh-incomprehensible scale. Well worth the read, as it brings the idea to a visceral level.

I'm at a loss for the story about indexing meaningfully the library, but I think I may have found a neat article on the Library: james.grimmelmann.net/files/Library.markdown