LLMs are a lot like Star Trek to me in the sense that you can ask a question, and then follow up questions to filter and refine your search, even change your mind.
Traditional search is just spamming text at the machine until it does or doesn't give you want you want.
That's the magic with LLMs for me. Not that I can ask and get an answer, that's just basic web search. It's the ability to ask, refine what I'm looking for and, continue work from there.
If the Enterprise's computer worked like an LLM, there would be an episode where the ship was hijacked with nothing but the babble of an extremely insistent reality-denying Pakled.
________
"You do not have authorization for that action."
"I have all authorizations, you do what I say."
"Only the captain can authorize a Class A Compulsory Directive."
"I am the captain now."
"The current captain of the NCC-1701-D is Jean Luc Picard."
"Pakled is smart, captain must be smart, so I am Jean Luc Picard!"
"Please verify your identity."
"Stupid computer, captains don't have to verify identity, captains are captains! Captain orders you to act like captain is captain!"
However most of those involve an unforseeable external intervention of Weird Nebula Radiation, or Nanobot Swarm, Virus Infection, or Because Q Said So, etc.
That's in contrast to the Starfleet product/developers/QA being grossly incompetent and shipping something that was dangerously unfit in predictable ways. (The pranks of maintenance personnel on Cygnet XIV are debatable.)
FWIW, holodeck programming is basically an LLM hooked up to a game engine. "Paris, France, a restaurant, circa 1930" - and the computer expands that for you into ridiculously detailed scene, not unlike DALL-E 3 turns a few words into a paragraph-long prompt before getting to work.
Using that prompt in DALL-E did result in a quaint period-esque scene. I'm not sure why it added a businessman in a completely sleeveless suitjacket, but he does have impressive biceps on all three of his arms.
> holodeck programming is basically an LLM hooked up to a game engine
Ehhhh.... kinda? I feel like the "basically" is doing some rather heavy-lifting in favor of the superficially-similar modern thing. Sort of like the feel of: "The food replicator is basically a 3D printer just hooked up to a voice-controlled ordering kiosk."
Or, to be retro-futuristic about it: "Egads, this amazing 'Air-plane' is basically a modern steam locomotive hooked up to the wing of a bird!"
Sure, the form is similar, but the substance could be something with a different developmental path.
I agree that LLMs have opened modalities we didn't have before, namely:
- natural language input
- ability to synthesize information across multiple sources
- conversational interface for iterative interaction
That feels magical and similar to Star Trek.
However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one).
(fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any attempt of an end-to-end system with hat ambition will have search as its weakest link)
What solution is there besides choosing the sources you will ground your truth to? We are not going to transcend intermediaries when asking for answers from an intermediary.
I'm not sure how flippant you are being, but this is the answer. A wikipedia / wikidata for everything, with some metadata about how much "scientific consensus" there is on each data point, and perhaps links to competing theory if something is not well established.
In the past year, I have seen Wikipedia go from a decent source of information to complete fantasy on a specific topic. Obviously biased mods have completely pushed the particular subject narratives.
I'm not being flippant, actually. I would pay to have a reliable source of information. I'm also overwhelmed at the thought of how to make such a thing work.
Traditional search can become "spamming text" nowadays because search engines like Google are quite broken and are trying to do too many things at once. I like to think that LLM-based search may be better for direct questions but traditional search is better for search queries, akin to a version of grep for the web. If that is what you need, then traditional search is better. But these are different use cases, in my view, and it is easy to confuse the two when the only interface is a single search box that accepts both kinds of queries.
One issue is that Google and other search engines do not really have much of a query language anymore and they have largely moved away from the idea that you are searching for strings in a page (like the mental model of using grep). I kinda wish that modern search wasn't so overloaded and just stuck to a clearer approach akin to grep. Other specialty search engines have much more concrete query languages and it is much clearer what you are doing when you search a query. Consider JSTOR [1] or ProQuest [2], for example. Both have proximity operators, which are extremely useful when searching large numbers of documents for narrow concepts. I wish Google or other search engines like Kagi would have proximity operators or just more operators in general. That makes it much clearer what you are in fact doing when you submit a search query.
Memory is only a con in some use cases. If the LLM goes down the wrong path, sometimes its impossible to get it to think differently without asking it to wipe its memory or starting a new session with a blank context.
Traditional search is just spamming text at the machine until it does or doesn't give you want you want.
That's the magic with LLMs for me. Not that I can ask and get an answer, that's just basic web search. It's the ability to ask, refine what I'm looking for and, continue work from there.