Hallucinations can be mostly eliminated with RAG and tools. I use NotebookLM all of the time to research through our internal artifacts, it includes citations/references from your documents.
Even with ChatGPT you can ask it to find web citations and if it uses the Python runtime to find answers, you can look at the code.
And to prevent the typical responses - my company uses GSuite so Google already has our IP, NotebookLM is specifically approved by my company and no Google doesn’t train on your documents
Facts can be checked with RAG, but the real value of AI isn't as a search replacement, but for reasoning/problem-solving where the answer isn't out there.
How do you, in general, fact check a chain of reasoning?
I can’t tell a search engine to summarize text for a technical audience and then another summary for a non technical audience.
I recently came into the middle of a cloud consulting project where a lot of artifacts, transcripts of discovery sessions, requirement docs, etc had already been created.
I asked NotebookLM all of the questions I would have asked a customer at the beginning of a project.
What it couldn’t answer, I then went back and asked the customer.
I was even able to get it to create a project plan with work streams and epics. Yes it wouldn’t have been effective if I didn’t already know project management, AWS and two decades+ of development experience.
Despite what people think, LLMs can also do a pretty good job at coding when well trained on the APIs. Fortunately, ChatGPT is well trained on the AWS CLI, SDKs in various languages and you can ask it to verify the SDK functions on the web.
I’ve been deep into AWS based development since LLMs have been a thing. My opinion may change if I get back into more traditional development
> I can’t tell a search engine to summarize text for a technical audience and then another summary for a non technical audience.
No, but, as amazing as that is, don't put too much trust in those summaries!
It's not summarizing based on grokking the key points of the text, but rather based on text vs summary examples found in the training set. The summary may pass a surface level comparison to the source material, while failing to capture/emphasize the key points that would come from having actually understood it.
I write the original content or I was in the meeting where I’m giving it the transcript. I know what points I need to get across to both audiences.
Just like I’m not randomly depending on it to do an Amazon style PRFAQ (I was indoctrinated as an Amazon employee for 3.5 years), create a project plan, etc, without being a subject matter expert in the areas. It’s a tool for an experienced writer, halfway decent project manager, AWS cloud application architect and developer.
Even with ChatGPT you can ask it to find web citations and if it uses the Python runtime to find answers, you can look at the code.
And to prevent the typical responses - my company uses GSuite so Google already has our IP, NotebookLM is specifically approved by my company and no Google doesn’t train on your documents