I'm not trying to be mean or break any HN rules, but did you read the article? It basically covers what you asked and was quite a revelation to me. Others in here point out that GPT-3 is not a chatbot which is good info, but I also wonder if there is anything out there that can even "remember" the context of a conversation it's supposed to be having.
Particularly interesting is the question from the article, "who is the current president". Seems like a slam dunk but it could not be answered. Interestingly this is a common question paramedics will give to people who have suffered head injuries to assess consciousness.
"chatscript" is a great dialogue engine where language patterns land the state machine into a "topic" which it gracefully backs out of (keeps a topic hierarchy) when it runs out of responses in that topic / branch of conversation.
It also includes the most robust, hand-tuned lemmer and stemmer + disambiguation language parser built on top of WordNet. really good stuff, written in C so its light on memory and responses are instantaneous.
Hidden in my comment is a question: has anybody even tried to included previous lines into the prompt, and does that not help? Asking here because there are a lot of ML nerds who could probably do a proof of concept in a few hours, if not a few lines of code, so there's a decent chance I'll get an informed response.
In most of the conversations with GPT-3 that you see online what people do is that they take the output from the first prompt and include it in the next prompt. That is how GPT-3 can keep the thread of the conversation going without changing subject constantly. This is also why those conversations are relatively short, because as you can imagine, if you keep feeding the output to the language model as input, you run out of input size very quickly.
A lot of chatbots from 20 years ago did that. They would even use it as a escape route, and would just bring back the previous topic of conversation (from say 10 lines ago) when they could not figure out something to come up for the current prompt. It really didn't help the illusion; rather it was even more obvious.
My guess (and I'm certainly no expert here) is that not only have people put a few hours or lines of code into it, but probably many a PHD thesis as well. It's an intensely researched topic which makes the article even more surprising to me.
I wonder what happened to IBM Watson's technology that was good at answering trivia -- it actually won a game of Jeopardy! a decade ago (although of course it answered things with a question as per the rules). I know that they weren't that successful at applying it to biomedical research as they had hoped, but it would seem it would be better at chatbots than GPT-3 and other deep-learning models.
The problems with chatbots have nothing to do with tech.
You implement chatbots to bend the demand curve to higher cost channels. Period.
If you can get 5% of the user base to wait 15 minutes by sending them to a chatbot tarpit, you will have eliminated the need for a dozen call center agents at peak times.
Providing good service with a chatbot is possible, but the work and organization required is extensive, and it’s cheaper to throw semi-skilled humans at it!
For anyone wondering, this is not an exaggeration. Watson actually had 15 (or 16, depending on the source) TB of RAM, as well as a 2880-core processor.
> I also wonder if there is anything out there that can even "remember" the context of a conversation it's supposed to be having.
IBM's Watson project involved an internal model of the world, though I have no idea if that got specialized into a local model of an ongoing conversation. Same type of thing though.
Having a context of this type available is actually necessary if you want to synthesize sentences in a language with definiteness-marking, such as English, because definiteness is marked according to whether the noun in question is or isn't already part of the context.
> Particularly interesting is the question from the article, "who is the current president". Seems like a slam dunk but it could not be answered. Interestingly this is a common question paramedics will give to people who have suffered head injuries to assess consciousness.
Not sure you'd still be able to use that question considered the insane QAnon followers that would answer "Trump".
Though I suppose even that answer would still at least prove some level of consciousness.
Particularly interesting is the question from the article, "who is the current president". Seems like a slam dunk but it could not be answered. Interestingly this is a common question paramedics will give to people who have suffered head injuries to assess consciousness.