I've been wrestling with these same challenges while building RA.Aid—trying to make tools that speak LLM. We have good tool integrations, but a lot of the tools were originally designed for human consumption. The LLMs seem to have their own idea of how they want to do something, which is what makes prompt optimization such an important factor.
> The LLMs seem to have their own idea of how they want to do something
exactly! what I'm experiencing is that prompt engineering has its limitations and comes with inconsistency issues...
by designing the tool from scratch tailored to LLMs, we can make the interface match what their "own idea of how to do" that particular task, which is more reliable and scalable
This is going to be really important. Designers, Product Managers, and Engineers are going to have to consider these machines as a new persona. I'll be interested to see how we try to build useful things for personas we don't understand! Sounds like a hard job but a worthwhile one.
Integrations between LLMs and real-world services are challenging because all our current interfaces were designed for humans.
While developing tools for LLMs, my team [1] and I came to the realization that we need a new engineering discipline. One that cares for the "machine experience", for building interfaces that are tailored to LLMs, having their 'preferences' and quirks in mind.
The LLM has to be seen as a consumer. A user itself.
We need a new breed of engineers dedicated to what we may call 'Machine Experience (MX) Engineering', just as we have UX Engineering, for instance.
reply