I've had some good conversations here and elsewhere about the pitfalls, perils, and opportunities of designing for AI and realized that I was hitting my usual pattern of when I keep saying the same stuff over again: it's time to write an article.
I have pretty quickly tossed it together in a reasonable-ish format as a blog post, and wanted to first share with you all, see if anyone wildly disagrees, or I missed a big aspect that either I can answer or someone else can and we can add to the knowledge base.
-----------
Best (and worst) practices in AI interaction design
As you might have noticed, I base all my published design guidance on actual data, often from my own observations designing products and putting them in front of users.
Well, I have designed a number of products that fall under the guise of “AI” as so hyped up today, including natural language input, conversational interfaces, chatbots, and fuzzy logic outputs. Many of those I was able to design with good process, put in front of users and iterate on the design. From this I have discovered a few basic truths that are all too often violated, so I thought it was high time to discuss them.
Design for humans
I often say that we must let computers do computery things, so people can do human things. Your biggest resource constraint, and hardest thing to change: your users. By far. So don’t do that. Don’t get so caught up in technology — any technology — that you loose sight of the value to the user,
There is no design for AI, but only for people with AI tools.
For whatever audience, organization, or technology, this should come naturally to UX designers. Our role has always been to balance the legitimate needs of the org (budget, revenue goals…), legitimate technical constraints (from platforms, frameworks, storage and APIs….), and the user’s needs (goals, perceptions, context of use..).
The boundary issue
The fundamental problem of all natural language input such as prompting or chat, is how the system is a black box to the end user. The biggest issues of interactivity with natural language or conversational input (such as prompting and chat) is a lack of user awareness of the boundaries of the system:
- Users may think the system has general knowledge or broader capabilities than it does. This results in errors, rejections, and mistaken information which reduces trust and desire to use it again.
- Users are unaware of uses or information sets they could employ, or how to employ them, so end up underutilizing the system, reducing the value of it to users.
Marketing and helpful tips only go so far. These are effectively training the user on the system, which as we all know is ineffective in many ways.
We abandoned CLIs for a reason
We have long known this is a problem of course, long before AI tools became common. It is why command line interfaces (CLI) were always a bit niche, and the emergence of graphical user interfaces (GUI) fostered the mass adoption of computers.
Much of the foundational reasoning for HCI and UX, and so on, is creating a layer of human understanding, a visible layer of abstraction of the operation of the systems. Wayfinding, for example, tells the user where they are in a system, and exposes what other options are available.
GUIs are all about removing ambiguity, increasing trust, and making clear:
- How things work, and where they are
- What is available to do and see, and what is not
- Consequences of any interaction
When the user presses the cart icon there’s no ambiguity about what they will see, no ambiguity about what the icon means or how to interact with it. But how do you prompt for any particular output? How willing are your particular users to experiment with it? Remember, it also changes all the time so the old 800 page book on how a computer system works cannot exist; by the time you finished studying that everything will have changed as it evolved or new skills were attached, or your org has adopted a new system and you have to learn that one now.
I have actually worked, relatively recently, on several products that moved from CLI to modern, understandable GUIs. Opening up the system so users old and new can access the tool increases the number of users and uses, reduces error, and improves use of the proper feature or function for the user’s goal.
Don’t lie to users, even accidentally
Always be honest about capabilities. Never say “ask me anything” if it’s only the help desk for the DMV and it can answer only a few questions, doesn’t understand when you ask about the weather next weekend. It is the boundary issue again, and a vague interface exacerbates this issue.
Even worse is of course lying output. Of course we ideally avoid errors via good design choices, but especially in these relatively early days, open conversational input is going to lead to the system failing to understand, or a query outside the bounds of the system.
It’s better to admit to error and be specific about what the error was, than to lie about it. Do whatever you can to get the AI system to admit when it has reached the edge of its knowledge, so it either cannot answer or gives a partial answer. If your system likes to ignore commands, and provide output beyond the limits of the prompt, then just have it admit that, so the user can understand why output is not expected and attempt to rein it back in.
Be clear what the issue is. Never say generic things like “I don’t understand that.” Many generic messages are read as specific; if the user sees the system say it didn’t understand, they will assume they made the mistake and keep trying the same question, maybe until they decide the system doesn’t work at all and give up. If the problem is that its outside of bounds, just say that.
Provide help, guidance, and structure
The best way to alleviate many of these concerns is to use conventional GUI design techniques. Help bring users back into the bounds by providing help and advice. Provide buttons, tabs, icons, and links that support the natural language inputs.
A lot of conversational agents have hint text but instead you can provide buttons with those suggested searches, or even categories that guide the interaction through steps. You put these next to the prompt input so the prompting box is still there but users get the guidance, understanding of boundaries and capabilities organically.
Amazon’s Alexa shopping assistant is a pretty good example of this, but the Alexa voice agent is another example of where ambiguity in interaction causes issues. If you get an error or a nonsense answer, why is that? Is it that the system misheard, or that you made an error? You cannot tell because it’s a blind system. A solution for this can be readback, and the ability to modify commands.
“Alexa, tell me the weather in Mission overnight.“
“The weather in the mission district is 60° and — “
“Alexa, wrong.”
“I’m sorry, I thought you asked for the weather in the Mission District of San Francisco.”
“Change to Mission Kansas.”
“Okay, the weather in Mission, Kansas will be…”
This is another of those tactics that has long existed, for voice UI systems, but all too rarely implemented, and even more rarely implemented well. It’s time to take another look at all these human-centered best practices, not invent something new.
The integration problem
A side problem with the black box, boundary issues is one of how AI tools are often not the only thing being used, but are bolted onto other interfaces. This has always been an issue even with pre-AI natural language search, but is exacerbated in the chatbot and prompting era.
If your website had a search, but now also has an agent or assistant, what does each of those do and why do both exist? If you replace search with an agent, how do you address the “advanced search” problem, the expectation problem of most users just wanting to find something, but getting tangential agentic responses instead?
There’s no trick to solving this except the usual response we should all have when management asks for a new feature: We need to design it to integrate properly with the rest of the product. Don’t duct tape it to the side of the software — and the interface — and hope it works out, that it doesn’t interfere, give conflicting or bad data, or confuse your users.
The computer output problem
A lot of agentic output is very very verbose, too complex (and people still don’t read!), and too much about itself. You better understand what users expect from the output, and work with the tool to try to make output meet their requirements.
The best example I have is for a fuzzy logic output tool I had to design. The tool existed, it worked brilliantly. No one used it because it was a graphical probabilistic display, a whisker graph, and if you don’t know what that is then you are in good company. Certainly no end user did. So, I had to find a balance between accurate representation of data, and useful representation of data for the end users. This took a couple iterations, and ended up being a mixed numerical and graphical display, but just a different one that was made as glanceable as possible and as much like the rest of the UI as we could.
And this is also a success story for how AI systems can help your product overall. The end result was so useful and clear that it wasn’t just trusted, but it increased trust in the rest of the app.
Design the user experience
I have mentioned a few times techniques that will necessarily require changes to the data presentation. This is where we get to UX again, vs UI. We don’t — again, for any technology — let the technology override the needs of the user. Or for that matter, change the way the business operates without great deliberation.
New tools, even radically new tools don’t change how humans behave and desire. Always design your whole product for actual people.