AI Experience Engineering
On designing AI products that feel human.
Most AI products built today are focused on showing off what the model can do. I think we need to shift focus to building experiences that cater to the human.
This is the most undervalued technical problem in AI right now, and I don't see many folks working on it.
When you use a great AI product, you don't think about the AI. You think better. The tool disappears. When you use a bad one, the AI is all you can think about. You're second-guessing it, re-checking its work, wrestling with the interface. The AI is present in exactly the way it shouldn't be.
The difference between these two experiences has almost nothing to do with model quality or capability. I've seen the same model produce both experiences depending on how the harness around it was designed. Designing the harness is not enough, we need to ensure that the harness aligns with the human ability. The harness is the product. The models are rapidly commoditizing — Claude, GPT, Gemini converge more with each generation — and yet almost all the talent and money is still going toward the model.
Take two AI tools that summarize your weekly fitness data (steps, sleep, resting heart rate), both powered by the same model and given the same numbers. One dumps the stats:
Average steps: 6,200/day. Resting heart rate: 72 bpm. Sleep: 6.4 hrs avg.
The other notices you started morning walks three weeks ago, sees your heart rate trending down, flags that your sleep is still short, and offers to adjust your routine. Same model, same data, same capabilities. One is a dashboard; the other is a coach. The difference is entirely in the harness.
It's obvious why. You can benchmark a model. You can put a number on it. The experience is a feeling, and engineers are uncomfortable optimizing for feelings. That's where an AI experience engineer comes into the picture. I think there are ways in which we can measure the experience. When should the agent work in the background, when should it ask for help, when should it refuse to do a task, etc. The current work on model alignment can only take the model so far. The harness around the model is what will define how the experience will be.
To design a good experience for the human, you have to understand what it's like to be the agent. I keep coming back to this.
You'd think we're supposed to be human-centered. And we are. But you can't design the human side without first understanding what the agent is actually doing, what it knows, what it doesn't know, and what it thinks it knows but is wrong about.
Think about what happens when you collaborate with a good colleague. They model your mental state. They know what you know and what you don't. They adjust how they present information based on that. When they're uncertain, they signal it with tone, with hedging, with the way they phrase things. Not with a confidence score.
An AI agent does something analogous internally. It operates with partial information. It has areas of high confidence and low confidence. It has places where its training makes it systematically wrong in predictable ways.
If you don't understand these things, you can't design the human's experience. You'll build an interface that presents uncertain information as certain. You'll build a workflow that asks the human to verify things the agent already knows, while silently passing through things the agent is guessing at. You'll create exactly the kind of mismatch that makes people distrust AI.
The human will trust the agent when the agent's actual confidence lines up with what the human perceives.
The best analogy I've found is what happens when a doctor explains a diagnosis to a patient. The doctor has a rich internal model. The patient has a different one. The doctor's job isn't to dump their model wholesale. It's to construct something useful for the patient, given what the patient already knows and what they need to do next.
That's the work of AI experience. The agent has a model of the world, messy and probabilistic and sometimes wrong. The human has a different model, experiential and contextual and full of judgment. The experience layer translates between them.
Most people skip this. They dump the agent's output into a UI. That's like handing someone raw lab results instead of explaining what they mean.
The person doing the translating has to understand both sides. What did the agent actually compute? How confident is it really? Where is its reasoning brittle? And on the other side: what is the human trying to do right now? What cognitive mode are they in? What kind of information can they absorb in this moment? The unknown and uncertainty of each human is what makes it such an interesting area of work.
You need two kinds of empathy at once. Empathy for the human and empathy for the machine. Yes, "empathy for the machine" is a deliberate contradiction — you can't literally empathize with something that doesn't have experiences. But that tension is the point. The work requires you to build an intuition for how the agent behaves as if you could empathize with it, while never forgetting that you can't.
Empathy for the machine means understanding its actual nature. An LLM doesn't know things the way you know things. Its confidence is not your confidence. Its reasoning is not your reasoning. When it produces the sentence "I think," it's doing something utterly different from what you do when you say those words.
If you don't grapple with this, you design dishonest experiences. Systems that perform certainty when the agent has none. Or systems that perform humility when the agent is actually reliable. Both erode trust. Both happen constantly in AI products because the people building them didn't bother to understand what the agent is actually doing under the hood.
The irony is that understanding the agent's nature is what lets you design something more human. When you know where the model is brittle, you route around the brittleness. When you know where it's strong, you lean into that strength. You need to choreograph a collaboration between two very different kinds of intelligence.
Something similar happened in the early web. The technical people in the late 90s were building for capabilities. Faster page loads. More concurrent connections. Better database queries. All of that mattered. But the companies that won were the ones that understood what people wanted to do on the internet and built experiences around that.
Google didn't win because they had the best search algorithm alone. The algorithm was genuinely superior — but it was the experience that made the capability accessible. One box. One button. Fast results. The algorithm enabled that experience. But the experience is what users chose.
We're at the same point with AI. The models are good enough. For many tasks they've been good enough for over a year. What's missing is the experience layer.
What does it mean in practice to engineer an AI experience?
You have to decide what the human should feel at every point in the interaction. Not what they should see. What they should feel. Confident? Curious? Skeptical? This sounds like design, and partly it is. But the feeling is downstream of the agent's actual behavior, not just its visual presentation. A dropdown menu can't fix a trust problem created by an overconfident model.
Then you have to understand the agent's real epistemic state and translate that into interaction patterns that produce the right feeling and outcome. This is the technical core. It requires understanding the model and the human deeply enough to build the bridge between them.
And then you close the loop. The human's response — or silence — tells you whether the translation is working. Are they verifying things they shouldn't need to? You're presenting uncertain things with too much authority, so they've learned to check everything. Are they blindly accepting things they should scrutinize? You've built too much trust in the wrong places.
The experience is never static. It's a continuous adjustment between two minds that think in very different ways.
I think AI Experience Engineering will become a recognized discipline within a few years. Right now it doesn't have a name. The people who do it well are scattered. Some call themselves product engineers, some AI engineers. Some are founders who just have good taste about this stuff.
But it's real. It might be more important right now than either software engineering or product design alone, because it's the bottleneck. We have plenty of model capability. We have plenty of UI frameworks. What we don't have is enough people who understand both the agent and the human well enough to build what goes between them.