Every assistant that can see your screen has the same failure mode, and it is not hallucination. It is confidence. You say “make this bold” while looking at a paragraph, and the model picks something. Most of the time it picks right, which is exactly what makes the wrong pick so expensive — you have stopped checking.
FormFactors is a desktop built around refusing that. The agent resolves what you are pointing at, names it back to you, and when it cannot resolve a referent it says so instead of choosing. That sounds like a small interaction detail. It changes the whole surface.
The decision everything follows from
Deixis binds to hover, not to selection. Pointing is where your cursor is, and questions inherit that referent at the moment you ask. It sounds obvious until you build it: I originally fired the suggestion chips on click, which physically drags the pointer off the thing the question is about. The chip row now fires on digits 1–9 so the pointer never moves. That bug is recorded in a comment in the source, dated, because it took a user session to notice and I did not want to rediscover it.
Once referents are first-class, the honesty falls out. The status pill can say “Pointing at: Document body” because there is a resolved entity to name. The action receipt can say ACTION / TARGET / DETAIL because those are three separate facts, not one sentence of model narration. The transparency trace is fed only from real seams — request entry, tool dispatch, acknowledgement — and never from the model describing itself, because a model describing its own actions is not evidence.
What it cost
Structure is slower than prose. A model can always produce a confident
paragraph; producing a typed card means the system has to actually know what
it did, and where it does not know, the card is empty and you can see the hole.
Several features died at that gate. ask_content is only offered where an
authorial field exists — an Excel cell or a photo edit is never authorial — so
the agent does not nag you to write things that have no author.
The honest path is also less impressive in a five-second demo. A confident assistant looks smarter right up until the moment it is wrong.
What I’d change next
The uncertainty vocabulary is still binary in too many places: it either resolves or it asks. The interesting middle is a ranked shortlist — “I think you mean the body, but it could be the heading” — with the cost of being wrong attached. That is where the design gets genuinely hard, because ranked uncertainty is only useful if the ranking is calibrated, and nothing in the current system measures whether it is.