Trust in automation

trust in automation · Lee & See (2004)

Bibliography checked ✓ Human-automation interaction · trust and reliance Tell the editor privately ▸

In three sentences People decide whether to use automation on the basis of trust, and trust can be mismatched. Too much and they keep relying on a system that has stopped working; too little and a usable tool sits idle. The design goal was never to push trust higher. It is to calibrate trust against what the system can actually do.

↳ Seen a counter-example?

Figure 1 · How trust forms and how it is calibrated. Information the system exposes feeds three psychological routes that form trust; trust is then calibrated and resolved against actual capability, which determines reliance. Two mismatches sit at the extremes: too much trust is misuse, too little is disuse. Redrawn from the framework in Lee and See (2004). View full size ▸

1. What happened

When designing a human-machine system, it feels natural to want users to trust it a little more. In 2004 Lee and See pulled scattered work on trust together into the unifying framework for the field[1]Lee, J. D., & See, K. A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors, 46(1), 50–80. Bibliographic details cross-checked against multiple sources. The summary of the framework's core claims follows secondary accounts that agree with one another.. Two of its central concepts for judging whether trust is appropriate are calibration, where the level of trust matches actual capability, and resolution, where trust can distinguish the situations in which the system is reliable from those in which it is not. Trust is fed by three kinds of information about the system, its purpose, its process, and its performance, and it forms through three psychological routes, analytic, analogical, and affective. What the interface exposes determines where trust goes. This page infers that when the information is unavailable, people are more likely to fall back on the affective and analogical shortcuts, and the risk of miscalibration rises with it.

Attribution where due: Lee and See (2004) act as synthesisers. Work on trust was scattered before this paper, and secondary sources credit the three information dimensions to earlier work, most often Lee and Moray (1992), with the exact origin unchecked. What the two of them contributed is the unified framework: calibration plus resolution, three information dimensions plus three psychological routes. Attribute the framework to them; the word "trust" needs no attribution.

2. Why this matters to you

"How do we make users trust the AI more" points in the wrong direction. The accidents caused by too much trust and the capacity wasted by too little are equally real. The question you can act on is this: in which situations is this system reliable, in which is it not, and can the user tell the difference? If they cannot, expose the process and the performance in the interface. Trust is something you calibrate. Trust you have talked up is just credulity.

Boundary: the goal is calibration and not elevation, and calibration has to resolve where the system is reliable and where it is not; trust is a dynamic state, not a trait; the framework here is confirmed at secondary-source grade only.

↳ Is your case inside this boundary or outside it?

Verification layer · Members

The guided layer of this node is open to all readers.

Verification details and failure boundaries are for members.

Subscribe yearly ▸Read the member edition ▸

Lab cases Written only where there is evidence

No field record for this node yet.

References

How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Follows secondary accounts” means the claim rests on sources that report it, not on the original document. Definitions as of 2026-09-20.

[1] Lee, J. D., & See, K. A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392 ↩

↳ Is a citation wrong?

Tell the editor privately: a correction, a counter-example or where it applies

An editor reviews submissions once a week. Once a note is handled, its summary and response are listed on the discussion and response record; nothing is published before then. No sign-up is needed, and the nickname may be left blank.

What are you submitting? Correction: a fact, citation or inference is wrong Counter-example: a case that contradicts this node's claim Applicability: it does not hold, or needs adjusting, in some setting

Where in this node (optional) Details Nickname (optional; seen only by the editor, never published)

If adopted, credit me by my public name (otherwise "an anonymous reader") Set a public name to be credited by name ▸

Submit anonymously, without signing in

Signing in lets you follow the status on your page. What you have written is kept.

You are signed in: this note is linked to your account, visible only to you and the editor, so you can follow its status on your page. The text is never published. Full privacy policy ▸

Submit anonymously instead, not linked to your account ▸

No email is collected. Your original text and nickname are read only by the editor and are never published; the discussion and response record carries only a summary the editor writes. The human check is provided by Cloudflare Turnstile. To limit abuse, the system keeps a hash of your IP address, not the address itself. Your original text is stored with our cloud provider and can be deleted on request. Full privacy policy ▸

Response record ▸

Knowledge graph Compare ▸Vigilance decrement Applied ▸Automation allocation discipline