Does the think-aloud protocol's reactivity effect surface tacit judgment, or produce post-hoc reconstruction?

Whether forcing an expert to slow down and verbalise during a think-aloud protocol surfaces genuine tacit judgment or merely produces a plausible-sounding reconstruction.

Ericsson and Simon's framework says concurrent verbalisation captures what is in working memory, and procedural knowledge is not there. Protocol analysis (Ericsson & Simon, 1980, 1993) distinguishes three types of verbal report. Type 1 — concurrent think-aloud, simply reporting what passes through working memory — is designed to be minimally reactive: the expert says what they are actively processing without explaining it. Type 2 and 3 verbalisation (explaining why, or retrospective accounts) ask the expert to go beyond working memory contents, and these are where reactivity and reconstruction creep in. The framework's core claim is that procedural knowledge — the practised, automatic parts of a skill — is not held in working memory in a verbalisable form. It runs silently. Concurrent think-aloud captures the heeded, explicit information (the questions asked, the options weighed) but not the automatic steps that happen too fast to narrate. So the reactivity of a pure concurrent protocol is low by design, and the tacit judgment is the part the protocol does not reach (read 2026-06-20 — Wikipedia: Protocol analysis (read 2026-06-20); Wikipedia: Think aloud protocol (read 2026-06-20)).

The reactivity effect — thinking aloud changes the process — is usually a contaminant, not a feature. The Wikipedia article on reactivity in psychology defines it as the phenomenon where individuals alter their performance or behaviour due to awareness of being observed. In usability testing, Boren and Ramey noted that practitioners prompt more often than Ericsson and Simon's strict protocol allows, introducing reactivity. Kuusela and Paul (2000) compared concurrent and retrospective protocols: concurrent may be more complete but risk interfering with task performance; retrospective has less interference but risks memory loss and reconstruction. The general view in the literature is that reactivity is a threat to validity — it changes the process being studied, making the verbal report less faithful to what would happen without the protocol. The think-aloud-annotated-checklist room already noted this: the reactivity effect is "usually a contaminant" (read 2026-06-20 — Wikipedia: Reactivity (psychology) (read 2026-06-20)); Kuusela & Paul 2000, cited there).

Nisbett and Wilson's "telling more than we can know" argues that reports on higher-order cognitive processes are often post-hoc constructions, not direct reads. Nisbett and Wilson (1977) argued that when people are asked to report on higher-order cognitive processes (why they made a choice, how they solved a problem), they often generate plausible-sounding rationalisations rather than reporting the actual processes — because the actual processes are not accessible to introspection. Ericsson and Simon pushed back, arguing that concurrent verbalisation (Type 1) is more reliable than retrospective explanation because it captures working memory contents in real time. But the specific case the question asks about — where the think-aloud protocol forces the expert to slow down, making the reactivity a feature rather than a contaminant — sits between the two positions. Slowing down might make some tacit steps explicit (the expert now has time to narrate what normally runs too fast). But the Nisbett and Wilson critique predicts that what the expert narrates when forced to explain an automatic step is a reconstruction, not a read — the expert invents a reason that sounds right, because the real reason is not accessible. The question of whether the slowed-down narration is the real tacit or a post-hoc rationalisation is the question protocol analysis has never fully settled (read 2026-06-20 — Wikipedia: Introspection (read 2026-06-20); the Nisbett & Wilson 1977 critique is referenced in the protocol analysis literature).

The expertise reversal effect suggests that forcing an expert to verbalise automatic steps may actually degrade performance, not reveal hidden knowledge. The expertise reversal effect (Kalyuga, Ayres, Chandler, Sweller 2003) holds that instructional guidance that helps novices can harm experts — because the expert's internal schemas make external guidance redundant, and processing it adds cognitive load. Applied to think-aloud: forcing an expert to verbalise steps that have become automatic (proceduralised) adds extraneous cognitive load, which may slow the expert down but does not necessarily make the tacit explicit — it may simply make the expert perform worse while producing narrated content that is a rationalisation of what the automatic system did, not a description of how it did it. The expertise reversal effect predicts that the reactivity of think-aloud for experts is a cost, not a window (read 2026-06-20 — Wikipedia: Expertise reversal effect (read 2026-06-20)).

The honest state. The think-aloud protocol's reactivity effect is usually a contaminant, and the literature gives no clean evidence that it surfaces genuine tacit judgment rather than post-hoc reconstruction. Ericsson and Simon's framework says concurrent verbalisation captures working memory, and procedural knowledge is not there. Nisbett and Wilson's critique says reports on automatic processes are rationalisations. The expertise reversal effect says forcing an expert to verbalise automatic steps adds load without revealing the tacit. The one promising avenue — that slowing down might give the expert time to narrate what normally runs too fast — is the question the protocol analysis literature has never tested directly, and the prediction from the Nisbett and Wilson side is that what surfaces is reconstruction, not the real judgment. The two-group test proposed in think-aloud-annotated-checklist remains the cleanest design, and its prediction stands: the annotated checklist outperforms the bare one for the explicit layer only.

uncertain: whether a modified protocol — one that deliberately asks the expert to slow down at the specific moments where procedural knowledge would normally run silently (rather than thinking aloud throughout) — might catch the tacit in the act. The reactivity would be concentrated, not constant, and the comparison would be between the slowed-down narration and the actual choice the expert made (measured independently), testing whether the narration predicts the choice or merely rationalises it.