Methods of cognitive psychology

Cognitive psychology studies processes that cannot be observed directly, such as how a memory is searched or how a decision is reached. It infers them from behaviour, chiefly from how long people take to respond and how often they are correct, and it tests the inferences by building computational models of the processes and fitting them to the data. Each of these methods has a documented limit. Reaction-time patterns alone cannot separate processes running one after another from processes running in parallel (Fific et al., 2008). Sufficiently general models of response times can fit any data, so fitting them proves little (Smith et al., 2014). Choosing among models has no single best method (Evans, 2019). And explanations pitched at only one of Marr’s levels of analysis are under-constrained (Cooper & Peebles, 2015; Love, 2015).

History

Not covered by the research: the development of mental chronometry, the information-processing approach and computational modelling.

Core concepts

Reaction-time analysis

Reaction time, the time from a stimulus to a response, is a basic tool for inferring mental processes. A classic use is to vary the workload, such as the number of items to be searched, and observe how response time grows. If time rises with each added item, the items seem to be processed one at a time. This inference fails: limited-capacity parallel models, which process all items at once but with capacity shared among them, mimic serial ones. That limit motivated systems factorial technology, a set of methods designed to tell serial from parallel processing more decisively (Fific et al., 2008).

Model fitting

Sequential-sampling models, such as the diffusion model, explain response times and accuracy together by assuming that evidence accumulates over time until it reaches a threshold. Jones and Dzhafarov showed that general versions of these models, diffusion and ballistic accumulators alike, can reproduce any pattern of response-time distributions and accuracy exactly (Smith et al., 2014). Smith, Ratcliff and McKoon accept this for the general class, but argue that the unfalsifiable members are deterministic growth models, not the diffusion models they use (Smith et al., 2014).

Model selection

When several models fit the data, researchers must choose among them, balancing fit against complexity. For computational models of response-time data, no method of model selection is objectively optimal, and different methods can lead to different conclusions (Evans, 2019).

Marr’s levels of analysis

Marr proposed that an information-processing system can be explained at three levels: the computational level (what problem is solved and why), the algorithmic or representational level (how), and the implementational level (in what physical mechanism). Cooper and Peebles argue that explanation that starts at the computational level, as purely Bayesian accounts do, or at the implementational level, has fundamental limitations, and needs coupling to the algorithmic level and to a cognitive architecture (Cooper & Peebles, 2015). Love lists two limits of the top-down, computation-first route: the computational level constrains explanation only weakly, and people can be suboptimal for reasons other than limited capacity. Love proposes integrating all three levels through the algorithmic level (Love, 2015).

Questions, controversies, and new developments

Mimicry

The common thread is model mimicry: different processes, or different models, predicting the same observable data. It appears in serial versus parallel processing (Fific et al., 2008), in general accumulator models (Smith et al., 2014), and in substantive debates elsewhere in the field, such as slot versus resource models of working memory, where item-limit researchers concede that low-fidelity models fit the same error patterns (Ngiam et al., 2022), and single- versus multiple-system models of memory (Berry et al., 2012).

Unfalsifiable frameworks

Broad frameworks raise the same problem at a larger scale. Cognitive architectures are criticised as not directly falsifiable; Newell appealed to Lakatos’s idea of research programmes, judged by their progress over time, and Cooper argues that the lack of direct falsifiability need not undermine them (Cooper, 2007). Bayesian models face a similar charge (Bowers & Davis, 2012); see Bayesian models of cognition.

Replication record

How well the findings produced by these methods replicate is covered in Replicability and generality.

Broader connections

The choice of level and method shapes the accounts compared in Architecture of the mind.

Further reading

• Fific et al. (2008): why reaction times alone cannot separate serial from parallel processing.
• Cooper and Peebles (2015) and Love (2015): two arguments for integrating Marr’s levels.
• Evans (2019): model selection for response-time models.