Retrieval beyond words
Retrieval beyond words

Retrieval beyond words

Kornell and Bjork (2008) famously presented learners with impressionist landscape artwork in blocked or interleaved formats and then tested their ability to identify the artist who had painted novel example paintings. There were two key findings: first, interleaving was superior to blocking in this test; and, second, students’ metacognitive awareness of their learning was faulty, as they tended to believe (incorrectly) that they had learned better via blocking. More recently, Krasne et al. (2020) report improvements in both accuracy and fluency of ECG diagnosis but also durability one year later.

These are the two key references I refer to when extending the use of retrieval practice for different ‘modalities.’ Yesterday I stumbled across another reference. Carpenter and Pashler (2007) found that subjects’ recall of map information was better after repeated testing than after conventional study, suggesting that the ‘testing effect’ can be applied to nonverbal learning tasks. Interestingly they signposted that covert (answers retrievals may be key in extending the testing effect to complex nonverbal learning.

The clear and obvious challenge: how to collect and score responses.


52 undergraduates learned two maps, each containing 12 features each. The top one third told that they would receive $10 (15$ today).

One through a testing and the other through an additional study opportunity. Four counterbalancing conditions were created:

  1. Map A was presented for study, followed by Map B for test/study
  2. Map A was presented for test/study, followed by Map B for study
  3. Map B was presented for study, followed by Map A for test/study
  4. Map B was presented for test/study, followed by Map A for study

Randomly assigned to one of these four counterbalancing conditions, after 30 minutes, participants were asked to draw both maps as well as they could. Two measures were recorded, absolute and relative accuracy.

Absolute accuracy: A point was awarded if the feature was placed within the correct quadrant.

Relative accuracy: A point was awarded if the feature was placed in the correct position relative to other features included in the drawing.


Map B was easier to learn than Map A. The test/study condition was significantly more beneficial than the study condition by all four scoring procedures. Drawings were better and more complete when learned through covert retrieval rather than additional study.

What did we learn?

It is encouraging to know that the effect is not limited to memory tasks that require writing, typing, or speaking a verbal response aloud.

Carpenter and Pashler (2007: 477)

At the time, the results of this study broadened the boundaries of the testing effect or retrieval practice to nonverbal learning tasks. It was also possibly one of the first studies to demonstrate the testing effect with a monetary incentive.

Still reading, still learning.

Not forgetting the various studies showing the application of retrieval for learning beyond paired associate learning or completion of word fragments, sentences and general knowledge; for example landscape pictures, locations, regions on maps, abstract symbols or visual images and even ECGs and three-dimensional virtual reality displays.

Carpenter, S.K., Pashler, H. Testing beyond words: Using tests to enhance visuospatial map learning. Psychonomic Bulletin & Review 14, 474–478 (2007).
Kornell, N. and Bjork, R. A. (2008) Learning concepts and categories: is spacing the ‘enemy of induction’? Psychological Science, 19(6): 585–592.
Krasne, S., Stevens, C. D., Kellman, P. J. and Niemann, J. T. (2020) Mastering electrocardiogram interpretation skills through a perceptual and adaptive learning module. AEM Education and Training, 5(2): e10454.

Leave a Reply