Semantic relatedness and the efficacy of retrieval practice · Gupta MW, Pan SC, Rickard TC

Methods

We recruited 242 students from the subject pool at the University of California, San Diego to participate in a laboratory experiment in exchange for partial course credit. All but 11 participants completed both sessions and data from the remaining 231 participants were analyzed. In the final sample there were 118 participants in the low relatedness group and 113 participants in the high relatedness group. An a priori power analysis that was conducted for a previous study in our laboratory (see Rickard & Pan, 2020; Experiment 1) indicated that at least 32 participants per group are needed to have a power of 0.8 to detect a difference in proportion correct between PCT and PCT-predicted on final test of at least 0.05. The current sample size far exceeded that value in an effort to minimize sampling variability in the cumulative proportion correct distributions. Participants provided informed consent via in-person signature. All procedures were approved by the institutional review board of the University of California, San Diego, IRB Protocol #141851 in compliance with the Declaration of Helsinki.

Materials and design

A cued-recall testing paradigm was used. The materials were 80 English word-pairs in each of two master lists (low and high semantic relatedness, respectively; see Supplementary Table 1). Here, “semantic relatedness” is operationalized as normative cue→target forward associative strength from the Nelson et al. norms. Forty of the low semantic relatedness word pairs were identical to those used by Rickard and Pan (2020). Forty additional low relatedness pairs were selected from the same normative data39 and using the same criteria as described in Rickard & Pan: nouns that were 3–7 letters, 1–3 syllables, high in concreteness (400–700), had frequency of at least 30 per million, and were weakly associated. The mean forward associative strength for these pairs was 0.028. The cue words for high relatedness pairs remained the same as those used for the low relatedness pairs. The 80 new target words were selected with the same criteria as for low relatedness pairs, with the exception that the target word selected for each cue was that with the highest associative strength in the database (mean = 0.238). For counterbalancing purposes, two versions of each master list were created. In version 1, half of the word pairs were randomly assigned to the test condition and the other half to the restudy condition, and in version 2 those assignments were reversed.

Procedure

The experiment was conducted in-person using laboratory desktop computers and the E-Prime 2.0 software program. Participants were randomly assigned to either the low or the high relatedness group and the training phase task (restudy vs. testing with feedback) was manipulated within-participants. The two versions of the master list for each group were counterbalanced using odd versus even participant numbers. Word pairs were presented horizontally at the center of the computer monitor, with left- or right-side word placement held constant across all experiment phases. On all test trials, the left-side word was the cue. Word pair presentation order was randomized in each phase, independently for each participant. In the training phase, restudy and test trials were randomly intermixed, a design that has been demonstrated to eliminate the forward testing effect confound during training24.

In the study phase of session one, participants in each group passively studied all 80 word-pairs one at a time for six seconds per trial, for a total of 80 trials. Each trial in the subsequent training phase lasted six seconds. In the restudy condition, each word-pair was restudied for six seconds. In the test condition, the cue-word was presented for five seconds, within which time the participants were asked to type the corresponding answer. Both the cue-word and the correct target were then presented for one second, constituting feedback. There was one trial for each word pair, yielding 80 total trials. The final test was administered in session two after a 24-h. delay. It was self-paced and involved one cued-recall test trial per item, with no feedback. If a participant was unable to recall the target-word, they were allowed to advance to the next cue-word without making a response.

Scoring and statistical analyses

We applied a scoring procedure wherein a response was counted as correct only if the target word spelling was entirely correct. Formal analyses involved t tests and ANOVAs, along with Bayes factors. We used the suggested default Cauchy prior value40, with a uniform r (prior) value of 0.707 applied to all Bayes factors tests. To evaluate Bayes factor values, we used Raftery’s criteria41, whereby Bayes factors falling within the range of 1–3 were considered weak evidence, 3–20 indicated positive evidence, 20–150 was strong evidence, and Bayes factors exceeding 150 were considered very strong evidence. For ease of interpretation, BF10 indicates in evidence favor of the alternative hypothesis and BF01 indicates evidence in favor of the null hypothesis.

Transparency and openness

We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study, and the study follows JARS (Appelbaum et al., 2018). All data, analysis and simulation code are available at the Open Science Framework (https://osf.io/dyv3h/). This study’s design and its analysis were not pre-registered.