Results
Final test results for proportion correct means, as well as the model predictions for the test conditions means, are shown in Fig. 4. A mixed factors ANOVA on the observed proportion correct (setting aside the predicted PCT), with factors of Training Task (restudy vs. test; within participants) and Relatedness (low vs. high relatedness; between participants), produced a significant (at α = 0.05) main effect of Training Task, F(1, 229) = 514.1, p < 0.0001, ηp2 = 0.69, BF10 = 5.91e56, a significant main effect of Relatedness, F(1, 229) = 126.6, p < 0.0001, ηp2 = 0.36, BF10 = 3.01e20, and a non-significant interaction, F(1, 229) = 0.108, p = 0.74, ηp2 = 0.0005, BF01 = 6.86.
The absence of a significant interaction between Relatedness and Training Task replicates the full dataset results of Walsh and Rissman (2023) and Experiment 1 of Carpenter (2009), as well as the meta-analytical results reported in Rowland (2014). However, a conclusion of equivalent relative testing efficacy is not warranted, because the absence of an ANOVA interaction is in this case uninterpretable. It does not necessarily indicate the absence of interaction at the psychological process level; that is, it does not demonstrate that the relative efficacy of testing is the same at low and high relatedness levels.
A preferred, model-based approach in this scenario is to test for an ANOVA interaction in a comparison of observed vs predicted test performance, with factors of Data Type (observed PCT vs. PCT-predicted; restudy data were excluded from this analysis) and Relatedness. The ANOVA yielded a non-significant main effect of Data Type, F(1, 229) = 2.89, p = 0.091, ηp2 = 0.01, BF01 = 2.67, a significant main effect of Relatedness, F(1,229) = 135.7, p < 0.0001, ηp2 = 0.37, BF10 = 3.47e21, and most critically, a significant interaction between Data Type and Relatedness, F(1, 229) = 16.2, p < 0.0001, ηp2 = 0.07, BF10 = 204.9. Pairwise t-tests were performed on the simple effects of Data Type. There was a non-significant difference in the low relatedness group, t(117) = 1.42, p = 0.16, BF01 = 3.65, but a robust difference in the high relatedness group, t(112) = –4.74, p < 0.0001, BF10 = 2411. In summary, there was evidence from the model fits that the efficacy of testing is reduced in the high relatedness group compared to the low relatedness group.
Mean final test PCR in the high relatedness group (0.47) is high relative to most of the literature, including the majority of our papers in which the dual memory model predictions were fitted to data. This fact raises the possibility that the failure of the model to fit well for the high relatedness group reflects a fundamental limitation of the model, even in the low relatedness case, when restudy proportion correct is in the range of 0.47. If so, then our results do not necessarily imply lower efficacy of testing in the high relatedness case. However, two sources of evidence argue against that possibility. First, in multiple cumulative distribution analyses of low relatedness data27, the model predictions have held across the entire range of restudy proportion correct. An example from prior work is the cumulative distribution shown in Fig. 3. See also the cumulative fit to the current low relatedness group in Fig. 5. Second, the model predictions for low relatedness materials have held in our prior work even when mean restudy correct exceeds 0.47. For example, in Gupta, Pan, and Rickard (2022), increasing initial study phase repetitions to eight produced final test mean restudy accuracy of 0.53, yet the model fit was almost exact. These findings support the validity of our approach of using the dual-memory model as a yardstick for estimating the relative efficacy of testing in the high vs. low relatedness groups.
Cumulative distributions
The cumulative distribution results are shown in Fig. 5 for both groups, along with the PCT predictions of the dual-memory model for both groups and the prediction of a modified version of that model for the high relatedness group (the correlated strengths version) that will be introduced in the General Discussion as a candidate account of our results. As expected based on prior work for low relatedness materials, the model fitted well across most of the distribution for the low-relatedness group. The predictions for that group deviated from the data primarily in the upper-middle range of distribution, where the model underpredicted the observed PCT magnitude. That form of fit deviation was not observed in prior distribution fits to low relatedness data4,24,27. See also Fig. 3 of the present paper. We conclude that the deviations likely reflect sampling variability.
On the other hand, the model systematically overpredicted the PCT values across all levels of PCR for the high relatedness group. Because the PC distributions reflect both proportion correct sampling variability and individual differences in task ability24,27, these results indicate that high relatedness materials reduce the efficacy of testing across a broad spectrum of individual differences in task ability. This result rules out the possibility that the mean TE was smaller in the high relatedness group merely because of a proportion correct ceiling effect limited to high performing participants.
The pivotal contribution of the distribution analysis in confirming the lower relative efficacy of testing in the high relatedness group can be appreciated by noting that (1) the full range of possible PCR values was exhibited across participants in both the low and high relatedness groups, and (2) at the great majority of those PCR values the TE magnitude was smaller in the high relatedness group, a result that is reflected in the consistently smaller PCT values in Fig. 5b than were predicted by the model. That strong distribution evidence of lower relative testing efficacy in the high relatedness group may appear incompatible with the small and non-significant difference between the group mean TEs (Fig. 4). However, that result is fully explained by the higher restudy proportion correct in the high relatedness group in the context of the quadratic relation between PCR and PCT. In the quadratic relation, the largest TEs occur when PCR is between about 0.4 and 0.6. In the high relatedness group, about 29.2% of participants had PCR values within that range, whereas in the low relatedness group only 10.1% of PCR values were within that range. Hence, the similar mean TE magnitudes in the two groups is the net effect of two opposing factors: (1) the greater efficacy of testing (at matched PCR levels) in the low relatedness group, countered by (2) attenuation of TEs in the low relatedness group due to the smaller percentage of participants with PCR values between about 0.4 and 0.6. Those two independent factors offset one another, masking the interaction between task and relatedness that is present at the psychological process level and yielding roughly equivalent group mean TEs. In summary, the cumulative distribution analysis can be understood as controlling for the quadratic effect and thus revealing the lower intrinsic efficacy of testing in the higher relatedness case.
Item-level analysis
Although the cumulative distribution analyses indicate no proportion correct ceiling effect across participants in the high relatedness group, it remains possible that there is an item-level ceiling effect that is widely distributed over participants, such that for a subset of items, proportion correct in both the restudy and test condition approached 1. If so, then the smaller TE magnitude in the high relatedness group may not reflect a fundamental effect of relatedness on the efficacy of testing, but rather a methodological limitation. However, two results appear to rule out that possibility. First, the largest item-level proportion correct in the restudy condition of the high relatedness group was 0.79 (for the item “mask – face”) and the largest proportion correct in the test condition was 0.93 (for “report – card”). The great majority of item proportions correct were well below those values, suggesting that there is no pervasive ceiling effect.
In a complementary approach involving all items in the high relatedness group, we performed a generalized linear model analysis using the Proc GLIMMIX procedure in the SAS 9.4 software. The model specified a binary response distribution (“1” for correct or “0” for incorrect) and random intercepts for subjects and items, with training condition (restudy vs test; categorical), item semantic relatedness (continuous), and their interaction, as within-subject predictors. The relatedness measure is based on associative norms for each item as listed in the Supplement. There were significant effects of both semantic relatedness, F(1, 8733) = 24.54, p < .0001, confirming the expected increase in overall accuracy with increasing item relatedness, and training condition, F(1, 8733) = 82.77, p < .0001, confirming the testing effect. The interaction, however, was not statistically significant, F(1, 8733) = 0.05, p = 0.82. Hence, there is no evidence for compression of the TE for higher relatedness items.
CC BY 4.0. "Semantic relatedness and the efficacy of retrieval practice" by Gupta MW, Pan SC, Rickard TC, NPJ science of learning (2026). https://doi.org/10.1038/s41539-026-00416-8. Licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Peer-reviewed and open access; full text from PubMed Central (PMC13230953). Headings and layout adapted for reading; figures and tables are in the original.