The Table 1 values reproduce Δ/SD_average (e.g., competence: 0.46/0.61 = 0.75; the corresponding d_z = t/√45 values would be 1.23, 1.10, 1.14, 0.89). In Table 3, however, d = 0.42 equals t/√45 (2.79/6.71), whereas Δ/SD_average = 1.56/2.48 = 0.63. Two different estimators appear to have been used without disclosure. Which formula was applied, and why does it differ between tables? The abstract’s headline d = 0.42 is not comparable with the motivational effects as reported — and per Cohen (1988), 0.42 is small-to-medium, not “moderate.”
The Methods report Cronbach’s α = 0.82 for the usability questionnaire; the Table 4 footnote reports α = 0.74 (10 items). Which is correct?
Implausible pre–post correlation for the concept test (Table 3). From t(44) = 2.79, Δ = 1.56, n = 45, the implied SD of difference scores is ≈ 3.75 — larger than either SD (2.16, 2.76) — implying r_pre,post ≈ −0.15. The motivation subscales imply r ≈ +0.72 to +0.82. A negative pre–post correlation on a knowledge test is highly unusual.
Bonferroni was applied only to the five primary tests, yet ten additional one-sample t-tests are reported (Table 4). At α = 0.05/10 = 0.005, Item 2 (t = 2.86, p ≈ 0.0065) and Item 7 (t = 2.80, p ≈ 0.0076) are not significant, contradicting the claim that “all items received ratings significantly above the neutral point.”
With all items reworded positively, standard SUS scoring is no longer applicable; moreover, “learnability, efficiency, memorability, errors, satisfaction” are Nielsen’s usability components, not SUS dimensions. Eliminating all reverse-coded items invites acquiescence bias — consistent with the .75–.86 inter-subscale correlations in Table 2 (relatedness–competence = .86), which raise serious discriminant-validity concerns for the four supposedly distinct IMI subscales.
Item 7 is “clarity of operational steps” in the Results but “interface clarity” in the Discussion (Item 3 is interface clarity). Table 1 mean differences do not match the tabulated means: relatedness 3.53 − 3.08 = 0.45 (reported 0.46); interest 3.48 − 3.05 = 0.43 (reported 0.42); the interest CI upper bound recomputes to 0.56, not 0.57.