When they do the database search, they use this Boolean string: (Education AND Reality) AND (Science OR Technology OR Engineering OR Mathematics) AND (“Systematic Review” OR “Meta-analysis”) AND (Virtual OR VR OR Augmented OR AR OR Extended OR XR OR Mixed OR MR)
But look at the first part; “(Education AND Reality)”. This is very strange and too narrow. Why they use only “Reality”? Maybe many important papers in education don’t use the word “Reality” directly in title or abstract? They might use “virtual environment”, “immersive learning”, “simulation”, or other terms. By forcing “Reality” they maybe miss many relevant systematic reviews, no?
Also, for the type of literature, they use only “Systematic Review” OR “Meta-analysis”. But an “umbrella review” itself can also include other types of reviews. Maybe some important reviews are not labelled strictly “systematic review” but are still relevant high-quality reviews.
So the main error is maybe the search string is too restrictive. It might exclude many good papers. If the foundation of the review, the collection of papers, is not completely representative, then all the trends and conclusions they find could be biased or wrong.
How they can be sure they catched all important reviews with such a narrow search? This is a big threat to their study’s validity, I think.