Hey, just read through this paper and there’s one pretty big head-scratcher in the methodology section that seems to undermine the whole “systematic” part of this systematic review.
You mentioned that for each of the three variable relationships, you selected only one paper per year from 2013-2023 to include in your final analysis tables. You state this was to “avoid bias” and “provide a balanced and comprehensive view.”
But… isn’t that the opposite of how a systematic review is supposed to work? The goal is to find all relevant, high-quality studies within your criteria, not to artificially limit yourself to one per year. This seems like a classic case of selection bias; you’re basically curating a small, convenient sample instead of systematically synthesizing the available evidence.
I have a few questions to clarify this:
1: What was the actual selection criteria for choosing the single paper from each year? The paper says it was for “unique contributions,” but that’s incredibly subjective. Who decided what was “unique,” and based on what specific, pre-defined criteria? Was it just the first one you found? The one with the most interesting findings? The one that best fit your hypothesis?
2: Wouldn’t this method automatically exclude seminal or highly influential papers if two were published in the same year? You might have missed the most important studies altogether because of this arbitrary rule.
3: How can you claim this provides a “comprehensive” view when you’ve intentionally excluded the vast majority of the literature you initially found? Your first dataset had 177 papers, but you analyzed only 10. That’s not a review; it’s a very small sample, and the conclusions can’t be considered representative of the whole field.
This approach seems to compromise the entire foundation of the study. The findings and the proposed “Culture-Incivility-Productivity Nexus” model are built on this hand-picked, non-representative sample, so it’s hard to have confidence in any of the conclusions. Was there a specific reason for using this non-standard method instead of a traditional systematic review process?