Study selection #2: the fine art of saying no … recording why!
Introduction
A systematic review depends on how well studies are selected. Defining eligibility criteria in advance and keeping full documentation of screening decisions are two of the most important steps in a systematic review. They make your process transparent, reproducible, and trustworthy — three values that are central to evidence-based research. By planning carefully, recording every step, and reporting results clearly, you make it easier for readers and policy-makers to understand and trust your findings.
To make the process efficient and effective, reviewers need to do two things from the start:
- Set clear eligibility criteria — what studies fit the review and what don’t.
- Keep full records of all decisions — especially the reasons why some studies were excluded.
These two practices are essential parts of good systematic review methods, recommended by Cochrane and PRISMA. Eligibility criteria tell you what kind of studies you want to include. They are written before the review starts — not after you’ve seen the results. Typical criteria may include:
- Population: who the study is about
- Intervention: what was done
- Study design: such as RCTs or cohort studies
- Limits: like language, time period, or setting
- Etc…
Writing these criteria early helps avoid bias. It keeps you from making decisions that might favour one outcome over another. You can find this guidance in the Cochrane Handbook (Chapter 3), which explains how to link eligibility criteria to your research question.
Why recording decisions matters
Once screening starts, every decision — include or exclude — should be documented. This shows readers exactly how you moved from thousands of search results to the final set of studies.
The PRISMA 2020 statement asks reviewers to report:
- How many records were found and screened
- How many full texts were reviewed
- How many studies were excluded (and why)
- The final number of included studies
This information usually appears in a PRISMA flow diagram. It is a simple, visual summary of the selection process. Recording reasons for exclusion (e.g. “wrong population” or “not a primary study”) helps others understand your choices and check the transparency of your work.
Rules of thumb
- Write a clear protocol
- Use a tracking sheet or review tool
- Use the PRISMA flow diagram
- Document any changes
- Make decisions consistent
In my own reviews, having well-defined eligibility criteria has saved a lot of time later. Here’s what I’ve learned:
- Writing short examples for each criterion helps reviewers apply them consistently
- Using a simple spreadsheet to record inclusion/exclusion decisions helps track progress
- Recording reasons for exclusion at the full-text stage (for example, “no economic outcome data”) avoids confusion later when writing the report
- Small pilot tests before full screening are useful to make sure everyone understands the criteria
- Using the PRISMA 2020 diagram makes the process transparent and helps readers trust your review
From my experience, the main risk would be: 1) defining unclear and unambiguous eligibility criteria; 2) setting criteria post-hoc or change it without justification. This can make a review inconsistent, raising concerns about bias and transparency.