Study selection #4: The hidden half of screening — beyond databases
When full-text screening becomes difficult
This post explains the workflow, the role of documentation, how snowballing works in practice, and why both steps remain essential in modern evidence reviews. Title and abstract screening reduces the bulk of search results, but many decisions cannot be made without reading the full paper. Full-text screening helps to:
- confirm eligibility when abstracts are unclear
- extract details on study design, population, outcomes, and methods
- identify reasons for exclusion in a transparent way
- support reproducibility and audit trails
According to the PRISMA 2020 statement, reviewers must document the number of records excluded at full-text stage and provide a reason for each exclusion. The reason must be specific (for example, ‘ineligible population’, ‘no relevant study design’, etc.), not general (for instance, ‘not relevant’). The Cochrane Handbook also emphasises that full-text screening should be done independently by at least two reviewers, with disagreements resolved through discussion or by a third reviewer.
Some common challenges:
- Ambiguous methods: Studies that claim to be RCTs but lack randomisation details.
- Mixed populations: Papers mixing eligible and ineligible groups without separate data.
- Multiple outcomes: Studies reporting only part of the required information.
- Non-English texts: Limited access or translation issues.
Snowballing: the hidden half of screening
Despite extensive database searches, important studies may still not appear in initial search results. Snowballing helps identify them by checking the references around known eligible papers. There are 3 main snowballing techniques:
1. Backward citation tracking (reference checking).Reviewers inspect the reference lists of included studies to identify older or foundational papers. This is often the simplest and most productive step. This method is particularly helpful in areas where indexing is inconsistent, such as qualitative research, global health, or emergent fields:
- Extract the reference list of each included study
- Screen titles/abstracts of references for relevance
- Retrieve full texts if potentially eligible
- Apply the same inclusion/exclusion criteria
2. Forward citation tracking (who cited this study?). Forward citation tracking identifies newer papers that cite the included study. Tools often used include ‘Google Scholar‘, ‘Pubmed‘, ‘Scopus‘ and ‘Web of Science‘:
- Search for the included paper in the citation database
- Identify all papers that have cited it
- Screen titles/abstracts
- Retrieve full texts for potentially eligible studies
3. PubMed ‘Similar/Related Articles‘ Search. PubMed’s Similar articles (formerly Related articles) function helps identify studies that share similar keywords, methods, or topics with an included paper. This method is especially useful in clinical, biomedical, and public health reviews where relevant studies may not share identical index terms:
- Open the included study in PubMed
- Select ‘Similar articles’ on the right-hand panel
- Review the automatically generated list of related studies
- Screen titles/abstracts for relevance
- Retrieve full texts for studies that might meet your criteria
Snowballing strengthens a systematic review by: 1) identifying studies not indexed in common databases; 2) finding studies using different terminology; 3) uncovering “grey zone” research; 4) capturing updates and follow-up studies; and 5) verifying that the database search strategy did not miss key papers. Cochrane recommends citation tracking as a supplementary method, especially in complex or rapidly evolving fields. PRISMA also recognises the value of additional searching techniques and encourages transparent reporting of all sources used. To properly document snowball searches you may track:
- which reference lists were checked
- which citation databases were used
- number of additional records identified
- number included/excluded
- reasons for exclusion
This information appears in the PRISMA flow diagram under “records identified through other methods”.
My experience: why snowballing often finds what databases miss
In my reviews, snowballing has repeatedly identified studies that were invisible in database searches (5–15% of final included studies, more and less). This is especially common in:
- qualitative studies
- economic evaluations
Snowballing has also helped resolve unclear eligibility decisions. For example, when a study seems borderline, checking its related citations can reveal linked publications, clarifying methods or outcomes. In academic and research environments, there is often pressure to reduce workload and speed up screening. Snowball searching is sometimes labelled “optional” or “extra”. Yet skipping it can create gaps in the evidence base and weaken the review’s claims. From my experience, there is simple balance between completness and omissions:
- Full-text screening checks what is directly in front of you
- Snowballing checks what sits just outside your initial view
Completeness takes time, but omissions are harder to fix later; together, they reduce the risk of missing important studies and support the credibility of the final review.
What might be the problems here?
- Incomplete retrieval of full texts
- Vague exclusion reasons
- Snowballing done too late
- Checking only some reference lists
- No documentation of the snowballing process
IT 
EN