Systematic reviews, evidence syntheses, meta-analyses. These are all terms for research methodologies that are growing in popularity in many disciplines as the push for evidence-based practice—and transparency, reproducibility, and bias-reduction in research more generally—becomes ever greater. But what are these methods?
Synthesis of information at its very core has always existed and is something we naturally do in our daily lives to make basic decisions. It has a historical place in the reviews of the literature included at the beginning of research papers to position a study within its broader context. Synthesis is also often part of reporting the results of a study. Authors synthesize their data points to form their conclusions. At its very basic definition, synthesis is a combination of “parts (such as data, concepts, or theories) to make a new whole” (Vogt, 2005).
Authors can conduct synthesis on primary data, as they do when conducting a study, or on secondary data, as with the literature review. Literature reviews and other syntheses of secondary data have a long history of serving as research methods in their own right. And just like other research methods, they have continued to develop and evolve.
Traditionally, a literature review synthesizes the findings of other papers that an author has subjectively chosen as relevant to their research. This is done through a method known as cherry picking. It provides ample room for confirmation bias to creep into the process. Researchers can select to include articles that support their hypotheses and ignore those that don’t.
The 1970s saw the start of a movement to make secondary data analyses (i.e., literature reviews) come more into alignment with the scientific method to help prevent confirmation and other kinds of bias from influencing our understanding of an issue (Feldman, 1971; Rosenthal, 1978; Jackson, 1980; Cooper, 1984).
The family of research methodologies commonly referred to as evidence synthesis, research synthesis, information synthesis, or knowledge synthesis, all stemmed from the development of the meta-analysis by Gene Glass (1976). Meta-analyses were designed specifically for quantitative secondary analysis. Or, as Glass described it, ‘an analysis of analyses” (p. 3).
While meta-analysis provided guidance for the synthesis of the data, it became clear that equally rigorous guidelines were necessary for how that data is collected. The quality of the data retrieved during the research process impacts the quality of the resulting conclusions. This is where the rigor of the data collection process becomes important.
In the 1980s, researchers began including more recommendations and guidelines for full research syntheses. These took the different methods, including meta-analyses and comprehensive search methods and put them together into larger methodologies—collections of methods.
In order to reduce confusion as much as possible, we at UNL Libraries have chosen to use the terminology of ‘advanced reviews’ to refer to methodologies that include multiple systematic methods for secondary analysis, focusing on the systematic nature specifically of the search process, aka data collection. This reflects the evolution from integrations (Feldman, 1971) to meta-analyses (Glass, 1976), integrative reviews (Cooper, 1982) and finally systematic reviews (Cochrane, 1989).
You can learn more about advanced reviews and contact a librarian for help on your project at https://unl.libguides.com/RP.
References:
Cochrane, A., (1989). Forward. In I. Chalmers, M. Enkin, & M. J. N. C. Keirse (Eds.), Effective Care in Pregnancy and Childbirth: Pregnancy (Vol. 1). Oxford University Press.
Cooper, H. M. (1982). Scientific Guidelines for Conducting Integrative Research Reviews. Review of Educational Research, 52(2), 291–302. https://doi.org/10.3102/00346543052002291
Cooper, H. (1984). The Integrative Research Review: A Systematic Approach. Educational Researcher, 15(8), 17–18. https://doi.org/10.3102/0013189X015008017
Feldman, K. A. (1971). Using the Work of Others: Some Observations on Reviewing and Integrating. Sociology of Education, 44(1), 86–102. https://doi.org/10.2307/2111964
Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3–8. https://doi.org/10.3102/0013189X005010003
Jackson, G. B. (1980). Methods for Integrative Reviews. Review of Educational Research, 50(3), 438–460. https://doi.org/10.3102/00346543050003438
Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin, 85(1), 185–193. https://doi.org/10.1037/0033-2909.85.1.185
Vogt, W. P. (2005). Synthesis. In Dictionary of Statistics & Methodology (3 ed.) 320-320. SAGE Publications, Inc. https://doi.org/10.4135/9781412983907