Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates, and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and its scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position your work in relation to other researchers and theorists
  • Show how your research addresses a gap or contributes to a debate
  • Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Literature review guide

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

empirical framework in literature review

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models, and methods?
  • Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

  • Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically evaluate: mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarize yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved March 4, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

U.S. flag

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Cover of Handbook of eHealth Evaluation: An Evidence-based Approach

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

  • formulating the research question(s) and objective(s),
  • searching the extant literature,
  • screening for inclusion,
  • assessing the quality of primary studies,
  • extracting data, and
  • analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

  • Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
  • Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
  • Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
  • Analyzing data using quantitative or qualitative methods.
  • Presenting results in summary of findings tables.
  • Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Table 9.1. Typology of Literature Reviews (adapted from Paré et al., 2015).

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

  • Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
  • Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
  • Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
  • Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
  • A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
  • Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
  • Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
  • Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
  • Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
  • Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
  • Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
  • Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
  • Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
  • Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
  • Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
  • Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
  • Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
  • Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
  • Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
  • Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
  • Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
  • Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
  • Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
  • Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
  • Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
  • Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
  • King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
  • Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
  • Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
  • Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
  • Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
  • Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
  • Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
  • Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
  • Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
  • Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
  • Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
  • Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
  • Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
  • Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
  • Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
  • Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
  • Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
  • Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
  • Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
  • Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
  • Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
  • Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
  • Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
  • Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
  • Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
  • Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
  • Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
  • Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
  • Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
  • Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
  • Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
  • Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
  • Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
  • Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
  • Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

  • Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
  • PDF version of this title (4.5M)
  • Disable Glossary Links

In this Page

  • Introduction
  • Overview of the Literature Review Process and Steps
  • Types of Review Articles and Brief Illustrations
  • Concluding Remarks

Related information

  • PMC PubMed Central citations
  • PubMed Links to PubMed

Recent Activity

  • Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

statistics

  • Open access
  • Published: 09 August 2022

A scoping review of frameworks in empirical studies and a review of dissemination frameworks

  • Ana A. Baumann   ORCID: orcid.org/0000-0002-4523-0147 1 ,
  • Cole Hooley 2 ,
  • Emily Kryzer 3 ,
  • Alexandra B. Morshed 4 ,
  • Cassidy A. Gutner 5 , 6 ,
  • Sara Malone 7 ,
  • Callie Walsh-Bailey 7 ,
  • Meagan Pilar 8 ,
  • Brittney Sandler 9 ,
  • Rachel G. Tabak 7 &
  • Stephanie Mazzucca 7  

Implementation Science volume  17 , Article number:  53 ( 2022 ) Cite this article

9405 Accesses

16 Citations

49 Altmetric

Metrics details

The field of dissemination and implementation (D&I) research has grown immensely in recent years. However, the field of dissemination research has not coalesced to the same degree as the field of implementation research. To advance the field of dissemination research, this review aimed to (1) identify the extent to which dissemination frameworks are used in dissemination empirical studies, (2) examine how scholars define dissemination, and (3) identify key constructs from dissemination frameworks.

To achieve aims 1 and 2, we conducted a scoping review of dissemination studies published in D&I science journals. The search strategy included manuscripts published from 1985 to 2020. Articles were included if they were empirical quantitative or mixed methods studies about the dissemination of information to a professional audience. Studies were excluded if they were systematic reviews, commentaries or conceptual papers, scale-up or scale-out studies, qualitative or case studies, or descriptions of programs. To achieve aim 1, we compiled the frameworks identified in the empirical studies. To achieve aim 2, we compiled the definitions from dissemination from frameworks identified in aim 1 and from dissemination frameworks identified in a 2021 review (Tabak RG, Am J Prev Med 43:337-350, 2012). To achieve aim 3, we compile the constructs and their definitions from the frameworks.

Out of 6017 studies, 89 studies were included for full-text extraction. Of these, 45 (51%) used a framework to guide the study. Across the 45 studies, 34 distinct frameworks were identified, out of which 13 (38%) defined dissemination. There is a lack of consensus on the definition of dissemination. Altogether, we identified 48 constructs, divided into 4 categories: process, determinants, strategies, and outcomes. Constructs in the frameworks are not well defined.

Implication for D&I research

This study provides a critical step in the dissemination research literature by offering suggestions on how to define dissemination research and by cataloging and defining dissemination constructs. Strengthening these definitions and distinctions between D&I research could enhance scientific reproducibility and advance the field of dissemination research.

Peer Review reports

Contributions to the literature

The field of dissemination research has not coalesced to the same degree as the field of implementation research. Clearly defining dissemination and identifying dissemination constructs will help enhance dissemination research.

In a review of 34 frameworks, we found a lack of consensus in the definition of dissemination and 48 constructs identified in the frameworks.

We provide a suggested definition of dissemination and a catalog of the constructs to advance the field of dissemination research.

The field of dissemination and implementation (D&I) research has grown extensively in the past years. While scholars from the field of implementation research have made substantial advances, the field of dissemination research has not coalesced to the same degree, limiting the ability to conduct rigorous, reproducible dissemination research. Dissemination research has broadly focused on examining how evidence-based information gets packaged into practices, policies, and programs. This information delivery is often targeted at providers in public health and clinical settings and policymakers to improve public health decision-making. Here, we use provider to refer to a person or group that provides something—in this case, information. The chasm between how evidence-based information is disseminated and how this information is used by providers and policymakers is well-documented [ 1 ] and further evidenced by the ongoing COVID-19 pandemic [ 2 , 3 ].

The definition of dissemination research has been modified over the years and is not consistent across various sources. Dissemination research could be advanced by further development of existing conceptual and theoretical work. In a previous review [ 4 ], nine D&I science frameworks were categorized as “dissemination only” frameworks (i.e., the explicit focus of the framework was on the spread of information about evidence-based interventions to a target audience) [ 4 ]. Frameworks are important because they provide a systematic way to develop, plan, manage and evaluate a study [ 5 , 6 ]. The extent to which dissemination scholars are using frameworks to inform their studies, and which frameworks are used, is unclear.

Building on previous compilations of dissemination frameworks [ 7 ], this paper intends to advance the knowledge of dissemination research by examining dissemination frameworks reported in the empirical literature, cataloging the constructs across different frameworks, and providing definitions for these constructs. A scoping review is ideal at this stage of the dissemination research literature because it helps map the existing frameworks from a body of emerging literature and identifies gaps in the field [ 8 ].

Specifically, this study has three aims: (1) to conduct a scoping review of the empirical dissemination literature and identify the dissemination frameworks informing those studies, (2) to examine how scholars define dissemination, and (3) to catalog and define the constructs from the dissemination frameworks identified in aim 1 and the frameworks categorized as dissemination only by Tabak et al. [ 4 ]

The methods section is divided into the three aims of this study. First, we report the methods for our scoping review to identify the frameworks used in empirical dissemination studies. Second, we report on how we identified the definitions of dissemination. Third, we report the methods for abstracting the dissemination constructs from the frameworks identified in the empirical literature (aim 1) and from the frameworks categorized as “dissemination only” by Tabak et al. [ 4 ] Tabak et al. [ 4 ] categorized models “on a continuum from dissemination to implementation” and acknowledge that “these divisions are intended to assist the reader in model selection, rather than to provide actual classifications for models.” For the current review, we selected only those categorized as dissemination-only because we aimed to examine whether there were any distinct components between the dissemination and implementation frameworks by coding the dissemination-only frameworks.

Scoping review of the literature

We conducted a scoping review to identify dissemination frameworks used in the empirical dissemination literature. A scoping review is appropriate as the goal of this work is to map the current state of the literature, not to evaluate evidence or provide specific recommendations as is the case with a systematic review [ 8 ]. We followed the method developed by Arksey and O’Malley [ 9 ] and later modified by Levac and colleagues [ 10 ]. In doing so, we first identified the research questions (i.e., “Which dissemination frameworks are used in the literature?” and “How are the dissemination constructs defined?”), identified relevant studies (see below), and charted the data to present a summary of our results.

We iteratively created a search strategy in Scopus with terms relevant to dissemination. We ran the search in 2017 and again in December 2020, using the following terms: TITLE-ABS-KEY (dissem* OR (knowledge AND trans*) OR diffuse* OR spread*) in the 20 most relevant journals for the D&I science field, identified by Norton et al. [ 11 ] We ran an identical search at a second time point due to several logistical reasons. This review was an unfunded project conducted by faculty and students who experienced numerous significant life transitions during the project period. We anticipated the original search would be out of date by the time of submission for publication, thus wanted to provide the most up-to-date literature feasible given the time needed to complete the review steps. This approach is appropriate for systematic and scoping reviews [ 12 ]

We included studies if they were (a) quantitative or mixed methods empirical studies, (b) if they were about the dissemination of information (e.g., guidelines) to targeted professional audiences, and (c) published since 1985. Articles were excluded if they were (a) systematic reviews, commentaries, or other non-empirical articles; (b) qualitative studies; (c) scale-up studies (i.e., expanding a program into additional delivery settings); (d) case studies or description of programs; and/or (e) dissemination of information to lay consumer audiences or the general public. Some of the exclusion criteria, specifically around distinguishing studies that were dissemination studies from scale-up or health communication studies, were refined as we reviewed the paper abstracts. In the “Definition of dissemination section, we explain our rationale and process to distinguish these types of studies.

The screening procedures were piloted among all coders with a random sample of articles. AB, SaM, CH, CG, EK, and CWB screened titles for inclusion/exclusion independently, then met to ensure a shared understanding of the criteria and to generate consensus. The same coders then reviewed titles based on the above inclusion/exclusion criteria. Any unclear records were retained for abstract review. Consistent with the previously utilized methodology, the abstract review was conducted sequentially to the title review [ 13 , 14 ]. This approach can improve efficiency while maintaining accuracy [ 15 ]. In this round of review, abstracts were single-screened for inclusion/exclusion. Then, 26% of the articles were independently co-screened by pairs of coders; coding pairs met to generate consensus on disagreements.

Articles that passed to full-text review were independently screened by two coders (AB, CH, EK, and CWB). Coders met to reach a consensus and a third reviewer was consulted if the pair could not reach an agreement. From included records, coders extracted bibliometric information about the article (authors, journal, and year of publication) and the name of the framework used in the study (if a framework was used). Coders met regularly to discuss any discrepancies in coding and to generate consensus; final decisions were made by a third reviewer if necessary.

Review of definitions of dissemination

First, we compiled the list of frameworks identified in the empirical studies. Because some frameworks categorized as dissemination-only by the review of frameworks in Tabak et al. [ 4 ] were not present in our sample, we added those to our list of frameworks to review. From the articles describing these frameworks, we extracted dissemination definitions, constructs, and construct definitions. AB, SM, AM, and MP independently abstracted and compared the constructs’ definitions.

Review of dissemination constructs

Once constructs were identified, the frequency of the constructs was counted, and definitions were abstracted. We then organized the constructs into four categories: dissemination processes, determinants, strategies, and outcomes. These categories were organized based on themes by AB and reviewed by all authors. We presented different versions of these categories to groups of stakeholders along our process, including posters at the 2019 and 2021 Conferences on the Science of Dissemination and Implementation in Health, the Washington University Network for Dissemination and Implementation Researchers (WUNDIR), and our network of D&I research peers. During these presentations and among our internal authorship group, we received feedback that the categorization of the constructs was helpful.

We defined the constructs in the dissemination process as constructs that relate to processes, stages, or events by which the dissemination process happens. The dissemination determinants construct encompasses constructs that may facilitate or obstruct the dissemination process (i.e., barriers or facilitators). The dissemination strategies constructs are those that describe the approaches or actions of a dissemination process. Finally, dissemination outcomes are the identified dissemination outcomes in the frameworks (distinct from health service, clinical, or population health outcomes). These categories are subjective and defined by the study team. The tables in Additional file 1 include our suggested labels and definitions for the constructs within these four categories, the definitions as provided by the articles describing the frameworks, and the total frequency of each construct from the frameworks reviewed.

The PRISMA Extension for Scoping Reviews (PRISMA-ScR) flowchart is shown in Fig. 1 . The combined searches yielded 6017 unique articles. Of those, 5622 were excluded during the title and abstract screening. Of the 395 full-text articles, we retained 89 in our final sample.

figure 1

PRISMA chart

Papers were excluded during the full-text review for several reasons. Many papers ( n = 101, 33%) were excluded because they did not meet the coding definitions for dissemination studies. For example, some studies were focused on larger quality improvement initiatives without a clear dissemination component while other studies reported disseminating findings tangentially. Many ( n = 61; 20%) were excluded because they reported a study testing approaches to spread information to the general public or lay audiences instead of to a group of professionals (e.g., disseminating information about HIV perinatal transmission to mothers, not healthcare providers.) Several articles ( n = 55, 18%) were related to the scale up of interventions and not the dissemination of information.

Frameworks identified

Table 1 shows the frameworks used in the included studies. We identified a total of 27 unique frameworks in the empirical studies. Out of the 27 frameworks identified, only three overlapped with the 11 frameworks cataloged as “dissemination only” in Tabak et al. [ 4 ] review. Two frameworks identified in the empirical studies were cataloged by Tabak et al [ 4 ] as “D = I,” one was cataloged as “D > I,” and one as “I only.” Additional file 1 : Table S1 shows all the frameworks, with frameworks 1–11 being “D only” from Tabak et al. [ 4 ], and frameworks 12–34 are the ones identified in our empirical sample. Rogers’ diffusion of innovation [ 16 ] was used most frequently (in 10 studies), followed by the Knowledge to Action Framework (in 4 studies) [ 17 ] and RE-AIM (in 3 studies) [ 18 ]. Dobbins’ Framework for the Dissemination and Utilization of Research for Health-Care Policy and Practice [ 19 ], the Interactive Systems Framework and Network Theory [ 20 ], and Kingdon’s Multiple Streams Framework [ 21 ] were each used by two studies. Thirty studies (33%) did not explicitly describe a dissemination framework that informed their work.

Definition of dissemination

Table 2 shows the definition of dissemination from the frameworks. Out of the 38 frameworks, only 12 (32%) defined dissemination. There is wide variability in the depth of the definitions, with some authors defining dissemination as a process “transferring research to the users,” [ 24 ] and others defining it as both a process and an outcome [ 19 , 23 ]. The definitions of dissemination varied among the 13 frameworks that defined dissemination; however, some shared characteristics were identified. In nine of the 13 frameworks, the definition of dissemination included language about the movement or spread of something, whether an idea, innovation, program, or research finding [ 16 , 23 , 24 , 25 , 26 , 27 , 28 , 31 , 32 ]. Seven of the frameworks described dissemination as active, intentional, or planned by those leading a dissemination effort [ 7 , 16 , 23 , 25 , 26 , 27 , 32 ]. Five frameworks specified some type of outcome as a result of dissemination (e.g., the adoption of an innovation or awareness of research results) [ 7 , 19 , 23 , 27 , 29 , 30 ]. Three of the frameworks’ definitions included the role of influential determinants of dissemination [ 19 , 27 , 29 , 30 ]. Only two frameworks highlighted dissemination as a process [ 23 , 25 ].

Definition of dissemination constructs

Below, we describe the results presented in Tables 3 , 4 , 5 , and 6 with constructs grouped by dissemination process, determinants, strategies, and outcomes. The definitions proposed for the constructs were based on a thematic review of the definitions provided in the articles, which can be found in the Additional file 1 : Tables S2-S5.

Table 3 shows the constructs that relate to the dissemination processes , i.e., the steps or processes through which dissemination happens. Seven constructs were categorized as processes: knowledge inquiry, knowledge synthesis, communication, interaction, persuasion, activation, and research transfer. That is, six frameworks suggest that the dissemination process starts with an inquiry of what type of information is needed to close the knowledge gap. Next, there is a process of gathering and synthesizing the information, including examining the context in which the information will be shared. After the information is identified and gathered, there is a process of communication, interaction with the information, and persuasion where the information is shared with the target users, where the users then engage with the information and activate towards action based on the information received. Finally, there is a process of research transfer, where the information sharing “becomes essentially independent of explicit intentional change activity.” [ 33 ]

Table 4 shows the 17 constructs categorized as dissemination determinants , which are constructs that reflect aspects that may facilitate or hinder the dissemination process. Determinants identified included content of the information, context, interpersonal networks, source of knowledge and audience, the medium of dissemination, opinion leaders, compatibility of the information with the setting, type of information, and capacity of the audience to adopt the innovation. Communication, the salience of communication, and users’ perceived attitudes towards the information were the most frequent constructs ( n = 14 each), followed by context ( n = 13), interpersonal networks ( n = 12), sources of knowledge, and audience ( n = 10 each).

Table 5 shows the nine constructs related to dissemination strategies , which are constructs that describe the approaches or actions to promote or support dissemination. Leeman and colleagues [ 34 ] conceptualize dissemination strategies as strategies that provide synthesis, translation, and support of information. The authors refer to dissemination as two broad strategies: developing materials and distributing materials. We identified several strategies related to the synthesis of information (e.g., identify the knowledge), translation of information (e.g., adapt information to context), and other constructs. Monitoring and evaluation were the most frequent constructs ( n = 10), with identify the quality gap and increase audience’s skills next ( n = 6).

Finally, Table 6 shows the dissemination outcomes , which are constructs related to the effects of the dissemination process. Fifteen constructs were categorized here, including awareness and changes in policy, decision and impact, adoption and cost, emotion reactions, knowledge gained, accountability, maintenance, persuasion, reception, confirmation, and fidelity. Knowledge utilization was the most cited construct across frameworks ( n = 11), followed by awareness and change in policy ( n = 8 each).

The goals of this study were threefold. First, we conducted a scoping review of the empirical literature to catalog the dissemination frameworks informing dissemination studies. Second, we compiled the definition of dissemination, and third, we cataloged and defined the constructs from the dissemination frameworks. During our review process, we found that clearly identifying dissemination studies was more complicated than anticipated. Defining the sample of articles to code for this study was a challenge because of the large variability of studies that use the word “dissemination” in the titles but that are actually scale-up or health communication studies.

The high variability in the definition of dissemination poses a challenge for the field because if we do not clearly define what we are doing, we are unable to set boundaries to distinguish dissemination research from other fields. Among the identified frameworks that defined dissemination, the definitions highlighted that dissemination involved the spread of something, whether knowledge, an innovation, or a program. Distinct from diffusion, several definitions described dissemination as an active process, using intentional strategies. Few definitions described the role of determinants, whether dissemination is a process or a discrete event, and what strategies and outcomes may be pertinent. Future work is needed to unify these distinct conceptualizations into a comprehensive definition that dissemination researchers can use.

While it is clear that dissemination differs from diffusion, as the latter has been considered the passive and “haphazard” spread of information [ 35 ], the distinction between dissemination and scale-up—as shown in the definitions identified in this study—is less clear. Some articles from our search not included in the review conceptualized dissemination as similar to scale-up. To clarify the distinction between dissemination and scale-up in our review, we used the WHO’s definition of scale-up [ 36 ] as “deliberate efforts to increase the impact of successfully tested health innovations to benefit more people and to foster policy and program development on a lasting basis.” In other words, based on these definitions, our team considered scale-up as referencing active efforts to spread evidence-based interventions , whereas diffusion is the passive spread of information. Dissemination, therefore, can be conceptualized as the active and planned spread of information.

Another helpful component in distinguishing dissemination science from other sciences is related to the target audience. Brownson et al. [ 1 ] define dissemination as an “active approach of spreading evidence-based information to the target audience via determined channels using planned strategies” (p. 9). Defining the target audience in the context of dissemination is important because it may help distinguish the field from social marketing. Indeed, several studies we excluded involved sharing information with the public (e.g., increasing the awareness of the importance of sunscreen in public swimming pools). Grier and Bryant define social marketing as a “program-planning process that applies commercial marketing concepts and techniques to promote voluntary behavior change ( … ) by groups of individuals, often referred to as the target audience.” [ 37 ] The target audience in the context of social marketing, the authors explain, is usually considered consumers but can also be policymakers [ 37 ]. To attempt to delineate a distinction between these two fields, dissemination work has traditionally identified professionals (e.g., clinicians, public health practitioners, policymakers) as the target audience of dissemination efforts, whereas the target audience in social marketing is conceptualized as a broader audience. Figure 2 shows how we conceptualize the distinct components of dissemination research from other fields. Based on these distinctions, we propose the following coalesced definition for dissemination research to guide this review: the scientific study of the targeted distribution of information to a specific professional person or group of professionals. Clearly distinguishing dissemination from scale-up as well as health communication will help further advance the dissemination research field.

figure 2

Proposed distinction of definitions between diffusion, scale-up, and dissemination

Our results show that of the empirical papers identified in this review, 51% used a framework to guide their study. This finding mirrors the suboptimal use of frameworks in the field of implementation research [ 38 , 39 ], with scholars recently putting forth guidance on how to select and use frameworks to enhance their use in implementation research studies [ 6 ]. Similarly, we provide a catalog of dissemination frameworks and their constructs identified in dissemination studies. It is necessary to move the dissemination research field forward by embedding frameworks in dissemination-focused studies.

Some empirical papers included in our review used frameworks based on the knowledge translation literature. Knowledge translation, a field most prominent outside the USA, has been defined as “a dynamic and iterative process that includes the synthesis, dissemination, exchange and ethically sound application of knowledge to improve health, provide more effective health services and products, and strengthen the health care system” [ 40 ]. As such, it conceptualizes an interactive relationship between the creation and the application of knowledge. In the USA, however, researchers tend to conceptualize dissemination as a concept discrete from implementation and use the acronym “D&I” to identify these two fields.

While one could state that there is a distinct set of outcomes, methods, and frameworks between dissemination and implementation fields, previous scholars have cataloged [ 4 ] a continuum, from dissemination only” to “implementation only” frameworks. Consistent with this, our findings show that scholars have adapted implementation frameworks to fit dissemination outcomes (e.g., Klesges’ adaptation of RE-AIM [ 41 ]), while other frameworks have both dissemination and implementation components (e.g., integrated Promoting Action on Research Implementation in Health Services [i-PARIHS] [ 42 ]). Additionally, behavioral change frameworks (e.g., theory of planned behavior) were cataloged in our study as they were used in included articles. The use of implementation frameworks in studies identified here as dissemination studies highlights at least three potential hypotheses. One possibility is the use of implementation frameworks in dissemination studies is due to the underdevelopment of the field of dissemination, as shown in the challenges that we found in the conceptual definition of dissemination. We hope that, by clearly outlining a definition of dissemination, scholars can start to empirically examine whether there are distinct components between implementation and dissemination outcomes and processes.

The second hypothesis is that we still do not have enough evidence in the dissemination or implementation fields to be dogmatic about the categorization of frameworks as either “dissemination” or “implementation.” Until we have more robust evidence about what is and what is not dissemination (or other continua along which frameworks may be categorized), we caution against holding too firm to characterizations of frameworks [ 38 , 43 , 44 , 45 ] Frameworks evolve as more empirical evidence is gathered [ 43 , 45 , 46 , 47 , 48 , 49 ], and they are applied in different settings and contexts. We could hypothesize that it is less important, as of now, to categorize a framework as an implementation or dissemination framework and instead clearly explain why a specific framework was selected and how it is applied in the study.

Selection and application of frameworks in dissemination and implementation research is still a challenge, especially considering scholars may often select frameworks in a haphazard way [ 6 , 50 , 51 ]. While scholars have put forward some guidance to select implementation frameworks [ 6 , 52 ], the challenge in the dissemination and implementation research fields is likely not only in the selection of the frameworks but perhaps more so in the misuse or misapplication of frameworks, theories, or models. A survey indicated that there is little consensus on the process that scholars use to select frameworks and that scholars select frameworks based on several criteria, including familiarity with the framework [ 50 ]. As such, Birken et al. [ 52 ] offer other criteria for the selection of frameworks, such as (a) usability (i.e., whether the framework includes relevant constructs and whether the framework provides an explanation of how constructs influence each other), (b) applicability (i.e., how a method, such as an interview, can be used with the framework; whether the framework is generalizable to different contexts), and (c) testability (i.e., whether the framework proposed a testable hypothesis and whether it contributes to an evidence-based or theory development). Moullin et al. [ 6 ] suggest that implementation frameworks should be selected based on their (a) purpose, (b), levels of analysis (e.g., provider, organizational, system), (c) degree of inclusion and depth of analysis or operationalization of implementation concepts, and (d) the framework’s orientation (e.g., setting and type of intervention).

More than one framework can be selected in one study, depending on the research question(s). The application of a framework can support a project in the planning stages (e.g., examining the determinants of a context, engaging with stakeholders), during the project (e.g., making explicit the mechanisms of action, tracking and exploring the process of change), and after the project is completed (e.g., use of the framework to report outcomes, to understand what happened and why) [ 6 , 51 , 53 ]. We believe that similar guidance can and should be applied to dissemination frameworks; further empirical work may be needed to help identify how to select and apply dissemination and/or implementation frameworks in dissemination research. The goal of this review is to support the advancement of the dissemination and implementation sciences by identifying constructs and frameworks that scholars can apply in their dissemination studies. Additional file 1 : Tables S6-S9 show the frequency of constructs per framework, and readers can see the variability in the frequency of constructs per framework to help in their selection of frameworks.

A third hypothesis is that the processes of dissemination and implementation are interrelated, may occur simultaneously, and perhaps support each other in the uptake of evidence-based interventions. For example, Leppin et al. [ 54 ] use the definition of implementation based on the National Institutes of Health: “the adoption and integration of evidence-based health interventions into clinical and community settings for the purposes of improving care delivery and efficiency, patient outcomes, and individual and population health” [ 55 ], and implementation research as the study of this process to develop a knowledge base about how interventions can be embedded in practices. In this sense, implementation aims to examine the “how” to normalize interventions in practices, to enhance uptake of these interventions, guidelines, or policies, whereas dissemination examines how to spread the information about these interventions, policies, and practices, intending to support their adoption (see Fig. 1 ). In other words, using Curran’s [ 56 ] simple terms, implementation is about adopting and maintaining “the thing” whereas dissemination is about intentionally spreading information to enable learning about “the thing.” As Leppin et al. argue, these two sciences [ 54 , 57 , 58 ], while separate, could co-occur in the process of supporting the uptake of evidence-based interventions. Future work may entail empirically understanding the role of these frameworks in dissemination research.

This review aimed to advance a critical step in the dissemination literature by defining and categorizing dissemination constructs. Constructs are subjective, socially constructed concepts [ 59 ], and therefore their definitions may be bounded by factors including, but not limited to, the researchers’ discipline and background, the research context, and time [ 60 ]. This is evident in the constructs’ lack of consistent, clear definitions (see Additional file 1 ). The inconsistency in the definitions of the constructs is problematic because it impairs measurement development and consequently validity and comparability across studies. The lack of clear definitions of the dissemination constructs may be due to the multidisciplinary nature of the D&I research field in general [ 61 , 62 ], which is a value of the field. However, not having consistency in terms and definitions makes it difficult to develop generalizable conclusions and synthesize scientific findings regarding dissemination research.

We identified a total of 48 constructs, which we separated into four categories: dissemination processes, determinants, strategies, and outcomes. By providing these categories, we can hope to help advance the field of dissemination research to ensure rigor and consistency. Process constructs are important to guide the critical steps and structure that scholars may need to take when doing dissemination research. Of note is that the processes identified in this study may not be unique to dissemination research but rather to the research process in general. As the field of dissemination research advances, it will be interesting to examine whether there are unique components in these process stages that are unique to the dissemination field. In addition to the process, an examination of dissemination determinants (i.e., barriers and facilitators) is essential in understanding how contextual factors occurring at different levels (e.g., information recipient, organizational setting, policy environment) influence dissemination efforts and impede or improve dissemination success [ 7 ]. Understanding the essential determinants will help to guide the selection and design of strategies that can support dissemination efforts. Finally, the constructs in the dissemination outcomes will help examine levels and processes to assess.

The categorization of the constructs was not without challenges. For example, persuasion was coded as a strategy (persuading) and as an outcome (persuasion). Likewise, the construct confirmation could be conceptualized as a stage [ 16 ] or as an outcome [ 19 ]. The constructs identified in this review provide an initial taxonomy for understanding and assessing dissemination outcomes, but more research and conceptualization are needed to fully describe dissemination processes, determinants, strategies, and outcomes. Given the recent interest in the dissemination literature [ 22 , 63 ], a future step for the field is examining the precise and coherent definition and operationalization of dissemination constructs, along with the identification or development of measures to assess them.

Limitations

A few limitations to this study should be noted. First, the search was limited to one bibliometric database and from journals publishing D&I in health studies. We limited our search to one database because we aimed to capture articles from Norton et al. [ 11 ], and therefore, our search methodology was focused on journals instead of on databases. Future work learning from other fields, and doing a broader search on other databases could provide different perspectives. Second, we did not include terms such as research utilization, research translation, knowledge exchange, knowledge mobilization, or translation science in our search, limiting the scope and potential generalizability of our search. Translation science has been defined as being a different science than dissemination, however. Leppin et al. [ 54 ], for example, offer the definition of translation science as the science that aims to identify and advance generalizable principles to expedite research translation, or the “process of turning observations into interventions that improve health” (see Fig. 1 ). Translation research, therefore, focuses on the determinants to achieve this end. Accordingly, Wilson et al. [ 7 ] used other terms in their search, including translation, diffusion of innovation, and knowledge mobilization and found different frameworks in their review. In their paper, Wilson and colleagues [ 7 ] provided a different analysis than ours in that they aimed to examine the theoretical underpinning of the frameworks identified by them. Our study is different from theirs in that we offer the definition of disseminating and a compilation of constructs and their definitions. A future study could combine the frameworks identified by our study with the ones identified by Wilson and colleagues and detail the theoretical origins of the frameworks, and the definitions of the constructs to support in the selection of frameworks for dissemination studies. Third, by being stringent in our inclusion criteria, we may have missed important work. Several articles were excluded from our scoping review because they were examining the spread of an evidence-based intervention (scale up) or of the spread of dissemination for the public (health communication). As noted above, however, clearly distinguishing dissemination from scale-up and from health communication will help further advance the dissemination research field and identify its mechanisms of action. Fourth, given the broad literatures in diffusion, dissemination, and social marketing, researchers may disagree with our definitions and how we conceptualized the constructs. Fifth, we did not code qualitative studies because we wanted to have boundaries in this study as it is a scoping study. Future studies could examine the application of frameworks in qualitative work. It is our hope that future research can build from this work to continue to define and test the dissemination constructs.

Conclusions

Based on the review of frameworks and the empirical literature, we defined dissemination research and outlined key constructs in the categories of dissemination process, strategies, determinants, strategies, and outcomes. Our data indicate that the field of dissemination research could be advanced with a more explicit focus on methods and a common understanding of constructs. We hope that our review will help guide the field in providing a narrative taxonomy of dissemination constructs that promote clarity and advance the dissemination research field. We hope that future stages of the dissemination research field can examine specific measures and empirically test the mechanisms of action of the dissemination process.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Dissemination and implementation

Integrated Promoting Action on Research Implementation in Health Services

Reach, Effectiveness, Adoption, Implementation, Maintenance

Consolidated Framework for Implementation Research

Brownson RC, Eyler AA, Harris JK, Moore JB, Tabak RG. Getting the Word Out: New Approaches for Disseminating Public Health Science. J Public Health Manag Pract. 2018;24(2):102-11. https://doi.org/10.1097/PHH.0000000000000673 .

Naeem S, bin, Bhatti R. The COVID-19 ‘infodemic’: a new front for information professionals. Health Inf Libr J. 2020;37(3):233–9. https://doi.org/10.1111/hir.12311 .

Article   Google Scholar  

Narayan KMV, Curran JW, Foege WH. The COVID-19 pandemic as an opportunity to ensure a more successful future for science and public health. JAMA. 2021;325(6):525–6. https://doi.org/10.1001/jama.2020.23479 .

Article   CAS   PubMed   Google Scholar  

Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50. https://doi.org/10.1016/j.amepre.2012.05.024 .

Article   PubMed   PubMed Central   Google Scholar  

Green L, Kreuter M. Health program planning: an educational and ecological approach: McGraw-Hill Education; 2005.

Google Scholar  

Moullin JC, Dickson KS, Stadnick NA, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. 2020;1(1):42. https://doi.org/10.1186/s43058-020-00023-7 .

Wilson PM, Petticrew M, Calnan MW, Nazareth I. Disseminating research findings: what should researchers do? A systematic scoping review of conceptual frameworks. Implement Sci. 2010;5(1):1–16.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143. https://doi.org/10.1186/s12874-018-0611-x .

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. https://doi.org/10.1080/1364557032000119616 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69. https://doi.org/10.1186/1748-5908-5-69 .

Norton WE, Lungeanu A, Chambers DA, Contractor N. Mapping the growing discipline of dissemination and implementation science in health. Scientometrics. 2017;112(3):1367–90. https://doi.org/10.1007/s11192-017-2455-2 .

Bramer W, Bain P. Updating search strategies for systematic reviews using EndNote. J Med Libr Assoc. 2017;105(3):285–9. https://doi.org/10.5195/jmla.2017.183 .

Hickey MD, Odeny TA, Petersen M, et al. Specification of implementation interventions to address the cascade of HIV care and treatment in resource-limited settings: a systematic review. Implement Sci. 2017;12(1):102. https://doi.org/10.1186/s13012-017-0630-8 .

Hooley C, Amano T, Markovitz L, Yaeger L, Proctor E. Assessing implementation strategy reporting in the mental health literature: a narrative review. Adm Policy Ment Health Ment Health Serv Res. 2020;47(1):19–35. https://doi.org/10.1007/s10488-019-00965-8 .

Mateen FJ, Oh J, Tergas AI, Bhayani NH, Kamdar BB. Titles versus titles and abstracts for initial screening of articles for systematic reviews. Clin Epidemiol. 2013;5(1):89–95. https://doi.org/10.2147/CLEP.S43118 .

Rogers EM, Singhal A, Quinlan MM. Diffusion of innovations: Routledge; 2014.

Graham ID, Tetroe JM. The Knowledge to Action Framework. Models Frameworks Implement Evid Based Pract. 2010;207:222.

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.

Article   CAS   Google Scholar  

Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A. A framework for the dissemination and utilization of research for health-care policy and practice. Worldviews Evid Based Nurs Presents Arch Online J Knowl Synth Nurs. 2002;9(1):149–60.

Wandersman A, Duffy J, Flaspohler P, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41(3):171–81.

Kingdon JW, Stano E. Agendas, alternatives, and public policies, vol. 45: Little, Brown Boston; 1984.

Winkler JD, Lohr KN, Brook RH. Persuasive communication and medical technology assessment. Arch Intern Med. 1985;145(2):314–7.

Scullion PA. Effective dissemination strategies. Nurse Res (through 2013). 2002;10(1):65–77.

Anderson M, Cosby J, Swan B, Moore H, Broekhoven M. The use of research in local health service agencies. Soc Sci Med. 1999;49(8):1007–19.

Elliott SJ, O’Loughlin J, Robinson K, et al. Conceptualizing dissemination research and activity: the case of the Canadian Heart Health Initiative. Health Educ Behav. 2003;30(3):267–82.

Owen N, Glanz K, Sallis JF, Kelder SH. Evidence-based approaches to dissemination and diffusion of physical activity interventions. Am J Prev Med. 2006;31(4):35–44.

Yuan CT, Nembhard IM, Stern AF, Brush JE Jr, Krumholz HM, Bradley EH. Blueprint for the dissemination of evidence-based practices in health care. Issue Brief (Commonw Fund). 2010;86:1–16.

Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

Oh CH, Rich RF. Explaining use of information in public policymaking. Knowledge Policy. 1996;9(1):3–35.

Landry R, Amara N, Lamari M. Utilization of social science research knowledge in Canada. Res Policy. 2001;30(2):333–49.

Birken SA, Lee SYD, Weiner BJ. Uncovering middle managers’ role in healthcare innovation implementation. Implement Sci. 2012;7(1):1–12.

Fairweather GW. Methods for experimental social innovation. 1967. Published online

Fairweather GWTLG. Experimental methods for social policy research. Pergamon Press; 1977.

Leeman J, Birken SA, Powell BJ, Rohweder C, Shea CM. Beyond “implementation strategies”: classifying the full range of strategies used in implementation science and practice. Implement Sci. 2017;12(1):125. https://doi.org/10.1186/s13012-017-0657-x .

Lomas J. Diffusion, dissemination, and implementation: who should do what? Ann N Y Acad Sci. 1993;703:226–35. https://doi.org/10.1111/j.1749-6632.1993.tb26351.x discussion 235-7.

World Health Organization, EXPAND NET. Nine steps for developing a scaling-up strategy.; 2010.

Grier S, Bryant CA. Social marketing in public health. Annu Rev Public Health. 2005;26(1):319–39. https://doi.org/10.1146/annurev.publhealth.26.021304.144610 .

Article   PubMed   Google Scholar  

Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1. https://doi.org/10.1186/s13012-018-0842-6 .

Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11(1):72. https://doi.org/10.1186/s13012-016-0437-z .

Straus SE, Tetroe J, Graham I. Defining knowledge translation. Can Med Assoc J. 2009;181(3-4):165–8. https://doi.org/10.1503/cmaj.081229 .

Dzewaltowski DA, Glasgow RE, Klesges LM, Estabrooks PA, Brock E. RE-AIM: evidence-based standards and a web resource to improve translation of research into practice. Ann Behav Med. 2004;28(2):75–80.

Rycroft-Malone J, Kitson A, Harvey G, et al. Ingredients for change: revisiting a conceptual framework. BMJ Qual Saf. 2002;11(2):174–80.

Glasgow RE, Estabrooks PA, Ory MG. Characterizing evolving frameworks: issues from Esmail et al. (2020) review. Implement Sci. 2020;15(1):53. https://doi.org/10.1186/s13012-020-01009-8 .

Esmail R, Hanson HM, Holroyd-Leduc J, et al. A scoping review of full-spectrum knowledge translation theories, models, and frameworks. Implement Sci. 2020;15(1):11. https://doi.org/10.1186/s13012-020-0964-5 .

Holtrop JS, Estabrooks PA, Gaglio B, et al. Understanding and applying the RE-AIM framework: clarifications and resources. J Clin Transl Sci. 2021;5(1):e126. https://doi.org/10.1017/cts.2021.789 .

Damschroder LJ, Reardon CM, Opra Widerquist MA, Lowery J. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):7. https://doi.org/10.1186/s13012-021-01181-5 .

Means AR, Kemp CG, Gwayi-Chore MC, et al. Evaluating and optimizing the Consolidated Framework for Implementation Research (CFIR) for use in low- and middle-income countries: a systematic review. Implement Sci. 2020;15(1):17. https://doi.org/10.1186/s13012-020-0977-0 .

Baumann AA. Commentary: Adapting and operationalizing the RE-AIM Framework for implementation science in environmental health: clean fuel cooking programs in low resource countries. Front Public Health. 2020;8. https://doi.org/10.3389/fpubh.2020.00218 .

Harvey G, Kitson A. Introducing the i-PARIHS framework. In: Implementing evidence-based practice in healthcare. 1st ed: Routledge; 2015. p. 25–46.

Chapter   Google Scholar  

Birken SA, Powell BJ, Shea CM, et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci. 2017;12(1):124. https://doi.org/10.1186/s13012-017-0656-y .

Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. “There is nothing so practical as a good theory”: a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857. https://doi.org/10.1186/s12913-018-3671-z .

Birken SA, Rohweder CL, Powell BJ, et al. T-CaST: an implementation theory comparison and selection tool. Implement Sci. 2018;13(1):143. https://doi.org/10.1186/s13012-018-0836-4 .

Stetler CB, Damschroder LJ, Helfrich CD, Hagedorn HJ. A guide for applying a revised version of the PARIHS framework for implementation. Implement Sci. 2011;6(1):99. https://doi.org/10.1186/1748-5908-6-99 .

Leppin AL, Mahoney JE, Stevens KR, et al. Situating dissemination and implementation sciences within and across the translational research spectrum. J Clin Transl Sci. 2020;4(3):152–8. https://doi.org/10.1017/cts.2019.392 .

National Institutes of Health. PAR 18-007 Dissemination and Implementation Research in Health FOA. 2017; October 12, 2017

Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. https://doi.org/10.1097/MLR.0b013e3182408812 .

Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1(1):1. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Dearing JW. Evolution of diffusion and dissemination theory. J Public Health Manag Pract. 2008;14(2):99–108. https://doi.org/10.1097/01.PHH.0000311886.98627.b7 .

Nunnally Bernstein Ira H. JC. Psychometric theory. McGraw-Hill; 1994.

Bergkvist L, Eisend M. The dynamic nature of marketing constructs. J Acad Mark Sci. 2021;49(3):521–41. https://doi.org/10.1007/s11747-020-00756-w .

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):103. https://doi.org/10.1186/s13012-019-0957-4 s13012-019-0957-4.

Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health Ment Health Serv Res. 2009;36(1):24–34.

Purtle J, Marzalik JS, Halfond RW, Bufka LF, Teachman BA, Aarons GA. Toward the data-driven dissemination of findings from psychological science. Am Psychol. 2020;75(8):1052–66. https://doi.org/10.1037/amp0000721 .

Baker EA, Brewer SK, Owens JS, Cook CR, Lyon AR. Dissemination science in school mental health: a framework for future research. Sch Ment Heal. 2021;13(4):791–807. https://doi.org/10.1007/s12310-021-09446-6 .

Lester JP. The utilization of policy analysis by state agency officials. Knowledge. 1993;14(3):267–90.

Kramer DM, Cole DC. Sustained, intensive engagement to promote health and safety knowledge transfer to and utilization by workplaces. Sci Commun. 2003;25(1):56–82.

Jacobson N, Butterill D, Goering P. Development of a framework for knowledge translation: understanding user context. J Health Serv Res Policy. 2003;8(2):94–9.

Zeldin S, Petrokubi J, MacNeil C. Youth-adult partnerships in decision making: disseminating and implementing an innovative idea into established organizations and communities. Am J Community Psychol. 2008;41(3):262–77.

Lipowski EE, Becker M. Presentation of drug prescribing guidelines and physician response. QRB Q Rev Bull. 1992;18(12):461–70.

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6(1):1–12.

Edquist C, Hommen L, McKelvey MD. Innovation and employment: process versus product innovation: Edward Elgar Publishing; 2001.

Book   Google Scholar  

Fairweather GW, Davidson WS. An introduction to community experimentation: McGraw-Hill; 1986.

Hovland CI, Janis IL, Kelley HH. Communication and persuasion. 1953. Published online.

Kumagai AK. A conceptual framework for the use of illness narratives in medical education. Acad Med. 2008;83(7):653–8.

de Silva MJ, Breuer E, Lee L, et al. Theory of change: a theory-driven approach to enhance the Medical Research Council’s framework for complex interventions. Trials. 2014;15(1):1–13.

Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9):S63–7.

Schmid TL, Pratt M, Witmer L. A framework for physical activity policy research. J Phys Act Health. 2006;3(s1):S20–9.

Redman S, Turner T, Davies H, et al. The SPIRIT Action Framework: a structured approach to selecting and testing strategies to increase the use of research in policy. Soc Sci Med. 2015;136:147–55.

Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manag Sci. 1989;35(8):982–1003.

Thacker SB, Stroup DF, Parrish RG, Anderson HA. Surveillance in environmental public health: issues, systems, and sources. Am J Public Health. 1996;86(5):633–8.

Ajzen I. From intentions to actions: a theory of planned behavior. In: Action control Springer; 1985. p. 11–39.

Kaegi L. From paper to practice to point of care: reports from a Zitter Group Conference on Implementing Practice Guidelines. Jt Comm J Qual Improv. 1996;22(8):551–83.

CAS   PubMed   Google Scholar  

Download references

Acknowledgements

The authors would like to thank the WUNDIR community for their invaluable feedback.

AB is supported by the National Heart, Lung and Blook Institute U24HL154426 and 5U01HL133994, the National Institute of Child Health and Human Development R01HD091218, and the National Institute of Mental Health P50MH122351. CWB is funded by NIMHD T37 MD014218. MP is supported by the National Institute of Allergy and Infectious Diseases (K24AI134413). ABM is supported by the National Heart, Lung, and Blood Institute (1T32HL130357). RGT and SM were supported by the National Institute of Diabetes and Digestive and Kidney Diseases P30DK092950 and by the Nutrition Obesity Research Center, P30 DK056341, and Cooperative Agreement number U48DP006395 from the Centers for Disease Control and Prevention. AB, SM, and TB were supported by the National Cancer Institute P50CA244431. AAB, ABM, MP, RGT, and SM were also supported by the National Center for Advancing Translational Sciences UL1TR002345. The findings and conclusions in this paper are those of the authors and do not necessarily represent the official positions of the National Institutes of Health or the Centers for Disease Control and Prevention.

Author information

Authors and affiliations.

Division of Public Health Sciences, Department of Surgery, Washington University in St. Louis, St. Louis, USA

Ana A. Baumann

School of Social Work, Brigham Young University, Provo, USA

Cole Hooley

BJC HealthCare, Community Health Improvement, St. Louis, USA

Emily Kryzer

Rollins School of Public Health, Emory University, Atlanta, USA

Alexandra B. Morshed

ViiV Healthcare, Research Triangle Park, NC, USA

Cassidy A. Gutner

Department of Psychiatry, Boston University School of Medicine, Boston, MA, USA

Brown School of Social Work, Washington University in St. Louis, St. Louis, USA

Sara Malone, Callie Walsh-Bailey, Rachel G. Tabak & Stephanie Mazzucca

Department of Infectious Diseases, Washington University School of Medicine, Washington University in St. Louis, St. Louis, USA

Meagan Pilar

Bernard Becker Medical Library, School of Medicine, Washington University in St. Louis, St. Louis, USA

Brittney Sandler

You can also search for this author in PubMed   Google Scholar

Contributions

AB developed the research question. AB, CH, AM, CAG, SM, and RT designed the study. AB, CH, EK, ABM, CAG, SaM, CWB, MP, RGT, and SM coded the data. BS supported with editing and references. All authors collaborated on writing the manuscript, and all approve the final version of the document.

Corresponding author

Correspondence to Ana A. Baumann .

Ethics declarations

Ethics approval and consent to participate.

Not applicable

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: table s1..

Definition of disseminations from frameworks. Table S2. Dissemination Process constructs, their definition, and frequency across frameworks. Table S3. Dissemination determinants constructs, their definition, and frequency across frameworks. Table S4. Dissemination strategy constructs, their definition, and frequency across frameworks. Table S5. Dissemination outcome constructs, their definitions, and frequency across frameworks. Table S6. Frequency of Process Constructs Across Frameworks. Table S7. Frequency of Determinant Constructs Across Frameworks. Table S8. Frequency of Strategy Constructs Across Frameworks. Table S9. Frequency of Determinant Constructs Across Frameworks. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist [ 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 ].

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Baumann, A.A., Hooley, C., Kryzer, E. et al. A scoping review of frameworks in empirical studies and a review of dissemination frameworks. Implementation Sci 17 , 53 (2022). https://doi.org/10.1186/s13012-022-01225-4

Download citation

Received : 14 April 2022

Accepted : 11 July 2022

Published : 09 August 2022

DOI : https://doi.org/10.1186/s13012-022-01225-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Dissemination
  • Dissemination research

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

empirical framework in literature review

Grad Coach

How To Structure Your Literature Review

3 options to help structure your chapter.

By: Amy Rommelspacher (PhD) | Reviewer: Dr Eunice Rautenbach | November 2020 (Updated May 2023)

Writing the literature review chapter can seem pretty daunting when you’re piecing together your dissertation or thesis. As  we’ve discussed before , a good literature review needs to achieve a few very important objectives – it should:

  • Demonstrate your knowledge of the research topic
  • Identify the gaps in the literature and show how your research links to these
  • Provide the foundation for your conceptual framework (if you have one)
  • Inform your own  methodology and research design

To achieve this, your literature review needs a well-thought-out structure . Get the structure of your literature review chapter wrong and you’ll struggle to achieve these objectives. Don’t worry though – in this post, we’ll look at how to structure your literature review for maximum impact (and marks!).

The function of the lit review

But wait – is this the right time?

Deciding on the structure of your literature review should come towards the end of the literature review process – after you have collected and digested the literature, but before you start writing the chapter. 

In other words, you need to first develop a rich understanding of the literature before you even attempt to map out a structure. There’s no use trying to develop a structure before you’ve fully wrapped your head around the existing research.

Equally importantly, you need to have a structure in place before you start writing , or your literature review will most likely end up a rambling, disjointed mess. 

Importantly, don’t feel that once you’ve defined a structure you can’t iterate on it. It’s perfectly natural to adjust as you engage in the writing process. As we’ve discussed before , writing is a way of developing your thinking, so it’s quite common for your thinking to change – and therefore, for your chapter structure to change – as you write. 

Need a helping hand?

empirical framework in literature review

Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components – an  introduction , a  body   and a  conclusion . 

Let’s take a closer look at each of these.

1: The Introduction Section

Just like any good introduction, the introduction section of your literature review should introduce the purpose and layout (organisation) of the chapter. In other words, your introduction needs to give the reader a taste of what’s to come, and how you’re going to lay that out. Essentially, you should provide the reader with a high-level roadmap of your chapter to give them a taste of the journey that lies ahead.

Here’s an example of the layout visualised in a literature review introduction:

Example of literature review outline structure

Your introduction should also outline your topic (including any tricky terminology or jargon) and provide an explanation of the scope of your literature review – in other words, what you  will   and  won’t   be covering (the delimitations ). This helps ringfence your review and achieve a clear focus . The clearer and narrower your focus, the deeper you can dive into the topic (which is typically where the magic lies). 

Depending on the nature of your project, you could also present your stance or point of view at this stage. In other words, after grappling with the literature you’ll have an opinion about what the trends and concerns are in the field as well as what’s lacking. The introduction section can then present these ideas so that it is clear to examiners that you’re aware of how your research connects with existing knowledge .

Free Webinar: Literature Review 101

2: The Body Section

The body of your literature review is the centre of your work. This is where you’ll present, analyse, evaluate and synthesise the existing research. In other words, this is where you’re going to earn (or lose) the most marks. Therefore, it’s important to carefully think about how you will organise your discussion to present it in a clear way. 

The body of your literature review should do just as the description of this chapter suggests. It should “review” the literature – in other words, identify, analyse, and synthesise it. So, when thinking about structuring your literature review, you need to think about which structural approach will provide the best “review” for your specific type of research and objectives (we’ll get to this shortly).

There are (broadly speaking)  three options  for organising your literature review.

The body section of your literature review is the where you'll present, analyse, evaluate and synthesise the existing research.

Option 1: Chronological (according to date)

Organising the literature chronologically is one of the simplest ways to structure your literature review. You start with what was published first and work your way through the literature until you reach the work published most recently. Pretty straightforward.

The benefit of this option is that it makes it easy to discuss the developments and debates in the field as they emerged over time. Organising your literature chronologically also allows you to highlight how specific articles or pieces of work might have changed the course of the field – in other words, which research has had the most impact . Therefore, this approach is very useful when your research is aimed at understanding how the topic has unfolded over time and is often used by scholars in the field of history. That said, this approach can be utilised by anyone that wants to explore change over time .

Adopting the chronological structure allows you to discuss the developments and debates in the field as they emerged over time.

For example , if a student of politics is investigating how the understanding of democracy has evolved over time, they could use the chronological approach to provide a narrative that demonstrates how this understanding has changed through the ages.

Here are some questions you can ask yourself to help you structure your literature review chronologically.

  • What is the earliest literature published relating to this topic?
  • How has the field changed over time? Why?
  • What are the most recent discoveries/theories?

In some ways, chronology plays a part whichever way you decide to structure your literature review, because you will always, to a certain extent, be analysing how the literature has developed. However, with the chronological approach, the emphasis is very firmly on how the discussion has evolved over time , as opposed to how all the literature links together (which we’ll discuss next ).

Option 2: Thematic (grouped by theme)

The thematic approach to structuring a literature review means organising your literature by theme or category – for example, by independent variables (i.e. factors that have an impact on a specific outcome).

As you’ve been collecting and synthesising literature , you’ll likely have started seeing some themes or patterns emerging. You can then use these themes or patterns as a structure for your body discussion. The thematic approach is the most common approach and is useful for structuring literature reviews in most fields.

For example, if you were researching which factors contributed towards people trusting an organisation, you might find themes such as consumers’ perceptions of an organisation’s competence, benevolence and integrity. Structuring your literature review thematically would mean structuring your literature review’s body section to discuss each of these themes, one section at a time.

The thematic structure allows you to organise your literature by theme or category  – e.g. by independent variables.

Here are some questions to ask yourself when structuring your literature review by themes:

  • Are there any patterns that have come to light in the literature?
  • What are the central themes and categories used by the researchers?
  • Do I have enough evidence of these themes?

PS – you can see an example of a thematically structured literature review in our literature review sample walkthrough video here.

Option 3: Methodological

The methodological option is a way of structuring your literature review by the research methodologies used . In other words, organising your discussion based on the angle from which each piece of research was approached – for example, qualitative , quantitative or mixed  methodologies.

Structuring your literature review by methodology can be useful if you are drawing research from a variety of disciplines and are critiquing different methodologies. The point of this approach is to question  how  existing research has been conducted, as opposed to  what  the conclusions and/or findings the research were.

The methodological structure allows you to organise your chapter by the analysis method  used - e.g. qual, quant or mixed.

For example, a sociologist might centre their research around critiquing specific fieldwork practices. Their literature review will then be a summary of the fieldwork methodologies used by different studies.

Here are some questions you can ask yourself when structuring your literature review according to methodology:

  • Which methodologies have been utilised in this field?
  • Which methodology is the most popular (and why)?
  • What are the strengths and weaknesses of the various methodologies?
  • How can the existing methodologies inform my own methodology?

3: The Conclusion Section

Once you’ve completed the body section of your literature review using one of the structural approaches we discussed above, you’ll need to “wrap up” your literature review and pull all the pieces together to set the direction for the rest of your dissertation or thesis.

The conclusion is where you’ll present the key findings of your literature review. In this section, you should emphasise the research that is especially important to your research questions and highlight the gaps that exist in the literature. Based on this, you need to make it clear what you will add to the literature – in other words, justify your own research by showing how it will help fill one or more of the gaps you just identified.

Last but not least, if it’s your intention to develop a conceptual framework for your dissertation or thesis, the conclusion section is a good place to present this.

In the conclusion section, you’ll need to present the key findings of your literature review and highlight the gaps that exist in the literature. Based on this, you'll  need to make it clear what your study will add  to the literature.

Example: Thematically Structured Review

In the video below, we unpack a literature review chapter so that you can see an example of a thematically structure review in practice.

Let’s Recap

In this article, we’ve  discussed how to structure your literature review for maximum impact. Here’s a quick recap of what  you need to keep in mind when deciding on your literature review structure:

  • Just like other chapters, your literature review needs a clear introduction , body and conclusion .
  • The introduction section should provide an overview of what you will discuss in your literature review.
  • The body section of your literature review can be organised by chronology , theme or methodology . The right structural approach depends on what you’re trying to achieve with your research.
  • The conclusion section should draw together the key findings of your literature review and link them to your research questions.

If you’re ready to get started, be sure to download our free literature review template to fast-track your chapter outline.

Literature Review Course

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

Literature review 101 - how to find articles

27 Comments

Marin

Great work. This is exactly what I was looking for and helps a lot together with your previous post on literature review. One last thing is missing: a link to a great literature chapter of an journal article (maybe with comments of the different sections in this review chapter). Do you know any great literature review chapters?

ISHAYA JEREMIAH AYOCK

I agree with you Marin… A great piece

Qaiser

I agree with Marin. This would be quite helpful if you annotate a nicely structured literature from previously published research articles.

Maurice Kagwi

Awesome article for my research.

Ache Roland Ndifor

I thank you immensely for this wonderful guide

Malik Imtiaz Ahmad

It is indeed thought and supportive work for the futurist researcher and students

Franklin Zon

Very educative and good time to get guide. Thank you

Dozie

Great work, very insightful. Thank you.

KAWU ALHASSAN

Thanks for this wonderful presentation. My question is that do I put all the variables into a single conceptual framework or each hypothesis will have it own conceptual framework?

CYRUS ODUAH

Thank you very much, very helpful

Michael Sanya Oluyede

This is very educative and precise . Thank you very much for dropping this kind of write up .

Karla Buchanan

Pheeww, so damn helpful, thank you for this informative piece.

Enang Lazarus

I’m doing a research project topic ; stool analysis for parasitic worm (enteric) worm, how do I structure it, thanks.

Biswadeb Dasgupta

comprehensive explanation. Help us by pasting the URL of some good “literature review” for better understanding.

Vik

great piece. thanks for the awesome explanation. it is really worth sharing. I have a little question, if anyone can help me out, which of the options in the body of literature can be best fit if you are writing an architectural thesis that deals with design?

S Dlamini

I am doing a research on nanofluids how can l structure it?

PATRICK MACKARNESS

Beautifully clear.nThank you!

Lucid! Thankyou!

Abraham

Brilliant work, well understood, many thanks

Nour

I like how this was so clear with simple language 😊😊 thank you so much 😊 for these information 😊

Lindiey

Insightful. I was struggling to come up with a sensible literature review but this has been really helpful. Thank you!

NAGARAJU K

You have given thought-provoking information about the review of the literature.

Vakaloloma

Thank you. It has made my own research better and to impart your work to students I teach

Alphonse NSHIMIYIMANA

I learnt a lot from this teaching. It’s a great piece.

Resa

I am doing research on EFL teacher motivation for his/her job. How Can I structure it? Is there any detailed template, additional to this?

Gerald Gormanous

You are so cool! I do not think I’ve read through something like this before. So nice to find somebody with some genuine thoughts on this issue. Seriously.. thank you for starting this up. This site is one thing that is required on the internet, someone with a little originality!

kan

I’m asked to do conceptual, theoretical and empirical literature, and i just don’t know how to structure it

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

  • Print Friendly

Empirical Research

Introduction, what is empirical research, attribution.

  • Finding Empirical Research in Library Databases
  • Designing Empirical Research
  • Case Sudies

Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief. 

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."  Ask yourself: Could I recreate this study and test these results?

Key characteristics to look for:

  • Specific research questions to be answered
  • Definition of the population, behavior, or   phenomena being studied
  • Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components:

  • Introduction : sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
  • Methodology: sometimes called "research design" -- how to recreate the study -- usually describes the population, research process, and analytical tools
  • Results : sometimes called "findings" -- what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
  • Discussion : sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies

Portions of this guide were built using suggestions from other libraries, including Penn State and Utah State University libraries.

  • Next: Finding Empirical Research in Library Databases >>
  • Last Updated: Jan 10, 2023 8:31 AM
  • URL: https://enmu.libguides.com/EmpiricalResearch

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 2 Chapter 3: What is Empirical Literature & Where can it be Found?

In Module 1, you read about the problem of pseudoscience. Here, we revisit the issue in addressing how to locate and assess scientific or empirical literature . In this chapter you will read about:

  • distinguishing between what IS and IS NOT empirical literature
  • how and where to locate empirical literature for understanding diverse populations, social work problems, and social phenomena.

Probably the most important take-home lesson from this chapter is that one source is not sufficient to being well-informed on a topic. It is important to locate multiple sources of information and to critically appraise the points of convergence and divergence in the information acquired from different sources. This is especially true in emerging and poorly understood topics, as well as in answering complex questions.

What Is Empirical Literature

Social workers often need to locate valid, reliable information concerning the dimensions of a population group or subgroup, a social work problem, or social phenomenon. They might also seek information about the way specific problems or resources are distributed among the populations encountered in professional practice. Or, social workers might be interested in finding out about the way that certain people experience an event or phenomenon. Empirical literature resources may provide answers to many of these types of social work questions. In addition, resources containing data regarding social indicators may also prove helpful. Social indicators are the “facts and figures” statistics that describe the social, economic, and psychological factors that have an impact on the well-being of a community or other population group.The United Nations (UN) and the World Health Organization (WHO) are examples of organizations that monitor social indicators at a global level: dimensions of population trends (size, composition, growth/loss), health status (physical, mental, behavioral, life expectancy, maternal and infant mortality, fertility/child-bearing, and diseases like HIV/AIDS), housing and quality of sanitation (water supply, waste disposal), education and literacy, and work/income/unemployment/economics, for example.

Image of the Globe

Three characteristics stand out in empirical literature compared to other types of information available on a topic of interest: systematic observation and methodology, objectivity, and transparency/replicability/reproducibility. Let’s look a little more closely at these three features.

Systematic Observation and Methodology. The hallmark of empiricism is “repeated or reinforced observation of the facts or phenomena” (Holosko, 2006, p. 6). In empirical literature, established research methodologies and procedures are systematically applied to answer the questions of interest.

Objectivity. Gathering “facts,” whatever they may be, drives the search for empirical evidence (Holosko, 2006). Authors of empirical literature are expected to report the facts as observed, whether or not these facts support the investigators’ original hypotheses. Research integrity demands that the information be provided in an objective manner, reducing sources of investigator bias to the greatest possible extent.

Transparency and Replicability/Reproducibility.   Empirical literature is reported in such a manner that other investigators understand precisely what was done and what was found in a particular research study—to the extent that they could replicate the study to determine whether the findings are reproduced when repeated. The outcomes of an original and replication study may differ, but a reader could easily interpret the methods and procedures leading to each study’s findings.

What is NOT Empirical Literature

By now, it is probably obvious to you that literature based on “evidence” that is not developed in a systematic, objective, transparent manner is not empirical literature. On one hand, non-empirical types of professional literature may have great significance to social workers. For example, social work scholars may produce articles that are clearly identified as describing a new intervention or program without evaluative evidence, critiquing a policy or practice, or offering a tentative, untested theory about a phenomenon. These resources are useful in educating ourselves about possible issues or concerns. But, even if they are informed by evidence, they are not empirical literature. Here is a list of several sources of information that do not meet the standard of being called empirical literature:

  • your course instructor’s lectures
  • political statements
  • advertisements
  • newspapers & magazines (journalism)
  • television news reports & analyses (journalism)
  • many websites, Facebook postings, Twitter tweets, and blog postings
  • the introductory literature review in an empirical article

You may be surprised to see the last two included in this list. Like the other sources of information listed, these sources also might lead you to look for evidence. But, they are not themselves sources of evidence. They may summarize existing evidence, but in the process of summarizing (like your instructor’s lectures), information is transformed, modified, reduced, condensed, and otherwise manipulated in such a manner that you may not see the entire, objective story. These are called secondary sources, as opposed to the original, primary source of evidence. In relying solely on secondary sources, you sacrifice your own critical appraisal and thinking about the original work—you are “buying” someone else’s interpretation and opinion about the original work, rather than developing your own interpretation and opinion. What if they got it wrong? How would you know if you did not examine the primary source for yourself? Consider the following as an example of “getting it wrong” being perpetuated.

Example: Bullying and School Shootings . One result of the heavily publicized April 1999 school shooting incident at Columbine High School (Colorado), was a heavy emphasis placed on bullying as a causal factor in these incidents (Mears, Moon, & Thielo, 2017), “creating a powerful master narrative about school shootings” (Raitanen, Sandberg, & Oksanen, 2017, p. 3). Naturally, with an identified cause, a great deal of effort was devoted to anti-bullying campaigns and interventions for enhancing resilience among youth who experience bullying.  However important these strategies might be for promoting positive mental health, preventing poor mental health, and possibly preventing suicide among school-aged children and youth, it is a mistaken belief that this can prevent school shootings (Mears, Moon, & Thielo, 2017). Many times the accounts of the perpetrators having been bullied come from potentially inaccurate third-party accounts, rather than the perpetrators themselves; bullying was not involved in all instances of school shooting; a perpetrator’s perception of being bullied/persecuted are not necessarily accurate; many who experience severe bullying do not perpetrate these incidents; bullies are the least targeted shooting victims; perpetrators of the shooting incidents were often bullying others; and, bullying is only one of many important factors associated with perpetrating such an incident (Ioannou, Hammond, & Simpson, 2015; Mears, Moon, & Thielo, 2017; Newman &Fox, 2009; Raitanen, Sandberg, & Oksanen, 2017). While mass media reports deliver bullying as a means of explaining the inexplicable, the reality is not so simple: “The connection between bullying and school shootings is elusive” (Langman, 2014), and “the relationship between bullying and school shooting is, at best, tenuous” (Mears, Moon, & Thielo, 2017, p. 940). The point is, when a narrative becomes this publicly accepted, it is difficult to sort out truth and reality without going back to original sources of information and evidence.

Wordcloud of Bully Related Terms

What May or May Not Be Empirical Literature: Literature Reviews

Investigators typically engage in a review of existing literature as they develop their own research studies. The review informs them about where knowledge gaps exist, methods previously employed by other scholars, limitations of prior work, and previous scholars’ recommendations for directing future research. These reviews may appear as a published article, without new study data being reported (see Fields, Anderson, & Dabelko-Schoeny, 2014 for example). Or, the literature review may appear in the introduction to their own empirical study report. These literature reviews are not considered to be empirical evidence sources themselves, although they may be based on empirical evidence sources. One reason is that the authors of a literature review may or may not have engaged in a systematic search process, identifying a full, rich, multi-sided pool of evidence reports.

There is, however, a type of review that applies systematic methods and is, therefore, considered to be more strongly rooted in evidence: the systematic review .

Systematic review of literature. A systematic reviewis a type of literature report where established methods have been systematically applied, objectively, in locating and synthesizing a body of literature. The systematic review report is characterized by a great deal of transparency about the methods used and the decisions made in the review process, and are replicable. Thus, it meets the criteria for empirical literature: systematic observation and methodology, objectivity, and transparency/reproducibility. We will work a great deal more with systematic reviews in the second course, SWK 3402, since they are important tools for understanding interventions. They are somewhat less common, but not unheard of, in helping us understand diverse populations, social work problems, and social phenomena.

Locating Empirical Evidence

Social workers have available a wide array of tools and resources for locating empirical evidence in the literature. These can be organized into four general categories.

Journal Articles. A number of professional journals publish articles where investigators report on the results of their empirical studies. However, it is important to know how to distinguish between empirical and non-empirical manuscripts in these journals. A key indicator, though not the only one, involves a peer review process . Many professional journals require that manuscripts undergo a process of peer review before they are accepted for publication. This means that the authors’ work is shared with scholars who provide feedback to the journal editor as to the quality of the submitted manuscript. The editor then makes a decision based on the reviewers’ feedback:

  • Accept as is
  • Accept with minor revisions
  • Request that a revision be resubmitted (no assurance of acceptance)

When a “revise and resubmit” decision is made, the piece will go back through the review process to determine if it is now acceptable for publication and that all of the reviewers’ concerns have been adequately addressed. Editors may also reject a manuscript because it is a poor fit for the journal, based on its mission and audience, rather than sending it for review consideration.

Word cloud of social work related publications

Indicators of journal relevance. Various journals are not equally relevant to every type of question being asked of the literature. Journals may overlap to a great extent in terms of the topics they might cover; in other words, a topic might appear in multiple different journals, depending on how the topic was being addressed. For example, articles that might help answer a question about the relationship between community poverty and violence exposure might appear in several different journals, some with a focus on poverty, others with a focus on violence, and still others on community development or public health. Journal titles are sometimes a good starting point but may not give a broad enough picture of what they cover in their contents.

In focusing a literature search, it also helps to review a journal’s mission and target audience. For example, at least four different journals focus specifically on poverty:

  • Journal of Children & Poverty
  • Journal of Poverty
  • Journal of Poverty and Social Justice
  • Poverty & Public Policy

Let’s look at an example using the Journal of Poverty and Social Justice . Information about this journal is located on the journal’s webpage: http://policy.bristoluniversitypress.co.uk/journals/journal-of-poverty-and-social-justice . In the section headed “About the Journal” you can see that it is an internationally focused research journal, and that it addresses social justice issues in addition to poverty alone. The research articles are peer-reviewed (there appear to be non-empirical discussions published, as well). These descriptions about a journal are almost always available, sometimes listed as “scope” or “mission.” These descriptions also indicate the sponsorship of the journal—sponsorship may be institutional (a particular university or agency, such as Smith College Studies in Social Work ), a professional organization, such as the Council on Social Work Education (CSWE) or the National Association of Social Work (NASW), or a publishing company (e.g., Taylor & Frances, Wiley, or Sage).

Indicators of journal caliber.  Despite engaging in a peer review process, not all journals are equally rigorous. Some journals have very high rejection rates, meaning that many submitted manuscripts are rejected; others have fairly high acceptance rates, meaning that relatively few manuscripts are rejected. This is not necessarily the best indicator of quality, however, since newer journals may not be sufficiently familiar to authors with high quality manuscripts and some journals are very specific in terms of what they publish. Another index that is sometimes used is the journal’s impact factor . Impact factor is a quantitative number indicative of how often articles published in the journal are cited in the reference list of other journal articles—the statistic is calculated as the number of times on average each article published in a particular year were cited divided by the number of articles published (the number that could be cited). For example, the impact factor for the Journal of Poverty and Social Justice in our list above was 0.70 in 2017, and for the Journal of Poverty was 0.30. These are relatively low figures compared to a journal like the New England Journal of Medicine with an impact factor of 59.56! This means that articles published in that journal were, on average, cited more than 59 times in the next year or two.

Impact factors are not necessarily the best indicator of caliber, however, since many strong journals are geared toward practitioners rather than scholars, so they are less likely to be cited by other scholars but may have a large impact on a large readership. This may be the case for a journal like the one titled Social Work, the official journal of the National Association of Social Workers. It is distributed free to all members: over 120,000 practitioners, educators, and students of social work world-wide. The journal has a recent impact factor of.790. The journals with social work relevant content have impact factors in the range of 1.0 to 3.0 according to Scimago Journal & Country Rank (SJR), particularly when they are interdisciplinary journals (for example, Child Development , Journal of Marriage and Family , Child Abuse and Neglect , Child Maltreatmen t, Social Service Review , and British Journal of Social Work ). Once upon a time, a reader could locate different indexes comparing the “quality” of social work-related journals. However, the concept of “quality” is difficult to systematically define. These indexes have mostly been replaced by impact ratings, which are not necessarily the best, most robust indicators on which to rely in assessing journal quality. For example, new journals addressing cutting edge topics have not been around long enough to have been evaluated using this particular tool, and it takes a few years for articles to begin to be cited in other, later publications.

Beware of pseudo-, illegitimate, misleading, deceptive, and suspicious journals . Another side effect of living in the Age of Information is that almost anyone can circulate almost anything and call it whatever they wish. This goes for “journal” publications, as well. With the advent of open-access publishing in recent years (electronic resources available without subscription), we have seen an explosion of what are called predatory or junk journals . These are publications calling themselves journals, often with titles very similar to legitimate publications and often with fake editorial boards. These “publications” lack the integrity of legitimate journals. This caution is reminiscent of the discussions earlier in the course about pseudoscience and “snake oil” sales. The predatory nature of many apparent information dissemination outlets has to do with how scientists and scholars may be fooled into submitting their work, often paying to have their work peer-reviewed and published. There exists a “thriving black-market economy of publishing scams,” and at least two “journal blacklists” exist to help identify and avoid these scam journals (Anderson, 2017).

This issue is important to information consumers, because it creates a challenge in terms of identifying legitimate sources and publications. The challenge is particularly important to address when information from on-line, open-access journals is being considered. Open-access is not necessarily a poor choice—legitimate scientists may pay sizeable fees to legitimate publishers to make their work freely available and accessible as open-access resources. On-line access is also not necessarily a poor choice—legitimate publishers often make articles available on-line to provide timely access to the content, especially when publishing the article in hard copy will be delayed by months or even a year or more. On the other hand, stating that a journal engages in a peer-review process is no guarantee of quality—this claim may or may not be truthful. Pseudo- and junk journals may engage in some quality control practices, but may lack attention to important quality control processes, such as managing conflict of interest, reviewing content for objectivity or quality of the research conducted, or otherwise failing to adhere to industry standards (Laine & Winker, 2017).

One resource designed to assist with the process of deciphering legitimacy is the Directory of Open Access Journals (DOAJ). The DOAJ is not a comprehensive listing of all possible legitimate open-access journals, and does not guarantee quality, but it does help identify legitimate sources of information that are openly accessible and meet basic legitimacy criteria. It also is about open-access journals, not the many journals published in hard copy.

An additional caution: Search for article corrections. Despite all of the careful manuscript review and editing, sometimes an error appears in a published article. Most journals have a practice of publishing corrections in future issues. When you locate an article, it is helpful to also search for updates. Here is an example where data presented in an article’s original tables were erroneous, and a correction appeared in a later issue.

  • Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2017). A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 12(8): e0181722. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5558917/
  • Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2018).Correction—A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 13(3): e0193937.  http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193937

Search Tools. In this age of information, it is all too easy to find items—the problem lies in sifting, sorting, and managing the vast numbers of items that can be found. For example, a simple Google® search for the topic “community poverty and violence” resulted in about 15,600,000 results! As a means of simplifying the process of searching for journal articles on a specific topic, a variety of helpful tools have emerged. One type of search tool has previously applied a filtering process for you: abstracting and indexing databases . These resources provide the user with the results of a search to which records have already passed through one or more filters. For example, PsycINFO is managed by the American Psychological Association and is devoted to peer-reviewed literature in behavioral science. It contains almost 4.5 million records and is growing every month. However, it may not be available to users who are not affiliated with a university library. Conducting a basic search for our topic of “community poverty and violence” in PsychINFO returned 1,119 articles. Still a large number, but far more manageable. Additional filters can be applied, such as limiting the range in publication dates, selecting only peer reviewed items, limiting the language of the published piece (English only, for example), and specified types of documents (either chapters, dissertations, or journal articles only, for example). Adding the filters for English, peer-reviewed journal articles published between 2010 and 2017 resulted in 346 documents being identified.

Just as was the case with journals, not all abstracting and indexing databases are equivalent. There may be overlap between them, but none is guaranteed to identify all relevant pieces of literature. Here are some examples to consider, depending on the nature of the questions asked of the literature:

  • Academic Search Complete—multidisciplinary index of 9,300 peer-reviewed journals
  • AgeLine—multidisciplinary index of aging-related content for over 600 journals
  • Campbell Collaboration—systematic reviews in education, crime and justice, social welfare, international development
  • Google Scholar—broad search tool for scholarly literature across many disciplines
  • MEDLINE/ PubMed—National Library of medicine, access to over 15 million citations
  • Oxford Bibliographies—annotated bibliographies, each is discipline specific (e.g., psychology, childhood studies, criminology, social work, sociology)
  • PsycINFO/PsycLIT—international literature on material relevant to psychology and related disciplines
  • SocINDEX—publications in sociology
  • Social Sciences Abstracts—multiple disciplines
  • Social Work Abstracts—many areas of social work are covered
  • Web of Science—a “meta” search tool that searches other search tools, multiple disciplines

Placing our search for information about “community violence and poverty” into the Social Work Abstracts tool with no additional filters resulted in a manageable 54-item list. Finally, abstracting and indexing databases are another way to determine journal legitimacy: if a journal is indexed in a one of these systems, it is likely a legitimate journal. However, the converse is not necessarily true: if a journal is not indexed does not mean it is an illegitimate or pseudo-journal.

Government Sources. A great deal of information is gathered, analyzed, and disseminated by various governmental branches at the international, national, state, regional, county, and city level. Searching websites that end in.gov is one way to identify this type of information, often presented in articles, news briefs, and statistical reports. These government sources gather information in two ways: they fund external investigations through grants and contracts and they conduct research internally, through their own investigators. Here are some examples to consider, depending on the nature of the topic for which information is sought:

  • Agency for Healthcare Research and Quality (AHRQ) at https://www.ahrq.gov/
  • Bureau of Justice Statistics (BJS) at https://www.bjs.gov/
  • Census Bureau at https://www.census.gov
  • Morbidity and Mortality Weekly Report of the CDC (MMWR-CDC) at https://www.cdc.gov/mmwr/index.html
  • Child Welfare Information Gateway at https://www.childwelfare.gov
  • Children’s Bureau/Administration for Children & Families at https://www.acf.hhs.gov
  • Forum on Child and Family Statistics at https://www.childstats.gov
  • National Institutes of Health (NIH) at https://www.nih.gov , including (not limited to):
  • National Institute on Aging (NIA at https://www.nia.nih.gov
  • National Institute on Alcohol Abuse and Alcoholism (NIAAA) at https://www.niaaa.nih.gov
  • National Institute of Child Health and Human Development (NICHD) at https://www.nichd.nih.gov
  • National Institute on Drug Abuse (NIDA) at https://www.nida.nih.gov
  • National Institute of Environmental Health Sciences at https://www.niehs.nih.gov
  • National Institute of Mental Health (NIMH) at https://www.nimh.nih.gov
  • National Institute on Minority Health and Health Disparities at https://www.nimhd.nih.gov
  • National Institute of Justice (NIJ) at https://www.nij.gov
  • Substance Abuse and Mental Health Services Administration (SAMHSA) at https://www.samhsa.gov/
  • United States Agency for International Development at https://usaid.gov

Each state and many counties or cities have similar data sources and analysis reports available, such as Ohio Department of Health at https://www.odh.ohio.gov/healthstats/dataandstats.aspx and Franklin County at https://statisticalatlas.com/county/Ohio/Franklin-County/Overview . Data are available from international/global resources (e.g., United Nations and World Health Organization), as well.

Other Sources. The Health and Medicine Division (HMD) of the National Academies—previously the Institute of Medicine (IOM)—is a nonprofit institution that aims to provide government and private sector policy and other decision makers with objective analysis and advice for making informed health decisions. For example, in 2018 they produced reports on topics in substance use and mental health concerning the intersection of opioid use disorder and infectious disease,  the legal implications of emerging neurotechnologies, and a global agenda concerning the identification and prevention of violence (see http://www.nationalacademies.org/hmd/Global/Topics/Substance-Abuse-Mental-Health.aspx ). The exciting aspect of this resource is that it addresses many topics that are current concerns because they are hoping to help inform emerging policy. The caution to consider with this resource is the evidence is often still emerging, as well.

Numerous “think tank” organizations exist, each with a specific mission. For example, the Rand Corporation is a nonprofit organization offering research and analysis to address global issues since 1948. The institution’s mission is to help improve policy and decision making “to help individuals, families, and communities throughout the world be safer and more secure, healthier and more prosperous,” addressing issues of energy, education, health care, justice, the environment, international affairs, and national security (https://www.rand.org/about/history.html). And, for example, the Robert Woods Johnson Foundation is a philanthropic organization supporting research and research dissemination concerning health issues facing the United States. The foundation works to build a culture of health across systems of care (not only medical care) and communities (https://www.rwjf.org).

While many of these have a great deal of helpful evidence to share, they also may have a strong political bias. Objectivity is often lacking in what information these organizations provide: they provide evidence to support certain points of view. That is their purpose—to provide ideas on specific problems, many of which have a political component. Think tanks “are constantly researching solutions to a variety of the world’s problems, and arguing, advocating, and lobbying for policy changes at local, state, and federal levels” (quoted from https://thebestschools.org/features/most-influential-think-tanks/ ). Helpful information about what this one source identified as the 50 most influential U.S. think tanks includes identifying each think tank’s political orientation. For example, The Heritage Foundation is identified as conservative, whereas Human Rights Watch is identified as liberal.

While not the same as think tanks, many mission-driven organizations also sponsor or report on research, as well. For example, the National Association for Children of Alcoholics (NACOA) in the United States is a registered nonprofit organization. Its mission, along with other partnering organizations, private-sector groups, and federal agencies, is to promote policy and program development in research, prevention and treatment to provide information to, for, and about children of alcoholics (of all ages). Based on this mission, the organization supports knowledge development and information gathering on the topic and disseminates information that serves the needs of this population. While this is a worthwhile mission, there is no guarantee that the information meets the criteria for evidence with which we have been working. Evidence reported by think tank and mission-driven sources must be utilized with a great deal of caution and critical analysis!

In many instances an empirical report has not appeared in the published literature, but in the form of a technical or final report to the agency or program providing the funding for the research that was conducted. One such example is presented by a team of investigators funded by the National Institute of Justice to evaluate a program for training professionals to collect strong forensic evidence in instances of sexual assault (Patterson, Resko, Pierce-Weeks, & Campbell, 2014): https://www.ncjrs.gov/pdffiles1/nij/grants/247081.pdf . Investigators may serve in the capacity of consultant to agencies, programs, or institutions, and provide empirical evidence to inform activities and planning. One such example is presented by Maguire-Jack (2014) as a report to a state’s child maltreatment prevention board: https://preventionboard.wi.gov/Documents/InvestmentInPreventionPrograming_Final.pdf .

When Direct Answers to Questions Cannot Be Found. Sometimes social workers are interested in finding answers to complex questions or questions related to an emerging, not-yet-understood topic. This does not mean giving up on empirical literature. Instead, it requires a bit of creativity in approaching the literature. A Venn diagram might help explain this process. Consider a scenario where a social worker wishes to locate literature to answer a question concerning issues of intersectionality. Intersectionality is a social justice term applied to situations where multiple categorizations or classifications come together to create overlapping, interconnected, or multiplied disadvantage. For example, women with a substance use disorder and who have been incarcerated face a triple threat in terms of successful treatment for a substance use disorder: intersectionality exists between being a woman, having a substance use disorder, and having been in jail or prison. After searching the literature, little or no empirical evidence might have been located on this specific triple-threat topic. Instead, the social worker will need to seek literature on each of the threats individually, and possibly will find literature on pairs of topics (see Figure 3-1). There exists some literature about women’s outcomes for treatment of a substance use disorder (a), some literature about women during and following incarceration (b), and some literature about substance use disorders and incarceration (c). Despite not having a direct line on the center of the intersecting spheres of literature (d), the social worker can develop at least a partial picture based on the overlapping literatures.

Figure 3-1. Venn diagram of intersecting literature sets.

empirical framework in literature review

Take a moment to complete the following activity. For each statement about empirical literature, decide if it is true or false.

Social Work 3401 Coursebook Copyright © by Dr. Audrey Begun is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

Book cover

Mutual Sustainability of Tubewell Farming and Aquifers pp 15–36 Cite as

Conceptual Framework and Literature Review

  • Ahmad Saeed Khattak 3  
  • First Online: 01 January 2014

1561 Accesses

Part of the Advances in Asian Human-Environmental Research book series (AAHER)

While ‘conceptual framework’ means a researcher’s own perceptions about the scope and structure of a problem, the literature review provides others’ ideas and work in areas close to that under study. With such a philosophy in mind, this chapter first constructs the author’s own thinking as to how the problem in question has originated, disseminated, and evolved; and how it can be resolved for sustainability if turned around. It also draws support from other like-minded models proposed in the fields of sustainable development. Next, the literature review thoroughly, but not exhaustively, scans the depth and breadth of the existing body of knowledge in the areas of sustainable agriculture, irrigation, and water resources, especially groundwater management. This chapter serves to elucidate the concept of sustainable development. It indicates the measurable indicators of sustainability in agriculture and irrigation systems, thus paving the way for designing the data tools at a subsequent stage. With the intention of giving due proportionate space to every significant prong of the aqua-agro issue, the whole conceptual discussion is organized under three sub-topics: (1) the general concepts of sustainability and sustainable development; (2) the focused area of sustainable agricultural development; and (3) the sustainable use of groundwater as an irrigation resource. To highlight the gap in indigenous research literature, the literature review is also organized in two levels of literature: international and national.

  • Sustainability
  • Sustainable development
  • Groundwater management
  • Over-exploitation
  • Efficient irrigation

This is a preview of subscription content, log in via an institution .

Buying options

  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
  • Available as EPUB and PDF
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
  • Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Achakzai GN, Toor AS (1989) Patterns of management in the Karez system of Balochistan. Pakistan Council of Research in Water Resources, Islamabad, pp 70–78

Google Scholar  

Adams B, Macdonald A (1995) Over-exploited aquifers. Final report, British Geological Survey, technical report WC/95/3, p 53

Alcamo J, Kreileman GJJ, Krol MS et al (1994a) Modeling the global society–biosphere–climate system: part 1: model description and testing. National Institute of Public Health and Environmental Protection, Bilthoven, Accessed at: http://sedac.ciesin.org/mva/KLUWER1994/AlcamoJosephed.1994.IMAGE2.0.html . Accessed 23 Sep 2012

Alcamo J, Van Den Born GJ, Bouwman AF et al (1994b) Modeling the global society–biosphere–climate system: part 2: computed scenarios. National Institute of Public Health and Environmental Protection, Bilthoven, Accessed at: http://sedac.ciesin.org/mva/KLUWER1994/image-2.0-kluwer-ch2.html . Accessed 23 Sep 2012

Ali A (2003) Water conservation in New Era. In: Water and wastewater perspectives of developing countries, Conference proceedings. Anamaya Publishers, New Delhi

Alley WM, Leake SA (2004) The journey from safe-yield to sustainability. Groundw J 42(1). National Groundwater Association, USA. Accessed at: http://info.ngwa.org/gwol/pdf/040678070.pdf . Accessed 24 Jul 2013

Alley WM, Reilly TE, Frank OL (1999) Sustainability of groundwater resources. U.S. Geological Survey, Circular 1186, Denver, CO

American Society of Agronomy (1989) Decision reached on sustainable agriculture. Agronomy News. Madison, Wisconsin, January 1989, p 15

Costanza R, Patten BC (1995) Defining and predicting sustainability. J Ecol Econ 15(3):193–196, Elsevier. Available at: http://www.sciencedirect.com/science/article/pii/0921800995000488 . 25/7/2013

Article   Google Scholar  

Custodio E (2002) Aquifer overexploitation: what does it mean? Geological survey of Spain. Hydrogeol J 10(2). Madrid. Accessed at: http://www.yemenwater.org/wp-content/uploads/2013/03/Custodio-Emilio-2002.pdf . Accessed 15 Dec 2010

Custodio E, Kretsinger V, Llamas MR (2004) Intensive development of groundwater: concept, facts and suggestions. Water Policy J 7. Accessed at: http://www.rac.es/ficheros/doc/00248.pdf . Accessed 15 Dec 2010

El Moujabber M (undated) Sustainable agriculture. Accessed at: http://www.sowamed.ird.fr/resource/RES310_WP6_IntroductionSustainableAgriculture.pdf . Accessed 15 May 2009

Evans M, Cant G (1981) The effect of irrigation on farm production and rural settlement in Mid-Canturbury: a comparison of the irrigated and dryland farming zones in the Lyndurst-Pendarves area, 1945–1946. J N Z Geogr 37(2):58–66, New Zealand

FAO (1989) Arid zone forestry. Conservation guide no. 20. Rome

FAO (1995) Survey sheet, land and water division (AQUASTAT). Accessed at: http://www.fao.org/docrep/015/i2490e/i2490e04b.pdf . Accessed 26 Jul 2013

Frank B, Indu JA (2005) Harvesting opportunities: rural development in the 21st century. Concept Papers, Fourth Regional Thematic Forum, World Bank

Gallopin GC (1997) Indicators and their use: information for decision-making. In: Billhartz SB, Matravers R (eds) A report on the project on indicators of sustainable development. Wiley, Chichester, pp 13–27

Gills VH, Baig MS (1992) Environmental profile of Balochistan. Land Resources and Urban Science Department/ Ecology Unit, Soil Survey of Pakistan, Netherlands/Lahore

GoB (2004) Integrated water resources management policy, Balochistan, Pakistan: final draft report, Component # 3, Balochistan Resource Management Programme

Hardi P, Zidan T (1997) Assessing sustainable development: principles in practice. International Institute for Sustainable Development, Winnipeg

Hassan Q, Garg NK (2003) Management of water resources: an integrated perspective. Printed in Water and Wastewater Perspectives of Developing Countries, conference proceedings, Devi R, Ahsan N (eds) . Anamaya Publishers, New Delhi

Kettle N, Harrington LMB, Harrington JA (2007) Groundwater depletion and agricultural landuse change in the high plains: a case study from Wichita county, Kansas. Prof Geogr J 59(2):221–235, Accessed at: http://krex.k-state.edu/dspace/bitstream/handle/2097/4947/KettlePG2007.pdf?sequence=1 . Accessed 18 Jun 2013

Khan MFK, Nawaz M (1990) Irrigation and landuse in the Kech Valley, Makran. Pakistan Council of Research in Water Resources, Islamabad, unpublished

Leffelaar PA (1999) On systems analysis and simulation of ecological processes; with examples in CSMP, FST and FORTRAN, vol 4, Current issues in production ecology. Kluwer Academic Publishers, New York, p 318

Book   Google Scholar  

MacRae RJ, Hill SB, Hennings J, Bentley AJ (1990) Policies, programs and regulations to support the transition to sustainable agriculture in Canada. Cambridge University Press, Cambridge. Am J Altern Agri (now known as Renewable Agriculture and Food System) 5(2)

Malafant KWJ, Fordham DP (1997) Integration frameworks in agricultural and resource planning and management. In: Munro RK, Leslie LM (eds) Climate prediction for agricultural and resource management. Australian Academy of Science, conference, Canberra, Bureau of Resource Sciences, Canberra, Australia. Accessed at: http://www.complexia.com.au/Documents/AAS_Paper/AAS_Pape.html . Accessed 20 July 2013

Marsh D (undated) Estimating the price of water in Pondicherry, India: a framework for policy analysis. Department of Economics, University of Waikato, Hamilton, New Zealand. Accessed at: http://www.mssanz.org.au/MODSIM99/Vol%202/Marsh.pdf . Accessed 28 Jun 2009

Meadows D (1998) Indicators and information systems for sustainable development. The Sustainability Institute, Hartland, Accessed at: http://www.iisd.org/pdf/s_ind_2.pdf . Accessed 12 July 2013

Morton J, Hoeflaken VH (1995) Some findings from a survey of flood irrigation schemes in Balochistan, Pakistan. Water Resour J. Economic and Social Commission for Asia and the Pacific, UNO

Mosher AT (undated) World food problems and prospects. http://ageconsearch.umn.edu/bitstream/17572/1/ar670021.pdf . Accessed 13 July 2008

Nawaz K (2004) Community participation and water management in Balochistan, Pakistan. http://archive.unu.edu/env/workshops/Aleppo/06%20-%20Nawaz.doc . Accessed 16 May 2008

Nikhel K, Sundararjan M, Misra DD (2003) Integrated water resource management for the Jhakrand State: a conceptual model, Printed in Water and wastewater perspectives of developing countries, Conference Proceedings, Devi R, Ahsan N (eds). Anamaya Publishers, New Delhi

Pennell JD (2006) Dryland salinity: economic, scientific, social and, policy dimensions. Aust J Agric Resour Econ 4(45):517–546

Ponce VM (2006) Groundwater utilisation and sustainability. Accessed at: http://groundwater.sdsu.edu/ . Accessed 24 Jul 2013

Ponce VM (2007) Sustainable yield of groundwater. Accessed at: http://ponce.sdsu.edu/groundwater_sustainable_yield.html . Accessed 24 Jul 2013

Postel SL (2003) Securing water for people, crops, and ecosystems: new mindset and new priorities. Nat Res Forum 27(2). Blackwell Publishing, Oxford. Accessed at: http://www.aseanenvironment.info/Abstract/41015659.pdf . Accessed 8 Jun 2010

Prasad Narasimha NB, James EJ (2003) Impact of an irrigation project on groundwater regime within a command area – a case study. Printed in Water and Wastewater Perspectives of Developing Countries, conference proceedings, Devi R, Ahsan N (eds). Anamaya Publishers, New Delhi

Pretty JN (1998) Toward more conducive policies for sustainable agriculture. Published in Agriculture and the Environment: Perspectives on Sustainable Rural Development, A World Bank symposium, p 37

Priyan K (2003) Emerging issues in water management and strategies for water security in Gujrat, India. Printed in Water and Wastewater Perspectives of Developing Countries, Conference Proceedings, Devi R, Ahsan N (eds). Anamaya Publishers, New Delhi

Reis S (2008) Analyzing land use/land cover changes using remote sensing and GIS in Rize, north-east Turkey. Sensors—Open Access Journal, published online by MDPI AG, Switzerland. Available at: http://www.mdpi.com/1424-8220/8/10/6188/pdf

Rosegrant MW, Cai X (2002) Water constraints and environmental impacts of agricultural growth. Am J Agric Econ 3(84):832–838, Oxford University Press, USA

Rosenzweig C, Hillel D (1995) Potential impacts of climate change on agriculture and world food supply. Consequences 1(2):23–32

Sadler BS, Cox WE (1987) Water resources management: the socio-political context. Water Res J Econ Soc Comm Asia Pac. UNO

Saeed M (Dr.) (2006) Promising crops and water efficient cropping patterns for irrigated farming systems of Balochistan, Pakistan. Summary report, TA-4560 (PAK), Project for Supporting Implementation of IWRM Policy in Balochistan. Govt. of Balochistan, ADB, and Royal Netherlands Govt

Saeed A (2012) Inter-effects of tubewell irrigated agriculture and the aquifer potential in Balochistan: a case study of Pishin Valley, 1981–2008. Ph.D. thesis, University of the Punjab, Lahore, Pakistan

Scanlon BR, Reedy RC, Stonestrom DA, Prudic DE, Dennehy KF (2005) Impact of land use and land cover change on groundwater recharge and quality in the southwestern US. Glob Chang Biol 11:1577–1593, Blackwell Publishing Ltd

Schoute JFT, Finke PA, Veeneklass FR, Wolfert HP (1995) Scenario studies for the rural environment. Kluwer Academic Publishers, Dordrecht

Sen PK (2004) Diagnosing irrigation water resources with multi-sensor remote sensing and GIS techniques: a case study of the Roxo Dame irrigation system, Portugal. M.Sc. Thesis, ITC, Enschede, The Netherlands

Sophocleous MA (1997) Managing water resources systems: why “safe yield” is not sustainable. J Ground Water 35(4):561, USA. Editorial available at: http://ponce.sdsu.edu/sophocleous_gw1997.pdf , 24/7/2013

Sophocleous MA (2000) From safe yield to sustainable development of water resources – the Kansas experience. J Hydrol 235(1–2):27–43, Elsevier, USA. Available at: http://people.ucalgary.ca/~hayashi/geog515/reading/Sophocleous2000.pdf , 24/7/2013

Spangenberg JH (2002) Environmental space and the prism of sustainability: frameworks for indicators measuring sustainable development. J Ecol Indic 2(3):295–309, Elsevier

Swart RJ, Raskin P, Robinson J (2004) The problem of the future: sustainability science and scenario analysis. J Glob Environ Chang Part A 14(2):137–146, Elsevier

Syed MA (1991) The diffusion of tube-well technology and karez abandonment in Balochistan. Selected papers, Agricultural Economy of Balochistan, Quetta Printing Press, Quetta

US Government (1990) Food, agriculture, conservation, and trade act of 1990, Public law 101-624. Title XVI, Subtitle A, Section 1603. Washington, DC

Van Ittersum MK, Rabbinge R, Van Latesteijn HC (1998) Exploratory landuse studies and their role in strategic policy making. J Agric Syst 58(3):309–330, Elsevier

Van Steenbergen F, Oliemans W (undated) Groundwater resource management in Pakistan. Accessed at: http://www.groundwatermanagement.org/documents/14locgrounwregulationpraxis.pdf . Accessed 12 Mar 2008

WCED (1987) Our common future: the Brundtland Report, World Commission on Environment and Development, UN, Oxford University Press

Wilson AJ, Tyrchniewicz A (1995) Agriculture and sustainable development: policy analysis on the great plains. International Institute for Sustainable Development, Winnipeg, Accessed at: http://www.iisd.org/pdf/agri_sd.pdf . Accessed 24 Nov 2009

Wolf P (1995) The problem of the sustainability of irrigation systems. Institute for Scientific Co-operation, Tubengen, J App Geogr Dev 45/46

Zandstra (1993) Preserving the options: International research for sustainable agriculture. In: Agriculture and environmental challenges. Proceedings of the 13th agriculture sector symposium, Srivastava and Alderman (eds). World Bank, Washington, DC, pp 101–107

Download references

Author information

Authors and affiliations.

Department of Geography, University of Balochistan, Quetta, Pakistan

Ahmad Saeed Khattak

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter.

Khattak, A.S. (2014). Conceptual Framework and Literature Review. In: Mutual Sustainability of Tubewell Farming and Aquifers. Advances in Asian Human-Environmental Research. Springer, Cham. https://doi.org/10.1007/978-3-319-02804-0_2

Download citation

DOI : https://doi.org/10.1007/978-3-319-02804-0_2

Published : 31 May 2014

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-02803-3

Online ISBN : 978-3-319-02804-0

eBook Packages : Earth and Environmental Science Earth and Environmental Science (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research
  • Advertise with us
  • Friday, March 08, 2024

Most Widely Read Newspaper

PunchNG Menu:

  • Special Features
  • Sex & Relationship

ID) . '?utm_source=news-flash&utm_medium=web"> Download Punch Lite App

Project Chapter Two: Literature Review and Steps to Writing Empirical Review

Writing an Empirical Review

Kindly share this story:

  • Conceptual review
  • Theoretical review,
  • Empirical review or review of empirical works of literature/studies, and lastly
  • Conclusion or Summary of the literature reviewed.
  • Decide on a topic
  • Highlight the studies/literature that you will review in the empirical review
  • Analyze the works of literature separately.
  • Summarize the literature in table or concept map format.
  • Synthesize the literature and then proceed to write your empirical review.

All rights reserved. This material, and other digital content on this website, may not be reproduced, published, broadcast, rewritten or redistributed in whole or in part without prior express written permission from PUNCH.

Contact: [email protected]

Stay informed and ahead of the curve! Follow The Punch Newspaper on WhatsApp for real-time updates, breaking news, and exclusive content. Don't miss a headline – join now!

Don't go and give your crypto to Pablo o. Use KOYN for a fast, secure and reliable transaction. Get started now Skip the Risk, Go with KOYN! Fast, safe, and reliable crypto trades. Start with KOYN! now.

BREAKING NEWS: Nigerians can now earn Dollars by flipping domain names, buy as low as $12 and sell for $1000-$5000. 100% practical guide/training available. Click here to start earning Dollars

ONLINE FOREX, BITCOIN, BINARY AND PROP FIRM CHALLENGE TRADING EXPERT.

Do you require the service of a tested, trusted and experienced trader, who can help you pass your Prop firm challenges and profitably trade your Forex, Bitcoin and Binary accounts for minimum of 10-20% weekly profit?. Click here for details; WhatsApp: +2348030797998

Nigerians can now work and earn US Dollars, acquire premium domains for as low as $1500, earn a profit up to ₦24 million,($17,000), click here to start now.

Experience the Thrill of Casino Gaming! Play Now at Mozzartbet.ng! Your Ticket to Unlimited Entertainment and Steading Winnings! Explore Mozzartbet.ng Casino Now!

BREAKING NEWS: 🚨 Could today be the day you change your life forever? 🌠 Dive into the chance to win a whopping $525 Million* in the US Megamillion Jackpot! 🎉💰 Imagine the possibilities! 🌍✨ Don't miss out! Join the excitement at 24Lottos.com - TAP HERE! 🚀🎯

Follow Punch on Whatsapp

Latest News

Afcfta: nigeria commences export to s’africa, others in april, n4bn fraud: anambra govt opposes obiano’s trial by efcc, $6.2m fraud: forensic expert confirms forgery of buhari’s signature, protesting residents warn against shaibu’s impeachment, reps demand rescue of 200 abducted borno women.

airtel-tenency-ad

Minimum wage: Workers differ, govs want revenue sharing reviewed

Lorem ipsum dolor sit amet, conse adipiscing elit.

pastor-adebayo

  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

  • Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
  • Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
  • Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

  • << Previous: Theoretical Framework
  • Next: Citation Tracking >>
  • Last Updated: Mar 6, 2024 12:59 PM
  • URL: https://libguides.usc.edu/writingguide

Penn State University Libraries

Empirical research in the social sciences and education.

  • What is Empirical Research and How to Read It
  • Finding Empirical Research in Library Databases
  • Designing Empirical Research
  • Ethics, Cultural Responsiveness, and Anti-Racism in Research
  • Citing, Writing, and Presenting Your Work

Contact the Librarian at your campus for more help!

Ellysa Cahoy

Introduction: What is Empirical Research?

Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief. 

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."  Ask yourself: Could I recreate this study and test these results?

Key characteristics to look for:

  • Specific research questions to be answered
  • Definition of the population, behavior, or   phenomena being studied
  • Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components:

  • Introduction : sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
  • Methodology: sometimes called "research design" -- how to recreate the study -- usually describes the population, research process, and analytical tools used in the present study
  • Results : sometimes called "findings" -- what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
  • Discussion : sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies

Reading and Evaluating Scholarly Materials

Reading research can be a challenge. However, the tutorials and videos below can help. They explain what scholarly articles look like, how to read them, and how to evaluate them:

  • CRAAP Checklist A frequently-used checklist that helps you examine the currency, relevance, authority, accuracy, and purpose of an information source.
  • IF I APPLY A newer model of evaluating sources which encourages you to think about your own biases as a reader, as well as concerns about the item you are reading.
  • Credo Video: How to Read Scholarly Materials (4 min.)
  • Credo Tutorial: How to Read Scholarly Materials
  • Credo Tutorial: Evaluating Information
  • Credo Video: Evaluating Statistics (4 min.)
  • Next: Finding Empirical Research in Library Databases >>
  • Last Updated: Feb 18, 2024 8:33 PM
  • URL: https://guides.libraries.psu.edu/emp

IMAGES

  1. Summary of empirical literature review

    empirical framework in literature review

  2. empirical study literature review

    empirical framework in literature review

  3. How to empirically review the literature?

    empirical framework in literature review

  4. Empirical Research: Definition, Methods, Types and Examples

    empirical framework in literature review

  5. 15 Empirical Evidence Examples (2024)

    empirical framework in literature review

  6. Notable Differences between Empirical Review and Literature Review

    empirical framework in literature review

VIDEO

  1. Methods L04

  2. Effective Review of Literature

  3. Literature review and its process

  4. Empirical Review (see attached video for more info.) #researchproject #literaturereview @GERHub4ALL

  5. Theoretical framework of theory, literature review, and academic rhetoric

  6. Background, Literature Review, and Theoretical Framework -- Sarah Lynne Bowman

COMMENTS

  1. Writing a Literature Review

    A literature review is a document or section of a document that collects key sources on a topic and discusses ... they're interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. ... Empirical versus theoretical scholarship; Divide the research ...

  2. How to Write a Literature Review

    Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.

  3. (PDF) Literature Reviews, Conceptual Frameworks, and Theoretical

    review or framework within an empirical study ... The purpose of this article is to conduct a semi-systematic framework-based literature review of the 200 most cited articles in public diplomacy ...

  4. Literature Reviews, Theoretical Frameworks, and Conceptual Frameworks

    While a literature review describes what is known and not known about the phenomenon, the conceptual framework leverages these gaps in describing the current study (Maxwell, 2012). In the example of Sabel et al. (2017) , the authors indicated there was a gap in the literature regarding how scaffolds engage students in metacognition to promote ...

  5. PDF Writing the literature review for empirical papers

    3. The literature review in an empirical paper In this section we discuss the literature review as a part of an empirical article. It plays the fundamental role of unveiling the theory, or theories, that underpin the paper argument, or, if there are no such theoretical background, which is the related extant knowledge.

  6. Guidance on Conducting a Systematic Literature Review

    This article is organized as follows: The next section presents the methodology adopted by this research, followed by a section that discusses the typology of literature reviews and provides empirical examples; the subsequent section summarizes the process of literature review; and the last section concludes the paper with suggestions on how to improve the quality and rigor of literature ...

  7. Writing the literature review for empirical papers

    A literature review uses ideas in the literature to define and develop a particular approach to a topic or method to provide something new [16]. The literature review begins with the topic's ...

  8. Literature review as a research methodology: An ...

    By integrating findings and perspectives from many empirical findings, a literature review can address research questions with a power that no single study has. ... Employee volunteering: A review and framework for future research. Journal of Management, 42 (2016), pp. 55-84, 10.1177/0149206315614374. View in Scopus Google Scholar.

  9. PDF Conceptualizing the Pathways of Literature Review in Research

    of theoretical and empirical literature, implication of the review, and theoretical and/or conceptual framework/s. Its implication is that any research work needs to pave its pathways distinctly for its successful execution. Keywords: Concept, Framework, Review, Roadmap, Theory 1. Introduction Conceptualizing literature review (LR) is at the ...

  10. Chapter 9 Methods for Literature Reviews

    9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.

  11. Writing the literature review for empirical papers

    Paper aims: The purpose of the paper is to offer guidance regarding how to write a Literature Review for empirical papers, that provides adequate background and convincing support. The literature review plays the fundamental role of unveiling the theory, or theories, that underpin the paper argument, sets its limits, and defines and clarifies the main concepts that will be used in the ...

  12. A scoping review of frameworks in empirical studies and a review of

    Scoping review of the literature. We conducted a scoping review to identify dissemination frameworks used in the empirical dissemination literature. A scoping review is appropriate as the goal of this work is to map the current state of the literature, not to evaluate evidence or provide specific recommendations as is the case with a systematic ...

  13. How To Structure A Literature Review (Free Template)

    Demonstrate your knowledge of the research topic. Identify the gaps in the literature and show how your research links to these. Provide the foundation for your conceptual framework (if you have one) Inform your own methodology and research design. To achieve this, your literature review needs a well-thought-out structure.

  14. PDF LITERATURE REVIEWS

    2. MOTIVATE YOUR RESEARCH in addition to providing useful information about your topic, your literature review must tell a story about how your project relates to existing literature. popular literature review narratives include: ¡ plugging a gap / filling a hole within an incomplete literature ¡ building a bridge between two "siloed" literatures, putting literatures "in conversation"

  15. Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks

    This essay starts with a discussion of the literature review, theoretical framework, and conceptual framework as components of a manuscript. This discussion includes similarities and distinctions among these components and their relation to other sections of a manuscript such as the problem statement, discussion, and implications.

  16. What is Empirical Research?

    Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components: Introduction: sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies

  17. Module 2 Chapter 3: What is Empirical Literature & Where can it be

    the introductory literature review in an empirical article; textbooks; You may be surprised to see the last two included in this list. Like the other sources of information listed, these sources also might lead you to look for evidence. But, they are not themselves sources of evidence. They may summarize existing evidence, but in the process of ...

  18. Conceptual Framework and Literature Review

    Abstract. While 'conceptual framework' means a researcher's own perceptions about the scope and structure of a problem, the literature review provides others' ideas and work in areas close to that under study. With such a philosophy in mind, this chapter first constructs the author's own thinking as to how the problem in question has ...

  19. Difference between theoretical literature review and empirical

    Theoretical literature review focuses on the existing theories, models and concepts that are relevant to a research topic. It does not collect or analyze primary data, but rather synthesizes and ...

  20. Project Chapter Two: Literature Review and Steps to Writing Empirical

    An empirical literature review process involves the evaluation of previous empirical studies to bring to rest a specific research issue. When conducting an empirical review, the researcher ...

  21. 5. The Literature Review

    A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...

  22. PDF Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks

    first section will discuss literature review, theoretical framework, and conceptual framework as components of a manuscript. The second section will discuss literature review, theoretical framework, and conceptual framework as types of manuscripts. The first area to be discussed is that these terms are used interchangeably.

  23. Empirical Research in the Social Sciences and Education

    Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components: Introduction: sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies