what are the five sources of literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Prevent plagiarism. Run a free check.

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved March 22, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Literature review sources

Sources for literature review can be divided into three categories as illustrated in table below. In your dissertation you will need to use all three categories of literature review sources:

Sources for literature review and examples

Generally, your literature review should integrate a wide range of sources such as:

• Books . Textbooks remain as the most important source to find models and theories related to the research area. Research the most respected authorities in your selected research area and find the latest editions of books authored by them. For example, in the area of marketing the most notable authors include Philip Kotler, Seth Godin, Malcolm Gladwell, Emanuel Rosen and others.
• Magazines . Industry-specific magazines are usually rich in scholarly articles and they can be effective source to learn about the latest trends and developments in the research area. Reading industry magazines can be the most enjoyable part of the literature review, assuming that your selected research area represents an area of your personal and professional interests, which should be the case anyways.
• Newspapers can be referred to as the main source of up-to-date news about the latest events related to the research area. However, the proportion of the use of newspapers in literature review is recommended to be less compared to alternative sources of secondary data such as books and magazines. This is due to the fact that newspaper articles mainly lack depth of analyses and discussions.
• Online articles . You can find online versions of all of the above sources. However, note that the levels of reliability of online articles can be highly compromised depending on the source due to the high levels of ease with which articles can be published online. Opinions offered in a wide range of online discussion blogs cannot be usually used in literature review. Similarly, dissertation assessors are not keen to appreciate references to a wide range of blogs, unless articles in these blogs are authored by respected authorities in the research area.

Your secondary data sources may comprise certain amount of grey literature as well. The term grey literature refers to type of literature produced by government, academics, business and industry in print and electronic formats, which is not controlled by commercial publishers. It is called ‘grey’ because the status of the information in grey literature is not certain. In other words, any publication that has not been peer reviewed for publication is grey literature.

The necessity to use grey literature arises when there is no enough peer reviewed publications are available for the subject of your study.

John Dudovskiy

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Literature Review: Lit Review Sources

• Lit Review Types
• GRADE System
• Do a Lit Review
• Citation Justice
• Lit Review Sources

Where do I find information for a literature review?

Research is done by...

...by way of...

...communicated through...

...and organized in...

Types of sources for a review...

• Primary source: Usually a report by the original researchers of a study (unfiltered sources)
• Secondary source: Description or summary by somebody other than the original researcher, e.g. a review article (filtered sources)
• Conceptual/theoretical: Papers concerned with description or analysis of theories or concepts associated with the topic
• Anecdotal/opinion/clinical: Views or opinions about the subject that are not research, review or theoretical (case studies or reports from clinical settings)

A Heirarchy of research information:

Source: SUNY Downstate Medical Center. Medical Research Library of Brooklyn. Evidence Based Medicine Course. A Guide to Research Methods: The Evidence Pyramid: http://library.downstate.edu/EBM2/2100.htm

Life Cycle of Publication

Click image to enlarge

Scientific information has a ‘life cycle’ of its own… it is born as an idea, and then matures and becomes more available to the public. First it appears within the so-called ‘invisible college’ of experts in the field, discussed at conferences and symposia or posted as pre-prints for comments and corrections. Then it appears in the published literature (the primary literature), often as a journal article in a peer-reviewed journal.

Researchers can use the indexing and alerting services of the secondary literature to find out what has been published in a field. Depending on how much information is added by the indexer or abstracter, this may take a few months (though electronic publication has sped up this process). Finally, the information may appear in more popular or reference sources, sometimes called the tertiary literature.

The person beginning a literature search may take this process in reverse: using tertiary sources for general background, then going to the secondary literature to survey what has been published, following up by finding the original (primary) sources, and generating their own research Idea.

(Original content by Wade Lee-Smith)

• << Previous: Citation Justice
• Next: Readings >>
• Last Updated: Feb 16, 2024 8:43 AM
• URL: https://libguides.utoledo.edu/litreview

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Mar 21, 2024 9:59 AM
• URL: https://libguides.usc.edu/writingguide

• Library Guides

The Literature Review

Primary and secondary sources, the literature review: primary and secondary sources.

• Searching the literature
• Grey literature
• Organising and analysing
• Systematic Reviews
• The Literature Review Toolbox

On this page

• Primary vs secondary sources: The differences explained 

Can something be both a primary and secondary source?

Research for your literature review can be categorised as either primary or secondary in nature. The simplest definition of primary sources is either original information (such as survey data) or a first person account of an event (such as an interview transcript). Whereas secondary sources are any publshed or unpublished works that describe, summarise, analyse, evaluate, interpret or review primary source materials. Secondary sources can incorporate primary sources to support their arguments.

Ideally, good research should use a combination of both primary and secondary sources. For example, if a researcher were to investigate the introduction of a law and the impacts it had on a community, he/she might look at the transcripts of the parliamentary debates as well as the parliamentary commentary and news reporting surrounding the laws at the time. 

Examples of primary and secondary sources

Primary vs secondary sources: The differences explained

Finding primary sources

• VU Special Collections  - The Special Collections at Victoria University Library are a valuable research resource. The Collections have strong threads of radical literature, particularly Australian Communist literature, much of which is rare or unique. Women and urban planning also feature across the Collections. There are collections that give you a picture of the people who donated them like Ray Verrills, John McLaren, Sir Zelman Cowen, and Ruth & Maurie Crow. Other collections focus on Australia's neighbours – PNG and Timor-Leste.
• POLICY - Sharing the latest in policy knowledge and evidence, this database supports enhanced learning, collaboration and contribution.
• Indigenous Australia  -  The Indigenous Australia database represents the collections of the Aboriginal and Torres Strait Islander Commission Library.
• Australian Heritage Bibliography - Aboriginal and Torres Strait Islander Subset (AHB-ATSIS)  - AHB is a bibliographic database that indexes and abstracts articles from published and unpublished material on Australia's natural and cultural environment. The AHB-ATSIS subset contains records that specifically relate to the Aboriginal and Torres Strait Islander peoples.include journal articles, unpublished reports, books, videos and conference proceedings from many different sources around Australia. Emphasis is placed on reports written or commissioned by government and non-government heritage agencies throughout the country.
• ATSIhealth  - The Aboriginal and Torres Strait Islander Health Bibliography (ATSIhealth), compiled by Neil Thomson and Natalie Weissofner at the School of Indigenous Australian Studies, Kurongkurl Katitjin, Edith Cowan University, is a bibliographic database that indexes published and unpublished material on Australian Indigenous health. Source documents include theses, unpublished articles, government reports, conference papers, abstracts, book chapters, books, discussion and working papers, and statistical documents. 
• National Archive of Australia  - The National Archives of Australia holds the memory of our nation and keeps vital Australian Government records safe. 
• National Library of Australia: Manuscripts  - Manuscripts collection that is wide ranging and provides rich evidence of the lives and activities of Australians who have shaped our society.
• National Library of Australia: Printed ephemera  - The National Library has been selectively collecting Australian printed ephemera since the early 1960s as a record of Australian life and social customs, popular culture, national events, and issues of national concern.
• National Library of Australia: Oral history and folklore - The Library’s Oral History and Folklore Collection dates back to the 1950’s and includes a rich and diverse collection of interviews and recordings with Australians from all walks of life.
• Historic Hansard - Commonwealth of Australia parliamentary debates presented in an easy-to-read format for historians and other lovers of political speech.
• The Old Bailey Online - A fully searchable edition of the largest body of texts detailing the lives of non-elite people ever published, containing 197,745 criminal trials held at London's central criminal court.

Whether or not a source can be considered both primary and  secondary, depends on the context. In some instances, material may act as a secondary source for one research area, and as a primary source for another. For example, Niccolò Machiavelli’s The Prince , published in 1513, is an important secondary source for any study of the various Renaissance princes in the Medici family; but the same book is also a primary source for the political thought that was characteristic of the sixteenth century because it reflects the attitudes of a person living in the 1500s.

Source: Craver, 1999, as cited in University of South Australia Library. (2021, Oct 6).  Can something be a primary and secondary source?.  University of South Australia Library. https://guides.library.unisa.edu.au/historycultural/sourcetypes

• << Previous: Overview
• Next: Searching the literature >>
• Last Updated: Feb 21, 2024 3:27 PM
• URL: https://libraryguides.vu.edu.au/the-literature-review

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 5: The Literature Review

5.3 Acceptable sources for literature reviews

Following are a few acceptable sources for literature reviews, listed in order from what will be considered most acceptable to less acceptable sources for your literature review assignments:

• Peer reviewed journal articles.
• Edited academic books.
• Articles in professional journals.
• Statistical data from government websites.
• Website material from professional associations (use sparingly and carefully). The following sections will explain and provide examples of these various sources.

Peer reviewed journal articles (papers)

A peer reviewed journal article is a paper that has been submitted to a scholarly journal, accepted, and published. Peer review journal papers go through a rigorous, blind review process of peer review. What this means is that two to three experts in the area of research featured in the paper have reviewed and accepted the paper for publication. The names of the author(s) who are seeking to publish the research have been removed (blind review), so as to minimize any bias towards the authors of the research (albeit, sometimes a savvy reviewer can discern who has done the research based upon previous publications, etc.). This blind review process can be long (often 12 to 18 months) and may involve many back and forth edits on the behalf of the researchers, as they work to address the edits and concerns of the peers who reviewed their paper. Often, reviewers will reject the paper for a variety of reasons, such as unclear or questionable methods, lack of contribution to the field, etc. Because peer reviewed journal articles have gone through a rigorous process of review, they are considered to be the premier source for research. Peer reviewed journal articles should serve as the foundation for your literature review.

The following link will provide more information on peer reviewed journal articles. Make sure you watch the little video on the upper left-hand side of your screen, in addition to reading the material at the following website:    http://guides.lib.jjay.cuny.edu/c.php?g=288333&p=1922599

Edited academic books

An edited academic book is a collection of scholarly scientific papers written by different authors. The works are original papers, not published elsewhere (“Edited volume,” 2018). The papers within the text also go through a process of review; however, the review is often not a blind review because the authors have been invited to contribute to the book. Consequently, edited academic books are fine to use for your literature review, but you also want to ensure that your literature review contains mostly peer reviewed journal papers.

Articles in professional journals

Articles from professional journals should be used with caution for your literature review. This is because articles in trade journals are not usually peer reviewed, even though they may appear to be. A good way to find out is to read the “About Us” section of the professional journal, which should state whether or not the papers are peer reviewed. You can also find out by Googling the name of the journal and adding “peer reviewed” to the search.

Statistical data from governmental websites

Governmental websites can be excellent sources for statistical data, e.g, Statistics Canada collects and publishes data related to the economy, society, and the environment (see https://www.statcan.gc.ca/eng/start ).

Website material from professional associations

Material from other websites can also serve as a source for statistics that you may need for your literature review. Since you want to justify the value of the research that interests you, you might make use of a professional association’s website to learn how many members they have, for example. You might want to demonstrate, as part of the introduction to your literature review, why more research on the topic of PTSD in police officers is important. You could use peer reviewed journal articles to determine the prevalence of PTSD in police officers in Canada in the last ten years, and then use the Ontario Police Officers´ Association website to determine the approximate number of police officers employed in the Province of Ontario over the last ten years. This might help you estimate how many police officers could be suffering with PTSD in Ontario. That number could potentially help to justify a research grant down the road. But again, this type of website- based material should be used with caution and sparingly.

Research Methods for the Social Sciences: An Introduction Copyright © 2020 by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• University of Texas Libraries

Literature Reviews

Steps in the literature review process.

• What is a literature review?
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• You may need to some exploratory searching of the literature to get a sense of scope, to determine whether you need to narrow or broaden your focus
• Identify databases that provide the most relevant sources, and identify relevant terms (controlled vocabularies) to add to your search strategy
• Finalize your research question
• Think about relevant dates, geographies (and languages), methods, and conflicting points of view
• Conduct searches in the published literature via the identified databases
• Check to see if this topic has been covered in other discipline's databases
• Examine the citations of on-point articles for keywords, authors, and previous research (via references) and cited reference searching.
• Save your search results in a citation management tool (such as Zotero, Mendeley or EndNote)
• De-duplicate your search results
• Make sure that you've found the seminal pieces -- they have been cited many times, and their work is considered foundational 
• Check with your professor or a librarian to make sure your search has been comprehensive
• Evaluate the strengths and weaknesses of individual sources and evaluate for bias, methodologies, and thoroughness
• Group your results in to an organizational structure that will support why your research needs to be done, or that provides the answer to your research question  
• Develop your conclusions
• Are there gaps in the literature?
• Where has significant research taken place, and who has done it?
• Is there consensus or debate on this topic?
• Which methodological approaches work best?
• For example: Background, Current Practices, Critics and Proponents, Where/How this study will fit in 
• Organize your citations and focus on your research question and pertinent studies
• Compile your bibliography

Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.

Videos Tutorials about Literature Reviews

This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.

Recommended Reading

• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

• Subject guides
• Researching for your literature review
• Literature sources

Researching for your literature review: Literature sources

• Literature reviews
• Before you start
• Develop a search strategy
• Keyword search activity
• Subject search activity
• Combined keyword and subject searching
• Online tutorials
• Apply search limits
• Run a search in different databases
• Supplementary searching
• Save your searches
• Manage results

Scholarly databases

It's important to make a considered decision as to where to search for your review of the literature. It's uncommon for a disciplinary area to be covered by a single publisher, so searching a single publisher platform or database is unlikely to give you sufficient coverage of studies for a review. A good quality literature review involves searching a number of databases individually.

The most common method is to search a combination of large inter-disciplinary databases such as Scopus & Web of Science Core Collection, and some subject-specific databases (such as PsycInfo or EconLit etc.). The Library databases are an excellent place to start for sources of peer-reviewed journal articles.

Depending on disciplinary expectations, or the topic of our review, you may also need to consider sources or search methods other than database searching. There is general information below on searching grey literature. However, due to the wide varieties of grey literature available, you may need to spend some time investigating sources relevant for your specific need.

Grey literature

Grey literature is information which has been published informally or non-commercially (where the main purpose of the producing body is not commercial publishing) or remains unpublished. One example may be Government publications.

Grey literature may be included in a literature review to minimise publication bias . The quality of grey literature can vary greatly - some may be peer reviewed whereas some may not have been through a traditional editorial process.

See the Grey Literature guide for further information on finding and evaluating grey sources.

See the Moodle book MNHS: Systematically searching the grey literature for a comprehensive module on grey literature for systematic reviews.

In certain disciplines (such as physics) there can be a culture of preprints being made available prior to submissions to journals. There has also been a noticeable rise in preprints in medical and health areas in the wake of Covid-19.

If preprints are relevant for you, you can search preprint servers directly. A workaround might be to utilise a search engine such as Google Scholar to search specifically for preprints, as Google Scholar has timely coverage of most preprint servers including ArXiv, RePec, SSRN, BioRxiv, and MedRxiv.

Articles in Press are not preprints, but are accepted manuscripts that are not yet formally published. Articles in Press have been made available as an early access online version of a paper that may not yet have received its final formatting or an allocation of a volume/issue number. As well as being available on a journal's website, Articles in Press are available in databases such as Scopus and Web of Science, and so (unlike preprints) don't necessarily require a separate search.

Conference papers

Conference papers are typically published in conference proceedings (the collection of papers presented at a conference), and may be found on an organisation or Society's website, as a journal, or as a special issue of journal.

In certain disciplines (such as computer science), conference papers may be highly regarded as a form of scholarly communication; the conferences are highly selective, the papers are generally peer reviewed, and papers are published in proceedings affiliated with high-quality publishing houses.

Conference papers may be indexed in a range of scholarly databases. If you only want to see conference papers, database limits can be used to filter results, or try a specific index such as the examples below:

• Conference proceedings citation index. Social science & humanities (CPCI-SSH)
• Conference proceedings citation index. Science (CPCI-S)
• ASME digital library conference proceedings

Honours students and postgraduates may request conference papers through Interlibrary Loans . However, conference paper requests may take longer than traditional article requests as they can be difficult to locate; they may have been only supplied to attendees or not formally published. Sometimes only the abstract is available.

If you are specifically looking for statistical data, try searching for the keyword statistics in a Google Advanced Search and limiting by a relevant site or domain. Below are some examples of sites, or you can try a domain such as .gov for government websites.

Statistical data can be found in the following selected sources:

• Australian Bureau of Statistics
• World Health Organization: Health Data and statistics
• Higher Education Statistics
• UNESCO Institute for Statistics
• Tourism Australia Statistics

For a list of databases that include statistics see: Databases by Subject: Statistics .

If you are specifically looking for information found in newspapers, the library has a large collection of Australian and overseas newspapers, both current and historical.

To search the full-text of newspapers in electronic format use a database such as  Newsbank.

Alternatively, see the Newspapers subject guide for comprehensive information on newspaper sources available via Monash University library and open source databases, as well as searching tips, online videos and more.

Dissertations and theses

The Monash University Library Theses subject guide provides resources and guidelines for locating and accessing theses (dissertations) produced by Monash University as well as other universities in Australia and internationally.  

International theses:

There are a number of theses databases and repositories.

A popular source is:

• ProQuest Dissertations & Theses Global  which predominantly, covers North American masters and doctoral theses. Full text is available for theses added since 1997. 

Australia and New Zealand theses:

Theses that are available in the library can be found using the  Search catalogue.

These include:

• Monash doctoral, masters and a small number of honours theses 
• other Australian and overseas theses that have been purchased for the collection.

Formats include print (not available for loan), microfiche and online (some may have access restrictions).

Trove includes doctoral, masters and some honours theses from all Australian and New Zealand universities, as well as theses awarded elsewhere but held by Australian institutions.

Tips:  

• Type in the title, author surname and/or keywords. Then on the results page refine your search to 'thesis'.
• Alternatively, use the Advanced search and include 'thesis' as a keyword or limi t your result to format = thesis
• << Previous: Literature reviews
• Next: Before you start >>

Literature review: your definitive guide

Joanna Wilkinson

This is our ultimate guide on how to write a narrative literature review. It forms part of our Research Smarter series . 

How do you write a narrative literature review?

Researchers worldwide are increasingly reliant on literature reviews. That’s because review articles provide you with a broad picture of the field, and help to synthesize published research that’s expanding at a rapid pace .

In some academic fields, researchers publish more literature reviews than original research papers. The graph below shows the substantial growth of narrative literature reviews in the Web of Science™, alongside the percentage increase of reviews when compared to all document types.

It’s critical that researchers across all career levels understand how to produce an objective, critical summary of published research. This is no easy feat, but a necessary one. Professionally constructed literature reviews – whether written by a student in class or an experienced researcher for publication – should aim to add to the literature rather than detract from it.

To help you write a narrative literature review, we’ve put together some top tips in this blog post.

Best practice tips to write a narrative literature review:

• Don’t miss a paper: tips for a thorough topic search
• Identify key papers (and know how to use them)
• Tips for working with co-authors
• Find the right journal for your literature review using actual data
• Discover literature review examples and templates

We’ll also provide an overview of all the products helpful for your next narrative review, including the Web of Science, EndNote™ and Journal Citation Reports™.

1. Don’t miss a paper: tips for a thorough topic search

Once you’ve settled on your research question, coming up with a good set of keywords to find papers on your topic can be daunting. This isn’t surprising. Put simply, if you fail to include a relevant paper when you write a narrative literature review, the omission will probably get picked up by your professor or peer reviewers. The end result will likely be a low mark or an unpublished manuscript, neither of which will do justice to your many months of hard work.

Research databases and search engines are an integral part of any literature search. It’s important you utilize as many options available through your library as possible. This will help you search an entire discipline (as well as across disciplines) for a thorough narrative review.

We provide a short summary of the various databases and search engines in an earlier Research Smarter blog . These include the Web of Science , Science.gov and the Directory of Open Access Journals (DOAJ).

Like what you see? Share it with others on Twitter:

[bctt tweet=”Writing a #LiteratureReview? Check out the latest @clarivateAG blog for top tips (from topic searches to working with coauthors), examples, templates and more”]

Searching the Web of Science

The Web of Science is a multidisciplinary research engine that contains over 170 million papers from more than 250 academic disciplines. All of the papers in the database are interconnected via citations. That means once you get started with your keyword search, you can follow the trail of cited and citing papers to efficiently find all the relevant literature. This is a great way to ensure you’re not missing anything important when you write a narrative literature review.

We recommend starting your search in the Web of Science Core Collection™. This database covers more than 21,000 carefully selected journals. It is a trusted source to find research papers, and discover top authors and journals (read more about its coverage here ).

Learn more about exploring the Core Collection in our blog, How to find research papers: five tips every researcher should know . Our blog covers various tips, including how to:

• Perform a topic search (and select your keywords)
• Explore the citation network
• Refine your results (refining your search results by reviews, for example, will help you avoid duplication of work, as well as identify trends and gaps in the literature)
• Save your search and set up email alerts

Try our tips on the Web of Science now.

2. Identify key papers (and know how to use them)

As you explore the Web of Science, you may notice that certain papers are marked as “Highly Cited.” These papers can play a significant role when you write a narrative literature review.

Highly Cited papers are recently published papers getting the most attention in your field right now. They form the top 1% of papers based on the number of citations received, compared to other papers published in the same field in the same year.

You will want to identify Highly Cited research as a group of papers. This group will help guide your analysis of the future of the field and opportunities for future research. This is an important component of your conclusion.

Writing reviews is hard work…[it] not only organizes published papers, but also positions t hem in the academic process and presents the future direction.   Prof. Susumu Kitagawa, Highly Cited Researcher, Kyoto University

3. Tips for working with co-authors

Writing a narrative review on your own is hard, but it can be even more challenging if you’re collaborating with a team, especially if your coauthors are working across multiple locations. Luckily, reference management software can improve the coordination between you and your co-authors—both around the department and around the world.

We’ve written about how to use EndNote’s Cite While You Write feature, which will help you save hundreds of hours when writing research . Here, we discuss the features that give you greater ease and control when collaborating with your colleagues.

Use EndNote for narrative reviews

Sharing references is essential for successful collaboration. With EndNote, you can store and share as many references, documents and files as you need with up to 100 people using the software.

You can share simultaneous access to one reference library, regardless of your colleague’s location or organization. You can also choose the type of access each user has on an individual basis. For example, Read-Write access means a select colleague can add and delete references, annotate PDF articles and create custom groups. They’ll also be able to see up to 500 of the team’s most recent changes to the reference library. Read-only is also an option for individuals who don’t need that level of access.

EndNote helps you overcome research limitations by synchronizing library changes every 15 minutes. That means your team can stay up-to-date at any time of the day, supporting an easier, more successful collaboration.

Start your free EndNote trial today .

4.Finding a journal for your literature review

Finding the right journal for your literature review can be a particular pain point for those of you who want to publish. The expansion of scholarly journals has made the task extremely difficult, and can potentially delay the publication of your work by many months.

We’ve written a blog about how you can find the right journal for your manuscript using a rich array of data. You can read our blog here , or head straight to Endnote’s Manuscript Matcher or Journal Citation Report s to try out the best tools for the job.

5. Discover literature review examples and templates

There are a few tips we haven’t covered in this blog, including how to decide on an area of research, develop an interesting storyline, and highlight gaps in the literature. We’ve listed a few blogs here that might help you with this, alongside some literature review examples and outlines to get you started.

Literature Review examples:

• Aggregation-induced emission
• Development and applications of CRISPR-Cas9 for genome engineering
• Object based image analysis for remote sensing

(Make sure you download the free EndNote™ Click browser plugin to access the full-text PDFs).

Templates and outlines:

• Learn how to write a review of literature , Univ. of Wisconsin – Madison
• Structuring a literature review , Australian National University
• Matrix Method for Literature Review: The Review Matrix , Duquesne University

Additional resources:

• Ten simple rules for writing a literature review , Editor, PLoS Computational Biology
• Video: How to write a literature review , UC San Diego Psychology

Related posts

Unlocking u.k. research excellence: key insights from the research professional news live summit.

For better insights, assess research performance at the department level

Getting the Full Picture: Institutional unification in the Web of Science

The Literature Review

• Publications: A World of Information

Description

Primary sources, secondary sources, tertiary sources.

• Types of Reviews and Their Differences
• Beginning Steps and Finishing a Review
• Information Sources: Where to Find Them
• Webinar Recording (20 Minutes, Slides and Quiz)
• Webinar Recording (50 Minutes, Slides and Quiz)

Scholarly, professional literature falls under 3 categories, primary, secondary, and tertiary.  Published works (also known as a publication) may fall into one or more of these categories, depending on the discipline.  See definitions and linked examples of primary, secondary, and tertiary sources.

Differences in Publishing Norms by Broader Discipline  

Scholarly and professional communication norms can be different among various disciplines.  For instance, scholars in political science or law will generally publish their knowledge and research differently than those in chemistry or physics.  To show these distinctions, links to examples are provided for primary, secondary and tertiary sources.  The broader disciplines of business, education, and social science tend to use publications from both the sciences and humanities.

If you are in STEM or Nursing, see these examples of primary, secondary and tertiary literature within your disciplines:

• Nursing - Primary, Secondary and Tertiary Sources (Using Library Resources for Information and Research)
• STEM: Primary, Secondary and Tertiary Sources (Guide to Science Information Sources)

Image:  Typewriter, by DanielMcCullough , Permission by Unsplash.com license .

A primary source is a document or work where its author had a direct interaction or was involved with what was studied or created.  These sources are recommended when you need to get information or findings that are a direct result or finding from a study, research, or creation.   A primary source can also be an actual creative work or original material.

Humanities:

• Magazine or newspaper articles
• Creative works ( literature , poetry , fiction books, film, works of art or design, performances)
• Autobiographies and memoirs
• Interviews and oral histories
• Laws, statutes and official documents

Science / Health Sciences:

• Reports of  original research :  journal articles, poster presentations, conference papers
• Theses & dissertations
• Technical reports

Image:   Library stacks.  Permission by Pixabay.com  license .

A secondary source is a document or work where its author had an indirect part in a study or creation; an author is usually writing about or reporting the work or research done by someone else.  Secondary sources can be used for additional or supporting information; they are not the direct product of research or the making of a creative work.

• Books (monographs) written about a topic
• Articles:  Criticism or interpretation of creative works
• Biographies
• Reviews of creative works
• Bibliographies
• Textbooks and books (monographs)  
• Reviews: Literature or systematic
• Articles in trade journals (not based on research)
• Opinion pieces or commentaries

Image:  Encyclopedias, by StockSnap, Permission by Pixabay.com  license .

A tertiary source provides agreed-upon facts like measurements, dates, and definitions.  They are usually known as reference works and include the following:

• Encyclopedias
• Standards :  a document with specifications that create rules, guidelines or characteristics to ensure that materials, processes, or services are fit for their purpose.  They are established by a professional organization to provide a baseline of acceptable quality.
• Handbooks and manuals :  a resource that summarizes major topics or processes within a field.  These often provide established measurements, definitions or research methods.
• Pathfinders ( Research Guides ): a list of recommended information sources on a topic or discipline.
• << Previous: Publications: A World of Information
• Next: Types of Reviews and Their Differences >>
• Last Updated: Mar 15, 2024 11:12 AM
• URL: https://libguides.fau.edu/literature-review

Florida Atlantic University Libraries 777 Glades Road Boca Raton, FL 33431 (561) 297-6911

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 5: Evaluating Sources

Learning objectives.

At the conclusion of this chapter, you will be able to:

• Critically evaluate the sources of the information you have found.
• Evaluate the content of selected material for your purposes.

5.1 Overview of evaluation of sources

Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops. ( Association of College & Research Libraries, 2016 ).

You developed a viable research question, compiled a list of subject headings and keywords and spent a great deal of time searching the literature of your discipline or topic for sources. It’s now time to evaluate all of the information you found. Not only do you want to be sure of the source and the quality of the information, but you also want to determine whether each item is appropriate fit for your own review. This is also the point at which you make sure that you have searched out publications for all areas of your research question and go back into the literature for another search, if necessary.

In general, when we discuss evaluation of sources we are talking about looking at quality, accuracy, relevance, bias, reputation, currency, and credibility factors in a specific work, whether it’s a book, ebook, article, website, or blog posting. Before you include a source in your literature review, you should clearly understand what it is and why you are including it. According to Bennard et al., ( 2014 ), “Using inaccurate, irrelevant, or poorly researched sources can affect the quality of your own work.” (para. 4).

When evaluating a work for inclusion in, or exclusion from, your literature review, ask yourself a series of questions about each source.

5.1.1 Evaluating books

For primary and secondary sources you located in your search, use the ASAP mnemonic to evaluate inclusion in your literature review:

5.1.1.1 Age

Is it outdated? The answer to this question depends on your topic. If you are comparing historical classroom management techniques, something from 1965 might be appropriate. In Nursing, unless you are doing a historical comparison, a textbook from 5 years ago might be too dated for your needs.

A General Rule of Thumb:

5 years, maximum: medicine, health, education, technology, science 10-20 years: history, literature, art

5.1.1.2 Sources

Check reference or bibliography sources as well as those listed in footnotes or endnotes. Skim the list to see what kinds of sources the author used. When were the sources published? If the author is primarily citing works from 10 or 15 years ago, the book may not be what you need.

5.1.1.3 Author

Does the author have the credentials to write on the topic? Does the author have an academic degree or research grant funding? What else has the author published on the topic?

5.1.1.4 Publisher

Look for academic presses, including university presses. Books published under popular press imprints (such as Random House or Macmillan, in the U.S.) will not present scholarly research in the same way as Sage, Oxford, Harvard, or the University of Washington Press.

Other questions to ask about the book you may want to include in your literature review:

• What is the book’s purpose? Why was it written? Who is the intended audience?
• What is the conclusion or argument? How well is the main argument or conclusion supported?
• Is it relevant to your research? How is it related to your research question?
• Do you see any evidence of bias or unsubstantiated data?

5.1.2 Evaluating websites

In your research, it is likely you will discover information on the web that you will want to include in your literature review. For example, if your review is related to the current policy issues in public education in the United States, a potentially relevant information source may be a document located on the National Center for Education Statistics (NCES) website titled The Condition of Education 2017 . Likewise, for nursing, an article titled Discussing Vaccination with Concerned Patients: An Evidence-Based Resource for Healthcare Providers is available through the nursingcenter.com website. How do you evaluate these resources, and others like them?

Use the RADAR mnemonic ( Mandalios, 2013 ) to evaluate internet sources:

5.1.2.1 Relevance

How did you find the website and how is it relevant to your topic?

• Was it recommended by a reliable source?
• Was it cited in a scholarly source, such as a peer-reviewed journal?
• Was it linked from a reputable site?

5.1.2.2 Authority

Look for the About page to find information about the purpose of the website . You may make a determination of its credibility based on what you find there. Does the page exhibit a particular point of view or bias? For example, a heart association or charter school may be promoting a particular perspective – how might that impact the objectivity of the information located on their site? Is there advertising or is there a product information attached to the content?

5.1.2.3 Date

• When was the page created?
• Is it kept up to date?
• Are the links current and functional?

5.1.2.4 Appearance

• Does the information presented appear to be factual?
• Is the language formal or academic?
• How does it compare to other information you have read on the topic?
• Are references or links to cited material included?

5.1.2.5 Reason

What is the web address or URL? This can give you a clue about the purpose of the website, which may be to debate, advocate, advertise or sell, campaign, or present information. Here are some common domains and their origins:

• .org – An advocacy website for an organization
• .com – A private or commercial site
• .net – A network organization or Internet provider/no longer frequently used
• .edu – The site of a higher educational institution
• .gov – A federal government site
• .wa.us – A state government site which may include public schools and community colleges
• .uk, .ca, .jm – A country site

Mike Caulfield ( 2017 ), the author of Web Literacy for Student Fact-Checkers , recommends a few simple strategies to evaluate a website (as well as social media):

• Check for previous work: Look around to see if someone else has already provided a synthesis of the research described.
• Go upstream to the source: Go “upstream” to the source of the claim. Most web content is not original. Get to the original source to understand the credibility and reliability of the information.
• Read laterally: Read laterally. Once you get to the source of a claim, read what other people say about the source (publication, author, etc.). The truth is in the network.

5.1.3 Evaluating journal articles

It is likely that most of the resources you locate for your review will be from the scholarly literature of your discipline or in your topic area. As we have already seen, peer-reviewed articles are written by and for experts in a field. They generally describe formal research studies or experiments with the purpose of providing insight on a topic. You may have located these articles through Google, Google Scholar, a subscription or open access database, or citation searching. You now may want to know how to evaluate the usefulness for your research. As with the other resources, you are again looking for authority, accuracy, reliability, relevance, currency, and scope. Looking at each article as a separate and unique artifact, consider these elements in your evaluation:

5.1.3.1 Credibility/Authority

ASK: Who is the author? Is this person considered an expert in their field?

• Search the author’s name in a general web search engine like Google.
• What are the researcher’s academic credentials?
• What else has this author written? Search by author in the databases and see how much they have published on any given subject.
• How often or frequently has this article been cited by other scholars?

Citation analysis is the study of the impact and assumed quality of an article, an author, or an institution, based on the number of times works and/or authors have been cited by others. Google Scholar is a good way to get at this information.

5.1.3.2 Accuracy

Check the facts. ASK:

• Can statistics be verified through other sources?
• Does this information seem to fit with what you have read in other sources?

5.1.3.3 Reliability/Objectivity

ASK: Is there an obvious bias? That doesn’t mean that you can’t use the information, it just means you need to take the bias into account.

• Is a particular point of view or bias immediately obvious, or does it seem objective at first glance?
• What point of view does the author represent? Are they clear about their point of view?
• Is the article an editorial that is trying to argue a position?
• Is the article in a publication with a particular editorial position?

5.1.3.4 Relevance

ASK: The hard questions:

• Is the information relevant to your topic/thesis?
• How does the article fit into the scope of the literature on this topic?
• Is the material too technical or too clinical?
• Is it too elementary or basic?
• Does the information support your thesis or help you answer your question, or is it a challenge to make some kind of connection?
• Does the information present an opposite point of view so you can show that you have addressed all sides of the argument in your paper?

5.1.3.5 Currency

• When was the source published?
• How important is current information to your topic, discipline, or paper type?
• Does older material add to the history of the research? Or do you need something more current to support your thesis?

5.1.3.6 Scope and Purpose

To determine and evaluate in this category, ASK:

• Is it a general work that provides an overview of the topic or is it specifically focused on only one aspect of your topic?
• Does the breadth of the work match your expectations?
• Is the article meant to inform, explain, persuade or sell something. Be aware of the purpose as you read the content and take that into consideration when deciding whether to use it or not.

For Nursing and other medical articles ASK:

• What are the research methods used in the article?
• Where does the method fall in the evidence pyramid? Systematic reviews and meta-analyses are the most credible, with articles that are opinions the least credible.

• Meta Analysis: A systematic review that uses quantitative methods to summarize the results.
• Systematic Review: An article in which the authors have systematically searched for, appraised, and summarized all of the medical literature on a specific topic.
• Randomized Controlled Trials (RCTs): RCTs include a randomized group of patients in an experimental group, as well as a control group. These groups are monitored for the variables/outcomes of interest.
• Cohort Study: Research identifies two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and follows these cohorts for a specified duration of time, in order to measure the outcome of interest.
• Case Study: Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looks to see if they had the exposure of interest.
• Animal Research / Lab Studies: Information creation begins at the bottom of the pyramid: This is where ideas and laboratory research take place. Ideas turn into therapies and diagnostic tools, which are then tested with lab models and animals.
• Background Information / Expert Opinion: Handbooks, encyclopedias, and textbooks often provide a good foundation or introduction and often include generalized information about a condition. While background information presents a convenient summary, it typically takes about three years for this type of literature to be published.

5.1.4 Evaluating Social Media

Although social media (for example, Twitter or Facebook) is generally treated as an object under study rather than a source of information on a topic, the prevalence of social media as communication and sharing platforms must be acknowledged. It’s important to be skeptical of these sources, especially for inclusion in a literature review. However, as with any other web resource,you can evaluate a social media posting for authenticity by asking the following questions:

• Location of the source – Is the author in the place they are tweeting or posting about?
• Network – Who is in the author’s network and who follows the account?
• Content – Can the information be corroborated from other sources?
• Contextual updates – Does the author usually post or tweet on this topic? If so, what did past or updated posts say? Do they fill in more details?
• Reliability – does the author cite sources and are those sources reliable? ( Sheridan Libraries, 2017 )

5.2 In Summary

Another way to think about evaluation of sources is to ask the 5W questions:

• What type of document is it?
• Who created it?
• Why was the material published?
• When was it published?
• Where was the resource published?
• How was the information gathered and presented? ( Radom, 2017 )

Locating sources for your literature review by using discovery layers, library catalogs, databases, search engines, and other search platforms may take a great deal of time and effort. Does everything you found and retrieved have value or worth to you as you write your own literature review? If the resource has not met the criteria above and you can’t justify its place in your literature review, it doesn’t deserve to be mentioned in your work. Include high-quality materials that are current, accurate, credible, and most importantly relevant to your research question, hypothesis, or topic.

Evaluate a Website

Watch this short video :

Using a search engine like Google, do a quick search for a topic that interests you. Select a website from your list of results and evaluate it using the elements of website evaluation listed earlier in this chapter.

• How did you find the website?
• What is the domain name (the URL) of the site?
• What can you learn about the author/s of the site?
• When was the site last updated?
• Is it accurate based on what you know about the topic?
• Are there references?
• Do you notice any bias?
• Is the site functional? (re links working? Or do they lead to non-functional pages?)

Evaluate a Book

Select a subject specific book or ebook that you can access quickly and evaluate it based on the ASAP criteria.

Evaluate an Article

You can practice evaluation using the attached articles. You don’t need to spend a lot of time with the article, but see if you can identify each of the elements of evaluation. Remember the elements of evaluation for articles are:

• Authority/Credibility or Study Design for Nursing
• Reliability/Objectivity
• Scope and Purpose

For Education: Quality standards in e-learning: A matrix of analysis ( Frydenberg, 2002 ).

For Nursing: Beliefs and attitudes towards participating in genetic research ( Kerath et al, 2013 ).

Test Yourself

Check the Answer Key

For Nursing students: Your topic is the relationship between autism and vaccinations. Which of the two resources would you include in your literature review? Why?

• Hviid, Anders, Michael Stellfield, Jan Wohlfart, and Mads Melbye. “Association Between Thimerosal-Containing Vaccine and Autism.” Journal of the American Medical Association 290, no. 13 (October 1, 2003): 1763–1766. http://jama.jamanetwork.com/article.aspx?articleid=197365
• Chepkemoi Maina, Lillian, Simon Karanja, and Janeth Kombich. “Immunization Coverage and Its Determinants among Children Aged 12–23 Months in a Peri-Urban Area of Kenya.” Pan-African Medical Journal 14, no.3 (February 1, 2013). http://www.panafrican-med-journal.com/content/article/14/3/full/

For Education students: Your topic is music therapy in kindergarten classrooms in the United States. Which of the two resources would you include in your literature review? Why?”

• Simpson, Kate, and Deb Keen. “Music Interventions for Children with Autism: Review of the Literature.” Journal of Autism and Developmental Disorders 41, no. 11 (November 2011): 1507-1514.
• Bowman, Robert. “Approaches for Counseling Children Through Music.” Elementary School Guidance and Counseling 21, no. 4 (April 1987): 284-91.

Image attribution

Literature Reviews for Education and Nursing Graduate Students Copyright © by Linda Frederiksen is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Literature Reviews: Types of Literature

• Library Basics
• 1. Choose Your Topic
• How to Find Books
• Types of Clinical Study Designs

Types of Literature

• 3. Search the Literature
• 4. Read & Analyze the Literature
• 5. Write the Review
• Keeping Track of Information
• Style Guides
• Books, Tutorials & Examples

Different types of publications have different characteristics.

Primary Literature Primary sources means original studies, based on direct observation, use of statistical records, interviews, or experimental methods, of actual practices or the actual impact of practices or policies. They are authored by researchers, contains original research data, and are usually published in a peer-reviewed journal. Primary literature may also include conference papers, pre-prints, or preliminary reports. Also called empirical research .

Secondary Literature Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature. Examples include review articles (such as meta-analysis and systematic reviews) and reference works. Professionals within each discipline take the primary literature and synthesize, generalize, and integrate new research.

Tertiary Literature Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.

Adapted from the Information Services Department of the Library of the Health Sciences-Chicago , University of Illinois at Chicago.

Types of Scientific Publications

These examples and descriptions of publication types will give you an idea of how to use various works and why you would want to write a particular kind of paper.

• Scholarly article aka empirical article
• Review article
• Conference paper

Scholarly (aka empirical) article -- example

Empirical studies use data derived from observation or experiment. Original research papers (also called primary research articles) that describe empirical studies and their results are published in academic journals.  Articles that report empirical research contain different sections which relate to the steps of the scientific method.

      Abstract - The abstract provides a very brief summary of the research.

     Introduction - The introduction sets the research in a context, which provides a review of related research and develops the hypotheses for the research.

     Method - The method section describes how the research was conducted.

     Results - The results section describes the outcomes of the study.

     Discussion - The discussion section contains the interpretations and implications of the study.

     References - A references section lists the articles, books, and other material cited in the report.

Review article -- example

A review article summarizes a particular field of study and places the recent research in context. It provides an overview and is an excellent introduction to a subject area. The references used in a review article are helpful as they lead to more in-depth research.

Many databases have limits or filters to search for review articles. You can also search by keywords like review article, survey, overview, summary, etc.

Conference proceedings, abstracts and reports -- example

Conference proceedings, abstracts and reports are not usually peer-reviewed.  A conference article is similar to a scholarly article insofar as it is academic. Conference articles are published much more quickly than scholarly articles. You can find conference papers in many of the same places as scholarly articles.

How Do You Identify Empirical Articles?

To identify an article based on empirical research, look for the following characteristics:

     The article is published in a peer-reviewed journal .

     The article includes charts, graphs, or statistical analysis .

     The article is substantial in size , likely to be more than 5 pages long.

     The article contains the following parts (the exact terms may vary): abstract, introduction, method, results, discussion, references .

• << Previous: Types of Clinical Study Designs
• Next: 3. Search the Literature >>
• Last Updated: Dec 29, 2023 11:41 AM
• URL: https://research.library.gsu.edu/litrev

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Strategies to Find Sources

Literature Review: The What, Why and How-to Guide — Strategies to Find Sources

• Getting Started
• Introduction
• How to Pick a Topic
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

The Research Process

Planning : Before searching for articles or books, brainstorm to develop keywords that better describe your research question.

Searching : While searching, take note of what other keywords are used to describe your topic, and use them to conduct additional searches

     ♠ Most articles include a keyword section

     ♠ Key concepts may change names throughout time so make sure to check for variations

Organizing : Start organizing your results by categories/key concepts or any organizing principle that make sense for you . This will help you later when you are ready to analyze your findings

Analyzing : While reading, start making notes of key concepts and commonalities and disagreement among the research articles you find.

♠ Create a spreadsheet  to record what articles you are finding useful and why.

♠ Create fields to write summaries of articles or quotes for future citing and paraphrasing .

Writing : Synthesize your findings. Use your own voice to explain to your readers what you learned about the literature on your topic. What are its weaknesses and strengths? What is missing or ignored?

Repeat : At any given time of the process, you can go back to a previous step as necessary.

Advanced Searching

All databases have Help pages that explain the best way to search their product. When doing literature reviews, you will want to take advantage of these features since they can facilitate not only finding the articles that you really need but also controlling the number of results and how relevant they are for your search. The most common features available in the advanced search option of databases and library online catalogs are:

• Boolean Searching (AND, OR, NOT): Allows you to connect search terms in a way that can either limit or expand your search results 
• Proximity Searching (N/# or W/#): Allows you to search for two or more words that occur within a specified number of words (or fewer) of each other in the database
• Limiters/Filters : These are options that let you control what type of document you want to search: article type, date, language, publication, etc.
• Question mark (?) or a pound sign (#) for wildcard: Used for retrieving alternate spellings of a word: colo?r will retrieve both the American spelling "color" as well as the British spelling "colour." 
• Asterisk (*) for truncation: Used for retrieving multiple forms of a word: comput* retrieves computer, computers, computing, etc.

Want to keep track of updates to your searches? Create an account in the database to receive an alert when a new article is published that meets your search parameters!

• EBSCOhost Advanced Search Tutorial Tips for searching a platform that hosts many library databases
• Library's General Search Tips Check the Search tips to better used our library catalog and articles search system
• ProQuest Database Search Tips Tips for searching another platform that hosts library databases

There is no magic number regarding how many sources you are going to need for your literature review; it all depends on the topic and what type of the literature review you are doing:

► Are you working on an emerging topic? You are not likely to find many sources, which is good because you are trying to prove that this is a topic that needs more research. But, it is not enough to say that you found few or no articles on your topic in your field. You need to look broadly to other disciplines (also known as triangulation ) to see if your research topic has been studied from other perspectives as a way to validate the uniqueness of your research question.

► Are you working on something that has been studied extensively? Then you are going to find many sources and you will want to limit how far back you want to look. Use limiters to eliminate research that may be dated and opt to search for resources published within the last 5-10 years.

• << Previous: How to Pick a Topic
• Next: Evaluating Sources & Lit. Reviews >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

To help you frame and write your literature review, think about these five c’s (Callahan, 2014):

• Cite the material you have referred to and used to help you define the research problem that you will study.
• Compare the various arguments, theories, methods, and findings expressed in the literature.For example, describe where the various researchers agree and where they disagree. Describe the similarities and dissimilarities in approaches to studying related research problems.
• Contrast the various arguments, themes, methods, approaches, and controversies apparent and/or described in the literature. For example, describe what major areas are contested, controversial and/or still in debate.
• Critique the literature. Describe which arguments you find more persuasive and explain why. Explain which approaches, findings, and methods seem most reliable, valid, appropriate, and/or most popular and why. Pay attention to the verbs you use to describe what previous researchers have stated (e.g., asserts, demonstrates, argues, clarifies, etc.).
• Connect the various research studies you reviewed. Describe how your work utilizes, draws upon, departs from, synthesizes, adds to or extends previous research studies.

Research Methods, Data Collection and Ethics Copyright © 2020 by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

What is the Purpose of a Literature Review?

4-minute read

• 23rd October 2023

If you’re writing a research paper or dissertation , then you’ll most likely need to include a comprehensive literature review . In this post, we’ll review the purpose of literature reviews, why they are so significant, and the specific elements to include in one. Literature reviews can:

1. Provide a foundation for current research.

2. Define key concepts and theories.

3. Demonstrate critical evaluation.

4. Show how research and methodologies have evolved.

5. Identify gaps in existing research.

6. Support your argument.

Keep reading to enter the exciting world of literature reviews!

What is a Literature Review?

A literature review is a critical summary and evaluation of the existing research (e.g., academic journal articles and books) on a specific topic. It is typically included as a separate section or chapter of a research paper or dissertation, serving as a contextual framework for a study. Literature reviews can vary in length depending on the subject and nature of the study, with most being about equal length to other sections or chapters included in the paper. Essentially, the literature review highlights previous studies in the context of your research and summarizes your insights in a structured, organized format. Next, let’s look at the overall purpose of a literature review.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

Literature reviews are considered an integral part of research across most academic subjects and fields. The primary purpose of a literature review in your study is to:

Provide a Foundation for Current Research

Since the literature review provides a comprehensive evaluation of the existing research, it serves as a solid foundation for your current study. It’s a way to contextualize your work and show how your research fits into the broader landscape of your specific area of study.  

Define Key Concepts and Theories

The literature review highlights the central theories and concepts that have arisen from previous research on your chosen topic. It gives your readers a more thorough understanding of the background of your study and why your research is particularly significant .

Demonstrate Critical Evaluation 

A comprehensive literature review shows your ability to critically analyze and evaluate a broad range of source material. And since you’re considering and acknowledging the contribution of key scholars alongside your own, it establishes your own credibility and knowledge.

Show How Research and Methodologies Have Evolved

Another purpose of literature reviews is to provide a historical perspective and demonstrate how research and methodologies have changed over time, especially as data collection methods and technology have advanced. And studying past methodologies allows you, as the researcher, to understand what did and did not work and apply that knowledge to your own research.  

Identify Gaps in Existing Research

Besides discussing current research and methodologies, the literature review should also address areas that are lacking in the existing literature. This helps further demonstrate the relevance of your own research by explaining why your study is necessary to fill the gaps.

Support Your Argument

A good literature review should provide evidence that supports your research questions and hypothesis. For example, your study may show that your research supports existing theories or builds on them in some way. Referencing previous related studies shows your work is grounded in established research and will ultimately be a contribution to the field.  

Literature Review Editing Services 

Ensure your literature review is polished and ready for submission by having it professionally proofread and edited by our expert team. Our literature review editing services will help your research stand out and make an impact. Not convinced yet? Send in your free sample today and see for yourself! 

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

What is market research.

No matter your industry, conducting market research helps you keep up to date with shifting...

8 Press Release Distribution Services for Your Business

In a world where you need to stand out, press releases are key to being...

3-minute read

How to Get a Patent

In the United States, the US Patent and Trademarks Office issues patents. In the United...

The 5 Best Ecommerce Website Design Tools 

A visually appealing and user-friendly website is essential for success in today’s competitive ecommerce landscape....

The 7 Best Market Research Tools in 2024

Market research is the backbone of successful marketing strategies. To gain a competitive edge, businesses...

Google Patents: Tutorial and Guide

Google Patents is a valuable resource for anyone who wants to learn more about patents, whether...

Make sure your writing is the best it can be with our expert English proofreading and editing.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Explore sell to government
• Ways you can sell to government
• How to access contract opportunities
• Conduct market research
• Register your business
• Certify as a small business
• Become a schedule holder
• Market your business
• Research active solicitations
• Respond to a solicitation
• What to expect during the award process
• Comply with contractual requirements
• Handle contract modifications
• Monitor past performance evaluations
• Explore real estate
• 3D-4D building information modeling
• Art in architecture | Fine arts
• Computer-aided design standards
• Commissioning
• Design excellence
• Engineering
• Project management information system
• Spatial data management
• Facilities operations
• Smart buildings
• Tenant services
• Utility services
• Water quality management
• Explore historic buildings
• Heritage tourism
• Historic preservation policy, tools and resources
• Historic building stewardship
• Videos, pictures, posters and more
• NEPA implementation
• Courthouse program
• Land ports of entry
• Prospectus library
• Regional buildings
• Renting property
• Visiting public buildings
• Real property disposal
• Reimbursable services (RWA)
• Rental policy and procedures
• Site selection and relocation
• For businesses seeking opportunities
• For federal customers
• For workers in federal buildings
• Explore policy and regulations
• Acquisition management policy
• Aviation management policy
• Information technology policy
• Real property management policy
• Relocation management policy
• Travel management policy
• Vehicle management policy
• Federal acquisition regulations
• Federal management regulations
• Federal travel regulations
• GSA acquisition manual
• Managing the federal rulemaking process
• Explore small business
• Explore business models
• Research the federal market
• Forecast of contracting opportunities
• Events and contacts
• Explore travel
• Per diem rates
• Transportation (airfare rates, POV rates, etc.)
• State tax exemption
• Travel charge card
• Conferences and meetings
• E-gov travel service (ETS)
• Travel category schedule
• Federal travel regulation
• Travel policy
• Explore technology
• Cloud computing services
• Cybersecurity products and services
• Data center services
• Hardware products and services
• Professional IT services
• Software products and services
• Telecommunications and network services
• Work with small businesses
• Governmentwide acquisition contracts
• MAS information technology
• Software purchase agreements
• Cybersecurity
• Digital strategy
• Emerging citizen technology
• Federal identity, credentials, and access management
• Mobile government
• Technology modernization fund
• Explore about us
• Annual reports
• Mission and strategic goals
• Role in presidential transitions
• Get an internship
• Launch your career
• Elevate your professional career
• Discover special hiring paths
• Events and training
• Agency blog
• Congressional testimony
• GSA does that podcast
• News releases
• Leadership directory
• Staff directory
• Office of the administrator
• Federal Acquisition Service
• Public Buildings Service
• Staff offices
• Board of Contract Appeals
• Office of Inspector General
• Region 1 | New England
• Region 2 | Northeast and Caribbean
• Region 3 | Mid-Atlantic
• Region 4 | Southeast Sunbelt
• Region 5 | Great Lakes
• Region 6 | Heartland
• Region 7 | Greater Southwest
• Region 8 | Rocky Mountain
• Region 9 | Pacific Rim
• Region 10 | Northwest/Arctic
• Region 11 | National Capital Region
• Per Diem Lookup

Privately owned vehicle (POV) mileage reimbursement rates

GSA has adjusted all POV mileage reimbursement rates effective January 1, 2024.

* Airplane nautical miles (NMs) should be converted into statute miles (SMs) or regular miles when submitting a voucher using the formula (1 NM equals 1.15077945 SMs).

For calculating the mileage difference between airports, please visit the U.S. Department of Transportation's Inter-Airport Distance website.

QUESTIONS: For all travel policy questions, email [email protected] .

Have travel policy questions? Use our ' Have a Question? ' site

PER DIEM LOOK-UP

1 choose a location.

Error, The Per Diem API is not responding. Please try again later.

No results could be found for the location you've entered.

Rates for Alaska, Hawaii, U.S. Territories and Possessions are set by the Department of Defense .

Rates for foreign countries are set by the State Department .

2 Choose a date

Rates are available between 10/1/2021 and 09/30/2024.

The End Date of your trip can not occur before the Start Date.

Traveler reimbursement is based on the location of the work activities and not the accommodations, unless lodging is not available at the work activity, then the agency may authorize the rate where lodging is obtained.

Unless otherwise specified, the per diem locality is defined as "all locations within, or entirely surrounded by, the corporate limits of the key city, including independent entities located within those boundaries."

Per diem localities with county definitions shall include "all locations within, or entirely surrounded by, the corporate limits of the key city as well as the boundaries of the listed counties, including independent entities located within the boundaries of the key city and the listed counties (unless otherwise listed separately)."

When a military installation or Government - related facility(whether or not specifically named) is located partially within more than one city or county boundary, the applicable per diem rate for the entire installation or facility is the higher of the rates which apply to the cities and / or counties, even though part(s) of such activities may be located outside the defined per diem locality.

• Research article
• Open access
• Published: 13 April 2021

Breastfeeding and the risk of childhood cancer: a systematic review and dose-response meta-analysis

• Qing Su 1   na1 ,
• Xiaohui Sun 1   na1 ,
• Liwen Zhu 2   na1 ,
• Qin Yan 1 ,
• Peiwen Zheng 3 ,
• Yingying Mao 1 &
• Ding Ye   ORCID: orcid.org/0000-0001-6654-7832 1  

BMC Medicine volume  19 , Article number:  90 ( 2021 ) Cite this article

16k Accesses

25 Citations

72 Altmetric

Metrics details

The aim of this study was to quantitatively summarize the available evidence on the association of breastfeeding with the risk of childhood cancer.

A literature search of PubMed and Embase databases was performed to identify eligible observational studies published from inception to July 17, 2020. The categorical and dose-response meta-analysis was conducted by pooling relative risk (RR) or odds ratio (OR) estimates with 95% confidence intervals (CIs). Potential sources of heterogeneity were detected by meta-regression and stratification analysis. Sensitivity analysis and publication bias test were also carried out.

Forty-five articles involving 475,579 individuals were included in the meta-analysis. Among the thirty-three studies on the association between breastfeeding and risk of childhood leukemia, the pooled risk estimates were 0.77 (95% CI, 0.65–0.91) and 0.77 (95% CI 0.63–0.94) for ever versus non/occasional breastfeeding and longest versus shortest breastfeeding duration group, respectively. There was clear indication for non-linear dose-response relationship between breastfeeding duration and the risk of childhood leukemia ( P non-linear < 0.001). The most protective effect (OR, 0.66, 95% CI 0.62–0.70) was observed at a breastfeeding duration of 9.6 months. Four studies examined, the association between breastfeeding and risk of childhood neuroblastoma, and significant inverse associations were consistently observed in both the comparisons of ever breastfeeding versus non/occasional breastfeeding (OR = 0.59, 95% CI 0.44–0.81) and longest versus shortest breastfeeding (OR = 0.61, 95% CI 0.44–0.83). However, no associations of breastfeeding with risk of other cancers were found.

Conclusions

Our study supports a protective role of breastfeeding on the risk of childhood leukemia, also suggesting a non-linear dose-response relationship. Further studies are warranted to confirm the association between breastfeeding and risk of childhood neuroblastoma.

Peer Review reports

Childhood cancer is emerging as a major cause of death in children worldwide, which is bleaker for children with cancer in lower-middle-income countries [ 1 ]. Although childhood cancer only accounted for 1% of the total cancer [ 2 ], while once it occurs, a range of medical, psychological, ethical, and societal concerns are raised. Moreover, the global age-standardized incidence rates of registered cancers in children aged 0–14 years have increased from 124.0 to 140.6 per million person-years since the 1980s [ 3 ]. To date, little is known about the etiology of childhood cancer, but maternal reproductive health are potential explanations for a fraction of the incidence [ 4 , 5 , 6 ].

It is widely accepted that breastfeeding may protect mothers against breast cancer [ 7 ] and ovarian cancer [ 8 ], but also bring multiple health benefits for the infants [ 9 , 10 ]. However, the relationships of breastfeeding with the risk of childhood cancer are inconsistent across studies and the associations may differ by cancer types. Several studies have shown that breastfeeding had a protective effect on childhood cancer and the protection increased with the duration of the breastfeeding [ 11 , 12 ]. However, some previous studies showed no evidence of protection from breastfeeding for childhood cancer, and the analyses by duration of breastfeeding also failed to support the protective hypothesis [ 13 , 14 ]. When specific types of childhood cancer were examined, Amitay and Keinan-Boker [ 15 ] showed that breastfeeding was inversely associated with the risk of childhood leukemia. However, Wang et al. [ 16 ] provided limited evidence for a protective role of breastfeeding in childhood Hodgkin's lymphoma. Other publications have even reported that prolonged breastfeeding was positively associated with the risk of childhood malignant germ cell tumors [ 17 ] and leukemia [ 18 ].

A previous meta-analysis of the association between breastfeeding and the risk of childhood cancer was based on 26 original studies published up to June 2004 [ 19 ]. The pooled effect estimates suggested that breastfeeding was associated with 9% (95% confidence interval (CI), 2–16%) lower risk of acute lymphoblastic leukemia (ALL), 24% (95% CI 3–40%) lower risk of Hodgkin’s disease, and 41% (95% CI 22–56%) lower risk of neuroblastoma, but no associations of breastfeeding with acute nonlymphoblastic leukemia, non-Hodgkin’s lymphoma, central nervous system cancers, malignant germ cell tumors, juvenile bone tumors, or other solid cancers. Since then, evidences regarding this association have accumulated rapidly and provided more answers to this question. For the dose-response relationship, a previous pooled analysis showed that the protective effect of breastfeeding on the risk of childhood ALL was lowest at the breastfeeding duration of 8–10 months [ 20 ]. However, the exact dose-response relationship has not yet been evaluated for other cancer types. Therefore, we conducted this updated systematic review and dose-response meta-analysis of epidemiological studies to quantify precisely the impact of breastfeeding on the incidence of childhood cancer.

The study was registered in the international prospective register of systematic reviews (PROSPERO: CRD42020199446). The Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) checklist for reporting the meta-analysis was shown in Additional file  1 : Table S1.

Search strategy

Original articles from PubMed and EMBASE databases were systematically searched from the inception to 17 July 2020 to identify potentially eligible studies on the association between breastfeeding and the risk of childhood cancer. The search strategy was as follows: (“child” OR “pediatric” OR “childhood” OR “children”) AND (“cancer” OR “tumor” OR “neoplasm” OR “carcinoma” OR “malignancy” OR “leukemia” OR “lymphoma” OR “neuroblastoma” OR “retinoblastoma” OR “melanoma”) AND (“breastfeeding” OR “infant feeding” OR “infant nutrition”). In addition, we conducted manual retrieval of the relevant references.

Inclusion and exclusion criteria

The eligible studies were included as follows: (1) study design of cohort or case-control, (2) clearly defining the outcome of interest as cancer of specific anatomical site [ 21 , 22 , 23 ], (3) reporting the relative risk (RR) or odds ratio (OR) and corresponding 95% CI to calculate the association between breastfeeding and cancer risk among children, or providing sufficient data to calculate them, (4) if study populations overlapped, we selected the one with larger sample size. The exclusion criteria were as follows: (1) systematic review or meta-analysis; (2) letter, meeting, or comment; (3) duplicate studies retrieved from various databases. Two reviewers (QS and XS) independently performed study review and inclusion, and discrepancies were solved by a third reviewer (DY).

Data extraction and quality assessment

We extracted crucial information from the final studies retained, including first author, year of publication, data collection years, country or region, sample size, age in years, source of participants, cancer site, method of assessing breastfeeding, breastfeeding category, variables adjusted or matched, and corresponding risk estimates with 95% CIs.

Two researchers (QY and PZ) independently rated the quality of the included studies using the Newcastle-Ottawa Scale with scores ranging from 0 to 9 points [ 24 ]. This scale evaluates studies on the following aspects: (I) selection of cases and controls (4 scores); (II) comparability of cases and controls (2 scores); (III) ascertainment of exposure and non-response rate (3 scores). Studies with a quality score more than 7 points were considered as high quality. Two researchers (QS and QY) independently assessed the potential risk of bias using the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool [ 25 ]. This tool encompasses seven domains: the presence of any confounding variables, selection bias, deviations from the exposure, misclassification of the exposure, missing data, measurement of outcomes, and selection of the reported results. In this approach, a study was categorized as “low risk,” “moderate risk,” “serious risk,” or “critical risk” of bias.

Statistical analysis

All statistical analyses were performed using STATA version 14.0. The multivariate-adjusted risk estimates were selected if they were reported in the original article; otherwise, the unadjusted risk estimates were calculated using the original data. For the studies reported risk estimates relative to a reference category other than shortest breastfeeding duration, the risk estimates were recalculated using the shortest breastfeeding duration as reference by using the Orsini method [ 26 ]. When pooling the risk estimates, we regraded the shortest breastfeeding duration as the reference group (non/occasional breastfeeding), i.e., some defined never and some defined less than 1 month, 2 months, or 6 months.

In the analysis of breastfeeding versus non/occasional breastfeeding, if the corresponding estimate had not been presented in a study, estimates associated with different breastfeeding categories were synthesized into a single estimate. We also combined the risk estimates comparing the longest with the shortest breastfeeding duration among the studies with equal or greater than three different breastfeeding categories. The forest plot of the association between breastfeeding and the risk of childhood cancer was generated for breastfeeding and non/occasional breastfeeding and longest versus shortest breastfeeding duration, respectively.

A two-stage dose-response meta-analysis [ 27 ] was conducted to investigate the potential non-linear dose-response relationship between breastfeeding and the risk of childhood cancer. Briefly, a restricted cubic splines model with four knots at fixed percentiles, 5%, 35%, 65%, and 95%, of exposure level was used, which had negligible influence on the estimates. We assigned a null value to the lower bound of the reference group. The midpoint of the range was adopted to represent the category. When the category was open-ended, we assigned the midpoint of the upper open-ended category assuming that they had the same interval as the adjacent category.

The heterogeneity was evaluated by Q-statistic test and I -squared ( I 2 ) [ 28 , 29 ]. The random-effects model was used to pool the effect estimates, as the approach can be used whether or not there is heterogeneity [ 30 ]. Subgroup analyses were performed by year of publication, geographic location, quality score, sample size, study design, and definition of reference category. Heterogeneity between strata by the above stratified factors was assessed by meta-regression analysis. Sensitivity analysis was performed to evaluate the effect of a particular study on the overall results by deleting one study at a time and combining the effect values of the remaining studies.

Funnel plot was generated, and the symmetry means no potential publication bias. Publication bias was assessed using Egger’s test [ 31 ] and Begg’s test [ 32 ].

Systematic search

The flowchart of study selection is presented in Additional file 1: Figure S1. The primary search strategy for PubMed and Embase yielded 2905 and 2771 articles, and manual search from the reference lists of original studies or relevant reviews and meta-analyses on this topic yielded 67 additional articles. After removal of duplicates, 5545 articles were retrieved for assessment based on title and abstract, of which 116 articles were included for full text evaluation. After exclusion of 8 articles with insufficient data, 4 articles only reporting data on all cancers, 47 articles as meta, review, comment or meeting abstract, 2 articles conducted not among children, and 10 articles with overlapping data sets (Additional file 1: Table S2), a total of 45 articles with 475,579 participants were included for this meta-analysis [ 13 , 14 , 17 , 18 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 ].

Characteristics of the included studies

The characteristics of selected studies are shown in Table 1 . The total number of participants (from 140 to 410,147) varied widely across the included studies. All articles represented a range of geographical areas in Europe ( n = 20), Asia ( n = 8), and North America or Oceania ( n = 17). There were 32 articles with population-based case-control design, 12 articles with hospital-based case-control design, and one article with cohort design. The median quality score of all included articles was 7, which resulted in 33 articles with a score of 7 or more and 12 articles with a score less than 7. Based on the ROBINS-I tool, 37 studies were considered at moderate risk of bias, and 8 studies were rated at serious risk of bias. Of the 45 articles, 33 studies provided the effect estimates for the association of breastfeeding on leukemia, 11 studies on lymphoma, 7 studies on brain tumors, 4 studies on neuroblastoma, 4 studies on soft-tissue sarcoma, 3 studies on nephroblastoma, 2 studies on retinoblastoma, and 2 studies on germ cell tumors. The details of included studies for the subsequent subgroup analysis are shown in Additional file 1: Table S3. Moreover, after excluding studies with the breastfeeding only as dichotomous variable, and no sufficient data of the number of cases and controls in each breastfeeding category, 23 studies were included for leukemia [ 13 , 14 , 18 , 34 , 35 , 36 , 38 , 39 , 40 , 41 , 42 , 43 , 46 , 47 , 54 , 56 , 58 , 61 , 63 , 64 , 70 , 71 , 72 ], 6 studies for lymphoma [ 14 , 33 , 36 , 40 , 43 , 46 ], and 6 studies for brain tumors [ 14 , 40 , 44 , 51 , 64 , 67 ] in dose-response meta-analysis.

Summary of main findings

The summary risk estimates of the risk of childhood leukemia were 0.77 (95% CI, 0.65–0.91) for breastfeeding versus non/occasional breastfeeding and 0.77 (95% CI, 0.63–0.94) for longest versus shortest breastfeeding duration (Fig.  1 ). A significant non-linear dose-response relationship between breastfeeding and the risk of childhood leukemia was found ( P  < 0.001 for non-linearity; Fig.  2 ). The overall dose-response relationship approximated to a U-shaped curve. Compared with never breastfeeding, the risk of leukemia was statistically significant at a duration of 4.4–15.0 months. The most protective effect (OR = 0.66, 95% CI 0.62–0.70) was observed at a duration of 9.6 months. Specifically, the average breastfeeding duration of 6 months and 12 months reduced 20% (95% CI 15%–25%) and 27% (95% CI 22%–33%) of the risk of childhood leukemia, respectively. Begg’s funnel plot was presented in Additional file 1 : Figure S2. The shape of the funnel plot revealed no asymmetric distribution with a P value of 0.075 by Begg’s test and 0.173 by Egger’s test.

Forest plots for pooled risk estimates and the corresponding 95% confidence intervals (CIs) of childhood leukemia risk for a breastfeeding vs. non/occasional breastfeeding and b longest vs. shortest breastfeeding duration

Risk estimates (solid line) and the corresponding 95% CIs (dash lines) for the dose-response relationship between breastfeeding and the risk of childhood leukemia

Subgroup analysis and meta-regression analysis

In the subgroup analysis for the comparison of breastfeeding versus non/occasional breastfeeding, there were significant differences in the heterogeneity between strata by geographic location ( P in meta-regression = 0.016) and definition of reference category ( P in meta-regression = 0.025) (Table  2 ). Protective effect of breastfeeding on the risk of childhood leukemia cancer was found in population from Asia (OR, 0.43; 95% CI, 0.19–0.98), but not Europe and North American or Oceania. The inverse association was more pronounced among the studies with defining occasional breastfeeding as reference group (OR, 0.47; 95% CI, 0.28–0.80), especially for the studies using breastfeeding less than 6 months as reference group (OR = 0.49, 95% CI 0.35–0.67). Subgroup analysis of publication year showed similar results with main analysis, and studies with small and larger sample size showed slightly different results. Significant associations were found in studies with high-quality score (OR, 0.72; 95% CI, 0.59–0.87) and population-based case-control studies (OR, 0.86; 95% CI, 0.76–0.96).

We also conducted subgroup analysis stratified by histologic type and found that breastfeeding was associated with a decreased risk for ALL at borderline statistical significance (OR = 0.92, 95% CI 0.84–1.00), but not associated with acute myeloid leukemia (AML) risk (Table  2 ). Moreover, all subgroup analysis of ALL were consistent with those of childhood leukemia (Additional file 1 : Table S4). Further stratified by immunophenotype in ALL, the results showed a weak evidence of borderline statistical significance that breastfeeding was associated with a small reduction in B cell ALL risk, but no evidence of the association between breastfeeding and risk of T cell ALL (Table  2 ). The forest plots of subgroup analysis are shown in Additional file 1 : Figure S3.

Sensitivity analysis

The one-study-removed analysis showed that exclusion of each study did not significantly change the results (Additional file 1 : Figure S4). Considering that the incidence of childhood leukemia varies with age, which may have an influence on the risk estimate, we performed the repeated analyses by only including the group of studies among children aged 0–14 years old. Consequently, we found that there was no significant effect on the pooled risk estimates (Additional file 1 : Figure S5).

There was no suggestive evidence of the association between breastfeeding and risk of childhood lymphoma, with the pooled risk estimates of 0.83 (95% CI, 0.68–1.02) and 0.77 (95% CI, 0.53–1.10) for the comparison of breastfeeding versus non/occasional breastfeeding and longest versus shortest breastfeeding duration, respectively (Fig.  3 ). The non-linear dose-response relationship curve showed that the association between breastfeeding and the risk of childhood lymphoma was significant in a narrow range of breastfeeding duration ( P  = 0.046 for non-linearity; Additional file 1 : Figure S6). The funnel plot showed symmetry distribution with a P value of 0.533 by Begg’s test and 0.267 by Egger’s test (Additional file 1 : Figure S7).

Forest plots for pooled risk estimates and the corresponding 95% confidence intervals (CIs) of childhood lymphoma risk for a breastfeeding vs. non/occasional breastfeeding and b longest vs. shortest breastfeeding duration

Heterogeneity between strata by geographic location ( P  = 0.041) and definition of reference group ( P  = 0.044) was identified by meta-regression analysis (Table  3 ). Only the studies conducted in Asia indicated a significant association between breastfeeding and risk of childhood lymphoma (OR = 0.53, 95% CI 0.37–0.76). In the subgroup of studies with defining occasional breastfeeding as reference category, the association between breastfeeding and risk of childhood lymphoma was significant (OR = 0.66, 95% CI 0.44–0.99). Comparing with the breastfeeding duration of less than 6 months, the pooled risk estimate of prolong breastfeeding for the risk of lymphoma was 0.50 (95% CI 0.34–0.75). Moreover, there were no significant associations of breastfeeding with both Hodgkin’s lymphoma and non-Hodgkin’s lymphoma (Table  3 ). The forest plots of stratified analysis are shown in Additional file 1 : Figure S8.

Omitting the studies of UKCCS Investigators et al. [ 43 ] and Hardell et al. [ 14 ] modified the pooled risk estimates in sensitivity analysis, suggesting the results may be unstable (Additional file 1 : Figure S9). Sensitivity analysis by only including the group of studies among children aged 0–14 years old showed the pooled risk estimate of 0.98 (95% CI 0.84–1.15; Additional file 1 : Figure S10).

Brain tumors

No significant association between breastfeeding and risk of childhood brain tumors was found (Fig.  4 ). Non-linear dose-response relationship was also not observed ( P  = 0.776 for non-linearity). There was no sign of asymmetry with a P value of 0.764 by Begg’s test and 0.261 by Egger’s test (Additional file 1 : Figure S11). All subgroups showed no significant association between breastfeeding and risk of childhood brain tumors (Table  4 ). The corresponding forest plots of subgroup analysis are presented in Additional file 1 : Figure S12. Sensitivity analysis excluding each study did not substantially change the results (Additional file 1 : Figure S13). Sensitivity analysis by excluding the studies considering upper age limits equal and higher than 15 years yielded similar result with the pooled risk estimate of 0.99 (95% CI 0.87–1.12; Additional file 1 : Figure S14).

Forest plots for pooled risk estimates and the corresponding 95% confidence intervals (CIs) of the risk of childhood brain tumors for a breastfeeding vs. non/occasional breastfeeding and b longest vs. shortest breastfeeding duration

Neuroblastoma, soft-tissue sarcoma, nephroblastoma, retinoblastoma, and germ cell tumors

The associations of breastfeeding with risk of childhood neuroblastoma, soft-tissue sarcoma, nephroblastoma, retinoblastoma, and germ cell tumors are presented in Table  5 . Significant association was consistently found in neuroblastoma for the two comparisons of breastfeeding versus non/occasional breastfeeding (OR = 0.59, 95% CI 0.44–0.81) and longest versus shortest breastfeeding (OR = 0.61, 95% CI 0.44–0.83).

Study quality and study design

Of the 45 studies included, 33 were sufficient to provide at least fair quality evidence regarding the association between breastfeeding and the risk of childhood cancer. The included studies are at risk for selection bias for cases and controls and the potential misclassification introduced by the lack of specificity in exposure definition. Confounding is also an important consideration on account of the nature of observational studies. Almost all studies matched cases with controls by sex (33/45) and age (37/45), and most studies also matched participants using geographic location (15/45), and a few additionally used race or ethnicity (6/45). These matching variables are crucial for the comparability of cases and controls. Moreover, parental socioeconomic status (SES) was commonly used as adjustment, because higher SES among controls may overestimate the protective effect of breastfeeding on the risk of childhood cancer [ 40 ]. Smoking during pregnancy was also considered necessary, since it is associated with breastfeeding [ 74 ], and it may be related with the risk of childhood cancer [ 75 , 76 ], even though the association may vary by cancer types [ 77 , 78 , 79 , 80 ]. Therefore, the imbalance in this factor between cases and controls may be contributory to confounding bias. Some other potential confounders were also taken into consideration, such as infectious exposures, day care, birth weight, and birth order. However, we cannot exclude the possibility of residual confounding, although most of the included studies had matching and adjustment variables. In the stratified analysis of study quality, we found that the results from studies with higher quality showed consistent association with overall risk estimates, which provides additional confidence in the findings of our meta-analysis.

Case-control study is the main study design that all included studies of the meta-analysis except one were case-control studies. Since most studies have collected exposure information through parental interview, case-control studies are susceptible to recall bias and selection bias. Cohort studies are considered to provide more robust estimates than case-control studies; therefore, further cohort studies are needed to provide more evidence on the association and risk of childhood cancer. An optimal study might be conducted within the framework of a large population-based registry or cohort with full access to baseline information regarding demographic characteristics, detailed data of breastfeeding including breastfeeding duration, the use of infant formulas, the main types of milk given, the age of introduction of a range of foods and so on, and collection of medical records to accurately identify all diagnosed cases [ 81 ]. However, cohort studies would require follow-up periods of several years consuming manpower, material, financial and time largely, and very large sample sizes to provide sufficient statistical power. To be noted, the findings are to be expected from International Childhood Cancer Consortium (I4C), which is the first cohort consortium to have published findings on childhood cancer to elaborate the association [ 82 ]. But for now, case-control studies are mainly reported, and results from population-based case-control studies are more reliable compared with those from hospital-based case-control studies. We also found similar association in population-based case-control studies with overall risk estimates.

There is sufficient evidence to show that breastfeeding was inversely associated with the risk by pooling a number of original articles, with 23% lower risk of childhood leukemia (95% CI 9–35%) for breastfeeding versus non/occasional breastfeeding and 23% lower risk (95% CI 6–37%) for longest versus shortest breastfeeding duration, respectively. Moreover, we found a protective effect of breastfeeding on the risk of childhood ALL among the studies with higher quality and population-based case-control studies.

We found high heterogeneity in the meta-analysis on the association between breastfeeding and the risk of childhood leukemia, and there exists significant heterogeneity in different regional groups by meta-regression analysis. In the subgroup analysis of geographic location, we found that breastfeeding was more strongly associated with the risk of childhood leukemia in Asia. In terms of risk of bias, Robins-I tool rated 4 European studies and 2 Asian studies at serious risk, and 10 European studies and 6 Asian studies were considered at moderate risk. At the original study level, one aspect that deserves particular attention is the difference in the number of cases and the source of population, which may at least in part explain the heterogeneity observed across the geographic region. For example, the Asian subgroup analyses included 8 studies with a total of 1739 cases, with only 1 study enrolled more than 200 cases. Moreover, few studies (3/8) used population-based case-control study design. However, European articles included 14 studies with 7518 cases and 71% studies used population controls. This is a similar situation to North America or Oceania, in that a majority (6/11) of articles included in the meta-analysis had more than 200 case numbers, and most (9/11) used population controls. Therefore, the differences in risk estimates could be related to study quality issues. On the other hand, there were several other potential explanations could be proposed. A great variation in breastfeeding duration across the countries may result in the heterogeneity. It should be noted that breastfeeding duration is shorter in high-income countries than in those that are resource-poor. It was estimated that 25% of infants in the USA and Europe are exclusively breastfed through 6 months [ 83 , 84 ], as compared with 43% in the South-East Asia region [ 85 ]. In particular, only three countries (France, Spain, and the USA) had rates below 80% for ever breastfeeding across all country groups [ 10 ]. It could be that varying cultural influences contribute to this region disparity, with for example protective Islamic beliefs, South Asian cultural teachings, and more extensive support networks. A study in UK provided the evidence that Pakistan-origin mothers had higher breastfeeding initiation rates and longer average breastfeeding durations than White British mothers [ 86 ]. Another cross-sectional study in the USA demonstrated that Asian women had the highest breastfeeding initiation rates relative to all other racial/ethnic groups [ 87 ]. On the other hand, the heterogeneity may also due to the breastfeeding pattern. The volume of breastfeeding differed considerably between the women who breastfed only and those who performed mixed feeding, even if these women had the same breastfeeding duration. Globally, the prevalence of exclusive breastfeeding varied widely, countries from Asia and the Pacific region had moderate to high rate of exclusive breastfeeding, while the rate of exclusive breastfeeding was lower in Europe and America [ 10 ].

Additionally, we performed the dose-response meta-analysis, which showed a specific non-linear dose-response relationship between breastfeeding and the risk of childhood leukemia. World Health Organization (WHO) and United Nations International Children’s Emergency Fund (UNICEF) developed the global strategy for infant and young child feeding that infants should be exclusively breastfed for the first 6 months of life to achieve optimal growth, development and health [ 88 ]. In the current study, we found that breastfeeding duration of 6 months could reduce 20% (95% CI 15–25%) risk of childhood leukemia. The U-shaped curve showed that breastfeeding duration of approximately 9.6 months might show the most significant protect effect on the risk of childhood leukemia. The decreased risk of childhood leukemia was statistically significant at a duration of 4.4–15.0 months. Interestingly, we found that there was more pronounced association between breastfeeding and risk of childhood leukemia when defining breastfeeding for less than 6 months as reference group, rather than never breastfeeding, which may due to that breastfeeding appeared a protective effect on the risk of leukemia after a certain period. The nonsignificant decreased risk at prolonged breastfeeding duration might derive from relatively small sample size but not real effect. Large-sample and well-designed studies should be developed in future to demonstrate this turning point.

There were several potential explanations why breastfeeding may decrease the risk of childhood leukemia. Breast milk contains high levels of immunologically active components and multifactorial anti-inflammatory defense mechanisms that influence the development of the immune system of the breastfed infant [ 89 , 90 ]. For example, soluble tumor necrosis factor (TNF)-related apoptosis-inducing ligand (TRAIL) in breast milk can control apoptosis and cell proliferation in various organs and tissues. Breastfeeding also provides the infants with human alpha-lactalbumin made lethal to tumor cells (HAMLET), which is a substance with anticancer activity in breast milk [ 91 ]. Besides, breast milk imparts the mother’s stem cells to the infant, where they potentially function to actively stimulate or modulate the immune system and promote its development early in life [ 92 ]. In particular, Greaves’ hypothesis proposed immunological model that breast milk could modify the immune response in the prevention of childhood ALL [ 93 ]. Moreover, accumulating evidence have demonstrated that breast milk has the potential of shaping the neonate’s gut microbiome, such as microbiota richness, diversity, and composition [ 94 , 95 ]. Recently, a large, multi-center study suggested that breastfeeding status was the most significant factor associated with microbiome structure in early life [ 96 ]. It is hypothesized that breastfeeding could decrease the risk of childhood leukemia by the recent discovery of the breast milk microbiome and its connections with immune factors [ 97 ].

The meta-analysis provided no convincing evidence of the association between breastfeeding and risk of lymphoma. The findings of the present study are consistent with previous observational studies although some early studies indicated negative association of breastfeeding with lymphoma risk, especially in Hodgkin’s lymphoma [ 19 , 43 ]. In addition, results from sensitivity analysis were unstable, which deserves more studies to clarify the association.

Our pooled analysis presented an inverse association between breastfeeding and risk of childhood lymphoma in Asia, but not Europe or North America, suggesting a potential region-specific effect. What is more, significant discrepancies across subgroups stratified by definition of reference category were found, and only the pooled estimate for the studies using occasional breastfeeding as reference group showed a significantly decreased risk of childhood lymphoma. It may be explained by the significant non-linear dose-response relationship. For example, Davis et al. [ 33 ] found that breastfeeding for more than 6 months was associated with a decreased risk of childhood lymphoma, but that breastfeeding for shorter durations was not associated with a reduced risk. The study using breastfeeding for less than 6 months as reference group suggested that a longer breastfeeding duration had a protective effect against Hodgkin’s lymphoma [ 53 ]. Another point we should make is that the incidence of lymphoma is highest among 10–14 years old (and even higher among 15–19 years old), while the low age of the cases in the included studies with a short tumor induction period might be too short to find a decreased risk for lymphoma.

Other cancers

We found significant association between breastfeeding and risk of childhood neuroblastoma, while no significant associations of breastfeeding with risk of soft-tissue sarcoma, brain tumors, nephroblastoma, retinoblastoma, and germ cell tumors. This is updated from the previous meta-analysis and the results were consistent [ 19 ]. However, the associations of breastfeeding and risk of these cancers may be underpowered because of the small number of studies in the meta-analysis.

Strength and limitation

The primary strength of our study is that the traditional categorical meta-analysis and dose-response analysis were applied simultaneously, which can provide more meaningful information. Another strength is the large sample size and number of included studies, which make the findings stable and reliable and enable us to conduct multiple subgroup analyses by geographic location, study quality score, study design, etc. In addition, we also performed stratified analyses on the association of breastfeeding and risk of the subtypes of individual cancers.

However, the present study has several limitations. First, there was high evidence of heterogeneity across studies in the categorical meta-analysis. However, meta-regression analyses suggested that geographic region and definition of reference category are the potential sources of the observed heterogeneity. Second, our meta-analysis included very limited studies from Oceania and no study from Africa. Third, the results of dose-response meta-analysis were prone to be influenced by possible exposure misclassification as the exposure dose was estimated with median for interval exposure, and the lower bound added to the half of the adjacent previous category for the highest open-ended exposure group. Fourth, the number of studies evaluating the associations of breastfeeding with risk of neuroblastoma, nephroblastoma, retinoblastoma, and germ cell tumors is small. Fifth, we were unable to assess differences by age or sex, because sufficiently age- or sex-specific studies are not available. Finally, limited by the lack information of breastfeeding pattern in most of included studies, such as exclusive breastfeeding and partial breastfeeding, we cannot evaluate the association between breastfeeding pattern and the risk of childhood cancer.

This meta-analysis demonstrates that breastfeeding was associated with the reduced the risk of childhood leukemia. The present study also provides suggestive evidence of the inverse association between breastfeeding and risk of neuroblastoma. In addition, given that the role of breastfeeding for the risk of childhood leukemia and lymphoma may be region-specific, further analyses are warranted to provide insights into the strategy of breastfeeding advocacy.

Availability of data and materials

All data generated or analyzed during this study are included in this article and its additional files.

Abbreviations

Confidence interval

Human alpha-lactalbumin made lethal to tumor cells

Preferred Reporting Items for Systematic reviews and Meta-Analysis

Relative risk

Tumor necrosis factor

TNF-related apoptosis-inducing ligand

United Nations International Children’s Emergency Fund

World Health Organization

Lam CG, Howard SC, Bouffet E, et al. Science and health for all children with cancer. Science. 2019;363:1182–6.

Article   CAS   PubMed   Google Scholar  

Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68:394–424.

Article   PubMed   Google Scholar  

Steliarova-Foucher E, Colombet M, Ries L, et al. International incidence of childhood cancer, 2001-10: a population-based registry study. Lancet Oncol. 2017;18:719–31.

Article   PubMed   PubMed Central   Google Scholar  

Belson M, Kingsley B, Holmes A. Risk factors for acute leukemia in children: a review. Environ Health Perspect. 2007;115:138–45.

Wigle DT, Arbuckle TE, Turner MC, et al. Epidemiologic evidence of relationships between reproductive and child health outcomes and environmental chemical contaminants. J Toxicol Environ Health B Crit Rev. 2008;11:373–517.

Spector LG, Pankratz N, Marcotte EL. Genetic and nongenetic risk factors for childhood cancer. Pediatr Clin N Am. 2015;62:11–25.

Article   Google Scholar  

Islami F, Liu Y, Jemal A, et al. Breastfeeding and breast cancer risk by receptor status--a systematic review and meta-analysis. Ann Oncol. 2015;26:2398–407.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Luan NN, Wu QJ, Gong TT, et al. Breastfeeding and ovarian cancer risk: a meta-analysis of epidemiologic studies. Am J Clin Nutr. 2013;98:1020–31.

Gartner LM, Morton J, Lawrence RA, et al. Breastfeeding and the use of human milk. Pediatrics. 2005;115:496–506.

Victora CG, Bahl R, Barros AJ, et al. Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect. Lancet. 2016;387:475–90.

Kucukcongar A, Oguz A, Pinarli FG, et al. Breastfeeding and childhood cancer: is breastfeeding preventative to childhood cancer? Pediatr Hematol Oncol. 2015;32:374–81.

PubMed   Google Scholar  

Ortega-Garcia JA, Ferris-Tortajada J, Torres-Cantero AM, et al. Full breastfeeding and paediatric cancer. J Paediatr Child Health. 2008;44:10–3.

Lancashire RJ, Sorahan T. Breastfeeding and childhood cancer risks: OSCC data. Br J Cancer. 2003;88:1035–7.

Hardell L, Dreifaldt AC. Breast-feeding duration and the risk of malignant diseases in childhood in Sweden. Eur J Clin Nutr. 2001;55:179–85.

Amitay EL, Keinan-Boker L. Breastfeeding and childhood leukemia incidence: A Meta-analysis and systematic review. JAMA Pediatr. 2015;169(6):e151025.

Wang KL, Liu CL, Zhuang Y, et al. Breastfeeding and the risk of childhood Hodgkin lymphoma: a systematic review and meta-analysis. Asian Pac J Cancer Prev. 2013;14(8):4733–7.

Shu XO, Nesbit ME, Buckley JD, et al. An exploratory analysis of risk factors for childhood malignant germ-cell tumors: report from the Childrens Cancer Group (Canada, United States). Cancer Causes Control. 1995;6:187–98.

Jimenez-Hernandez E, Fajardo-Gutierrez A, Nunez-Enriquez JC, et al. A greater birthweight increases the risk of acute leukemias in Mexican children-experience from the Mexican Interinstitutional Group for the identification of the causes of childhood leukemia (MIGICCL). Cancer Med. 2018;7:1528–36.

Martin RM, Gunnell D, Owen CG, et al. Breast-feeding and childhood cancer: a systematic review with metaanalysis. Int J Cancer. 2005;117:1020–31.

Rudant J, Lightfoot T, Urayama KY, et al. Childhood acute lymphoblastic leukemia and indicators of early immune stimulation: a Childhood Leukemia International Consortium study. Am J Epidemiol. 2015;181(8):549–62.

Birch JM, Marsden HB. A classification scheme for childhood cancer. Int J Cancer. 1987;40(5):620–4.

Kramárová E, Stiller CA. The international classification of childhood cancer. Int J Cancer. 1996;68(6):759–65.

Steliarova-Foucher E, Stiller C, Lacour B, et al. International classification of childhood cancer, third edition. Cancer. 2005;103(7):1457–67.

Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25:603–5.

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.

Orsini N. From floated to conventional confidence intervals for the relative risks based on published dose-response data. Comput Methods Prog Biomed. 2010;98:90–3.

Orsini N, Li R, Wolk A, et al. Meta-analysis for linear and nonlinear dose-response relations: examples, an evaluation of approximations, and software. Am J Epidemiol. 2012;175:66–73.

Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–60.

Chen B, Benedetti A. Quantifying heterogeneity in individual participant data meta-analysis with binary outcomes. Syst Rev. 2017;6:243.

DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88.

Hayashino Y, Noguchi Y, Fukui T. Systematic evaluation and comparison of statistical tests for publication bias. J Epidemiol. 2005;15:235–43.

Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–101.

CAS   PubMed   Google Scholar  

Davis MK, Savitz DA, Graubard BI. Infant feeding and childhood cancer. Lancet. 1988;2:365–8.

Magnani C, Pastore G, Terracini B. Infant feeding and childhood cancer. Lancet. 1988;2:1136.

van Duijn CM, van Steensel-Moll HA, van der Does-vd BA, et al. Infant feeding and childhood cancer. Lancet. 1988;2:796–7.

Shu XO, Clemens J, Zheng W, et al. Infant breastfeeding and the risk of childhood lymphoma and leukaemia. Int J Epidemiol. 1995;24:27–32.

Petridou E, Trichopoulos D, Kalapothaki V, et al. The risk profile of childhood leukaemia in Greece: a nationwide case-control study. Br J Cancer. 1997;76:1241–7.

Schuz J, Kaletsch U, Meinert R, et al. Association of childhood leukaemia with factors related to the immune system. Br J Cancer. 1999;80:585–90.

Shu XO, Linet MS, Steinbuch M, et al. Breast-feeding and risk of childhood acute leukemia. J Natl Cancer Inst. 1999;91:1765–72.

Smulevich VB, Solionova LG, Belyakova SV. Parental occupation and other factors and cancer risk in children: I. Study methodology and non-occupational factors. Int J Cancer. 1999;83:712–7.

Dockerty JD, Skegg DC, Elwood JM, et al. Infections, vaccinations, and the risk of childhood leukaemia. Br J Cancer. 1999;80:1483–9.

Infante-Rivard C, Fortier I, Olson E. Markers of infection, breast-feeding and childhood acute lymphoblastic leukaemia. Br J Cancer. 2000;83:1559–64.

UK Childhood Cancer Study Investigators. Breastfeeding and childhood cancer. Br J Cancer. 2001;85:1685–94.

Schuz J, Kaletsch U, Kaatsch P, et al. Risk factors for pediatric tumors of the central nervous system: results from a German population-based case-control study. Med Pediatr Oncol. 2001;36:274–82.

Daniels JL, Olshan AF, Pollock BH, et al. Breast-feeding and neuroblastoma, USA and Canada. Cancer Causes Control. 2002;13:401–5.

Murray L, McCarron P, Bailie K, et al. Association of early life factors and acute lymphoblastic leukaemia in childhood: historical cohort study. Br J Cancer. 2002;86:356–61.

Perrillat F, Clavel J, Jaussent I, et al. Breast-feeding, fetal loss and childhood acute leukaemia. Eur J Pediatr. 2002;161:235–7.

Jourdan-Da SN, Perel Y, Mechinaud F, et al. Infectious diseases in the first year of life, perinatal characteristics and childhood acute leukaemia. Br J Cancer. 2004;90:139–45.

Altinkaynak S, Selimoglu MA, Turgut A, et al. Breast-feeding duration and childhood acute leukemia and lymphomas in a sample of Turkish children. J Pediatr Gastroenterol Nutr. 2006;42:568–72.

Saddlemire S, Olshan AF, Daniels JL, et al. Breast-feeding and Wilms tumor: a report from the Children’s Oncology Group. Cancer Causes Control. 2006;17:687–93.

Shaw AK, Li P, Infante-Rivard C. Early infection and risk of childhood brain tumors (Canada). Cancer Causes Control. 2006;17:1267–74.

Petridou E, Andrie E, Dessypris N, et al. Incidence and characteristics of childhood Hodgkin's lymphoma in Greece: a nationwide study (Greece). Cancer Causes Control. 2006;17:209–15.

Bener A, Hoffmann GF, Afify Z, et al. Does prolonged breastfeeding reduce the risk for childhood leukemia and lymphomas? Minerva Pediatr. 2008;60:155–61.

MacArthur AC, McBride ML, Spinelli JJ, et al. Risk of childhood leukemia associated with vaccination, infection, and medication use in childhood: the cross-Canada childhood leukemia study. Am J Epidemiol. 2008;167:598–606.

Flores-Lujano J, Perez-Saldivar ML, Fuentes-Panana EM, et al. Breastfeeding and early infection in the aetiology of childhood leukaemia in Down syndrome. Br J Cancer. 2009;101:860–4.

Rudant J, Orsi L, Menegaux F, et al. Childhood acute leukemia, early common infections, and allergy: the ESCALE study. Am J Epidemiol. 2010;172:1015–27.

Rudant J, Orsi L, Monnereau A, et al. Childhood Hodgkin’s lymphoma, non-Hodgkin's lymphoma and factors related to the immune system: the Escale study (SFCE). Int J Cancer. 2011;129:2236–47.

Waly MI, Ali A, Al-Saadoon M, et al. Breastfeeding is not associated with risk of developing childhood leukemia in the Sultanate of Oman. Asian Pac J Cancer Prev. 2011;12:2087–91.

Urayama KY, Chokkalingam AP, Metayer C, et al. HLA-DP genetic variation, proxies for early life immune modulation and childhood acute lymphoblastic leukemia risk. Blood. 2012;120:3039–47.

Lupo PJ, Zhou R, Skapek SX, et al. Allergies, atopy, immune-related factors and childhood rhabdomyosarcoma: a report from the Children’s Oncology Group. Int J Cancer. 2014;134:431–6.

Article   PubMed   CAS   Google Scholar  

Mattioli S, Farioli A, Legittimo P, et al. Tobacco smoke and risk of childhood acute non-lymphocytic leukemia: findings from the SETIL study. PLoS One. 2014;9:e111028.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Schraw JM, Dong YQ, Okcu MF, et al. Do longer formula feeding and later introduction of solids increase risk for pediatric acute lymphoblastic leukemia? Cancer Causes Control. 2014;25:73–80.

Ajrouche R, Rudant J, Orsi L, et al. Childhood acute lymphoblastic leukaemia and indicators of early immune stimulation: the Estelle study (SFCE). Br J Cancer. 2015;112:1017–26.

Greenop KR, Bailey HD, Miller M, et al. Breastfeeding and nutrition to 2 years of age and risk of childhood acute lymphoblastic leukemia and brain tumors. Nutr Cancer. 2015;67:431–41.

Heck JE, Omidakhsh N, Azary S, et al. A case-control study of sporadic retinoblastoma in relation to maternal health conditions and reproductive factors: a report from the Children’s Oncology group. BMC Cancer. 2015;15:735.

Amitay EL, Dubnov RG, Keinan-Boker L. Breastfeeding, other early life exposures and childhood leukemia and lymphoma. Nutr Cancer. 2016;68:968–77.

Rios P, Bailey HD, Orsi L, et al. Risk of neuroblastoma, birth-related characteristics, congenital malformations and perinatal exposures: a pooled analysis of the ESCALE and ESTELLE French studies (SFCE). Int J Cancer. 2016;139:1936–48.

Bailey HD, Rios P, Lacour B, et al. Factors related to pregnancy and birth and the risk of childhood brain tumours: the ESTELLE and ESCALE studies (SFCE, France). Int J Cancer. 2017;140:1757–69.

Schraw JM, Scheurer ME, Forman MR. A vulnerable age for the introduction of solid foods in pediatric acute lymphoblastic leukemia. Nutr Cancer. 2017;69:261–6.

Gao Z, Wang R, Qin ZX, et al. Protective effect of breastfeeding against childhood leukemia in Zhejiang Province, P. R. China: a retrospective case-control study. Libyan J Med. 2018;13:1508273.

PubMed   PubMed Central   Google Scholar  

Lingappa ALKSRS. Breastfeeding and its associated risk in children with acute leukemia: a retrospective study. Indian J Med Paediatr Oncol. 2018;39:312–5.

Rafieemehr H, Calhor F, Esfahani H, et al. Risk of acute lymphoblastic leukemia: results of a case-cntrol study. Asian Pac J Cancer Prev. 2019;20:2477–83.

Bauer H, Rios P, Schleiermacher G, et al. Maternal and perinatal characteristics, congenital malformations and the risk of wilms tumor: the ESTELLE study. Cancer Causes Control. 2020;31:491–501.

Cohen SS, Alexander DD, Krebs NF, et al. Factors associated with breastfeeding initiation and continuation: a meta-analysis. J Pediatr. 2018;203:190–6.

Stjernfeldt M, Berglund K, Lindsten J, et al. Maternal smoking during pregnancy and risk of childhood cancer. Lancet. 1986;1(8494):1350–2.

Rumrich IK, Viluksela M, Vahakangas K, et al. Maternal smoking and the risk of cancer in early life - a meta-analysis. PLoS One. 2016;11(11):e0165040.

Antonopoulos CN, Sergentanis TN, Papadopoulou C, et al. Maternal smoking during pregnancy and childhood lymphoma: a meta-analysis. Int J Cancer. 2011;129(11):2694–703.

Chu P, Wang H, Han S, et al. Maternal smoking during pregnancy and risk of childhood neuroblastoma: systematic review and meta-analysis. J Cancer Res Ther. 2016;12(2):999–1005.

Bailey HD, Lacour B, Guerrini-Rousseau L, et al. Parental smoking, maternal alcohol, coffee and tea consumption and the risk of childhood brain tumours: the ESTELLE and ESCALE studies (SFCE, France). Cancer Causes Control. 2017;28:719–32.

Metayer C, Petridou E, Arangure JM, et al. Parental tobacco smoking and acute myeloid leukemia: The Childhood Leukemia International Consortium. Am J Epidemiol. 2016;184(4):261–73.

Guise JM, Austin D, Morris CD. Review of case-control studies related to breastfeeding and reduced risk of childhood leukemia. Pediatrics. 2005;116:e724–31.

Tikellis G, Dwyer T, Paltiel O, et al. The International Childhood Cancer Cohort Consortium (I4C): a research platform of prospective cohorts for studying the aetiology of childhood cancers. Paediatr Perinat Epidemiol. 2018;32(6):568–83.

Division of Nutrition, Physical Activity, and Obesity, National Center for Chronic Disease Prevention and Health Promotion. Breastfeeding Report Card: United States. 2020. https://www.cdc.gov/breastfeeding/pdf/2020-Breastfeeding-Report-Card-H.pdf .

Google Scholar  

Bagci Bosi AT, Eriksen KG, Sobko T, et al. Breastfeeding practices and policies in WHO European region member states. Public Health Nutr. 2016;19:753–64.

World Health Organization. World health statistics 2013. Geneva: WHO; 2013.

Brown LJ, Sear R. Are mothers less likely to breastfeed in harsh environments? Physical environmental quality and breastfeeding in the born in Bradford study. Matern Child Nutr. 2019;15(4):e12851.

Singh GK, Kogan MD, Dee DL. Nativity/immigrant status, race/ethnicity, and socioeconomic determinants of breastfeeding initiation and duration in the United States, 2003. Pediatrics. 2007;119(Suppl 1):S38–46.

World Health Organization and UNICEF. Global strategy for infant and young child feeding. https://www.who.int/maternal_child_adolescent/documents/9241562218/en/ . (Accessed 10 Sept 2020).

World Health Organization. Global initiative for childhood cancer. https://www.who.int/cancer/childhood-cancer/en/ .

Hanson LA. The mother-offspring dyad and the immune system. Acta Paediatr. 2000;89:252–8.

Mossberg AK, Hun MK, Morozova-Roche LA, et al. Structure and function of human alpha-lactalbumin made lethal to tumor cells (HAMLET)-type complexes. FEBS J. 2010;277:4614–25.

Hassiotou F, Hartmann PE. At the dawn of a new discovery: the potential of breast milk stem cells. Adv Nutr. 2014;5:770–8.

Greaves MF. Aetiology of acute leukaemia. Lancet. 1997;349:344–9.

Rogier EW, Frantz AL, Bruno ME, et al. Secretory antibodies in breast milk promote long-term intestinal homeostasis by regulating the gut microbiota and host gene expression. Proc Natl Acad Sci U S A. 2014;111(8):3074–9.

Andreas NJ, Kampmann B, Mehring L-DK. Human breast milk: a review on its composition and bioactivity. Early Hum Dev. 2015;91(11):629–35.

Stewart CJ, Ajami NJ, O'Brien JL, et al. Temporal development of the gut microbiome in early childhood from the TEDDY study. Nature. 2018;562(7728):583–8.

DeWeerdt S. How baby's first microbes could be crucial to future health. Nature. 2018;555(7695):S18–9.

Download references

Acknowledgements

The authors thank all the participants and researchers of the studies included for the meta-analysis.

This study was funded by the National Natural Science Foundation of China (No. 81973663), Natural Science Foundation of Zhejiang Province (No. LQ20H260008), and Medical Health Science and Technology Project of Zhejiang Provincial Health Commission (2020KY195).

Author information

Qing Su, Xiaohui Sun and Liwen Zhu contributed equally to this work.

Authors and Affiliations

Department of Epidemiology and Biostatistics, School of Public Health, Zhejiang Chinese Medical University, 548 Binwen Road, Hangzhou, 310053, China

Qing Su, Xiaohui Sun, Qin Yan, Yingying Mao & Ding Ye

Department of Hematology and Oncology, Children’s hospital of Nanjing Medical University, Nanjing, 210008, China

Department of Medical Adiministration, Women’s Hospital, Zhejiang University School of Medicine, Hangzhou, 310006, China

Peiwen Zheng

You can also search for this author in PubMed   Google Scholar

Contributions

DY and YM conceived the study. QS and QY searched the databases and checked them according to the eligible criteria and exclusion criteria. XS and PZ helped develop search strategies. DY, QS, XS, and LZ analyzed the data and wrote the draft of the paper. LZ, YM, and DY contributed to reviewing or revising the paper. DY and YM are the guarantors of this work. All authors read and approve the final manuscript.

Corresponding authors

Correspondence to Yingying Mao or Ding Ye .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors have declared no conflict of interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: table s1..

The PRISMA 2009 checklist for this meta-analysis. Figure S1. Flowchart of study selection. Table S2. List of excluded studies along with reason. Table S3. Details of included studies for subgroup analysis. Figure S2. Begg’s funnel plots identifying the publication bias for the association between breastfeeding and risk of childhood leukemia. Figure S3. Forest plots of subgroup analysis of association between breastfeeding and childhood leukemia risk in the order listed in Table 2 . Table S4. Subgroup analyses of the association between breastfeeding and acute lymphoblastic leukemia risk. Figure S4. One-study-removed analysis on the association of breastfeeding with risk of (A) childhood leukemia and (B) acute lymphoblastic leukemia. Figure S5. Pooled analysis of studies including only children aged 0-14 years old for the for the association of breastfeeding with risk of (A) childhood leukemia and (B) acute lymphoblastic leukemia. Figure S6. Risk estimates (solid line) and the corresponding 95% CIs (dash lines) for the dose-response relationship between breastfeeding and the risk of childhood lymphoma. Figure S7. Begg’s funnel plots identifying the publication bias for the association between breastfeeding and risk of childhood lymphoma. Figure S8. Forest plots of subgroup analysis of association between breastfeeding and childhood lymphoma risk in the order listed in Table 3 . Figure S9. One-study-removed analysis on the association of breastfeeding with risk of childhood lymphoma. Figure S10. Pooled analysis of studies including only children aged 0-14 years old for the for the association of breastfeeding with risk of childhood lymphoma. Figure S11. Begg’s funnel plots identifying the publication bias for the association between breastfeeding and risk of childhood brain tumors. Figure S12. Forest plots of subgroup analysis of association between breastfeeding and risk of childhood brain tumors in the order listed in Table 4 . Figure S13. One-study-removed analysis on the association of breastfeeding with risk of childhood brain tumors. Figure S14. Pooled analysis of studies including only children aged 0–14 years old for the for the association of breastfeeding with risk of childhood brain tumors.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Su, Q., Sun, X., Zhu, L. et al. Breastfeeding and the risk of childhood cancer: a systematic review and dose-response meta-analysis. BMC Med 19 , 90 (2021). https://doi.org/10.1186/s12916-021-01950-5

Download citation

Received : 07 October 2020

Accepted : 24 February 2021

Published : 13 April 2021

DOI : https://doi.org/10.1186/s12916-021-01950-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Breastfeeding
• Childhood cancer
• Meta-analysis
• Dose-response

BMC Medicine

ISSN: 1741-7015

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 14, Issue 3
• Evolution of the data and methods in real-world COVID-19 vaccine effectiveness studies on mortality: a scoping review protocol
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-5397-228X Paulina Stehlik 1 , 2 ,
• http://orcid.org/0000-0001-7734-9436 Caroline Dowsett 1 ,
• Ximena Camacho 3 ,
• http://orcid.org/0000-0001-6444-7272 Michael O Falster 3 ,
• http://orcid.org/0000-0003-4135-2523 Renly Lim 4 ,
• http://orcid.org/0000-0002-9793-113X Sharifa Nasreen 5 ,
• Nicole L Pratt 6 ,
• Sallie-Anne Pearson 3 ,
• http://orcid.org/0000-0003-2934-2242 David Henry 2 , 3
• 1 School of Pharmacy and Medical Sciences , Griffith University Faculty of Health , Gold Coast , Queensland , Australia
• 2 Institute for Evidence-Based Healthcare , Bond University Faculty of Health Sciences and Medicine , Gold Coast , Queensland , Australia
• 3 School of Population Health , University of New South Wales Medicine & Health , Sydney , New South Wales , Australia
• 4 Quality Use of Medicines and Pharmacy Research Centre , University of South Australia Division of Health Sciences , Adelaide , South Australia , Australia
• 5 SUNY Downstate Health Sciences, University School of Public Health , New York , New York , USA
• 6 Quality Use of Medicines and Pharmacy Research Centre , University of South Australia Clinical & Health Sciences Academic Unit , Adelaide , South Australia , Australia
• Correspondence to Paulina Stehlik; p.stehlik{at}griffith.edu.au

Background Early evidence on COVID-19 vaccine efficacy came from randomised trials. Many important questions subsequently about vaccine effectiveness (VE) have been addressed using real-world studies (RWS) and have informed most vaccination policies globally. As the questions about VE have evolved during the pandemic so have data, study design, and analytical choices. This scoping review aims to characterise this evolution and provide insights for future pandemic planning—specifically, what kinds of questions are asked at different stages of a pandemic, and what data infrastructure and methods are used?

Methods and analysis We will identify relevant studies in the Johns Hopkins Bloomberg School of Public Health VIEW-hub database, which curates both published and preprint VE RWS identified from PubMed, Embase, Scopus, Web of Science, the WHO COVID Database, MMWR, Eurosurveillance, medRxiv, bioRxiv, SSRN, Europe PMC, Research Square, Knowledge Hub, and Google. We will include RWS of COVID-19 VE that reported COVID-19-specific or all-cause mortality (coded as ‘death’ in the ‘effectiveness studies’ data set).

Information on study characteristics; study context; data sources; design and analytic methods that address confounding will be extracted by single reviewer and checked for accuracy and discussed in a small group setting by methodological and analytic experts. A timeline mapping approach will be used to capture the evolution of this body of literature.

By describing the evolution of RWS of VE through the COVID-19 pandemic, we will help identify options for VE studies and inform policy makers on the minimal data and analytic infrastructure needed to support rapid RWS of VE in future pandemics and of healthcare strategies more broadly.

Ethics and dissemination As data is in the public domain, ethical approval is not required. Findings of this study will be disseminated through peer-reviewed publications, conference presentations, and working-papers to policy makers.

Registration https://doi.org/10.17605/OSF.IO/ZHDKR

• STATISTICS & RESEARCH METHODS
• Protocols & guidelines
• INFECTIOUS DISEASES
• PUBLIC HEALTH

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjopen-2023-079071

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

We will use a comprehensive curated database (Johns Hopkins Bloomberg School of Public Health VIEW-hub) that compiles relevant studies on a weekly basis from multiple databases, preprint servers, and the grey literature.

While use of a curated database may lead to some studies being missed, this is unlikely to change the overall findings of this scoping review.

All extraction will be conducted by a single author to ensure consistency in extraction and checked by a second author to ensure accuracy.

Weekly group discussions about the individual studies and coding of data will strengthen data integrity.

End users have been involved in the design of this study and will continue to be consulted throughout its conduct.

Introduction

The COVID-19 pandemic has been unprecedented in terms of its direct health impacts and disruption of many aspects of modern society. It has also been remarkable in the speed with which scientists and industry collaborated in the production and testing of a range of vaccines.

It became apparent quickly that the COVID-19 vaccines did not stimulate sterilising immunity but provided protection against severe illness and death, most importantly in those with underlying risk factors. 1 2 The randomised trials that formed the evidence base for the initial deployment of vaccines included few subjects who were elderly, very young, pregnant, had immunodeficiency or severe comorbidity states. 3 Although quite large, the randomised trials documented few deaths and could not provide precise estimates of the effectiveness of the vaccines in reducing COVID-related and all-cause mortality.

The subsequent evaluation of vaccine effectiveness (VE) using controlled observational studies has been complicated by changes in the infectiousness and virulence of the SARS-CoV-2 virus, and rising background levels of vaccine-induced or naturally acquired immunity. Case fatality rates have fallen substantially, particularly in highly vaccinated countries. 4 Deaths are now concentrated in a group of older patients, those with obesity and those who have serious comorbidities or are immunocompromised. 5 This rapidly changing landscape created a need for continuous ‘real-world’ studies (RWS) of VE in susceptible groups, against emerging viral variants and after repeated vaccine doses. 6 These studies use data collected outside of clinical trial settings to define exposures, endpoints, and relevant covariates. This is achieved by analysing data from electronic medical records, administrative records, death registries, and registries established specifically to record infection status and vaccine receipt. 6

Most VE studies of COVID-19 vaccines have employed large population-scale linked routinely collected data sets. However, countries have varied in the timeliness of their response to this major challenge. In some countries, for instance Israel and UK, collaborations between researchers, health service providers, and government agencies enabled rapid analyses of large data sets using sophisticated techniques to adjust for confounding and other sources of bias. In contrast, other countries, for instance Australia and Aotearoa/New Zealand, were slow to conduct effectiveness studies, in part because of low infection rates early in the pandemic, and in Australia because of difficulties in accessing the necessary linked data sets. 7 8

Systematic reviews of VE studies have concentrated, appropriately, on the vaccines’ ability to prevent serious illness and death. 9–12 They have been consistent in confirming that multiple doses of the available vaccines have been associated with large reductions in mortality, with quite rapid waning (over months) in protection, mandating a need for repeated booster doses. As the impacts of vaccines on infection and transmission have been limited and transient, 13 it diminishes the value of infection as the principal study endpoint. The decline in PCR testing and registration of antigen test results have reduced the value of test results as the basis for test negative designs. 14 15 The nature of COVID-19-related hospitalisations has changed during the pandemic with an increase in incidental findings of infection through routine testing of patients admitted for other reasons. 15 On the other hand, there has been an increasing focus on excess all-cause mortality as a measure of the success of countries in controlling the spread of the virus and mitigating its negative impacts on healthcare systems. 16 17

The COVID-19 pandemic has been a historic event that we must learn from. The rapid deployment of vaccines, followed by studies of their effectiveness, represents the largest and most important healthcare intervention in recent history and one that was evaluated largely using non-randomised studies. The sense of pandemic urgency led to rapid development of strategies to establish data sets, designs, and analytic approaches. This evolution of study questions, data designs, and methods through the course of the pandemic provides a unique learning opportunity for policy makers and researchers alike.

We plan to conduct a scoping review of the evidence base on real-world COVID-19 VE, focusing on studies that report on death as an outcome, to document this evolution. Specifically we will explore: how policy-relevant questions changed over the course of the pandemic, and how these affected the choices of data sources, designs, and analytical methods. By analysing these, we hope to provide information that is useful to the following stakeholders:

Policy makers and health system managers: by indicating what data sets will have to be created de novo and the need for linkage to existing routinely collected data in responding to future pandemics.

Clinicians and laboratory scientists: by identifying the disease manifestations and clinical and demographic vulnerability factors that will be required to inform the designs and analyses needed to evaluate the effectiveness of vaccines and other interventions, how these may change over the course of a future pandemic, and how the clinical community can advocate for the appropriate data elements to be linked and made available to researchers.

Data scientists and methodologists: to provide guidance as to study designs, analytical and adjustment techniques that are most often used in providing rapid estimates of VE early in a future pandemic; to advocate for the data elements required to deal with confounding to be collected and available in a linked analysable form.

Vaccine manufacturers: to understand better the postlicensing requirements for vaccines and pharmaceutical products under pandemic conditions and contribute appropriately to the necessary evaluations.

The pharmacoepidemiology community generally: the rapid evaluation of VE during the COVID-19 pandemic provides lessons for the timely investigation of a range of pharmaceutical treatments for emerging health threats.

We will conduct a scoping review, following the methods published by the Joanna Briggs Institute 18 and report the results according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement for scoping review (PRISMA-ScR). 19 This scoping review is registered with the Open Science Framework (OSF; https://doi.org/10.17605/OSF.IO/ZHDKR ). Data extraction has begun (25 September 2023, after protocol registration), and will continue for approximately 12 months.

Information sources and data selection

We will retrieve relevant studies from the VIEW-hub database, 20 maintained by Johns Hopkins Bloomberg School of Public Health. This database includes a wide range of study types including vaccine efficacy trials, VE studies, impact studies, and safety studies. As our principal aim is to describe the evolution of observational VE studies using real-world data, we used the VIEW-hub ‘effectiveness studies’ data set.

The VIEW-hub search strategy and inclusion criteria for this data set have been described in detail elsewhere (see online supplemental file ). 21 Briefly, the ‘effectiveness studies’ data set includes both published and preprint studies of VE identified from PubMed, Embase, Scopus, Web of Science, the WHO COVID Database, MMWR, Eurosurveillance, medRxiv, bioRxiv, SSRN, Europe PMC, Research Square, and Knowledge Hub, as well as Google alerts for COVID-19 VE studies. Studies are screened weekly by the same two epidemiologists at Johns Hopkins Bloomberg School of Public Health, and the following data elements are extracted for studies included in the data set: study author, title, date published, link to paper, country of origin, vaccine studied, variant studied, population, study start and end date, and outcomes of interest. Studies in the data set can be filtered by the vaccine, variant, outcomes, study population, and region variables through drop-down menus.

Supplemental material

Studies must also meet minimum criteria for causal inference studies using real-world data. The studies must include both vaccinated and unvaccinated (or other control) subjects, drawn from a comparable time period, capturing the relevant endpoints in both groups, having a secure record of vaccination (not relying on recall) and be free of obvious major methodological flaws. The latter judgement was not based on a strict risk of bias assessment.

To identify studies in the VIEW-hub’s ‘effectiveness studies’ data set that examine mortality (either all-cause or cause-specific), we will use the drop-down menu feature to select study outcomes coded as ‘death’. No additional eligibility criteria will be applied.

At the time of writing this protocol (1 August 2023), the VIEW-hub database lists 495 observational studies of VE from 50 countries, and 92 (~19%) list ‘death’ as an endpoint.

Data extraction

We will extract data on:

Study characteristics : country, study design, publication status, protocol available, funding sources (including whether the study was funded by an independent source or manufacturer), study ethics approval (or waiver), consent requirements (or waiver).

Study context : reported vaccine policies in place, reported dominant viral variant at time of study.

PICO-T : inclusion and exclusion criteria, exposure (ie, vaccine(s)) and definition of exposure, control group, outcome definitions, outcomes collection period duration of follow-up and number of deaths.

Data sources and additional variables : the types of data sources used (eg, survey, electronic medical records, and administrative data), which were linked at an individual level and which were not, baseline confounders collected, and for adjusted outcomes which variables they were adjusted for.

Analytical strategies to minimise bias : methods for minimising baseline confounding (eg, propensity score analysis, instrumental variable analysis, covariate adjustment, self-controlled designs, etc) and further details of how the methods were implemented as appropriate, such as how the propensity score was implemented (matching, stratification, or inverse probability of treatment weights) and which variables were included in the propensity score model. Additionally, we will extract details on whether a sensitivity analyses was conducted, subgroups analysed, methods used for dealing with missing data, and methods used for dealing with time varying environmental risk.

We anticipate that there will be a few data points where it will be difficult to provide an exhaustive list of potential categories for some of the variables of interest a priori. We will therefore take an inductive approach to categorising variables such as ‘data sources’, ‘inclusion criteria’, and ‘adjustment techniques’ by entering them in free text and then developing categories through group discussion.

The lead author (PS) will develop a purpose-built data-extraction form in SharePoint Lists and a blank copy of the form and data dictionary will be provided on our OSF site. PS will also develop a validation set using a random sample of seven papers and verified by experts in pharmacoepidemiology (DH) and analysis (XC). A single author (CD) will independently extract data on the validation set until 80% agreement is achieved, at which stage they will continue with data extraction. A second reviewer (PS) will check the accuracy of all data extractions, and a core team (DH, CD, PS, and XC) will meet regularly to discuss each study, ensure it meets the inclusion criteria, and the main messages that it provides. The broader study team will meet less frequently to address issues arising and ensure data are categorised in a meaningful way that helps to inform decision making.

All data will be made publicly available via our study’s OSF page ( https://osf.io/m4cbf/ ).

Assessment of risk of bias

We aim to describe the evolution of the literature and will therefore not conduct a formal assessment of the risk of bias in the included studies. However, all included studies in the VIEW-hub database must meet a minimal set of quality criteria, and while this does not mean that they are free of bias, the process aims to ensure a baseline level of quality.

Data synthesis

To describe the evolution of RWS of COVID-19 VE over the course of the pandemic, we will use descriptive statistics to quantify study characteristics—including evolution of study designs (eg, test-negative designs, cohorts, and regression discontinuity), research questions asked (eg, comparisons of two doses vs boosters, effectiveness, and waning effect), data sources (eg, regularly collected population data and registry data), analytic approaches (eg, by design or form of adjustment), populations included, countries studied, outcome definitions, and event rates.

We will provide a temporal sequence of these characteristics overall, and where there are sufficient data within countries, present them visually (eg, as annotated stacked area graphs) to establish a template that enables anticipation of study questions and therefore supports planning for data availability in future pandemics.

We plan to develop interactive visuals as outputs so that stakeholders can interrogate the data further. All data manipulation, analysis, and visualisation will occur using Python and R and we will share all code via OSF.

Review team and consultation

Our review team and reference group consist of content experts in review methodology, vaccine and drug effectiveness studies, biostatistics, and data science. Several have been involved directly in the conduct of VE studies during the COVID-19 pandemic and have a good working knowledge of the relevant literature. Most of the team members are actively involved in the National Health and Medical Research Council (NHMRC)-funded Centre for Research Excellence in Medicines Intelligence, which aims to accelerate real-world evidence development to inform medicines policy decision making. 22 Our reference group also comprises end users in infectious diseases and pandemic management, vaccine epidemiology, and medicines and vaccine policy.

All authors and advisory group members have provided comment on this protocol, and the appropriateness of the research questions and data elements. The advisory group will be consulted on how best to present the data so that it is usable and helps with decision making in each member’s respective area.

In addition, we anticipate that the data we collect can be used for future review automation work and improve the efficiency of research. Our advisory group also includes an expert in review methodology and automation who will provide advice on future-proofing our dataset.

Ethics and dissemination

As this scoping review will only include data in the public domain, ethics review is not required.

Findings of this review will be relevant to several stakeholders, including those involved in pandemic response, data infrastructure, and health technology evaluation. As such, we will disseminate our findings in five ways: (1) working papers for policy makers in Australia; (2) open access publication of findings in peer-reviewed journals; (3) presentation of findings at local and international infectious disease, vaccine, health systems, and health management conferences; (4) online interactive visual to allow interrogation of the extracted data; and (5) open access to our data, code, and preprints via OSF.

Ethics statements

Patient consent for publication.

Not applicable.

Acknowledgments

We would like to acknowledge the advisory group and Oyungerel Byambasuren to their feedback and comments on draft versions of this document, particularly on the methodology and which variables to collect to provide meaningful information to decision makers.

• Polack FP ,
• Thomas SJ ,
• Kitchin N , et al
• Clemens SAC ,
• Madhi SA , et al
• Liu F , et al
• Guitian J , et al
• Ponzo S , et al
• White C , et al
• Stehlik P ,
• Camacho X , et al
• Stehlik P , et al
• Liu J , et al
• Kalafat E ,
• Blakeway H , et al
• Rahmani K ,
• Shavaleh R ,
• Forouhi M , et al
• Rodríguez-Barraquer I , et al
• Mukherjee B
• UK Health Security Agency working with the Medicines & Healthcare products Regulatory Agency
• ↵ The true death toll of COVID-19: estimating global excess mortality: World Health Organization , Available : https://www.who.int/data/stories/the-true-death-toll-of-covid-19-estimating-global-excess-mortality [Accessed 7 Mar 2023 ].
• Aromataris E ,
• Tricco AC ,
• Zarin W , et al
• ↵ International vaccine access center (IVAC), Johns Hopkins Bloomberg school of public health. VIEW-Hub ; 2023 .
• International Vaccine Access Center (IVAC), Johns Hopkins Bloomberg School of Public Health, World Health Organization, Coalition for Epidemic Preparedness Innovation
• Camacho X ,
• Vajdic C , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Twitter @DrRenly

Contributors PS and DH conceptualised the project, acquired the funding, and are acting as project supervisors. PS, CD, XC, MOF, RL, SN, NLP, S-AP, and DH contributed to the methodology. PS developed the resources and database and will oversee database and project management. PS, DH, and XC piloted the database and extraction tool and developed the validation set. CD is conducting the data extraction which will be checked by PS. PS, XC, and MOF developed the data synthesis plan. PS and DH wrote the original draft of this manuscript, and PS, CD, XC, MOF, RL, SN, NLP, S-AP, and DH all edited and reviewed the draft and final revisions.

Funding Medicines Intelligence Centre for Research Excellence (MI-CRE) 2022 Project Incubator Grant; The MI-CRE is supported by the National Health and Medical Research Council’s Centres of Research Excellence (CRE) scheme (ID 1196900). RL is supported by a National Health and Medical Research Council (NHMRC) Early Career Fellowship (APP1156368). MOF is supported by a Future Leader Fellowship from the National Heart Foundation of Australia (ID: 105609). XC is supported by a NHMRC Postgraduate Scholarship (ID: 2005259).

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: