synthesis in research rrl example

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• 6. Synthesize
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate

Synthesis Visualization

Synthesis matrix example.

• 7. Write a Literature Review

• Synthesis Worksheet

About Synthesis

Approaches to synthesis.

You can sort the literature in various ways, for example:

How to Begin?

Read your sources carefully and find the main idea(s) of each source

Look for similarities in your sources – which sources are talking about the same main ideas? (for example, sources that discuss the historical background on your topic)

Use the worksheet (above) or synthesis matrix (below) to get organized

This work can be messy. Don't worry if you have to go through a few iterations of the worksheet or matrix as you work on your lit review!

Four Examples of Student Writing

In the four examples below, only ONE shows a good example of synthesis: the fourth column, or  Student D . For a web accessible version, click the link below the image.

Long description of "Four Examples of Student Writing" for web accessibility

• Download a copy of the "Four Examples of Student Writing" chart

Click on the example to view the pdf.

From Jennifer Lim

• << Previous: 5. Critically Analyze and Evaluate
• Next: 7. Write a Literature Review >>
• Last Updated: Jan 10, 2024 4:46 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Find This link opens in a new window
• Evaluate This link opens in a new window

Get Organized

• Lit Review Prep Use this template to help you evaluate your sources, create article summaries for an annotated bibliography, and a synthesis matrix for your lit review outline.

Synthesize your Information

Synthesize: combine separate elements to form a whole.

Synthesis Matrix

A synthesis matrix helps you record the main points of each source and document how sources relate to each other.

After summarizing and evaluating your sources, arrange them in a matrix or use a citation manager to help you see how they relate to each other and apply to each of your themes or variables.  

By arranging your sources by theme or variable, you can see how your sources relate to each other, and can start thinking about how you weave them together to create a narrative.

• Step-by-Step Approach
• Example Matrix from NSCU
• Matrix Template
• << Previous: Summarize
• Next: Integrate >>
• Last Updated: Sep 26, 2023 10:25 AM
• URL: https://guides.library.jhu.edu/lit-review

Literature Syntheis 101

How To Synthesise The Existing Research (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Eunice Rautenbach (DTech) | August 2023

One of the most common mistakes that students make when writing a literature review is that they err on the side of describing the existing literature rather than providing a critical synthesis of it. In this post, we’ll unpack what exactly synthesis means and show you how to craft a strong literature synthesis using practical examples.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Literature Synthesis

• What exactly does “synthesis” mean?
• Aspect 1: Agreement
• Aspect 2: Disagreement
• Aspect 3: Key theories
• Aspect 4: Contexts
• Aspect 5: Methodologies
• Bringing it all together

What does “synthesis” actually mean?

As a starting point, let’s quickly define what exactly we mean when we use the term “synthesis” within the context of a literature review.

Simply put, literature synthesis means going beyond just describing what everyone has said and found. Instead, synthesis is about bringing together all the information from various sources to present a cohesive assessment of the current state of knowledge in relation to your study’s research aims and questions .

Put another way, a good synthesis tells the reader exactly where the current research is “at” in terms of the topic you’re interested in – specifically, what’s known , what’s not , and where there’s a need for more research .

So, how do you go about doing this?

Well, there’s no “one right way” when it comes to literature synthesis, but we’ve found that it’s particularly useful to ask yourself five key questions when you’re working on your literature review. Having done so,  you can then address them more articulately within your actual write up. So, let’s take a look at each of these questions.

1. Points Of Agreement

The first question that you need to ask yourself is: “Overall, what things seem to be agreed upon by the vast majority of the literature?”

For example, if your research aim is to identify which factors contribute toward job satisfaction, you’ll need to identify which factors are broadly agreed upon and “settled” within the literature. Naturally, there may at times be some lone contrarian that has a radical viewpoint , but, provided that the vast majority of researchers are in agreement, you can put these random outliers to the side. That is, of course, unless your research aims to explore a contrarian viewpoint and there’s a clear justification for doing so. 

Identifying what’s broadly agreed upon is an essential starting point for synthesising the literature, because you generally don’t want (or need) to reinvent the wheel or run down a road investigating something that is already well established . So, addressing this question first lays a foundation of “settled” knowledge.

Need a helping hand?

2. Points Of Disagreement

Related to the previous point, but on the other end of the spectrum, is the equally important question: “Where do the disagreements lie?” .

In other words, which things are not well agreed upon by current researchers? It’s important to clarify here that by disagreement, we don’t mean that researchers are (necessarily) fighting over it – just that there are relatively mixed findings within the empirical research , with no firm consensus amongst researchers.

This is a really important question to address as these “disagreements” will often set the stage for the research gap(s). In other words, they provide clues regarding potential opportunities for further research, which your study can then (hopefully) contribute toward filling. If you’re not familiar with the concept of a research gap, be sure to check out our explainer video covering exactly that .

3. Key Theories

The next question you need to ask yourself is: “Which key theories seem to be coming up repeatedly?” .

Within most research spaces, you’ll find that you keep running into a handful of key theories that are referred to over and over again. Apart from identifying these theories, you’ll also need to think about how they’re connected to each other. Specifically, you need to ask yourself:

• Are they all covering the same ground or do they have different focal points  or underlying assumptions ?
• Do some of them feed into each other and if so, is there an opportunity to integrate them into a more cohesive theory?
• Do some of them pull in different directions ? If so, why might this be?
• Do all of the theories define the key concepts and variables in the same way, or is there some disconnect? If so, what’s the impact of this ?

Simply put, you’ll need to pay careful attention to the key theories in your research area, as they will need to feature within your theoretical framework , which will form a critical component within your final literature review. This will set the foundation for your entire study, so it’s essential that you be critical in this area of your literature synthesis.

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of our limited-time offer to get 60% off the standard price.

4. Contexts

The next question that you need to address in your literature synthesis is an important one, and that is: “Which contexts have (and have not) been covered by the existing research?” .

For example, sticking with our earlier hypothetical topic (factors that impact job satisfaction), you may find that most of the research has focused on white-collar , management-level staff within a primarily Western context, but little has been done on blue-collar workers in an Eastern context. Given the significant socio-cultural differences between these two groups, this is an important observation, as it could present a contextual research gap .

In practical terms, this means that you’ll need to carefully assess the context of each piece of literature that you’re engaging with, especially the empirical research (i.e., studies that have collected and analysed real-world data). Ideally, you should keep notes regarding the context of each study in some sort of catalogue or sheet, so that you can easily make sense of this before you start the writing phase. If you’d like, our free literature catalogue worksheet is a great tool for this task.

5. Methodological Approaches

Last but certainly not least, you need to ask yourself the question: “What types of research methodologies have (and haven’t) been used?”

For example, you might find that most studies have approached the topic using qualitative methods such as interviews and thematic analysis. Alternatively, you might find that most studies have used quantitative methods such as online surveys and statistical analysis.

But why does this matter?

Well, it can run in one of two potential directions . If you find that the vast majority of studies use a specific methodological approach, this could provide you with a firm foundation on which to base your own study’s methodology . In other words, you can use the methodologies of similar studies to inform (and justify) your own study’s research design .

On the other hand, you might argue that the lack of diverse methodological approaches presents a research gap , and therefore your study could contribute toward filling that gap by taking a different approach. For example, taking a qualitative approach to a research area that is typically approached quantitatively. Of course, if you’re going to go against the methodological grain, you’ll need to provide a strong justification for why your proposed approach makes sense. Nevertheless, it is something worth at least considering.

Regardless of which route you opt for, you need to pay careful attention to the methodologies used in the relevant studies and provide at least some discussion about this in your write-up. Again, it’s useful to keep track of this on some sort of spreadsheet or catalogue as you digest each article, so consider grabbing a copy of our free literature catalogue if you don’t have anything in place.

Bringing It All Together

Alright, so we’ve looked at five important questions that you need to ask (and answer) to help you develop a strong synthesis within your literature review.  To recap, these are:

• Which things are broadly agreed upon within the current research?
• Which things are the subject of disagreement (or at least, present mixed findings)?
• Which theories seem to be central to your research topic and how do they relate or compare to each other?
• Which contexts have (and haven’t) been covered?
• Which methodological approaches are most common?

Importantly, you’re not just asking yourself these questions for the sake of asking them – they’re not just a reflection exercise. You need to weave your answers to them into your actual literature review when you write it up. How exactly you do this will vary from project to project depending on the structure you opt for, but you’ll still need to address them within your literature review, whichever route you go.

The best approach is to spend some time actually writing out your answers to these questions, as opposed to just thinking about them in your head. Putting your thoughts onto paper really helps you flesh out your thinking . As you do this, don’t just write down the answers – instead, think about what they mean in terms of the research gap you’ll present , as well as the methodological approach you’ll take . Your literature synthesis needs to lay the groundwork for these two things, so it’s essential that you link all of it together in your mind, and of course, on paper.

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

excellent , thank you

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Working with sources
• Synthesizing Sources | Examples & Synthesis Matrix

Synthesizing Sources | Examples & Synthesis Matrix

Published on July 4, 2022 by Eoghan Ryan . Revised on May 31, 2023.

Synthesizing sources involves combining the work of other scholars to provide new insights. It’s a way of integrating sources that helps situate your work in relation to existing research.

Synthesizing sources involves more than just summarizing . You must emphasize how each source contributes to current debates, highlighting points of (dis)agreement and putting the sources in conversation with each other.

You might synthesize sources in your literature review to give an overview of the field or throughout your research paper when you want to position your work in relation to existing research.

Table of contents

Example of synthesizing sources, how to synthesize sources, synthesis matrix, other interesting articles, frequently asked questions about synthesizing sources.

Let’s take a look at an example where sources are not properly synthesized, and then see what can be done to improve it.

This paragraph provides no context for the information and does not explain the relationships between the sources described. It also doesn’t analyze the sources or consider gaps in existing research.

Research on the barriers to second language acquisition has primarily focused on age-related difficulties. Building on Lenneberg’s (1967) theory of a critical period of language acquisition, Johnson and Newport (1988) tested Lenneberg’s idea in the context of second language acquisition. Their research seemed to confirm that young learners acquire a second language more easily than older learners. Recent research has considered other potential barriers to language acquisition. Schepens, van Hout, and van der Slik (2022) have revealed that the difficulties of learning a second language at an older age are compounded by dissimilarity between a learner’s first language and the language they aim to acquire. Further research needs to be carried out to determine whether the difficulty faced by adult monoglot speakers is also faced by adults who acquired a second language during the “critical period.”

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To synthesize sources, group them around a specific theme or point of contention.

As you read sources, ask:

• What questions or ideas recur? Do the sources focus on the same points, or do they look at the issue from different angles?
• How does each source relate to others? Does it confirm or challenge the findings of past research?
• Where do the sources agree or disagree?

Once you have a clear idea of how each source positions itself, put them in conversation with each other. Analyze and interpret their points of agreement and disagreement. This displays the relationships among sources and creates a sense of coherence.

Consider both implicit and explicit (dis)agreements. Whether one source specifically refutes another or just happens to come to different conclusions without specifically engaging with it, you can mention it in your synthesis either way.

Synthesize your sources using:

• Topic sentences to introduce the relationship between the sources
• Signal phrases to attribute ideas to their authors
• Transition words and phrases to link together different ideas

To more easily determine the similarities and dissimilarities among your sources, you can create a visual representation of their main ideas with a synthesis matrix . This is a tool that you can use when researching and writing your paper, not a part of the final text.

In a synthesis matrix, each column represents one source, and each row represents a common theme or idea among the sources. In the relevant rows, fill in a short summary of how the source treats each theme or topic.

This helps you to clearly see the commonalities or points of divergence among your sources. You can then synthesize these sources in your work by explaining their relationship.

If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.

• ChatGPT vs human editor
• ChatGPT citations
• Is ChatGPT trustworthy?
• Using ChatGPT for your studies
• What is ChatGPT?
• Chicago style
• Paraphrasing

 Plagiarism

• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Consequences of plagiarism
• Common knowledge

Scribbr Citation Checker New

The AI-powered Citation Checker helps you avoid common mistakes such as:

• Missing commas and periods
• Incorrect usage of “et al.”
• Ampersands (&) in narrative citations
• Missing reference entries

Synthesizing sources means comparing and contrasting the work of other scholars to provide new insights.

It involves analyzing and interpreting the points of agreement and disagreement among sources.

You might synthesize sources in your literature review to give an overview of the field of research or throughout your paper when you want to contribute something new to existing research.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

Topic sentences help keep your writing focused and guide the reader through your argument.

In an essay or paper , each paragraph should focus on a single idea. By stating the main idea in the topic sentence, you clarify what the paragraph is about for both yourself and your reader.

At college level, you must properly cite your sources in all essays , research papers , and other academic texts (except exams and in-class exercises).

Add a citation whenever you quote , paraphrase , or summarize information or ideas from a source. You should also give full source details in a bibliography or reference list at the end of your text.

The exact format of your citations depends on which citation style you are instructed to use. The most common styles are APA , MLA , and Chicago .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Ryan, E. (2023, May 31). Synthesizing Sources | Examples & Synthesis Matrix. Scribbr. Retrieved March 26, 2024, from https://www.scribbr.com/working-with-sources/synthesizing-sources/

Is this article helpful?

Eoghan Ryan

Other students also liked, signal phrases | definition, explanation & examples, how to write a literature review | guide, examples, & templates, how to find sources | scholarly articles, books, etc., "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 7: Synthesizing Sources

Learning objectives.

At the conclusion of this chapter, you will be able to:

• synthesize key sources connecting them with the research question and topic area.

7.1 Overview of synthesizing

7.1.1 putting the pieces together.

Combining separate elements into a whole is the dictionary definition of synthesis.  It is a way to make connections among and between numerous and varied source materials.  A literature review is not an annotated bibliography, organized by title, author, or date of publication.  Rather, it is grouped by topic to create a whole view of the literature relevant to your research question.

Your synthesis must demonstrate a critical analysis of the papers you collected as well as your ability to integrate the results of your analysis into your own literature review.  Each paper collected should be critically evaluated and weighed for “adequacy, appropriateness, and thoroughness” ( Garrard, 2017 ) before inclusion in your own review.  Papers that do not meet this criteria likely should not be included in your literature review.

Begin the synthesis process by creating a grid, table, or an outline where you will summarize, using common themes you have identified and the sources you have found. The summary grid or outline will help you compare and contrast the themes so you can see the relationships among them as well as areas where you may need to do more searching. Whichever method you choose, this type of organization will help you to both understand the information you find and structure the writing of your review.  Remember, although “the means of summarizing can vary, the key at this point is to make sure you understand what you’ve found and how it relates to your topic and research question” ( Bennard et al., 2014 ).

As you read through the material you gather, look for common themes as they may provide the structure for your literature review.  And, remember, research is an iterative process: it is not unusual to go back and search information sources for more material.

At one extreme, if you are claiming, ‘There are no prior publications on this topic,’ it is more likely that you have not found them yet and may need to broaden your search.  At another extreme, writing a complete literature review can be difficult with a well-trod topic.  Do not cite it all; instead cite what is most relevant.  If that still leaves too much to include, be sure to reference influential sources…as well as high-quality work that clearly connects to the points you make. ( Klingner, Scanlon, & Pressley, 2005 ).

7.2 Creating a summary table

Literature reviews can be organized sequentially or by topic, theme, method, results, theory, or argument.  It’s important to develop categories that are meaningful and relevant to your research question.  Take detailed notes on each article and use a consistent format for capturing all the information each article provides.  These notes and the summary table can be done manually, using note cards.  However, given the amount of information you will be recording, an electronic file created in a word processing or spreadsheet is more manageable. Examples of fields you may want to capture in your notes include:

• Authors’ names
• Article title
• Publication year
• Main purpose of the article
• Methodology or research design
• Participants
• Measurement
• Conclusions

  Other fields that will be useful when you begin to synthesize the sum total of your research:

• Specific details of the article or research that are especially relevant to your study
• Key terms and definitions
• Strengths or weaknesses in research design
• Relationships to other studies
• Possible gaps in the research or literature (for example, many research articles conclude with the statement “more research is needed in this area”)
• Finally, note how closely each article relates to your topic.  You may want to rank these as high, medium, or low relevance.  For papers that you decide not to include, you may want to note your reasoning for exclusion, such as ‘small sample size’, ‘local case study,’ or ‘lacks evidence to support assertion.’

This short video demonstrates how a nursing researcher might create a summary table.

7.2.1 Creating a Summary Table

  Summary tables can be organized by author or by theme, for example:

For a summary table template, see http://blogs.monm.edu/writingatmc/files/2013/04/Synthesis-Matrix-Template.pdf

7.3 Creating a summary outline

An alternate way to organize your articles for synthesis it to create an outline. After you have collected the articles you intend to use (and have put aside the ones you won’t be using), it’s time to identify the conclusions that can be drawn from the articles as a group.

  Based on your review of the collected articles, group them by categories.  You may wish to further organize them by topic and then chronologically or alphabetically by author.  For each topic or subtopic you identified during your critical analysis of the paper, determine what those papers have in common.  Likewise, determine which ones in the group differ.  If there are contradictory findings, you may be able to identify methodological or theoretical differences that could account for the contradiction (for example, differences in population demographics).  Determine what general conclusions you can report about the topic or subtopic as the entire group of studies relate to it.  For example, you may have several studies that agree on outcome, such as ‘hands on learning is best for science in elementary school’ or that ‘continuing education is the best method for updating nursing certification.’ In that case, you may want to organize by methodology used in the studies rather than by outcome.

Organize your outline in a logical order and prepare to write the first draft of your literature review.  That order might be from broad to more specific, or it may be sequential or chronological, going from foundational literature to more current.  Remember, “an effective literature review need not denote the entire historical record, but rather establish the raison d’etre for the current study and in doing so cite that literature distinctly pertinent for theoretical, methodological, or empirical reasons.” ( Milardo, 2015, p. 22 ).

As you organize the summarized documents into a logical structure, you are also appraising and synthesizing complex information from multiple sources.  Your literature review is the result of your research that synthesizes new and old information and creates new knowledge.

7.4 Additional resources:

Literature Reviews: Using a Matrix to Organize Research / Saint Mary’s University of Minnesota

Literature Review: Synthesizing Multiple Sources / Indiana University

Writing a Literature Review and Using a Synthesis Matrix / Florida International University

 Sample Literature Reviews Grid / Complied by Lindsay Roberts

Select three or four articles on a single topic of interest to you. Then enter them into an outline or table in the categories you feel are important to a research question. Try both the grid and the outline if you can to see which suits you better. The attached grid contains the fields suggested in the video .

Literature Review Table  

Test yourself.

• Select two articles from your own summary table or outline and write a paragraph explaining how and why the sources relate to each other and your review of the literature.
• In your literature review, under what topic or subtopic will you place the paragraph you just wrote?

Image attribution

Literature Reviews for Education and Nursing Graduate Students Copyright © by Linda Frederiksen is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Library Guides

Literature reviews: synthesis.

• Criticality

Synthesise Information

So, how can you create paragraphs within your literature review that demonstrates your knowledge of the scholarship that has been done in your field of study?  

You will need to present a synthesis of the texts you read.  

Doug Specht, Senior Lecturer at the Westminster School of Media and Communication, explains synthesis for us in the following video:  

Synthesising Texts  

What is synthesis? 

Synthesis is an important element of academic writing, demonstrating comprehension, analysis, evaluation and original creation.  

With synthesis you extract content from different sources to create an original text. While paraphrase and summary maintain the structure of the given source(s), with synthesis you create a new structure.  

The sources will provide different perspectives and evidence on a topic. They will be put together when agreeing, contrasted when disagreeing. The sources must be referenced.  

Perfect your synthesis by showing the flow of your reasoning, expressing critical evaluation of the sources and drawing conclusions.  

When you synthesise think of "using strategic thinking to resolve a problem requiring the integration of diverse pieces of information around a structuring theme" (Mateos and Sole 2009, p448). 

Synthesis is a complex activity, which requires a high degree of comprehension and active engagement with the subject. As you progress in higher education, so increase the expectations on your abilities to synthesise. 

How to synthesise in a literature review: 

Identify themes/issues you'd like to discuss in the literature review. Think of an outline.  

Read the literature and identify these themes/issues.  

Critically analyse the texts asking: how does the text I'm reading relate to the other texts I've read on the same topic? Is it in agreement? Does it differ in its perspective? Is it stronger or weaker? How does it differ (could be scope, methods, year of publication etc.). Draw your conclusions on the state of the literature on the topic.  

Start writing your literature review, structuring it according to the outline you planned.  

Put together sources stating the same point; contrast sources presenting counter-arguments or different points.  

Present your critical analysis.  

Always provide the references. 

The best synthesis requires a "recursive process" whereby you read the source texts, identify relevant parts, take notes, produce drafts, re-read the source texts, revise your text, re-write... (Mateos and Sole, 2009). 

What is good synthesis?  

The quality of your synthesis can be assessed considering the following (Mateos and Sole, 2009, p439):  

Integration and connection of the information from the source texts around a structuring theme. 

Selection of ideas necessary for producing the synthesis. 

Appropriateness of the interpretation.  

Elaboration of the content.  

Example of Synthesis

Original texts (fictitious): 

Synthesis: 

Animal experimentation is a subject of heated debate. Some argue that painful experiments should be banned. Indeed it has been demonstrated that such experiments make animals suffer physically and psychologically (Chowdhury 2012; Panatta and Hudson 2016). On the other hand, it has been argued that animal experimentation can save human lives and reduce harm on humans (Smith 2008). This argument is only valid for toxicological testing, not for tests that, for example, merely improve the efficacy of a cosmetic (Turner 2015). It can be suggested that animal experimentation should be regulated to only allow toxicological risk assessment, and the suffering to the animals should be minimised.   

Bibliography

Mateos, M. and Sole, I. (2009). Synthesising Information from various texts: A Study of Procedures and Products at Different Educational Levels. European Journal of Psychology of Education,  24 (4), 435-451. Available from https://doi.org/10.1007/BF03178760 [Accessed 29 June 2021].

• << Previous: Structure
• Next: Criticality >>
• Last Updated: Nov 18, 2023 10:56 PM
• URL: https://libguides.westminster.ac.uk/literature-reviews

CONNECT WITH US

• Herron School of Art
• Ruth Lilly Law
• Ruth Lilly Medical
• School of Dentistry

Literature Review - A Self-Guided Tutorial

• Literature Reviews: A Recap
• Reading Journal Articles
• Does it describe a Literature Review?
• 1. Identify the question
• 2. Review discipline styles
• Searching article databases - video
• Finding the article full-text
• Citation chaining
• When to stop searching
• 4. Manage your references
• 5. Critically analyze and evaluate
• 6. Synthesize
• 7. Write literature review

Who's My Librarian?

Locate your University Library's subject librarian  for personalized assistance.

Students doing research in specific areas may also request assistance at other IUPUI libraries:

• IU School of Dentistry Library
• Ruth Lilly Law Library
• Ruth Lilly Medical Library  

You can sort the literature in various ways, for example:

Synthesis Matrix Example

From Jennifer Lim

• << Previous: 5. Critically analyze and evaluate
• Next: 7. Write literature review >>
• Last Updated: Jun 15, 2023 8:24 AM
• URL: https://iupui.libguides.com/literaturereview

Educational resources and simple solutions for your research journey

How to Write Review of Related Literature (RRL) in Research

A review of related literature (a.k.a RRL in research) is a comprehensive review of the existing literature pertaining to a specific topic or research question. An effective review provides the reader with an organized analysis and synthesis of the existing knowledge about a subject. With the increasing amount of new information being disseminated every day, conducting a review of related literature is becoming more difficult and the purpose of review of related literature is clearer than ever.  

All new knowledge is necessarily based on previously known information, and every new scientific study must be conducted and reported in the context of previous studies. This makes a review of related literature essential for research, and although it may be tedious work at times , most researchers will complete many such reviews of varying depths during their career. So, why exactly is a review of related literature important?    

Table of Contents

Why a review of related literature in research is important  

Before thinking how to do reviews of related literature , it is necessary to understand its importance. Although the purpose of a review of related literature varies depending on the discipline and how it will be used, its importance is never in question. Here are some ways in which a review can be crucial.  

• Identify gaps in the knowledge – This is the primary purpose of a review of related literature (often called RRL in research ). To create new knowledge, you must first determine what knowledge may be missing. This also helps to identify the scope of your study.  
• Avoid duplication of research efforts – Not only will a review of related literature indicate gaps in the existing research, but it will also lead you away from duplicating research that has already been done and thus save precious resources.  
• Provide an overview of disparate and interdisciplinary research areas – Researchers cannot possibly know everything related to their disciplines. Therefore, it is very helpful to have access to a review of related literature already written and published.  
• Highlight researcher’s familiarity with their topic 1  – A strong review of related literature in a study strengthens readers’ confidence in that study and that researcher.

Tips on how to write a review of related literature in research

Given that you will probably need to produce a number of these at some point, here are a few general tips on how to write an effective review of related literature 2 .

• Define your topic, audience, and purpose: You will be spending a lot of time with this review, so choose a topic that is interesting to you. While deciding what to write in a review of related literature , think about who you expect to read the review – researchers in your discipline, other scientists, the general public – and tailor the language to the audience. Also, think about the purpose of your review of related literature .  
• Conduct a comprehensive literature search: While writing your review of related literature , emphasize more recent works but don’t forget to include some older publications as well. Cast a wide net, as you may find some interesting and relevant literature in unexpected databases or library corners. Don’t forget to search for recent conference papers.
• Review the identified articles and take notes: It is a good idea to take notes in a way such that individual items in your notes can be moved around when you organize them. For example, index cards are great tools for this. Write each individual idea on a separate card along with the source. The cards can then be easily grouped and organized.  
• Determine how to organize your review: A review of related literature should not be merely a listing of descriptions. It should be organized by some criterion, such as chronologically or thematically.  
• Be critical and objective: Don’t just report the findings of other studies in your review of related literature . Challenge the methodology, find errors in the analysis, question the conclusions. Use what you find to improve your research. However, do not insert your opinions into the review of related literature. Remain objective and open-minded.  
• Structure your review logically: Guide the reader through the information. The structure will depend on the function of the review of related literature. Creating an outline prior to writing the RRL in research is a good way to ensure the presented information flows well.  

As you read more extensively in your discipline, you will notice that the review of related literature appears in various forms in different places. For example, when you read an article about an experimental study, you will typically see a literature review or a RRL in research , in the introduction that includes brief descriptions of similar studies. In longer research studies and dissertations, especially in the social sciences, the review of related literature will typically be a separate chapter and include more information on methodologies and theory building. In addition, stand-alone review articles will be published that are extremely useful to researchers.  

The review of relevant literature or often abbreviated as, RRL in research , is an important communication tool that can be used in many forms for many purposes. It is a tool that all researchers should befriend.  

• University of North Carolina at Chapel Hill Writing Center. Literature Reviews.  https://writingcenter.unc.edu/tips-and-tools/literature-reviews/  [Accessed September 8, 2022]
• Pautasso M. Ten simple rules for writing a literature review. PLoS Comput Biol. 2013, 9. doi: 10.1371/journal.pcbi.1003149.

Q:  Is research complete without a review of related literature?

A research project is usually considered incomplete without a proper review of related literature. The review of related literature is a crucial component of any research project as it provides context for the research question, identifies gaps in existing literature, and ensures novelty by avoiding duplication. It also helps inform research design and supports arguments, highlights the significance of a study, and demonstrates your knowledge an expertise.

Q: What is difference between RRL and RRS?

The key difference between an RRL and an RRS lies in their focus and scope. An RRL or review of related literature examines a broad range of literature, including theoretical frameworks, concepts, and empirical studies, to establish the context and significance of the research topic. On the other hand, an RRS or review of research studies specifically focuses on analyzing and summarizing previous research studies within a specific research domain to gain insights into methodologies, findings, and gaps in the existing body of knowledge. While there may be some overlap between the two, they serve distinct purposes and cover different aspects of the research process.

Q: Does review of related literature improve accuracy and validity of research?

Yes, a comprehensive review of related literature (RRL) plays a vital role in improving the accuracy and validity of research. It helps authors gain a deeper understanding and offers different perspectives on the research topic. RRL can help you identify research gaps, dictate the selection of appropriate research methodologies, enhance theoretical frameworks, avoid biases and errors, and even provide support for research design and interpretation. By building upon and critically engaging with existing related literature, researchers can ensure their work is rigorous, reliable, and contributes meaningfully to their field of study.

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$39 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

What is Inductive Reasoning? Definition, Types and Examples

How R Discovery achieved 2X engagement with GetFTR

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• AIMS Public Health
• v.3(1); 2016

What Synthesis Methodology Should I Use? A Review and Analysis of Approaches to Research Synthesis

Kara schick-makaroff.

1 Faculty of Nursing, University of Alberta, Edmonton, AB, Canada

Marjorie MacDonald

2 School of Nursing, University of Victoria, Victoria, BC, Canada

Marilyn Plummer

3 College of Nursing, Camosun College, Victoria, BC, Canada

Judy Burgess

4 Student Services, University Health Services, Victoria, BC, Canada

Wendy Neander

Associated data, additional file 1.

When we began this process, we were doctoral students and a faculty member in a research methods course. As students, we were facing a review of the literature for our dissertations. We encountered several different ways of conducting a review but were unable to locate any resources that synthesized all of the various synthesis methodologies. Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use ‘research synthesis’ as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.

We conducted an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis. We searched five databases, reviewed websites of key organizations, hand-searched several journals, and examined relevant texts from the reference lists of the documents we had already obtained.

We identified four broad categories of research synthesis methodology including conventional, quantitative, qualitative, and emerging syntheses. Each of the broad categories was compared to the others on the following: key characteristics, purpose, method, product, context, underlying assumptions, unit of analysis, strengths and limitations, and when to use each approach.

Conclusions

The current state of research synthesis reflects significant advancements in emerging synthesis studies that integrate diverse data types and sources. New approaches to research synthesis provide a much broader range of review alternatives available to health and social science students and researchers.

1. Introduction

Since the turn of the century, public health emergencies have been identified worldwide, particularly related to infectious diseases. For example, the Severe Acute Respiratory Syndrome (SARS) epidemic in Canada in 2002-2003, the recent Ebola epidemic in Africa, and the ongoing HIV/AIDs pandemic are global health concerns. There have also been dramatic increases in the prevalence of chronic diseases around the world [1] – [3] . These epidemiological challenges have raised concerns about the ability of health systems worldwide to address these crises. As a result, public health systems reform has been initiated in a number of countries. In Canada, as in other countries, the role of evidence to support public health reform and improve population health has been given high priority. Yet, there continues to be a significant gap between the production of evidence through research and its application in practice [4] – [5] . One strategy to address this gap has been the development of new research synthesis methodologies to deal with the time-sensitive and wide ranging evidence needs of policy makers and practitioners in all areas of health care, including public health.

As doctoral nursing students facing a review of the literature for our dissertations, and as a faculty member teaching a research methods course, we encountered several ways of conducting a research synthesis but found no comprehensive resources that discussed, compared, and contrasted various synthesis methodologies on their purposes, processes, strengths and limitations. To complicate matters, writers use terms interchangeably or use different terms to mean the same thing, and the literature is often contradictory about various approaches. Some texts [6] , [7] – [9] did provide a preliminary understanding about how research synthesis had been taken up in nursing, but these did not meet our requirements. Thus, in this article we address the need for a comprehensive overview of research synthesis methodologies to guide public health, health care, and social science researchers and practitioners.

Research synthesis is relatively new in public health but has a long history in other fields dating back to the late 1800s. Research synthesis, a research process in its own right [10] , has become more prominent in the wake of the evidence-based movement of the 1990s. Research syntheses have found their advocates and detractors in all disciplines, with challenges to the processes of systematic review and meta-analysis, in particular, being raised by critics of evidence-based healthcare [11] – [13] .

Our purpose was to conduct an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis [14] – [15] . We synthesize and critique the main approaches to research synthesis that are relevant for public health, health care, and social scientists. Research synthesis is the overarching term we use to describe approaches to combining, aggregating, integrating, and synthesizing primary research findings. Each synthesis methodology draws on different types of findings depending on the purpose and product of the chosen synthesis (see Additional File 1 ).

3. Method of Review

Based on our current knowledge of the literature, we identified these approaches to include in our review: systematic review, meta-analysis, qualitative meta-synthesis, meta-narrative synthesis, scoping review, rapid review, realist synthesis, concept analysis, literature review, and integrative review. Our first step was to divide the synthesis types among the research team. Each member did a preliminary search to identify key texts. The team then met to develop search terms and a framework to guide the review.

Over the period of 2008 to 2012 we extensively searched the literature, updating our search at several time points, not restricting our search by date. The dates of texts reviewed range from 1967 to 2015. We used the terms above combined with the term “method* (e.g., “realist synthesis” and “method*) in the database Health Source: Academic Edition (includes Medline and CINAHL). This search yielded very few texts on some methodologies and many on others. We realized that many documents on research synthesis had not been picked up in the search. Therefore, we also searched Google Scholar, PubMed, ERIC, and Social Science Index, as well as the websites of key organizations such as the Joanna Briggs Institute, the University of York Centre for Evidence-Based Nursing, and the Cochrane Collaboration database. We hand searched several nursing, social science, public health and health policy journals. Finally, we traced relevant documents from the references in obtained texts.

We included works that met the following inclusion criteria: (1) published in English; (2) discussed the history of research synthesis; (3) explicitly described the approach and specific methods; or (4) identified issues, challenges, strengths and limitations of the particular methodology. We excluded research reports that resulted from the use of particular synthesis methodologies unless they also included criteria 2, 3, or 4 above.

Based on our search, we identified additional types of research synthesis (e.g., meta-interpretation, best evidence synthesis, critical interpretive synthesis, meta-summary, grounded formal theory). Still, we missed some important developments in meta-analysis, for example, identified by the journal's reviewers that have now been discussed briefly in the paper. The final set of 197 texts included in our review comprised theoretical, empirical, and conceptual papers, books, editorials and commentaries, and policy documents.

In our preliminary review of key texts, the team inductively developed a framework of the important elements of each method for comparison. In the next phase, each text was read carefully, and data for these elements were extracted into a table for comparison on the points of: key characteristics, purpose, methods, and product; see Additional File 1 ). Once the data were grouped and extracted, we synthesized across categories based on the following additional points of comparison: complexity of the process, degree of systematization, consideration of context, underlying assumptions, unit of analysis, and when to use each approach. In our results, we discuss our comparison of the various synthesis approaches on the elements above. Drawing only on documents for the review, ethics approval was not required.

We identified four broad categories of research synthesis methodology: Conventional, quantitative, qualitative, and emerging syntheses. From our dataset of 197 texts, we had 14 texts on conventional synthesis, 64 on quantitative synthesis, 78 on qualitative synthesis, and 41 on emerging syntheses. Table 1 provides an overview of the four types of research synthesis, definitions, types of data used, products, and examples of the methodology.

Although we group these types of synthesis into four broad categories on the basis of similarities, each type within a category has unique characteristics, which may differ from the overall group similarities. Each could be explored in greater depth to tease out their unique characteristics, but detailed comparison is beyond the scope of this article.

Additional File 1 presents one or more selected types of synthesis that represent the broad category but is not an exhaustive presentation of all types within each category. It provides more depth for specific examples from each category of synthesis on the characteristics, purpose, methods, and products than is found in Table 1 .

4.1. Key Characteristics

4.1.1. what is it.

Here we draw on two types of categorization. First, we utilize Dixon Woods et al.'s [49] classification of research syntheses as being either integrative or interpretive . (Please note that integrative syntheses are not the same as an integrative review as defined in Additional File 1 .) Second, we use Popay's [80] enhancement and epistemological models .

The defining characteristics of integrative syntheses are that they involve summarizing the data achieved by pooling data [49] . Integrative syntheses include systematic reviews, meta-analyses, as well as scoping and rapid reviews because each of these focus on summarizing data. They also define concepts from the outset (although this may not always be true in scoping or rapid reviews) and deal with a well-specified phenomenon of interest.

Interpretive syntheses are primarily concerned with the development of concepts and theories that integrate concepts [49] . The analysis in interpretive synthesis is conceptual both in process and outcome, and “the product is not aggregations of data, but theory” [49] , [p.12]. Interpretive syntheses involve induction and interpretation, and are primarily conceptual in process and outcome. Examples include integrative reviews, some systematic reviews, all of the qualitative syntheses, meta-narrative, realist and critical interpretive syntheses. Of note, both quantitative and qualitative studies can be either integrative or interpretive

The second categorization, enhancement versus epistemological , applies to those approaches that use multiple data types and sources [80] . Popay's [80] classification reflects the ways that qualitative data are valued in relation to quantitative data.

In the enhancement model , qualitative data adds something to quantitative analysis. The enhancement model is reflected in systematic reviews and meta-analyses that use some qualitative data to enhance interpretation and explanation. It may also be reflected in some rapid reviews that draw on quantitative data but use some qualitative data.

The epistemological model assumes that quantitative and qualitative data are equal and each has something unique to contribute. All of the other review approaches, except pure quantitative or qualitative syntheses, reflect the epistemological model because they value all data types equally but see them as contributing different understandings.

4.1.2. Data type

By and large, the quantitative approaches (quantitative systematic review and meta-analysis) have typically used purely quantitative data (i.e., expressed in numeric form). More recently, both Cochrane [81] and Campbell [82] collaborations are grappling with the need to, and the process of, integrating qualitative research into a systematic review. The qualitative approaches use qualitative data (i.e., expressed in words). All of the emerging synthesis types, as well as the conventional integrative review, incorporate qualitative and quantitative study designs and data.

4.1.3. Research question

Four types of research questions direct inquiry across the different types of syntheses. The first is a well-developed research question that gives direction to the synthesis (e.g., meta-analysis, systematic review, meta-study, concept analysis, rapid review, realist synthesis). The second begins as a broad general question that evolves and becomes more refined over the course of the synthesis (e.g., meta-ethnography, scoping review, meta-narrative, critical interpretive synthesis). In the third type, the synthesis begins with a phenomenon of interest and the question emerges in the analytic process (e.g., grounded formal theory). Lastly, there is no clear question, but rather a general review purpose (e.g., integrative review). Thus, the requirement for a well-defined question cuts across at least three of the synthesis types (e.g., quantitative, qualitative, and emerging).

4.1.4. Quality appraisal

This is a contested issue within and between the four synthesis categories. There are strong proponents of quality appraisal in the quantitative traditions of systematic review and meta-analysis based on the need for strong studies that will not jeopardize validity of the overall findings. Nonetheless, there is no consensus on pre-defined criteria; many scales exist that vary dramatically in composition. This has methodological implications for the credibility of findings [83] .

Specific methodologies from the conventional, qualitative, and emerging categories support quality appraisal but do so with caveats. In conventional integrative reviews appraisal is recommended, but depends on the sampling frame used in the study [18] . In meta-study, appraisal criteria are explicit but quality criteria are used in different ways depending on the specific requirements of the inquiry [54] . Among the emerging syntheses, meta-narrative review developers support appraisal of a study based on criteria from the research tradition of the primary study [67] , [84] – [85] . Realist synthesis similarly supports the use of high quality evidence, but appraisal checklists are viewed with scepticism and evidence is judged based on relevance to the research question and whether a credible inference may be drawn [69] . Like realist, critical interpretive syntheses do not judge quality using standardized appraisal instruments. They will exclude fatally flawed studies, but there is no consensus on what ‘fatally flawed’ means [49] , [71] . Appraisal is based on relevance to the inquiry, not rigor of the study.

There is no agreement on quality appraisal among qualitative meta-ethnographers with some supporting and others refuting the need for appraisal. [60] , [62] . Opponents of quality appraisal are found among authors of qualitative (grounded formal theory and concept analysis) and emerging syntheses (scoping and rapid reviews) because quality is not deemed relevant to the intention of the synthesis; the studies being reviewed are not effectiveness studies where quality is extremely important. These qualitative synthesis are often reviews of theoretical developments where the concept itself is what is important, or reviews that provide quotations from the raw data so readers can make their own judgements about the relevance and utility of the data. For example, in formal grounded theory, the purpose of theory generation and authenticity of data used to generate the theory is not as important as the conceptual category. Inaccuracies may be corrected in other ways, such as using the constant comparative method, which facilitates development of theoretical concepts that are repeatedly found in the data [86] – [87] . For pragmatic reasons, evidence is not assessed in rapid and scoping reviews, in part to produce a timely product. The issue of quality appraisal is unresolved across the terrain of research synthesis and we consider this further in our discussion.

4.2. Purpose

All research syntheses share a common purpose -- to summarize, synthesize, or integrate research findings from diverse studies. This helps readers stay abreast of the burgeoning literature in a field. Our discussion here is at the level of the four categories of synthesis. Beginning with conventional literature syntheses, the overall purpose is to attend to mature topics for the purpose of re-conceptualization or to new topics requiring preliminary conceptualization [14] . Such syntheses may be helpful to consider contradictory evidence, map shifting trends in the study of a phenomenon, and describe the emergence of research in diverse fields [14] . The purpose here is to set the stage for a study by identifying what has been done, gaps in the literature, important research questions, or to develop a conceptual framework to guide data collection and analysis.

The purpose of quantitative systematic reviews is to combine, aggregate, or integrate empirical research to be able to generalize from a group of studies and determine the limits of generalization [27] . The focus of quantitative systematic reviews has been primarily on aggregating the results of studies evaluating the effectiveness of interventions using experimental, quasi-experimental, and more recently, observational designs. Systematic reviews can be done with or without quantitative meta-analysis but a meta-analysis always takes place within the context of a systematic review. Researchers must consider the review's purpose and the nature of their data in undertaking a quantitative synthesis; this will assist in determining the approach.

The purpose of qualitative syntheses is broadly to synthesize complex health experiences, practices, or concepts arising in healthcare environments. There may be various purposes depending on the qualitative methodology. For example, in hermeneutic studies the aim may be holistic explanation or understanding of a phenomenon [42] , which is deepened by integrating the findings from multiple studies. In grounded formal theory, the aim is to produce a conceptual framework or theory expected to be applicable beyond the original study. Although not able to generalize from qualitative research in the statistical sense [88] , qualitative researchers usually do want to say something about the applicability of their synthesis to other settings or phenomena. This notion of ‘theoretical generalization’ has been referred to as ‘transferability’ [89] – [90] and is an important criterion of rigour in qualitative research. It applies equally to the products of a qualitative synthesis in which the synthesis of multiple studies on the same phenomenon strengthens the ability to draw transferable conclusions.

The overarching purpose of emerging syntheses is challenging the more traditional types of syntheses, in part by using data from both quantitative and qualitative studies with diverse designs for analysis. Beyond this, however, each emerging synthesis methodology has a unique purpose. In meta-narrative review, the purpose is to identify different research traditions in the area, synthesize a complex and diverse body of research. Critical interpretive synthesis shares this characteristic. Although a distinctive approach, critical interpretive synthesis utilizes a modification of the analytic strategies of meta-ethnography [61] (e.g., reciprocal translational analysis, refutational synthesis, and lines of argument synthesis) but goes beyond the use of these to bring a critical perspective to bear in challenging the normative or epistemological assumptions in the primary literature [72] – [73] . The unique purpose of a realist synthesis is to amalgamate complex empirical evidence and theoretical understandings within a diverse body of literature to uncover the operative mechanisms and contexts that affect the outcomes of social interventions. In a scoping review, the intention is to find key concepts, examine the range of research in an area, and identify gaps in the literature. The purpose of a rapid review is comparable to that of a scoping review, but done quickly to meet the time-sensitive information needs of policy makers.

4.3. Method

4.3.1. degree of systematization.

There are varying degrees of systematization across the categories of research synthesis. The most systematized are quantitative systematic reviews and meta-analyses. There are clear processes in each with judgments to be made at each step, although there are no agreed upon guidelines for this. The process is inherently subjective despite attempts to develop objective and systematic processes [91] – [92] . Mullen and Ramirez [27] suggest that there is often a false sense of rigour implied by the terms ‘systematic review’ and ‘meta-analysis’ because of their clearly defined procedures.

In comparison with some types of qualitative synthesis, concept analysis is quite procedural. Qualitative meta-synthesis also has defined procedures and is systematic, yet perhaps less so than concept analysis. Qualitative meta-synthesis starts in an unsystematic way but becomes more systematic as it unfolds. Procedures and frameworks exist for some of the emerging types of synthesis [e.g., [50] , [63] , [71] , [93] ] but are not linear, have considerable flexibility, and are often messy with emergent processes [85] . Conventional literature reviews tend not to be as systematic as the other three types. In fact, the lack of systematization in conventional literature synthesis was the reason for the development of more systematic quantitative [17] , [20] and qualitative [45] – [46] , [61] approaches. Some authors in the field [18] have clarified processes for integrative reviews making them more systematic and rigorous, but most conventional syntheses remain relatively unsystematic in comparison with other types.

4.3.2. Complexity of the process

Some synthesis processes are considerably more complex than others. Methodologies with clearly defined steps are arguably less complex than the more flexible and emergent ones. We know that any study encounters challenges and it is rare that a pre-determined research protocol can be followed exactly as intended. Not even the rigorous methods associated with Cochrane [81] systematic reviews and meta-analyses are always implemented exactly as intended. Even when dealing with numbers rather than words, interpretation is always part of the process. Our collective experience suggests that new methodologies (e.g., meta-narrative synthesis and realist synthesis) that integrate different data types and methods are more complex than conventional reviews or the rapid and scoping reviews.

4.4. Product

The products of research syntheses usually take three distinct formats (see Table 1 and Additional File 1 for further details). The first representation is in tables, charts, graphical displays, diagrams and maps as seen in integrative, scoping and rapid reviews, meta-analyses, and critical interpretive syntheses. The second type of synthesis product is the use of mathematical scores. Summary statements of effectiveness are mathematically displayed in meta-analyses (as an effect size), systematic reviews, and rapid reviews (statistical significance).

The third synthesis product may be a theory or theoretical framework. A mid-range theory can be produced from formal grounded theory, meta-study, meta-ethnography, and realist synthesis. Theoretical/conceptual frameworks or conceptual maps may be created in meta-narrative and critical interpretive syntheses, and integrative reviews. Concepts for use within theories are produced in concept analysis. While these three product types span the categories of research synthesis, narrative description and summary is used to present the products resulting from all methodologies.

4.5. Consideration of context

There are diverse ways that context is considered in the four broad categories of synthesis. Context may be considered to the extent that it features within primary studies for the purpose of the review. Context may also be understood as an integral aspect of both the phenomenon under study and the synthesis methodology (e.g., realist synthesis). Quantitative systematic reviews and meta-analyses have typically been conducted on studies using experimental and quasi-experimental designs and more recently observational studies, which control for contextual features to allow for understanding of the ‘true’ effect of the intervention [94] .

More recently, systematic reviews have included covariates or mediating variables (i.e., contextual factors) to help explain variability in the results across studies [27] . Context, however, is usually handled in the narrative discussion of findings rather than in the synthesis itself. This lack of attention to context has been one criticism leveled against systematic reviews and meta-analyses, which restrict the types of research designs that are considered [e.g., [95] ].

When conventional literature reviews incorporate studies that deal with context, there is a place for considering contextual influences on the intervention or phenomenon. Reviews of quantitative experimental studies tend to be devoid of contextual considerations since the original studies are similarly devoid, but context might figure prominently in a literature review that incorporates both quantitative and qualitative studies.

Qualitative syntheses have been conducted on the contextual features of a particular phenomenon [33] . Paterson et al. [54] advise researchers to attend to how context may have influenced the findings of particular primary studies. In qualitative analysis, contextual features may form categories by which the data can be compared and contrasted to facilitate interpretation. Because qualitative research is often conducted to understand a phenomenon as a whole, context may be a focus, although this varies with the qualitative methodology. At the same time, the findings in a qualitative synthesis are abstracted from the original reports and taken to a higher level of conceptualization, thus removing them from the original context.

Meta-narrative synthesis [67] , [84] , because it draws on diverse research traditions and methodologies, may incorporate context into the analysis and findings. There is not, however, an explicit step in the process that directs the analyst to consider context. Generally, the research question guiding the synthesis is an important factor in whether context will be a focus.

More recent iterations of concept analysis [47] , [96] – [97] explicitly consider context reflecting the assumption that a concept's meaning is determined by its context. Morse [47] points out, however, that Wilson's [98] approach to concept analysis, and those based on Wilson [e.g., [45] ], identify attributes that are devoid of context, while Rodgers' [96] , [99] evolutionary method considers context (e.g., antecedents, consequences, and relationships to other concepts) in concept development.

Realist synthesis [69] considers context as integral to the study. It draws on a critical realist logic of inquiry grounded in the work of Bhaskar [100] , who argues that empirical co-occurrence of events is insufficient for inferring causation. One must identify generative mechanisms whose properties are causal and, depending on the situation, may nor may not be activated [94] . Context interacts with program/intervention elements and thus cannot be differentiated from the phenomenon [69] . This approach synthesizes evidence on generative mechanisms and analyzes contextual features that activate them; the result feeds back into the context. The focus is on what works, for whom, under what conditions, why and how [68] .

4.6. Underlying Philosophical and Theoretical Assumptions

When we began our review, we ‘assumed’ that the assumptions underlying synthesis methodologies would be a distinguishing characteristic of synthesis types, and that we could compare the various types on their assumptions, explicit or implicit. We found, however, that many authors did not explicate the underlying assumptions of their methodologies, and it was difficult to infer them. Kirkevold [101] has argued that integrative reviews need to be carried out from an explicit philosophical or theoretical perspective. We argue this should be true for all types of synthesis.

Authors of some emerging synthesis approaches have been very explicit about their assumptions and philosophical underpinnings. An implicit assumption of most emerging synthesis methodologies is that quantitative systematic reviews and meta-analyses have limited utility in some fields [e.g., in public health – [13] , [102] ] and for some kinds of review questions like those about feasibility and appropriateness versus effectiveness [103] – [104] . They also assume that ontologically and epistemologically, both kinds of data can be combined. This is a significant debate in the literature because it is about the commensurability of overarching paradigms [105] but this is beyond the scope of this review.

Realist synthesis is philosophically grounded in critical realism or, as noted above, a realist logic of inquiry [93] , [99] , [106] – [107] . Key assumptions regarding the nature of interventions that inform critical realism have been described above in the section on context. See Pawson et al. [106] for more information on critical realism, the philosophical basis of realist synthesis.

Meta-narrative synthesis is explicitly rooted in a constructivist philosophy of science [108] in which knowledge is socially constructed rather than discovered, and what we take to be ‘truth’ is a matter of perspective. Reality has a pluralistic and plastic character, and there is no pre-existing ‘real world’ independent of human construction and language [109] . See Greenhalgh et al. [67] , [85] and Greenhalgh & Wong [97] for more discussion of the constructivist basis of meta-narrative synthesis.

In the case of purely quantitative or qualitative syntheses, it may be an easier matter to uncover unstated assumptions because they are likely to be shared with those of the primary studies in the genre. For example, grounded formal theory shares the philosophical and theoretical underpinnings of grounded theory, rooted in the theoretical perspective of symbolic interactionism [110] – [111] and the philosophy of pragmatism [87] , [112] – [114] .

As with meta-narrative synthesis, meta-study developers identify constructivism as their interpretive philosophical foundation [54] , [88] . Epistemologically, constructivism focuses on how people construct and re-construct knowledge about a specific phenomenon, and has three main assumptions: (1) reality is seen as multiple, at times even incompatible with the phenomenon under consideration; (2) just as primary researchers construct interpretations from participants' data, meta-study researchers also construct understandings about the primary researchers' original findings. Thus, meta-synthesis is a construction of a construction, or a meta-construction; and (3) all constructions are shaped by the historical, social and ideological context in which they originated [54] . The key message here is that reports of any synthesis would benefit from an explicit identification of the underlying philosophical perspectives to facilitate a better understanding of the results, how they were derived, and how they are being interpreted.

4.7. Unit of Analysis

The unit of analysis for each category of review is generally distinct. For the emerging synthesis approaches, the unit of analysis is specific to the intention. In meta-narrative synthesis it is the storyline in diverse research traditions; in rapid review or scoping review, it depends on the focus but could be a concept; and in realist synthesis, it is the theories rather than programs that are the units of analysis. The elements of theory that are important in the analysis are mechanisms of action, the context, and the outcome [107] .

For qualitative synthesis, the units of analysis are generally themes, concepts or theories, although in meta-study, the units of analysis can be research findings (“meta-data-analysis”), research methods (“meta-method”) or philosophical/theoretical perspectives (“meta-theory”) [54] . In quantitative synthesis, the units of analysis range from specific statistics for systematic reviews to effect size of the intervention for meta-analysis. More recently, some systematic reviews focus on theories [115] – [116] , therefore it depends on the research question. Similarly, within conventional literature synthesis the units of analysis also depend on the research purpose, focus and question as well as on the type of research methods incorporated into the review. What is important in all research syntheses, however, is that the unit of analysis needs to be made explicit. Unfortunately, this is not always the case.

4.8. Strengths and Limitations

In this section, we discuss the overarching strengths and limitations of synthesis methodologies as a whole and then highlight strengths and weaknesses across each of our four categories of synthesis.

4.8.1. Strengths of Research Syntheses in General

With the vast proliferation of research reports and the increased ease of retrieval, research synthesis has become more accessible providing a way of looking broadly at the current state of research. The availability of syntheses helps researchers, practitioners, and policy makers keep up with the burgeoning literature in their fields without which evidence-informed policy or practice would be difficult. Syntheses explain variation and difference in the data helping us identify the relevance for our own situations; they identify gaps in the literature leading to new research questions and study designs. They help us to know when to replicate a study and when to avoid excessively duplicating research. Syntheses can inform policy and practice in a way that well-designed single studies cannot; they provide building blocks for theory that helps us to understand and explain our phenomena of interest.

4.8.2. Limitations of Research Syntheses in General

The process of selecting, combining, integrating, and synthesizing across diverse study designs and data types can be complex and potentially rife with bias, even with those methodologies that have clearly defined steps. Just because a rigorous and standardized approach has been used does not mean that implicit judgements will not influence the interpretations and choices made at different stages.

In all types of synthesis, the quantity of data can be considerable, requiring difficult decisions about scope, which may affect relevance. The quantity of available data also has implications for the size of the research team. Few reviews these days can be done independently, in particular because decisions about inclusion and exclusion may require the involvement of more than one person to ensure reliability.

For all types of synthesis, it is likely that in areas with large, amorphous, and diverse bodies of literature, even the most sophisticated search strategies will not turn up all the relevant and important texts. This may be more important in some synthesis methodologies than in others, but the omission of key documents can influence the results of all syntheses. This issue can be addressed, at least in part, by including a library scientist on the research team as required by some funding agencies. Even then, it is possible to miss key texts. In this review, for example, because none of us are trained in or conduct meta-analyses, we were not even aware that we had missed some new developments in this field such as meta-regression [117] – [118] , network meta-analysis [119] – [121] , and the use of individual patient data in meta-analyses [122] – [123] .

One limitation of systematic reviews and meta-analyses is that they rapidly go out of date. We thought this might be true for all types of synthesis, although we wondered if those that produce theory might not be somewhat more enduring. We have not answered this question but it is open for debate. For all types of synthesis, the analytic skills and the time required are considerable so it is clear that training is important before embarking on a review, and some types of review may not be appropriate for students or busy practitioners.

Finally, the quality of reporting in primary studies of all genres is variable so it is sometimes difficult to identify aspects of the study essential for the synthesis, or to determine whether the study meets quality criteria. There may be flaws in the original study, or journal page limitations may necessitate omitting important details. Reporting standards have been developed for some types of reviews (e.g., systematic review, meta-analysis, meta-narrative synthesis, realist synthesis); but there are no agreed upon standards for qualitative reviews. This is an important area for development in advancing the science of research synthesis.

4.8.3. Strengths and Limitations of the Four Synthesis Types

The conventional literature review and now the increasingly common integrative review remain important and accessible approaches for students, practitioners, and experienced researchers who want to summarize literature in an area but do not have the expertise to use one of the more complex methodologies. Carefully executed, such reviews are very useful for synthesizing literature in preparation for research grants and practice projects. They can determine the state of knowledge in an area and identify important gaps in the literature to provide a clear rationale or theoretical framework for a study [14] , [18] . There is a demand, however, for more rigour, with more attention to developing comprehensive search strategies and more systematic approaches to combining, integrating, and synthesizing the findings.

Generally, conventional reviews include diverse study designs and data types that facilitate comprehensiveness, which may be a strength on the one hand, but can also present challenges on the other. The complexity inherent in combining results from studies with diverse methodologies can result in bias and inaccuracies. The absence of clear guidelines about how to synthesize across diverse study types and data [18] has been a challenge for novice reviewers.

Quantitative systematic reviews and meta-analyses have been important in launching the field of evidence-based healthcare. They provide a systematic, orderly and auditable process for conducting a review and drawing conclusions [25] . They are arguably the most powerful approaches to understanding the effectiveness of healthcare interventions, especially when intervention studies on the same topic show very different results. When areas of research are dogged by controversy [25] or when study results go against strongly held beliefs, such approaches can reduce the uncertainty and bring strong evidence to bear on the controversy.

Despite their strengths, they also have limitations. Systematic reviews and meta-analyses do not provide a way of including complex literature comprising various types of evidence including qualitative studies, theoretical work, and epidemiological studies. Only certain types of design are considered and qualitative data are used in a limited way. This exclusion limits what can be learned in a topic area.

Meta-analyses are often not possible because of wide variability in study design, population, and interventions so they may have a narrow range of utility. New developments in meta-analysis, however, can be used to address some of these limitations. Network meta-analysis is used to explore relative efficacy of multiple interventions, even those that have never been compared in more conventional pairwise meta-analyses [121] , allowing for improved clinical decision making [120] . The limitation is that network meta-analysis has only been used in medical/clinical applications [119] and not in public health. It has not yet been widely accepted and many methodological challenges remain [120] – [121] . Meta-regression is another development that combines meta-analytic and linear regression principles to address the fact that heterogeneity of results may compromise a meta-analysis [117] – [118] . The disadvantage is that many clinicians are unfamiliar with it and may incorrectly interpret results [117] .

Some have accused meta-analysis of combining apples and oranges [124] raising questions in the field about their meaningfulness [25] , [28] . More recently, the use of individual rather than aggregate data has been useful in facilitating greater comparability among studies [122] . In fact, Tomas et al. [123] argue that meta-analysis using individual data is now the gold standard although access to the raw data from other studies may be a challenge to obtain.

The usefulness of systematic reviews in synthesizing complex health and social interventions has also been challenged [102] . It is often difficult to synthesize their findings because such studies are “epistemologically diverse and methodologically complex” [ [69] , p.21]. Rigid inclusion/exclusion criteria may allow only experimental or quasi-experimental designs into consideration resulting in lost information that may well be useful to policy makers for tailoring an intervention to the context or understanding its acceptance by recipients.

Qualitative syntheses may be the type of review most fraught with controversy and challenge, while also bringing distinct strengths to the enterprise. Although these methodologies provide a comprehensive and systematic review approach, they do not generally provide definitive statements about intervention effectiveness. They do, however, address important questions about the development of theoretical concepts, patient experiences, acceptability of interventions, and an understanding about why interventions might work.

Most qualitative syntheses aim to produce a theoretically generalizable mid-range theory that explains variation across studies. This makes them more useful than single primary studies, which may not be applicable beyond the immediate setting or population. All provide a contextual richness that enhances relevance and understanding. Another benefit of some types of qualitative synthesis (e.g., grounded formal theory) is that the concept of saturation provides a sound rationale for limiting the number of texts to be included thus making reviews potentially more manageable. This contrasts with the requirements of systematic reviews and meta-analyses that require an exhaustive search.

Qualitative researchers debate about whether the findings of ontologically and epistemological diverse qualitative studies can actually be combined or synthesized [125] because methodological diversity raises many challenges for synthesizing findings. The products of different types of qualitative syntheses range from theory and conceptual frameworks, to themes and rich descriptive narratives. Can one combine the findings from a phenomenological study with the theory produced in a grounded theory study? Many argue yes, but many also argue no.

Emerging synthesis methodologies were developed to address some limitations inherent in other types of synthesis but also have their own issues. Because each type is so unique, it is difficult to identify overarching strengths of the entire category. An important strength, however, is that these newer forms of synthesis provide a systematic and rigorous approach to synthesizing a diverse literature base in a topic area that includes a range of data types such as: both quantitative and qualitative studies, theoretical work, case studies, evaluations, epidemiological studies, trials, and policy documents. More than conventional literature reviews and systematic reviews, these approaches provide explicit guidance on analytic methods for integrating different types of data. The assumption is that all forms of data have something to contribute to knowledge and theory in a topic area. All have a defined but flexible process in recognition that the methods may need to shift as knowledge develops through the process.

Many emerging synthesis types are helpful to policy makers and practitioners because they are usually involved as team members in the process to define the research questions, and interpret and disseminate the findings. In fact, engagement of stakeholders is built into the procedures of the methods. This is true for rapid reviews, meta-narrative syntheses, and realist syntheses. It is less likely to be the case for critical interpretive syntheses.

Another strength of some approaches (realist and meta-narrative syntheses) is that quality and publication standards have been developed to guide researchers, reviewers, and funders in judging the quality of the products [108] , [126] – [127] . Training materials and online communities of practice have also been developed to guide users of realist and meta-narrative review methods [107] , [128] . A unique strength of critical interpretive synthesis is that it takes a critical perspective on the process that may help reconceptualize the data in a way not considered by the primary researchers [72] .

There are also challenges of these new approaches. The methods are new and there may be few published applications by researchers other than the developers of the methods, so new users often struggle with the application. The newness of the approaches means that there may not be mentors available to guide those unfamiliar with the methods. This is changing, however, and the number of applications in the literature is growing with publications by new users helping to develop the science of synthesis [e.g., [129] ]. However, the evolving nature of the approaches and their developmental stage present challenges for novice researchers.

4.9. When to Use Each Approach

Choosing an appropriate approach to synthesis will depend on the question you are asking, the purpose of the review, and the outcome or product you want to achieve. In Additional File 1 , we discuss each of these to provide guidance to readers on making a choice about review type. If researchers want to know whether a particular type of intervention is effective in achieving its intended outcomes, then they might choose a quantitative systemic review with or without meta-analysis, possibly buttressed with qualitative studies to provide depth and explanation of the results. Alternately, if the concern is about whether an intervention is effective with different populations under diverse conditions in varying contexts, then a realist synthesis might be the most appropriate.

If researchers' concern is to develop theory, they might consider qualitative syntheses or some of the emerging syntheses that produce theory (e.g., critical interpretive synthesis, realist review, grounded formal theory, qualitative meta-synthesis). If the aim is to track the development and evolution of concepts, theories or ideas, or to determine how an issue or question is addressed across diverse research traditions, then meta-narrative synthesis would be most appropriate.

When the purpose is to review the literature in advance of undertaking a new project, particularly by graduate students, then perhaps an integrative review would be appropriate. Such efforts contribute towards the expansion of theory, identify gaps in the research, establish the rationale for studying particular phenomena, and provide a framework for interpreting results in ways that might be useful for influencing policy and practice.

For researchers keen to bring new insights, interpretations, and critical re-conceptualizations to a body of research, then qualitative or critical interpretive syntheses will provide an inductive product that may offer new understandings or challenges to the status quo. These can inform future theory development, or provide guidance for policy and practice.

5. Discussion

What is the current state of science regarding research synthesis? Public health, health care, and social science researchers or clinicians have previously used all four categories of research synthesis, and all offer a suitable array of approaches for inquiries. New developments in systematic reviews and meta-analysis are providing ways of addressing methodological challenges [117] – [123] . There has also been significant advancement in emerging synthesis methodologies and they are quickly gaining popularity. Qualitative meta-synthesis is still evolving, particularly given how new it is within the terrain of research synthesis. In the midst of this evolution, outstanding issues persist such as grappling with: the quantity of data, quality appraisal, and integration with knowledge translation. These topics have not been thoroughly addressed and need further debate.

5.1. Quantity of Data

We raise the question of whether it is possible or desirable to find all available studies for a synthesis that has this requirement (e.g., meta-analysis, systematic review, scoping, meta-narrative synthesis [25] , [27] , [63] , [67] , [84] – [85] ). Is the synthesis of all available studies a realistic goal in light of the burgeoning literature? And how can this be sustained in the future, particularly as the emerging methodologies continue to develop and as the internet facilitates endless access? There has been surprisingly little discussion on this topic and the answers will have far-reaching implications for searching, sampling, and team formation.

Researchers and graduate students can no longer rely on their own independent literature search. They will likely need to ask librarians for assistance as they navigate multiple sources of literature and learn new search strategies. Although teams now collaborate with library scientists, syntheses are limited in that researchers must make decisions on the boundaries of the review, in turn influencing the study's significance. The size of a team may also be pragmatically determined to manage the search, extraction, and synthesis of the burgeoning data. There is no single answer to our question about the possibility or necessity of finding all available articles for a review. Multiple strategies that are situation specific are likely to be needed.

5.2. Quality Appraisal

While the issue of quality appraisal has received much attention in the synthesis literature, scholars are far from resolution. There may be no agreement about appraisal criteria in a given tradition. For example, the debate rages over the appropriateness of quality appraisal in qualitative synthesis where there are over 100 different sets of criteria and many do not overlap [49] . These differences may reflect disciplinary and methodological orientations, but diverse quality appraisal criteria may privilege particular types of research [49] . The decision to appraise is often grounded in ontological and epistemological assumptions. Nonetheless, diversity within and between categories of synthesis is likely to continue unless debate on the topic of quality appraisal continues and evolves toward consensus.

5.3. Integration with Knowledge Translation

If research syntheses are to make a difference to practice and ultimately to improve health outcomes, then we need to do a better job of knowledge translation. In the Canadian Institutes of Health Research (CIHR) definition of knowledge translation (KT), research or knowledge synthesis is an integral component [130] . Yet, with few exceptions [131] – [132] , very little of the research synthesis literature even mentions the relationship of synthesis to KT nor does it discuss strategies to facilitate the integration of synthesis findings into policy and practice. The exception is in the emerging synthesis methodologies, some of which (e.g., realist and meta-narrative syntheses, scoping reviews) explicitly involve stakeholders or knowledge users. The argument is that engaging them in this way increases the likelihood that the knowledge generated will be translated into policy and practice. We suggest that a more explicit engagement with knowledge users in all types of synthesis would benefit the uptake of the research findings.

Research synthesis neither makes research more applicable to practice nor ensures implementation. Focus must now turn seriously towards translation of synthesis findings into knowledge products that are useful for health care practitioners in multiple areas of practice and develop appropriate strategies to facilitate their use. The burgeoning field of knowledge translation has, to some extent, taken up this challenge; however, the research-practice gap continues to plague us [133] – [134] . It is a particular problem for qualitative syntheses [131] . Although such syntheses have an important place in evidence-informed practice, little effort has gone into the challenge of translating the findings into useful products to guide practice [131] .

5.4. Limitations

Our study took longer than would normally be expected for an integrative review. Each of us were primarily involved in our own dissertations or teaching/research positions, and so this study was conducted ‘off the sides of our desks.’ A limitation was that we searched the literature over the course of 4 years (from 2008–2012), necessitating multiple search updates. Further, we did not do a comprehensive search of the literature after 2012, thus the more recent synthesis literature was not systematically explored. We did, however, perform limited database searches from 2012–2015 to keep abreast of the latest methodological developments. Although we missed some new approaches to meta-analysis in our search, we did not find any new features of the synthesis methodologies covered in our review that would change the analysis or findings of this article. Lastly, we struggled with the labels used for the broad categories of research synthesis methodology because of our hesitancy to reinforce the divide between quantitative and qualitative approaches. However, it was very difficult to find alternative language that represented the types of data used in these methodologies. Despite our hesitancy in creating such an obvious divide, we were left with the challenge of trying to find a way of characterizing these broad types of syntheses.

6. Conclusion

Our findings offer methodological clarity for those wishing to learn about the broad terrain of research synthesis. We believe that our review makes transparent the issues and considerations in choosing from among the four broad categories of research synthesis. In summary, research synthesis has taken its place as a form of research in its own right. The methodological terrain has deep historical roots reaching back over the past 200 years, yet research synthesis remains relatively new to public health, health care, and social sciences in general. This is rapidly changing. New developments in systematic reviews and meta-analysis, and the emergence of new synthesis methodologies provide a vast array of options to review the literature for diverse purposes. New approaches to research synthesis and new analytic methods within existing approaches provide a much broader range of review alternatives for public health, health care, and social science students and researchers.

Acknowledgments

KSM is an assistant professor in the Faculty of Nursing at the University of Alberta. Her work on this article was largely conducted as a Postdoctoral Fellow, funded by KRESCENT (Kidney Research Scientist Core Education and National Training Program, reference #KRES110011R1) and the Faculty of Nursing at the University of Alberta.

MM's work on this study over the period of 2008-2014 was supported by a Canadian Institutes of Health Research Applied Public Health Research Chair Award (grant #92365).

We thank Rachel Spanier who provided support with reference formatting.

List of Abbreviations (in Additional File 1 )

Conflict of interest: The authors declare that they have no conflicts of interest in this article.

Authors' contributions: KSM co-designed the study, collected data, analyzed the data, drafted/revised the manuscript, and managed the project.

MP contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

JB contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

WN contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

All authors read and approved the final manuscript.

Additional Files: Additional File 1 – Selected Types of Research Synthesis

This Additional File is our dataset created to organize, analyze and critique the literature that we synthesized in our integrative review. Our results were created based on analysis of this Additional File.

Literature Review Basics

• What is a Literature Review?
• Synthesizing Research
• Using Research & Synthesis Tables
• Additional Resources

About the Research and Synthesis Tables

Research Tables and Synthesis Tables are useful tools for organizing and analyzing your research as you assemble your literature review. They represent two different parts of the review process: assembling relevant information and synthesizing it. Use a Research table to compile the main info you need about the items you find in your research -- it's a great thing to have on hand as you take notes on what you read! Then, once you've assembled your research, use the Synthesis table to start charting the similarities/differences and major themes among your collected items.

We've included an Excel file with templates for you to use below; the examples pictured on this page are snapshots from that file.

• Research and Synthesis Table Templates This Excel workbook includes simple templates for creating research tables and synthesis tables. Feel free to download and use!

Using the Research Table

This is an example of a  research table,  in which you provide a basic description of the most important features of the studies, articles, and other items you discover in your research. The table identifies each item according to its author/date of publication, its purpose or thesis, what type of work it is (systematic review, clinical trial, etc.), the level of evidence it represents (which tells you a lot about its impact on the field of study), and its major findings. Your job, when you assemble this information, is to develop a snapshot of what the research shows about the topic of your research question and assess its value (both for the purpose of your work and for general knowledge in the field).

Think of your work on the research table as the foundational step for your analysis of the literature, in which you assemble the information you'll be analyzing and lay the groundwork for thinking about what it means and how it can be used.

Using the Synthesis Table

This is an example of a  synthesis table  or  synthesis matrix , in which you organize and analyze your research by listing each source and indicating whether a given finding or result occurred in a particular study or article ( each row lists an individual source, and each finding has its own column, in which X = yes, blank = no). You can also add or alter the columns to look for shared study populations, sort by level of evidence or source type, etc. The key here is to use the table to provide a simple representation of what the research has found (or not found, as the case may be). Think of a synthesis table as a tool for making comparisons, identifying trends, and locating gaps in the literature.

How do I know which findings to use, or how many to include?  Your research question tells you which findings are of interest in your research, so work from your research question to decide what needs to go in each Finding header, and how many findings are necessary. The number is up to you; again, you can alter this table by adding or deleting columns to match what you're actually looking for in your analysis. You should also, of course, be guided by what's actually present in the material your research turns up!

• << Previous: Synthesizing Research
• Next: Additional Resources >>
• Last Updated: Sep 26, 2023 12:06 PM
• URL: https://usi.libguides.com/literature-review-basics

Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design (2020)

Chapter: chapter 2 - literature review and synthesis.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4 Literature Review and Synthesis Literature Review Purpose of Literature Review Performance-based seismic design (PBSD) for infrastructure in the United States is a developing field, with new research, design, and repair technologies; definitions; and method- ologies being advanced every year. A synthesis report, NCHRP Synthesis 440: Performance- Based Seismic Bridge Design (Marsh and Stringer 2013), was created to capture PBSD understanding up to that point. This synthesis report described the background, objec- tives, and research up until 2011 to 2012 and synthesized the information, including areas where knowledge gaps existed. The literature review in this research report focuses on new infor mation developed after the efforts of NCHRP Synthesis 440. The intention is that this research report will fuel the next challenge: developing a methodology to implement PBSD for bridge design. Literature Review Process Marsh and Stringer (2013) performed an in-depth bridge practice review by sending a questionnaire to all 50 states, with particular attention to regions with higher seismic hazards. The survey received responses from a majority of those agencies. This process was continued in the current project with a request for new information or research that the state depart- ment of transportation (DOT) offices have participated in or are aware of through other organizations. The research team reached out to the list of states and researchers in Table 1. An X within a box is placed in front of their names if they responded. The team also examined the websites of the state DOTs that participated to investigate whether something was studied locally, especially work being developed in California. The research team made an additional effort to perform a practice review of bridge designs, research, and other design industries, specifically in the building industry. The building industry has been developing PBSD for more than 20 years, and some of their developments are appli- cable to bridge design. These combined efforts have allowed the research team to assemble an overview of the state of PBSD engineering details and deployment since Marsh and Stringer’s (2013) report was published. NCHRP Synthesis 440 primarily dealt with the effects of strong ground motion shaking. Secondary effects such as tsunami/seiche, ground failure (surface rupture, liquefaction, or slope failure), fire, and flood were outside the scope of this study. Regardless, their impact on bridges may be substantial, and investigation into their effects is undoubtedly important. C H A P T E R 2

Literature Review and Synthesis 5 The following e-mail was sent to the owners and researchers. Dear (individual): We are assisting Modjeski & Masters with the development of proposed guidelines for Performance- Based Seismic Bridge Design, as part of NCHRP [Project] 12-106. Lee Marsh and our Team at BergerABAM are continuing our efforts from NCHRP Synthesis 440, which included a literature review up to December of 2011. From this timeframe forward, we are looking for published research, contractual language, or owner documents that deal with the following categories: 1. Seismic Hazards (seismic hazard levels, hazard curves, return periods, geo-mean vs. maximum direc- tion, probabilistic vs. deterministic ground motions, conditional mean spectrum, etc.) 2. Structure Response (engineering design parameters, materials and novel columns, isolation bearings, modeling techniques, etc.) 3. Damage Limit States (performance descriptions, displacement ductility, drift ratios, strain limits, rotation curvature, etc.) 4. Potential for Loss (damage descriptions, repairs, risk of collapse, economical loss, serviceability loss, etc.) 5. Performance Design Techniques (relating hazard to design to performance to risk, and how to assess [these] levels together) If you are aware of this type of resource, please provide a contact that we can work with to get this information or provide a published reference we can gather. Your assistance is appreciated. We want to minimize your time, and ask that you respond by Wednesday, 8 February 2017. Thank you again, Research Team Synthesis of PBSD (2012–2016) Objectives of NCHRP Synthesis 440 The synthesis gathered data from a number of different but related areas. Marsh and Stringer (2013), herein referred to as NCHRP Synthesis 440, set the basis for this effort. The research report outline follows what has been added to the NCHRP Synthesis 440 effort since 2012. The information gathered that supplements NCHRP Synthesis 440 includes, but is not limited to, the following topics. • Public and engineering expectations of seismic design and the associated regulatory framework Participation State Alaska DOT Arkansas DOT California DOT (Caltrans) Illinois DOT Indiana DOT Missouri DOT Montana DOT Nevada DOT Oregon DOT South Carolina DOT Utah DOT Washington State DOT Table 1. List of state DOT offices and their participation.

6 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design • Seismic hazard analysis • Structural analysis and design • Damage analysis • Loss analysis • Organization-specific criteria for bridges • Project-specific criteria Where new or updated information is available for these areas, a summary is included. Marsh and Stringer (2013) also identified gaps in the knowledge base of PBSD, current as of 2012, that need to be closed. Knowledge gaps certainly exist in all facets of PBSD; however, key knowledge gaps that should be closed in order to implement PBSD are covered. • Gaps related to seismic hazard prediction • Gaps related to structural analysis • Gaps related to damage prediction • Gaps related to performance • Gaps related to loss prediction • Gaps related to regulatory oversight and training • Gaps related to decision making These knowledge gaps have been filled in somewhat in this research report but, for the most part, these areas are still the key concepts that require additional development to further the development of a PBSD guide specification. Public and Engineering Expectations of Seismic Design and the Associated Regulatory Framework The public expectation of a structure, including a bridge, is that it will withstand an earthquake, but there is a limited understanding of what that actually means. Decision makers struggle to understand how a bridge meeting the current requirements of the AASHTO Guide Specifications for LRFD Seismic Bridge Design (2011), herein referred to as AASHTO guide specifications, will perform after either the expected (design) or a higher level earthquake. Decision makers understand the basis of life safety, wherein the expectation is that no one will perish from a structure collapsing, but often mistakenly believe that the structure will also be usable after the event. In higher level earthquakes, even in some lower level events, this is not true without repair, retrofit, or replacement. In the past decade, there has been an increased awareness by owners and decision makers as to the basis of seismic design. As a result, a need has developed for performance criteria so that economic and social impacts can be interwoven with seismic design into the decision processes (see Figure 1). Several states, including California, Oregon, and the State of Washington, are working toward resiliency plans, although these are developed under different titles or programs within the states. Resiliency has been defined in several ways: (1) amount of damage from an event measured in fatalities, structural replacement cost, and recovery time and (2) the time to resto- ration of lifelines, reoccupation of homes and structures, and, in the short term, resumption of normal living routines. The California DOT Caltrans has generated risk models and is in the process of developing a new seismic design specification to address PBSD in bridge design. The risk models and specifications are not published yet, but the use in PBSD is discussed in greater detail later in this chapter.

Literature Review and Synthesis 7 The State of Washington The State of Washington’s resiliency plan, outlined in Washington State Emergency Management Council–Seismic Safety Committee (2012), works to identify actions and policies before, during, and after an earthquake event that can leverage existing policies, plans, and initiatives to realize disaster resilience within a 50-year life cycle. The hazard level used for trans- portation planning is the 1000 year event. The goals for transportation systems vary depending on the type of service a route provides, as shown in following components of the plan. For major corridors such as Interstates 5, 90, and 405 and floating bridges SR 520, I-90, and Hood Canal, the target timeframe for response and recovery is between 1 to 3 days and 1 to 3 months, depending on location. The current anticipated timeframe based on current capacity and without modifications is between 3 months to 1 year and 1 to 3 years, depending on location. The actual response and recovery time will depend on a number of factors. For example: 1. The number of Washington State DOT personnel who are able to report to work may be limited by a variety of circumstances, including where personnel were at the time of the earthquake and whether they sustained injuries. 2. Bridges and roadways in earthquake-affected areas must be inspected. How long this takes will depend on the number and accessibility of the structures and the availability of qualified inspectors. 3. Some bridges and segments of road may be rendered unusable or only partially usable as a result of the earthquake or secondary effects. The response and recovery timeframe will depend on the number, the location, and the extent of the damage. 4. Certain earthquake scenarios could result in damage to the Ballard Locks and cause the water level in Lake Washington to drop below the level required to operate the floating bridges. 5. Depending on the scenario and local conditions, liquefaction and slope failure could damage both interstates and planned detours. During the first 3 days after the event, the Washington State Department of Transportation (Washington State DOT) will inspect bridges and begin repairs as needed. Washington State DOT’s first priority will be to open key routes for emergency response vehicles. Subsequent phases of recovery will include setting up detours where necessary and regulating the type and Figure 1. PBSD decision-making process (Guidelines Figure 2.0-1). References to guidelines figures and tables within parentheses indicate the proposed AASHTO guidelines.

8 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design volume of traffic, to give the public as much access as possible while damaged roads and bridges are repaired. For major and minor arterials, which encompass arterial roadways (including bridges) other than the interstates (so therefore includes state highways and many city and county roads), the target timeframe for response and recovery is between 0 to 24 hours and 3 months to 1 year, depending on location; the percentage of roadways that are open for use will increase over this period. Anticipated timeframe based on current capacity is between 1 week to 1 month and 1 to 3 years, depending on location; the percentage of roadways that are open for use will increase over this period. The goal of Washington State Emergency Management Council’s resiliency plan is to establish a means to coordinate agencies, public–private partnerships, and standards toward these resiliency goals. The plan outlines goals for recovery times for transportation systems in terms of hours, days, weeks, months, and years, with targets to achieve different levels of recovery (see Table 2) as follows. Similar recovery timeframe processes were established for service sectors (e.g., hospitals, law enforcement, and education); utilities; ferries, airports, ports, and navigable waterways; mass transit; and housing. The overall resiliency plan also discusses the degree to which the recovery of one component or sector would depend on the restoration of another. The key interdependencies that the participants identified include information and communication technologies, transportation, electricity, fuel, domestic water supplies, wastewater systems, finance and banking, and planning and community development. It appears that the implementation of the Washington State Emergency Management Council’s initiative, originally assumed to take 2.5 to 3 years in 2012, has not seen significant development since then. However, the State’s initiative to develop a more resilient community has been extended down to the county level, with King County’s efforts referenced in Rahman et al. (2014) and, at the city level, with the City of Seattle referenced in CEMP (2015). This reflects the commitment needed not only by the legislature and the state departments but also by other agencies (e.g., county, city, or utilities) and the public to take an interest in, and provide funding for, the development of a resiliency plan. The recovery continuum is presented graphically in Figure 2. Developing this relationship with other agency plans is an iterative process that will take time, as shown in Figure 3. Identifying the critical sectors of the agency is necessary to develop a resiliency model and determine how to approach a disaster recovery framework. King County worked from Washington State’s initiative to develop Figure 4. The Oregon DOT Oregon DOT has developed a variation of the approach identified by the State of Wash- ington; further discussion is found later in this chapter. Other Resilience Documents The building industry has recently seen the development of two additional documents that address PBSD in terms of expectations and process. The REDi Rating System from REDi (2013) sets an example for incorporating resilience- based design into the PBSD process. This document outlines structural resilience objectives for organizational resilience, building resilience, loss assessment, and ambient resilience to evaluate and rate the decision making and design methodology using PBSD for a specific project.

Literature Review and Synthesis 9 The document is one of the only references that addresses a system to develop probabilistic methods to estimate downtime. The overall intent is to provide a roadmap to resilience. This roadmap is intended to allow owners to resume business operation and to provide livable conditions quickly after an earthquake. The Los Angeles Tall Buildings Structural Design Council (LATBSDC 2014) created an alter- native procedure specific to their location. Design specification criteria are identified and modi- fications are described as appropriate for the PBSD approach to tall buildings in this localized Minimal (A minimum level of service is restored, primarily for the use of emergency responders, repair crews, and vehicles transporting food and other critical supplies.) Functional (Although service is not yet restored to full capacity, it is sufficient to get the economy moving again—for example, some truck/freight traffic can be accommodated. There may be fewer lanes in use, some weight restrictions, and lower speed limits.) Operational (Restoration is up to 80 to 90 percent of capacity: A full level of service has been restored and is sufficient to allow people to commute to school and to work.) Time needed for recovery to 80 to 90 percent operational given current conditions. Source: Washington State Emergency Management Council–Seismic Safety Committee (2012). Table 2. Washington State’s targets of recovery.

10 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Source: Adapted from FHWA by CEMP (2015). Figure 2. Recovery continuum process. Source: CEMP (2015). Figure 3. Relationship of disaster recovery framework to other city plans. region. This procedure is a good example of how PBSD criteria and methodology need to be established locally, with a knowledge of risk, resources, and performance needs in order to set the criteria for true PBSD. Seismic Hazard Prediction As outlined in NCHRP Synthesis 440, the seismic hazard includes the regional tectonics and the local site characteristics from either a deterministic or probabilistic viewpoint. The deterministic form allows the assessment of shaking at a site as a function of the controlling earthquake that can occur on all the identified faults or sources. The probabilistic approach

Literature Review and Synthesis 11 defines an acceleration used in design that would be exceeded during a given window of time (e.g., a 7% chance of exceedance in 75 years). The following subsections provide a summary of procedures currently used within AASHTO, as well as new issues that should be eventually addressed in light of approaches used by the building industry. AASHTO Probabilistic Approach As summarized in the AASHTO guide specifications, the current approach used by AASHTO involves the use of a probabilistic hazard model with a nominal return period of 1000 years. Baker (2013) noted that the probabilistic seismic hazard analysis involves the following five steps: 1. Identify all earthquake sources capable of producing damaging ground motions. 2. Characterize the distribution of earthquake magnitudes (the rates at which earthquakes of various magnitudes are expected to occur). 3. Characterize the distribution of source-to-site distances associated with potential earthquakes. 4. Predict the resulting distribution of ground motion intensity as a function of earthquake magnitude, distance, and so forth. 5. Combine uncertainties in earthquake size, location, and ground motion intensity, using a calculation known as the total probability theorem. While implementation of the five steps in the probabilistic approach is beyond what most practicing bridge engineers can easily perform, AASHTO, working through the U.S. Geological Survey, developed a website hazard tool that allows implementation of the probabilistic proce- dure based on the latitude and longitude of a bridge site. The product of the website includes peak ground acceleration (PGA), spectral acceleration at 0.2 s (Ss), and spectral acceleration at 1 s (S1). These values are for a reference-site condition comprising soft rock/stiff soil, having a time-averaged shear wave velocity (Vs) over the upper 100 feet of soil profile equal to 2500 feet per second (fps). The Geological Survey website can also correct for local site conditions following procedures in the AASHTO Guide Specifications for LRFD Seismic Bridge Design. One of the limitations of the current U.S. Geological Survey hazard website is that it is based on a seismic hazard model developed in 2002. The Geological Survey updated its seismic model in 2008 and then in 2014; however, these updates are currently not implemented within the AASHTO hazard model on the Geological Survey’s website. Oregon and the State of Washington have updated the seismic hazard map used by the Oregon DOT and the Washington State Source: Rahman et al. (2014). Figure 4. Resilient King County critical sectors and corresponding subsectors.

12 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design DOT to include the 2014 U.S. Geological Survey hazard model; however, most state DOTs are still using the out-of-date hazard model. Use of the outdated hazard model introduces some inconsistencies in ground motion prediction, relative to the current Geological Survey hazard website tool at some locations. Discussions are ongoing between NCHRP and the U.S. Geological Survey to update the 2002 website tool. Another issue associated with the current AASHTO probabilistic method is that it is based on the geomean of the ground motion. In other words, the ground motion prediction equations in the hazard model are based on the geomean of recorded earthquake motions. These motions are not necessarily the largest motion. The building industry recognized that the maximum direction could result in larger ground motions and introduced maximum direction corrections. These corrections increase spectral acceleration by a factor of 1.1 and S1 by a factor of 1.3. The relevance of this correction to bridges is discussed in the next subsection of this review. The building industry also introduced a risk-of-collapse correction to the hazard model results. This correction is made to Ss and S1. The size of the correction varies from approximately 0.8 to 1.2 within the continental United States. It theoretically adjusts the hazard curves to provide a 1% risk of collapse in 50 years. The risk-of-collapse corrections were developed by the U.S. Geological Survey for a range of building structures located throughout the United States. Although no similar corrections have been developed for bridges, the rationale for the adjust- ment needs to be further evaluated to determine if the rationale should be applied to bridge structures. As a final point within this discussion of probabilistic methods within the AASHTO guide specifications, there are several other areas of seismic response that need to be considered. These include near-fault and basin effects on ground motions, as well as a long-period transition factor. The near-fault and basin adjustments correct the Ss and S1 spectral accelerations for locations near active faults and at the edge of basins, respectively. These adjustments typically increase spectral accelerations at longer periods (> 1 s) by 10% to 20%, depending on specifics of the site. The long-period transition identifies the point at which response spectral ordinates are no longer proportional to the 1/T decay with increasing period. These near-fault, basin, and long-period adjustments have been quantified within the building industry guidance documents but remain, for the most part, undefined within the AASHTO guide specifications. As bridge discussions and research move closer to true probabilistic format for PBSD, these issues need to be addressed as part of a future implementation process. Correction for Maximum Direction of Motion Over the last decade, a debate has been under way within the building industry regarding the appropriate definition of design response spectra (Stewart et al. 2011). The essence of the argument relates to the representation of bidirectional motion via response spectra. In both the AASHTO LRFD Bridge Design Specifications (2014), as well as the AASHTO Guide Specifications for LRFD Seismic Bridge Design (SGS), response spectra are established by defining spectral ordinates at two or three different periods from design maps developed by the U.S. Geological Survey for a return period of 1000 years. The resulting spectra are then adjusted for local site conditions, resulting in the final design spectra. In establishing the design maps for parameters such as Ss and S1, the U.S. Geological Survey has traditionally relied upon probabilistic seismic hazard analysis, which utilizes ground motion prediction equations (GMPEs) defined by the geometric mean of the two principal directions of recorded motion. In 2006, Boore introduced a new rotation independent geometric mean definition termed GMRotI50 (Boore et al. 2006). Then, in 2010, Boore developed a new defini- tion that does not rely upon the geometric mean termed RotD50 spectra, which can be generi- cally expressed as RotDNN spectra, where NN represents the percentile of response (i.e., 50 is

Literature Review and Synthesis 13 consistent with the median, 0 is the minimum, and 100 is the maximum). The NGA–West2 project GMPEs utilized RotD50 spectra for the ground motion models; however, the 2009 National Earthquake Hazards Reduction Program (NEHRP) provisions adopted a factor to modify the median response, RotD50, to the maximum possible response, RotD100 as the spectra for the design maps (Stewart et al. 2011). Introducing RotD100 resulted in a 10% to 30% increase in spectral ordinates results relative to the geometric mean, which has traditionally been used as a basis of seismic design. In order to appreciate the impact of these choices, a brief discussion of RotDNN spectra is warranted. As described in Boore (2010), for a given recording station, the two orthogonal- component time series are combined into a single time series corresponding to different rotation angles, as shown in Equation 1: aROT(t ; θ) = a1(t)cos(θ) + a2(t)sin(θ) (1) where a1(t ) and a2(t ) are the orthogonal horizontal component acceleration time series and θ is the rotation angle. For example, consider the two orthogonal horizontal component time series, H1 and H2, shown in Figure 5. The single time series corresponding to the rotation angle θ is created by combining the Direction 1 and Direction 2 time series. Then, the response spectrum for that single time series can be obtained, as shown in the figure. The process is repeated for a range of azimuths from 0° to one rotation-angle increment less than 180°. If the rotation-angle increment is θ, then there will be 180/θ single time series, as well as 180/θ corresponding response spectra. For example, if θ = 30°, then there will be six single time series (the original two, as well as four generated time series), as well as six response spectra, as shown in Figure 6. Once the response spectra for all rotation angles are obtained, then the nth percentile of the spectral amplitude over all rotation angles for each period is computed (e.g., RotD50 is the median value and RotD100 is the largest value for all rotation angles). For example, at a given period of 1 s, the response spectra values for all rotation angles are sorted, and the RotD100 value would be the largest value from all rotation angles while RotD50 would be the median. This is repeated for all periods, with potentially different rotation angles, producing the largest Source: Palma (2019). Figure 5. Combination of time series to generate rotation dependent spectra.

14 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design response at any given period (period-dependent rotation angle.) Figure 7 shows an example of the two orthogonal horizontal components, as well as the RotD50 and RotD100 spectra for the as-recorded ground motion from the 2011 Christchurch, New Zealand, earthquake at Kaiapoi North School station. As can be seen in the sample spectra (see Figure 7), the RotD100 spectrum represents a sub- stantial increase in demand when compared with the RotD50 spectrum. The main question facing the bridge community from this point onward is the appropriate selection of response spectra definition. This question can only be answered by developing sample designs to both the RotD50 and RotD100 spectra, which would then be evaluated via no-linear time history analysis. Such a study will require multiple bridge configurations and multiple ground motions. As an example of the potential impact, Figure 8 shows the results of a single-degree-of- freedom bridge column designed according to both RotD50 and RotD100 spectra, along with the resulting nonlinear time history analysis. The column was designed using direct displacement- based design to achieve a target displacement of 45 cm. It is clear from the results in Figure 8d that the nonlinear response of the column designed to the RotD100 spectrum matches the target Source: Palma (2019). Figure 6. Example of time series rotations with an angle increment (p) of 30ç. Source: Palma (2019). Figure 7. Sample spectra for a recorded ground motion pair.

Literature Review and Synthesis 15 reasonably well, while designing to the RotD50 spectrum results in displacements that are much greater than expected. This is, of course, only one result of an axisymmetric system. In the future (and outside the scope of this project), a systematic study could be conducted for both single degree of freedom and multiple degrees of freedom systems. The literature on this topic can be divided into two categories: (1) response spectra definitions and (2) impact on seismic response. The majority of the literature addresses the former. For example, Boore et al. (2006) and Boore (2010) introduced orientation-independent measures of seismic intensity from two horizontal ground motions. Boore et al. (2006) proposed two measures of the geometric mean of the seismic intensity, which are independent of the in-situ orientations of the sensors. One measure uses period-dependent rotation angles to quantify the spectral intensity, denoted GMRotDnn. The other measure is the GMRotInn, where I stands for period-independent. The ground motion prediction equations of Abrahamson and Silva (1997), Figure 8. Single bridge column designed according to both RotD50 and RotD100 spectra (Tabas EQ = Tabas earthquake and displ. = displacement).

16 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Boore et al. (1997), Campbell and Bozorgnia (2003), and Sadigh et al. (1997) have been updated using GMRotI50 as the dependent variable. Since more users within the building industry expressed the desire to use the maximum spec- tral response over all the rotation angles without geometric means, Boore (2010) introduced the measures of ground-shaking intensity irrespective of the sensor orientation. The measures are RotDnn and RotInn, whose computation is similar to GMRotDnn and GMRotInn without computing the geometric means. With regard to impact on seismic response, the opinion paper by Stewart et al. (2011) and the work by Mackie et al. (2011) on the impact of incidence angle on bridge response are relevant. Specifically, Stewart et al. (2011) noted the importance of computational analysis of structures (which had not been done as of 2011) in proposing appropriate spectra definitions. Other Methodologies for Addressing Seismic Ground Motion Hazards There are several other reports that address the question of the methodology that may be utilized in developing the seismic hazard. These recent studies endeavored to create a method- ology that is easier for engineers, as users, to understand how to tie the seismic hazard to the performance expectation. The variability of these approaches also demonstrates the broad range of options and therefore a limited understanding by practitioners in the bridge design industry. Following are some examples that apply to PBSD. Wang et al. (2016) performed a probabilistic seismic risk analysis (SRA) based on a single ground motion parameter (GMP). For structures whose responses can be better predicted using multiple GMPs, a vector-valued SRA (VSRA) gives accurate estimates of risk. A simplified approach to VSRA, which can substantially improve computational efficiency without losing accuracy, and a new seismic hazard de-aggregation procedure are proposed. This approach and the new seismic hazard de-aggregation procedure would allow an engineer to determine a set of controlling earthquakes in terms of magnitude, source–site distance, and occurrence rate for the site of interest. Wang et al. presented two numerical examples to validate the effectiveness and accuracy of the simplified approach. Factors affecting the approximations in the simplified approach were discussed. Kwong and Chopra (2015) investigated the issue of selecting and scaling ground motions as input excitations for response history analyses of buildings in performance-based earthquake engineering. Many ground motion selection and modification procedures have been developed to select ground motions for a variety of objectives. This report focuses on the selection and scaling of single, horizontal components of ground motion for estimating seismic demand hazard curves of multistory frames at a given site. Worden et al. (2012) used a database of approximately 200,000 modified Mercalli intensity (MMI) observations of California earthquakes collected from U.S. Geological Survey reports, along with a comparable number of peak ground motion amplitudes from California seismic networks, to develop probabilistic relationships between MMI and peak ground velocity (PGV), PGA, and 0.3-s, 1-s, and 3-s 5% damped pseudo-spectral acceleration. After associating each ground motion observation with an MMI computed from all the seismic responses within 2 kilometers of the observation, a joint probability distribution between MMI and ground motion was derived. A reversible relationship was then derived between MMI and each ground motion parameter by using a total least squares regression to fit a bilinear function to the median of the stacked probability distributions. Among the relationships, the fit-to-peak ground velocity has the smallest errors, although linear combinations of PGA and PGV give nominally better results. The magnitude and distance terms also reduce the overall residuals and are justifiable on an information theoretical basis.

Literature Review and Synthesis 17 Another approach to developing the appropriate seismic hazard comes out of Europe. Delavaud et al. (2012) presented a strategy to build a logic tree for ground motion prediction in European countries. Ground motion prediction equations and weights have been determined so that the logic tree captures epistemic uncertainty in ground motion prediction for six different tectonic regions in Europe. This includes selecting candidate GMPEs and simultaneously running them through a panel of six experts to generate independent logic trees and rank the GMPEs on available test data. The collaboration of this information is used to set a weight to the GMPEs and create a consensus logic tree. This output then is run through a sensitivity analysis of the proposed weights on the seismic hazard before setting a final logic tree for the GMPEs. Tehrani and Mitchell (2014) used updated seismic hazard maps for Montreal, Canada to develop a uniform hazard spectra for Site Class C and a seismic hazard curve to analyze bridges in the localized area. Kramer and Greenfield (2016) evaluated three case studies following the 2011 Tohoku earthquake to better understand and design for liquefaction. Existing case history databases are incomplete with respect to many conditions for which geotechnical engineers are often required to evaluate liquefaction potential. These include liquefaction at depth, liquefaction of relatively dense soils, and liquefaction of gravelly soils. Kramer and Greenfield’s investigation of the three case histories will add to the sparse existing data for those conditions, and their interpretations will aid in the validation and development of predictive procedures for liquefaction potential evaluation. Structural Analysis and Design Predicting the structural response to the earthquake ground motions is critical for the PBSD process. NCHRP Synthesis 440 outlined several analysis methods that can be used to accomplish this task. The multimodal linear dynamic procedures are outlined in AASHTO LRFD Bridge Design Specifications (AASHTO 2014) and AASHTO Guide Specifications for LRFD Seismic Bridge Design (AASHTO 2011), although the Guide Specifications also include the parameters for performing a model pushover analysis in addition to prescriptive detail practices to ensure energy-dissipating systems behave as intended and other elements are capacity-protected. Other methods of analysis may be better suited for PBSD, but the initial PBSD approach will likely follow the procedures of the AASHTO guide specifications, with multi-level hazards and performance expectations. Limited research and code development have been accomplished since NCHRP Synthesis 440, but one new analysis method, outlined in Babazadeh et al. (2015), includes a three-dimensional finite element model simulation that is used to efficiently predict intermediate damage limit states in a consistent manner, with the experimental observations extracted from the actual tested columns. Other recent articles of structural analysis identified areas of improvement in the current design methodology that may be beneficial to PBSD. Huff and Pezeshk (2016) compared the substitute structure method methodology for isolated bearings with the displacement-based design methodology for ordinary bridges and showed that these two methodologies vary in estimating inelastic displacements. Huff (2016a) identified issues that are generally simplified or ignored in current practice of predicting inelastic behavior of bridges during earthquakes, both on the capacity (in the section of the element type and geometric nonlinearities) and demand (issues related to viscous dampening levels) sides of the process. The current SGS methodology for nonlinear static procedures were compared in Hajihashemi et al. (2017) with recent methodologies for multimodal pushover procedures that take into account all significant modes of the structure and with modified equivalent linearization procedures developed for

18 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design FEMA-440 (FEMA 2005). All of these analysis articles identify areas of current discussion on how to improve the analytical procedures proposed in the SGS. NCHRP Synthesis 440 focused primarily on new analysis methods, but a recent increased focus, in both academia and industry, has to do with new materials and systems and their impacts on PBSD. The evolution of enhanced seismic performance has been wrapped into several research topics, such as accelerated bridge construction (ABC), novel columns, and PBSD. The following are several aspects, though not all-encompassing, which have been improved upon in the last 6 years or so. Improving Structural Analysis Through Better Material Data The analysis and performance of a bridge are controlled with material property parameters incorporated into the seismic analysis models, specifically for the push-over analysis method. AASHTO Guide Specifications for LRFD Seismic Bridge Design (AASHTO 2011) specifies the strain limits to use for ASTM A706 (Grade 60) and ASTM A615 Grade 60 reinforcement. These strain limits come from Caltrans study of 1,100 mill certificates for ASTM A706 Grade 60 in the mid-1990s for projects in Caltrans bridge construction. The results were reported as elongation—not strain—at peak stress, so select bar pull tests were performed to correlate elongation to strain at peak stress. This was assumed to be a conservative approach, though it has recently been validated with a new ASTM A706 Grade 80 study at North Carolina State University by Overby et al. (2015a), which showed Caltrans numbers, by comparison, for Grade 60 are reasonable and conservative. Overby et al. (2015b) developed stress strain parameters for ASTM A706 Grade 80 reinforcing steel. Approximately 800 tests were conducted on bars ranging from #4 to #18 from multiple heats from three producing mills. Statistical results were presented for elastic modulus, yield strain and stress, strain-hardening strain, strain at maximum stress, and ultimate stress. Research is currently under way at North Carolina State University that aims to identify strain limit states, plastic hinge lengths, and equivalent viscous damping models for bridge columns constructed from A706 Grade 80 reinforcing steel. Work is also under way at the University of California, San Diego, on applications of Grade 80 rebar for capacity-protected members such as bridge cap beams. Design Using New Materials and Systems Structural analysis and design are fundamentally about structural response to the earthquake ground motion and the analysis methods used to develop this relationship. The complexity of the analysis depends on the geometry of the structure and elements and the extent of inelastic behavior. This is coupled with the damage, or performance criteria but has been broken out for the purposes of this report and NCHRP Synthesis 440. Next generation bridge columns, often referred to as novel columns, are improving as a tool for engineers to control both the structural analysis, as the make-up of the material changes the inelastic behavior, and the element performance of bridges in higher seismic hazards. The energy-dissipating benefits of low damage materials—such as ultrahigh-performance concrete (UHPC), engineered cementi- tious composites (ECC), and shape memory alloy, fiber-reinforced polymer (FRP) wraps and tubes, elastomeric bearings, and post-tensioned strands or bars—can be utilized by engineers to improve seismic performance and life-cycle costs after a significant seismic event. Recent (Saiidi et al. 2017) studies tested various combinations of these materials to determine if there are columns that can be built with these materials that are equivalent to, or better than, conventional reinforced concrete columns (in terms of cost, complexity, and construction duration) but that improve seismic performance, provide greater ductility, reduce damage, and accommodate a quicker recovery time and reduce loss in both the bridge and the economic environment.

Literature Review and Synthesis 19 Accelerated bridge construction is also a fast-developing field in bridge engineering, with draft guide specifications for design and construction currently being developed for adop- tion by AASHTO for AASHTO LRFD Bridge Design Specifications (AASHTO 2014). ABC has economic impacts that go beyond seismic engineering, but research is focusing on details and connections for accelerated construction in higher seismic regions, moving two research paths forward at the same time. Tazarv and Saiidi (2014) incorporated ABC research with novel column research to evaluate combined novel column materials that can be constructed quickly. The research focused on the performance of materials and how to incorporate them into practice. Key mechanical properties of reinforcing SMA were defined as follows: • Observed yield strength (fyo) is the stress at the initiation of nonlinearity on the first cycle of loading to the upper plateau. • Austenite modulus (k1) is the average slope between 15% to 70% of fyo. • Post yield stiffness (k2) is the average slope of curve between 2.5% and 3.5% of strain on the upper plateau of the first cycle of loading to 6% strain. • Austenite yield strength (fy) is the stress at the intersection of line passing through origin with slope of k1 and line passing through stress at 3% strain with slope of k2. • Lower plateau inflection strength (fi) is the stress at the inflection point of lower plateau during unloading from the first cycle to 6% strain. • Lower plateau stress factor, β = 1 – (fi/fy). • Residual strain (eres) is the tensile strain after one cycle to 6% and unloading to 1 ksi (7 MPa). • Recoverable super-elastic strain (er) is maximum strain with at least 90% strain recovery capacity. Using the ASTM standard for tensile testing, er ≤ 6%. • Martensite modulus (k3) is the slope of the curve between 8% to 9% strain, subsequent to one cycle of loading to 6% strain, unloading to 1 ksi (7 MPa) and reloading to the ultimate stress. • Secondary post-yield stiffness ratio, α = k3/k1. • Ultimate strain (eu) is strain at failure. A graphical representation is shown in Figure 9, and minimum and expected mechanical properties are listed in Table 3. Other researchers, such as at the University of Washington, are currently testing grouted bars using conventional grouts and finding that these development lengths can be reduced greatly. However, it is the force transfer of the grouted duct to the reinforcing outside the duct that may Figure 9. NiTi SE SMA nonlinear model.

20 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design require additional length to adequately develop the energy-dissipating or capacity-protecting system that was intended by the designer for performance of the bridge in a high seismic event. Tazarv and Saiidi (2014) identified other material properties such as UHPC and ECC, shown in Tables 4 and 5, respectively. Tazarv and Saiidi (2014) also addressed grouted splice sleeve couplers, self-consolidating concrete (SCC), and other connection types that could be used in ABC and novel column configurations, testing these materials in the laboratory to see if various combinations produced a logical system to be carried forward in research, design, and implementation. Trono et al. (2015) studied a rocking post-tensioned hybrid fiber-reinforced concrete (HyFRC) bridge column that was designed to limit damage and residual drifts and that was tested dynamically under earthquake excitation. The column utilized post-tensioned strands, HyFRC, and a combination of unbonded and headed longitudinal reinforcement. There have been two projects related to the field of novel columns and ABC through the National Cooperative Highway Research Program. One project was NCHRP Project 12-101, which resulted in NCHRP Report 864, 2 volumes (Saiidi et al. 2017), and the other project was NCHRP Project 12-105, which resulted in NCHRP Research Report 935 (Saiidi et al. 2020). NCHRP Project 12-101 identified three novel column systems—specifically, SMA and ECC, ECC and FRP, and hybrid rocking column using post-tensioned strands and fiber-reinforced Parameter Tensile Compressive,ExpectedbExpectedbMinimuma Table 3. Minimum expected reinforcing NiTi SE SMA mechanical properties. Properties Range Poisson’s Ratio 0.2 Creep Coefficient* 0.2 to 0.8 Total Shrinkage** *Depends on curing conditions and age of loading. up to 900x10-6 Equation Compressive Strength (f'UHPC) f'UHPC 20 to 30 ksi, (140 to 200 MPa) Coefficient of Thermal Expansion (5.5 to 8.5)x10 -6/°F, (10 to 15)x10-6/°C Specific Creep* (0.04 to 0.3)x10 -6/psi, (6 to 45)x10-6/MPa A time-dependent equation for UHPC strength is available. Tensile Cracking Strength (ft,UHPC) ft,UHPC = 6.7 (psi) f'UHPCEUHPC = 49000 (psi) 0.9 to 1.5 ksi, (6 to 10 MPa) Modulus of Elasticity (EUHPC) 6000 to 10000 ksi, (40 to 70 GPa) **Combination of drying shrinkage and autogenous shrinkage and depends on curing method. Table 4. UHPC mechanical properties.

Literature Review and Synthesis 21 polymer confinement—and compared them to a conventional reinforced column. The research and properties of the material are provided; incorporating laboratory tests and calibration, design examples are created to help engineers understand how to use these advanced materials in a linear elastic seismic demand model and to determine performance using a pushover analysis. It is worth noting that ductility requirements do not accurately capture the perfor- mance capabilities of these novel columns, and drift ratio limits are being used instead, similar to the building industry. NCHRP Project 12-101 also provided evaluation criteria that can be evaluated and incorporated by AASHTO into a guide specification or into AASHTO Guide Specifications for LRFD Seismic Bridge Design (AASHTO 2011) directly. NCHRP Project 12-105 synthesized research, design codes, specifications, and contract language throughout all 50 states and combined the knowledge base and lessons learned for ABC into proposed guide specifications for both design and construction. This work focused on connections, and most of that information is related to seismic performance of ABC elements and systems. Earthquake resisting elements (ERE) and earthquake resisting systems (ERS) are specifically identified, defined, and prescribed for performance in AASHTO guide specifica- tions (AASHTO 2011) but only implicitly applied in AASHTO LRFD Bridge Design Specifications (AASHTO 2014). Since NCHRP Project 12-105 is applicable to both of these design resources, ERE and ERS are discussed in terms of how to apply performance to the force-based seismic design practice of AASHTO LRFD Bridge Design Specifications (AASHTO 2014). The proposed guide specification language also identifies when performance of materials have to be incor- porated into the design, say in higher seismic hazards, and when it is acceptable to apply ABC connections and detailing practices with prescriptive design methodologies. As the industry’s understanding of performance increases, the engineering industry is accepting the benefits that come from a more user-defined engineering practice that is implemented by identifying material properties; evaluating hazards and soil and structural responses; and verifying performance through strain limits, damage limits states, moment curvature, displacements, and ductility. These tools and advancements in ABC and novel column designs, including other material property performance and analytical methodologies, are allowing PBSD to advance in other areas, such as hazard prediction, loss prediction, and the owner decision-making process. Feng et al. (2014a) studied the application of fiber-based analysis to predict the nonlinear response of reinforced concrete bridge columns. Specifically considered were predictions of overall force-deformation hysteretic response and strain gradients in plastic hinge regions. The authors also discussed the relative merits of force-based and displacement-based fiber elements and proposed a technique for prediction of nonlinear strain distribution based on the modified compression field theory. Fulmer et al. (2013) developed a new steel bridge system that is based upon ABC techniques that employ an external socket to connect a circular steel pier to a cap beam through the use of grout and shear studs. The resulting system develops a plastic hinge in the pipe away from the column-to-cap interface. An advantage of the design is ease of construction, as no field welding Properties Range Flexural Strength 1.5 to 4.5 ksi (10 to 30 MPa) Modulus of Elasticity 2600 to 5000 ksi (18 to 34 GPa) Ultimate Tensile Strain 1 to 8% Ultimate Tensile Strength 0.6 to 1.7 ksi (4 to 12 MPa) First Crack Strength 0.4 to 1.0 ksi (3 to 7 MPa) Compressive Strength 3 to 14 ksi (20 to 95 MPa) Table 5. ECC mechanical properties.

22 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design is required: the two assemblies are placed together and the annular space between the column and cap filled with grout. Figure 10 shows the details of this connection, and Figure 11 shows a test of the system. Another system being investigated is isolation bearings or dampening devices. Xie and Zhiang (2016) investigated the effectiveness and optimal design of protective devices for the seismic protection of highway bridges. Fragility functions are first derived by probabilistic seismic demand analysis, repair cost ratios are then derived using a performance-based methodol- ogy, and the associated component failure probability. Subsequently, the researchers tried to identify the optimal design parameters of protective devices for six design cases with various combinations of isolation bearings and fluid dampers and discussed the outcomes. Damage mitigation through isolation and energy dissipation devices is continually improving based on research, development, and implementation in the field. Recent events within the State of Washington, Alaska, and other state agencies have shown that the benefits of these tools can be compromised if the intended performance cannot be sustained for the 75-year design life of the structure. Mackie and Stojadinovic (2015) outlined performance criteria for fabrica- tion and construction that need to be administered properly, and engineers should consider the effects of moisture, salts, or other corrosive environmental conditions that can affect the performance of the isolation or energy-dissipating system. Another constraint with these systems can be the proprietary nature that occurs as a specific isolation or energy-dissipating system is utilized to develop a specific performance expectation that can only be accomplished with the prescribed system. This proprietary nature of these systems can create issues for certain funding sources that require equal bidding opportunities and the project expense that can accompany a proprietary system. To address this type of design constraint, Illinois DOT has been developing an earthquake-resisting system (ERS) to leverage the displacement capacity available at typical bearings in order to provide seismic protection to substructures of typical bridges. LaFave et al. (2013a) identified the effects and design parameters, Source: Fulmer et al. (2013). 5" 4 at 5" O.C. A A A-A Connection Details 45° UT 100% 3 8" 12 Studs Spaced Around Cross Section 30°Typ. 15° Offset Studs Inside Pipe from Cap Beam CL HSS16x0.500 Pipe 24x0.500 2'-0"2 14 " 4 at 5" O.C. 212"-34 "Ø Shear Studs 1'-11" Pipe Stud Detail Grout Provided By and Placed by NCSU Figure 10. Grouted shear stud bridge system.

Literature Review and Synthesis 23 such as fuse capacity, shear response, and sliding response, which can be used to account for more standard bearing configurations in seismic analysis, especially lower seismic hazard regions. A variation on the use of bearings in order to improve seismic performance of a pier wall configuration was outlined in Bignell et al. (2006). Historically, pinned, rocking, and sliding bearings have been used with interior pier walls and steel girder superstructures. These bearing configurations were compared with replacement elastomeric bearing configurations and details for structural analysis techniques, damage limit states, and structural fragility, and performance through probability distributions were utilized as a PBSD process for determining solutions to seismic isolation and enhanced seismic performance. The foundation conditions, pier wall effects, bearing type, and even embankment effects to structural performance were included in this evaluation. Another approach to enhanced performance is modifications to foundation elements or increased understanding and modeling of soil–structure interaction, specifically where lateral spread or liquefaction design conditions make conventional bridge design and elements imprac- tical. One example of this is the seismic design and performance of bridges constructed with rocking foundations, as evaluated in Antonellis and Panagiotou (2013). This type of rocking goes beyond the loss of contact area currently allowed in the guide specifications. The applica- tion of columns supported on rocking foundations accommodates large deformations, while there is far less damage, and can re-center after large earthquakes. Another approach is to tie a tolerable displacement of an individual deep foundation element to a movement that would cause adverse performance, excessive maintenance issues, or functionality problems with the bridge structure. Roberts et al. (2011) established a performance-based soil–structure–interaction design approach for drilled shafts. Chiou and Tsai (2014) evaluated displacement ductility of an in-ground hinging of a fixed head pile. Assessment formulas were developed for the displacement ductility capacity of a fixed-head pile in cohesion-less soils. The parameters in the formulas included the sectional over-strength ratio and curvature ductility capacity, as well as a modification factor for consider- ing soil nonlinearity. The modification factor is a function of the displacement ratio of the pile’s ultimate displacement to the effective soil yield displacement, which is constructed through a number of numerical pushover analyses. Source: Fulmer et al. (2013). Figure 11. Photograph of completed system before seismic testing showing hinge locations.

24 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Damage Analysis As stated in NCHRP Synthesis 440, it is a fundamental need for the PBSD methodology to determine the type of damage and the likelihood that such damage will occur in the particular components of the structural system. This determination is of vital importance, as the damage sustained by a structure (and its nonstructural components) is directly relatable to the use or loss of a system after an earthquake. Therefore, there is a need to be able to reliably link structural and nonstructural response (internal forces, deformations, accelerations, and displacements) to damage. This is the realm of damage analyses, where damage is defined as discrete observable damage states (e.g., yield, spalling, longitudinal bar buckling, and bar fracture). Although the primary focus of the discussions is on structural components, similar considerations must be made for nonstructural components as well. NCHRP Synthesis 440 outlined an initial discussion on types of structural damage observed during historic earthquakes and laboratory experiments, prefaced the methods that have been developed to predict damage, identified structural details and concepts that could be used to reduce damage even in strong ground shaking, and reviewed post-event inspection tools. The new materials discussed in previous sections also apply to this discussion but are not repeated herein. Accurate damage prediction relies upon accurate definitions of performance limit states at the material level (i.e., strain limits) and the corresponding relationship between strain and displacement. Examples of recent research follow. Research by Feng et al. (2014b, 2014c) used finite element analysis validated by experimental test results to develop a model for predicting the tension strain corresponding to bar buckling. The model considers the impact of loading history on the boundary conditions of longitudinal bar restraint provided by the transverse steel. Goodnight et al. (2016a) identified strain limits to initiate bar buckling based on experimental results from 30 column tests (Equation 2). Following additional bidirectional tests on 12 columns, Equation 2 was revised to Equation 3. In addition, strain limit state equations were proposed for the compression strain in concrete to cause spiral yielding (Goodnight et al. 2017a). Goodnight et al. (2016b) also developed a new plastic hinge length model based on the data collected during those tests, which accounts for the actual curvature distribution in RC bridge columns. The revised model separates the strain penetration component from the flexural component while also recognizing that the hinge length for compression is smaller than that for tension. Brown et al. (2015) developed strain limit state (Equation 4) (tube wall local buckling) and equivalent viscous damping equations for reinforced concrete filled steel tubes (RCFSTs). The recommendations of the authors were based upon reversed cyclic tests of 12 RCFSTs of variable D/t (diameter to thickness) ratios. 0.03 700 0.1 (2)bucklingbar f E P f A s s yhe s ce g ε = + ρ − ′ 0.032 790 0.14 (3)bucklingbar f E P f A s s yhe s ce g ε = + ρ − ′ 0.021 9100 (4)tension buckling D t yε = − ≥ ε

Literature Review and Synthesis 25 where rs = reinforcement ratio, fyhe = expected yield strength of the steel tube (ksi), Es = elastic modulus of steel (ksi), P = axial load (kip), f ′ce = expected concrete strength (ksi), Ag = gross area of concrete (in.2), D = diameter of tube (in.), t = thickness of tube (in.), and ey = yield strain for steel (in./in.). Loss Analysis The PBSD combines the seismic hazard, structural, and damage analysis into a performance matrix that can be estimated into a loss metric. There are many loss metrics that can be used by, and that are important to, stakeholders and decision makers (discussed in detail in NCHRP Synthesis 440), but all these metrics can be boiled down to three main categories: deaths, dollars, and downtime. Bertero (2014) discussed earthquake lessons, in terms of loss, to be considered in both design and construction of buildings. At the beginning of 2010, two large earthquakes struck the Americas. The January 12, 2010, Haiti earthquake with a magnitude 7.0 produced about 300,000 deaths (second by the number of fatalities in world history after the 1556 Shaanxi, China earthquake). A month later, the February 27, 2010, Maule Chilean earthquake with a magnitude 8.8 (an energy release 500 times bigger than that from the Haiti earthquake) produced 500 deaths, most due to the resulting tsunami. However, the Chilean earthquake caused more than $30 billion of direct damage, left dozens of hospitals and thousands of schools nonoperational, and caused a general blackout for several hours, as well as the loss of service of essential communications facilities, crucial to take control of the chaotic after-earthquake situ- ation. Bertero (2014) compared the severity of both earthquakes and comments on their effects to life and the economy of the affected countries, as well as the features of the seismic codes or the absence of codes. An example of risk analysis with PBSD is utilized in Bensi et al. (2011), with the development of a Bayesian network (BN) methodology for performing infrastructure seismic risk assessment and providing decision support with an emphasis on immediate post-earthquake applications. A BN is a probabilistic graphical model that represents a set of random variables and their probabilistic dependencies. The proposed methodology consists of four major components: (1) a seismic demand model of ground motion intensity as a spatially distributed random field, accounting for multiple sources and including finite fault rupture and directivity effects; (2) a model for seismic performance of point-site and distributed components; (3) models of system performance as a function of component states; and (4) models of post-earthquake decision making for inspection and operation or shutdown of components. The use of the term Bayesian to describe this approach comes from the well-known Bayes rule, attributed to the 18th-century mathematician and philosopher Thomas Bayes: A B AB B B A B A( ) ( )( ) ( ) ( ) ( )= =Pr Pr Pr Pr Pr Pr (5) Pr(AB) is the probability of joint occurrence of Events A and B; Pr(A) is the marginal probability of Event A; Pr(A|B) is the conditional probability of Event A, given that Event B

26 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design has occurred; and Pr(B) is the marginal probability of Event B. The quantity Pr(B | A) is known as the likelihood of the observed Event B. Note that the probability of Event A appears on both sides of Equation 5. The Bayes rule describes how the probability of Event A changes given information gained about the occurrence of Event B. For discrete nodes, a conditional probability table is attached to each node that provides the conditional probability mass function (PMF) of the random variable represented by the node, given each of the mutually exclusive combinations of the states of its parents. For nodes without parents (e.g., X1 and X2 in Figure 12), known as root nodes, a marginal probability table is assigned. The joint PMF of all random variables X in the BN is constructed as the product of the conditional PMFs: (6) 1 p x p x Pa xi ii n∏( ) ( )( )= = Bensi et al. (2011) goes on to introduce BN models further and discusses how to incorporate BN-based seismic demand models into bridge design. The BN methodology is applied to modeling of random fields, construction of an approximate transformation matrix, and numer- ical investigation of approximation methods, including a discussion on the effect of correlation approximations on system reliability. Modeling component performance with BNs to capture seismic fragility of point-site components and distributed components, as well as modeling system performance of BNs with both qualitative and conventional methods, is explained. This reference goes on to identify efficient minimal link set (MLS), minimal cut set (MCS) formulations, optimal ordering of efficient MLS and MCS formulations, and heuristic augmen- tation that can be utilized with the BN methodology. Bensi et al. (2011) continues the PBSD process by addressing the owner decision-making process (see more discussion later in the report) and how to incorporate this model into that process. Two example problems are provided utilizing this methodology, including a California high-speed rail system that incorporates the bridge modeling into the example. Similarly, in Tehrani and Mitchell (2014), the seismic performance of 15 continuous four- span bridges with different arrangements of column heights and diameters was studied using incremental dynamic analysis (IDA). These bridges were designed using the Canadian Highway Bridge Design Code provisions (CSA 2006). The IDA procedure has been adopted by some guidelines to determine the seismic performance, collapse capacity, and fragility of buildings. Similar concepts can be used for the seismic assessment of bridges. Fragility curves can be devel- oped using the IDA results to predict the conditional probability that a certain damage state is exceeded at a given intensity measure value. Assuming that the IDA data are lognormally distributed, it is possible to develop the fragility curves at collapse (or any other damage state) by computing only the median collapse capacity and the logarithmic standard deviation of the IDA results for any given damage state. The fragility curves can then be analytically computed using Equation 7 as follows: ln ln (7)50% TOT P failure S x x S a a C( )( ) ( )= = Φ − β     where function F = cumulative normal distribution function, SCa 50% = median capacity determined from the IDA, and βTOT = total uncertainty caused by record-to-record variability, design requirements, test data, and structural modeling. Figure 12. A simple BN.

Literature Review and Synthesis 27 The seismic risk associated with exceeding different damage states in the columns, includ- ing yielding, cover spalling, bar buckling, and structural collapse (i.e., dynamic instability) was predicted. Some simplified equations were derived for Montreal, Quebec, Canada, to estimate the mean annual probability of exceeding different damage states in the columns using the IDA results. Repair and retrofit procedures are linked to loss predictions, as outlined in the FHWA’s retro- fitting manual (Buckle et al. 2006). Several chapters/articles address analysis, methodologies, effects, analytical tools, and costs for retrofit and repairs to mitigate damage or return a structure to a serviceable condition. Zimmerman et al. (2013) is one example, in which numerical techniques and seismic retrofit solutions for shear-critical reinforced concrete columns was investigated, utilizing test data of a reinforced concrete column with widely spaced transverse reinforcement. The study focused on the analysis method of nonlinear trusses and the retrofit option known as supplemental gravity columns, which is an example of how loss prediction and the analysis process are linked and should be iterated through PBSD. Organization-Specific Criteria for Bridges and Project-Specific Criteria NCHRP Synthesis 440 has two sections of criteria: organization-specific criteria for bridges and project-specific criteria. New information for both of these sections since NCHRP Synthesis 440 published is combined. The California DOT (Caltrans) Caltrans is currently updating their Seismic Design Criteria (SDC) to specify requirements to meet the performance goals for newly designed Ordinary Standard and Recovery Standard con- crete bridges. Nonstandard bridges require Project-Specific Seismic Design Criteria, in addition to the SDC, to address their nonstandard features. For both standard and nonstandard bridges, Caltrans is also categorizing their inventory in terms of Ordinary Bridges, Recovery Bridges, and Important Bridges. Some states have had issues with terms like Important or Essential, as a bridge is considered important to those that utilize each bridge. Caltrans is using these terms to correlate with loss analysis of an owner’s infrastructure and the time to reopen the bridge to support lifeline and recovery corridors. The bridge performance is also evaluated using a dual-seismic hazard; for Caltrans SDC they are listed as a Safety Evaluation Earthquake (SEE) for Ordinary Bridges. Both SEE and Functional Evaluation Earthquake (FEE) for Recovery Bridges are summarized in Table 6. Caltrans SDC revisions will also provide updates to the design parameters in Chapter 3 of the SDC and updates to both the analysis methods and displacement ductility demand values in Chapter 4 of the SDC. The adjustments to the displacement ductility demand values are revised to limit the bridge displacements beyond the initial yielding point of the ERE, specifically if a recovery standard bridge is being designed. The revisions to their SDC is an example of how PBSD is being gradually introduced as a better method of dealing with the hazards, soil–structure interaction, analysis tools, methodologies, material properties, damage states, performance, and loss. Similar revisions are being made to Seismic Design Specifications of Highway Bridges, as detailed in Japan Road Association (JRA) revisions in 2012. A synopsis of the revisions is provided in Kuwabara et al. (2013). The JRA specifications apply to Japanese road bridges and consist of five parts: Part I, Common; Part II, Steel Bridges; Part III, Concrete Bridges; Part IV, Substruc- tures; and Part V, Seismic Design. The revisions are based on improvements in terms of safety,

28 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design serviceability, and durability of bridges. Based on those lessons, design earthquake ground motions corresponding to the subduction-type earthquake were revised, and the requirements for easy and secure maintenance (inspection and repair works) for the bridges were clearly specified. JRA has clarified their performance of ERE conventionally reinforced columns for a dual-level (SPL 2 and SPL 3) seismic performance evaluation, as summarized in Table 7. The JRA 2012 revisions also address connection failures between reinforced concrete steel piles and the pile-supported spread footing to improve structural detailing and performance at the head of the piles. This is similar to research performed by the University of Washington, see Stephens et al. (2015) and Stephens et al. (2016) for both Caltrans and Washington State DOT, respectively, to evaluate capacity protecting this region and even considering the development of plastic hinges at these locations for combined hazard events or large lateral spreading and liquefaction occurrences. Caltrans also funded a study by Saini and Saiidi (2014) to address probabilistic seismic design of bridge columns using a probabilistic damage control approach and reliability analysis. Source: Caltrans. BRIDGE CATEGORY SEISMIC HAZARD EVALUATION LEVEL POST EARTHQUAKE DAMAGE STATE EXPECTED POST EARTHQUAKE SERVICE LEVEL Table 6. Caltrans draft proposed seismic design bridge performance criteria. SPL2 SPL3 Note: SPL1: Fully operational is required. Limit state of bridge is serviceability limit state. Negligible structural damage and nonstructural damage are allowed. Table 7. Seismic performance of bridge and limit states of conventionally reinforced concrete bridge column.

Literature Review and Synthesis 29 The probabilistic damage control approach uses the extent of lateral displacement nonlinearity defined by Damage Index (DI) to measure the performance of bridge columns. DI is a measure of damage from the lower measure of zero damage to the ultimate measure of a collapse mecha- nism for an element that has been subjected to base excitations. The performance objective was defined based on predefined apparent Damage States (DS), and the DS were correlated to DIs based on a previous study at the University of Nevada, Reno (Figure 13) (Vosooghi and Saiidi 2010). A statistical analysis of the demand damage index (DIL) was performed to develop fragility curves (load model) and to determine the reliability index for each DS. The results of the reliability analyses were analyzed, and a direct probabilistic damage control approach was developed to calibrate design DI to obtain a desired reliability index against failure. The calculated reliability indices and fragility curves showed that the proposed method could be effectively used in seismic design of new bridges, as well as in seismic assessment of existing bridges. The DS and DI are summarized with performance levels defined by Caltrans in Table 8, which shows the correlation between DS and DI. Figure 14 shows a fragility curve using lognormal distribution. Figure 15 shows both the fragility curves (upper two graphs) and reliability indices (lower two graphs) for four column bents (FCBs), with 4-foot diameter columns that are 30 feet in length in Site D for both the 1000 year and 2500 year seismic events. Note: O-ST = ordinary standard bridge, O-NST = ordinary nonstandard bridge, Rec. = recovery bridge, Imp. = important bridge, and NA = not applicable. Damage State (DS) Service to Public Service to Emergency Emergency Repair Design Damage Index (DI) Earthquake Levels (Years) Table 8. Design performance levels. DI P (D I { D S) Figure 13. Correlation between DS and DI.

30 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Figure 14. Fragility curve. 100% 80% 60% 40% 20% 0% 0.00 0.20 0.40 0.60 0.80 1.00 P (D I L ) DIL 4.0 3.0 2.0 1.0 0.0 R el ia bi lit y In de x | D S DS3 DS4 DS5 DS6 Damage State (DS) 6.0 5.0 4.0 3.0 2.0 1.0 0.0 R el ia bi lit y In de x | D S DS3 DS4 DS5 DS6 Damage State (DS) (a) (b) (d)(c) 0.00 0.20 0.40 0.60 0.80 1.00 DIL 100% 80% 60% 40% 20% 0% P (D I L ) Figure 15. Fragility curves and reliability indices for FCBs with 4-foot columns in Site D. The Oregon DOT The Oregon DOT is developing a global plan for addressing resiliency in order to improve recovery for the next Cascadia Earthquake and Tsunami, using PBSD in terms of applying applicable hazards, identifying critical services, developing a comprehensive assessment of structures and systems, and updating public policies. The resilience goals are similar to those discussed at the beginning of this chapter, with the following statement: Oregon citizens will not only be protected from life-threatening physical harm, but because of risk reduction measures and pre-disaster planning, communities will recover more quickly and with less continuing vulnerability following a Cascadia subduction zone earthquake and tsunami.

Literature Review and Synthesis 31 Research has shown that the next great (magnitude 9.0) Cascadia subduction zone earth- quake is pending, as shown in Figure 16. This comparison of historical subduction zone earthquakes in northern California, Oregon, and Washington covers 10000 years of seismic history. The evidence of a pending event has made decision makers and the public take notice and put forth resources to develop strategies revolving around PBSD. Oregon’s performance-based features are modified from NCHRP Synthesis 440 to account for a third hazard condition: Cascadia Subduction Zone Earthquake (CSZE) in Oregon DOT’s Bridge Design and Drafting Manual—Section 1, Design (Oregon DOT 2016a; see also Oregon DOT 2016b). Design of new bridges on and west of US 97 references two levels of perfor- mance criteria: life safety and operational. Design of new bridges east of US 97 requires life safety criteria only. Seismic design criteria for life safety and operational criteria are described as follows. • “Life Safety” Criteria: Design all bridges for a 1,000-year return period earthquake (7 percent prob- ability of exceedance in 75 years) to meet the “Life Safety” criteria using the 2014 USGS Hazard Maps. The probabilistic hazard maps for an average return period of 1,000 years and 500 years are available at ODOT Bridge Section website, but not available on USGS website. To satisfy the “Life Safety” criteria, use Response Modification Factors from LRFD Table 3.10.7.1-1 using an importance category of “other.” • “Operational” Criteria: Design all bridges on and west of US 97 to remain “Operational” after a full rupture of Cascadia Subduction Zone Earthquake (CSZE). The full-rupture CSZE hazard maps are available at the ODOT Bridge Section website. To satisfy the “Operational” criteria, use Response Modification Factors from LRFD Table 3.10.7.1-1 using an importance category of “essential.” When requested in writing by a local agency, the “Operational” criteria for local bridges may be waived. The CSZE is a deterministic event, and a deterministic design response spectrum must be generated. To allow for consistency and efficiency in design for the CSZE, an application for generating the design response spectra has been developed by Portland State University (Nako et al. 2009). AASHTO guide specifications values for Table 3.4.2.3-1 are modified into two tables for (1) values of Site Factor, Fpga, at zero-period on the acceleration spectrum and (2) values of Site Factor, Fa, for short-period range of acceleration spectrum. Table 3.4.2.3-2 is replaced with values of Site Factor, Fv, for long-period range of acceleration spectrum. For seismic retrofit projects, the lower level ground motion is modified to be the CSZE with full rupture, as seen in Table 9. Performance levels, including performance level zero (PL0), are specified based on bridge importance and the anticipated service life (ASL) category required. Source: OSSPAC (2013). Figure 16. Cascadia earthquake timeline.

32 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design The South Carolina DOT South Carolina Department of Transportation (South Carolina DOT) has updated its geo- technical design manual (South Carolina DOT 2019). Chapters 12, 13, and 14 for geo technical seismic analysis, hazard, and design, respectively, have been updated to current practices and research, including incorporation of PBSD hazard prediction. South Carolina DOT is also updating their site coefficients to be more appropriate for South Carolina’s geologic and seismic conditions; see Andrus et al. (2014). Note that with the revisions, South Carolina DOT issued a design memorandum in November 2015 that revised the substructure unit quantitative damage criteria (maximum ductility demand) table (Table 7.1 of the SCDOT Seismic Design Specifications for Highway Bridges). See Table 10. The Utah DOT The Utah DOT and Brigham Young University (see Franke et al. 2014a, 2014b, 2015a, 2015b, 2015c, 2016) are researching the ability for engineers to apply the benefits of the full performance- based probabilistic earthquake analysis without requiring specialized software, training, or education. There is an emphasis on differences between deterministic and performance-based procedures for assessing liquefaction hazards and how the output can vary significantly with these two methodologies, especially in areas of low seismicity. Guidance is provided regarding when to use each of the two methodologies and how to bind the analysis effort. Additionally, a simplified performance-based procedure for assessment of liquefaction triggering using liquefaction loading maps was developed with this research. The components of this tool, as well as step-by-step procedures for the liquefaction initiation and lateral spread displacement models, are provided. The tool incorporates the simplified performance-based procedures determined with this research. National Highway Institute Marsh et al. (2014) referenced a manual for the National Highway Institute’s training course for engineers to understand displacement-based LRFD seismic analysis and design of bridges, which is offered through state agencies and open to industry engineers and geotechnical engi- neers. This course helps designers understand the principles behind both force-based AASHTO (AASHTO 2014) and displacement-based AASHTO (AASHTO 2011) methodologies, including a deeper understanding of what performance means in a seismic event. Other similar courses are also being offered to industry and are improving the understanding of practicing engineers. Federal Emergency Management Agency The Federal Emergency Management Agency (FEMA) has developed a series of design guidelines for seismic performance assessment of buildings and three of the five documents EARTHQUAKE GROUND MOTION BRIDGE IMPORTANCE and SERVICE LIFE CATEGORY Table 9. Modifications to minimum performance levels for retrofitted bridges.

Literature Review and Synthesis 33 are referenced in FEMA (2012a, 2012b, 2012c). A step-by-step methodology and explanation of implementation are provided for an intensity-based assessment and for a time-based assess- ment. The process of identifying and developing appropriate fragility curves is demonstrated. A software program called Performance Assessment Calculation Tool has also been developed with a user manual that is included in the FEMA documents to help engineers apply PBSD to the building industry. Japan Road Association The Japan Road Association (JRA) Design Specifications have been revised based on the performance-based design code concept in response to the international harmonization of design codes and the flexible employment of new structures and new construction methods. Figure 17 shows the code structure for seismic design using the JRA Design Specifications. The performance matrix is based on a two-level ground motion (Earthquakes 1 and 2), with the first one based on an interpolate-type earthquake and magnitude of around 8, and the second one with a magnitude of around 7 with a short distance to the structure. Kuwabara et al. (2013) outlined the incremental revisions from the JRA Design Specif i- cations between 2002 and 2012. These revisions include, but are not limited to, the ductility design method of reinforced concrete bridges, plastic hinge length equation, evaluation of hollow columns, and the introduction of high-strength steel reinforcement. Following the 2016 earthquake in Kumamoto, Japan, a new version of the JRA Design Specifications is in the works. Note: Analysis for FEE is not required for OC III bridges. Source: South Carolina DOT (2015). Design Earthquake Operational Classification (OC)Bridge Systems Table 10. South Carolina DOT substructure unit quantitative damage criteria (maximum ductility demand ld).

34 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design Identification of Knowledge Gaps The resources to develop guide specifications for PBSD are improving with examples such as the upcoming Seismic Design Criteria, Version 2 from Caltrans, which will address aspects of PBSD and the building industry’s efforts to develop practices in PBSD and tools for engineers and owners to collaborate on solutions based on performance criteria and expectations. There is still a perception that the bridge industry could better predict likely performance in large, damaging earthquakes than is being done at the present, and there are still gaps in that knowledge base that need to be closed. Most of the knowledge gaps listed in Marsh and Stringer (2013) are still applicable today; see Table 11. The technology readiness levels represent what has been developed and used; what research is done, ongoing, and being discussed; and what only exists in concept. Knowledge gaps certainly exist in all facets of PBSD; however, other key knowledge gaps beyond those listed in NCHRP Synthesis 440 (Marsh and Stringer 2013) that should be closed in order to improve the implementation of PBSD are covered. Objectives of Codes Mandated Specifications Overall Goals Functional Requirements (Basic Requirements) Performance Requirement Level Verification Methods and Acceptable Solutions Can be Modified or May be Selected with Necessary Verifications Importance, Loads, Design Ground Motion, Limit States Principles of Performance Verification Verifications of Seismic Performances (Static and Dynamic Verifications) Evaluation of Limit States of Members (RC and Steel Columns, Bearings, Foundations and Superstructure) Unseating Prevention Systems Principles of Seismic Design Figure 17. Code structure for seismic design using JRA design specifications. TRL Description 0-25 25-50 50-75 75-100 1 PBSD concept exists 2 Seismic hazard deployable 3 Structural analysis deployable 4 Damage analysis deployable 5 Loss analysis deployable 6 Owners willing and skilled in PBSD 7 Design guidelines 8 Demonstration projects 9 Proven effectiveness in earthquake Technology Readiness Level (TRL) % of Development Complete Table 11. Technology readiness levels for PBSD.

Literature Review and Synthesis 35 Gaps related to structural analysis can include minimum and expected properties for reinforcing greater than Grade 80, stainless steel, and other materials that can improve serviceability and in some conditions performance. Oregon DOT has been using stainless steel in their bridges located along the coastline and other highly corrosive environments to extend the service life of the bridge; however, many of these locations are also prone to large CSZE and the use of these materials in earthquake resisting elements is still being developed. In the State of Washington’s resiliency plan, outlined in Washington State Emergency Management Council–Seismic Safety Committee (2012), what is missing is a link between damage levels and return to service. This is a knowledge gap given what we know structurally and what this report is suggesting as a desired goal for post-earthquake recovery. Gaps related to decision makers can include bridge collapse. It is not intended that the PBSD guide specifications will address tsunami events, but the JRA specifications do address tsunami as well as landslide effects. Figures 18 and 19 are examples of these other types of failure systems and show the collapse of bridges caused by effects other than ground motion (Kuwabara et al. 2013). The decision to combine these types of effects with a seismic hazard, even combining liquefaction, down drag, and lateral spreading effects, needs additional clarification and is currently left up to the owner to assess implications of probability, safety, and cost ramifications. Liang and Lee (2013) summarized that in order to update the extreme event design limit states in the AASHTO 2014, combinations of all nonextreme and extreme loads need to be formulated on the same probability-based platform. Accounting for more than one-time variable load creates a complex situation, in which all of the possible load combinations, even many that are not needed for the purpose of bridge design, have to be determined. A formulation of a criterion to determine if a specific term is necessary to be included or rejected is described, and a comparison of the value of a given failure probability to the total pre-set permissible design failure probability can be chosen as this criterion. Figure 18. Collapse of bridge due to landslide. (Note: Reprinted courtesy of the National Institute of Standards and Technology, U.S. Department of Commerce. Not copyrightable in the United States). Source: Kuwabara et al. (2013).

36 Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design While the seismic hazard definition was once thought to be relatively well understood, there is a growing knowledge gap related to the effect of rotation angle on intensity of ground motions and how the use of a geometric mean of the motions, or other methods of including the effect of rotation angle (RotDxx), should be incorporated into seismic design. This issue is not specific to PBSD; like all seismic design methods, PBSD is reliant on a full understanding of the hazard definition for proper implementation. The knowledge gaps identified in NCHRP Synthesis 440 are still applicable. Many of these knowledge gaps will become evident to both engineers and decision makers as the PBSD guidelines are developed. Overall, the baseline information to develop PBSD guide specifica- tions are in place. Industry’s end goal of understanding the relationship between risk-based decision making and design decisions and methodologies to meet performance goals is going to be an iterative process. Figure 19. Collapse of bridge due to tsunami. (Note: Reprinted courtesy of the National Institute of Standards and Technology, U.S. Department of Commerce. Not copyrightable in the United States). Source: Kuwabara et al. (2013).

Performance-based seismic design (PBSD) for infrastructure in the United States is a developing field, with new research, design, and repair technologies; definitions; and methodologies being advanced every year.

The TRB National Cooperative Highway Research Program's NCHRP Research Report 949: Proposed AASHTO Guidelines for Performance-Based Seismic Bridge Design presents a methodology to analyze and determine the seismic capacity requirements of bridge elements expressed in terms of service and damage levels of bridges under a seismic hazard. The methodology is presented as proposed AASHTO guidelines for performance-based seismic bridge design with ground motion maps and detailed design examples illustrating the application of the proposed guidelines and maps.

Supplemental materials to the report include an Appendix A - SDOF Column Investigation Sample Calculations and Results and Appendix B - Hazard Comparison.

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

• Introduction for Types of Dissertations
• Overview of the Dissertation
• Self-Assessment Exercise
• What is a Dissertation Committee
• Different Types of Dissertations
• Introduction for Overview of the Dissertation Process
• Responsibilities: the Chair, the Team and You
• Sorting Exercise
• Stages of a Dissertation
• Managing Your Time
• Create Your Own Timeline
• Working with a Writing Partner
• Key Deadlines
• Self Assessment Exercise
• Additional Resources
• Purpose and Goals
• Read and Evaluate Chapter 1 Exemplars
• Draft an Introduction of the Study
• Outline the Background of the Problem
• Draft your Statement of the Problem
• Draft your Purpose of the Study
• Draft your Significance of the Study
• List the Possible Limitations and Delimitations
• Explicate the Definition of Terms
• Outline the Organization of the Study
• Recommended Resources and Readings
• Purpose of the Literature Review
• What is the Literature?
• Article Summary Table
• Writing a Short Literature Review
• Outline for Literature Review
• Synthesizing the Literature Review
• Purpose of the Methodology Chapter
• Topics to Include
• Preparing to Write the Methodology Chapter
• Confidentiality
• Building the Components for Chapter Three
• Preparing for Your Qualifying Exam (aka Proposal Defense)
• What is Needed for Your Proposal Defense?
• Submitting Your Best Draft
• Preparing Your Abstract for IRB
• Use of Self-Assessment
• Preparing Your PowerPoint
• During Your Proposal Defense
• After Your Proposal Defense
• Pre-observation – Issues to consider
• During Observations
• Wrapping Up
• Recommended Resources and Readings (Qualitative)
• Quantitative Data Collection
• Recommended Resources and Readings (Quantitative)
• Qualitative: Before you Start
• Qualitative: During Analysis
• Qualitative: After Analysis
• Qualitative: Recommended Resources and Readings
• Quantitative: Deciding on the Right Analysis
• Quantitative: Data Management and Cleaning
• Quantitative: Keep Track of your Analysis
• The Purpose of Chapter 4
• The Elements of Chapter 4
• Presenting Results (Quantitative)
• Presenting Findings (Qualitative)
• Chapter 4 Considerations
• The Purpose of Chapter 5
• Preparing Your Abstract for the Graduate School
• Draft the Introduction for Chapter 5
• Draft the Summary of Findings
• Draft Implications for Practice
• Draft your Recommendations for Research
• Draft your Conclusions
• What is Needed
• What Happens During the Final Defense?
• What Happens After the Final Defense?

Synthesizing the Literature Review Topic 4: Literature Review

Writing in the Health and Social Sciences: Literature Reviews and Synthesis Tools

• Journal Publishing
• Style and Writing Guides
• Readings about Writing
• Citing in APA Style This link opens in a new window
• Resources for Dissertation Authors
• Citation Management and Formatting Tools
• What are Literature Reviews?
• Conducting & Reporting Systematic Reviews
• Finding Systematic Reviews
• Tutorials & Tools for Literature Reviews

Systematic Literature Reviews: Steps & Resources

These steps for conducting a systematic literature review are listed below . 

Also see subpages for more information about:

• The different types of literature reviews, including systematic reviews and other evidence synthesis methods
• Tools & Tutorials

Literature Review & Systematic Review Steps

• Develop a Focused Question
• Scope the Literature  (Initial Search)
• Refine & Expand the Search
• Limit the Results
• Download Citations
• Abstract & Analyze
• Create Flow Diagram
• Synthesize & Report Results

1. Develop a Focused   Question 

Consider the PICO Format: Population/Problem, Intervention, Comparison, Outcome

Focus on defining the Population or Problem and Intervention (don't narrow by Comparison or Outcome just yet!)

"What are the effects of the Pilates method for patients with low back pain?"

Tools & Additional Resources:

• PICO Question Help
• Stillwell, Susan B., DNP, RN, CNE; Fineout-Overholt, Ellen, PhD, RN, FNAP, FAAN; Melnyk, Bernadette Mazurek, PhD, RN, CPNP/PMHNP, FNAP, FAAN; Williamson, Kathleen M., PhD, RN Evidence-Based Practice, Step by Step: Asking the Clinical Question, AJN The American Journal of Nursing : March 2010 - Volume 110 - Issue 3 - p 58-61 doi: 10.1097/01.NAJ.0000368959.11129.79

2. Scope the Literature

A "scoping search" investigates the breadth and/or depth of the initial question or may identify a gap in the literature. 

Eligible studies may be located by searching in:

• Background sources (books, point-of-care tools)
• Article databases
• Trial registries
• Grey literature
• Cited references
• Reference lists

When searching, if possible, translate terms to controlled vocabulary of the database. Use text word searching when necessary.

Use Boolean operators to connect search terms:

• Combine separate concepts with AND  (resulting in a narrower search)
• Connecting synonyms with OR  (resulting in an expanded search)

Search:  pilates AND ("low back pain"  OR  backache )

Video Tutorials - Translating PICO Questions into Search Queries

• Translate Your PICO Into a Search in PubMed (YouTube, Carrie Price, 5:11) 
• Translate Your PICO Into a Search in CINAHL (YouTube, Carrie Price, 4:56)

3. Refine & Expand Your Search

Expand your search strategy with synonymous search terms harvested from:

• database thesauri
• reference lists
• relevant studies

Example: 

(pilates OR exercise movement techniques) AND ("low back pain" OR backache* OR sciatica OR lumbago OR spondylosis)

As you develop a final, reproducible strategy for each database, save your strategies in a:

• a personal database account (e.g., MyNCBI for PubMed)
• Log in with your NYU credentials
• Open and "Make a Copy" to create your own tracker for your literature search strategies

4. Limit Your Results

Use database filters to limit your results based on your defined inclusion/exclusion criteria.  In addition to relying on the databases' categorical filters, you may also need to manually screen results.  

• Limit to Article type, e.g.,:  "randomized controlled trial" OR multicenter study
• Limit by publication years, age groups, language, etc.

NOTE: Many databases allow you to filter to "Full Text Only".  This filter is  not recommended . It excludes articles if their full text is not available in that particular database (CINAHL, PubMed, etc), but if the article is relevant, it is important that you are able to read its title and abstract, regardless of 'full text' status. The full text is likely to be accessible through another source (a different database, or Interlibrary Loan).  

• Filters in PubMed
• CINAHL Advanced Searching Tutorial

5. Download Citations

Selected citations and/or entire sets of search results can be downloaded from the database into a citation management tool. If you are conducting a systematic review that will require reporting according to PRISMA standards, a citation manager can help you keep track of the number of articles that came from each database, as well as the number of duplicate records.

In Zotero, you can create a Collection for the combined results set, and sub-collections for the results from each database you search.  You can then use Zotero's 'Duplicate Items" function to find and merge duplicate records.

• Citation Managers - General Guide

6. Abstract and Analyze

• Migrate citations to data collection/extraction tool
• Screen Title/Abstracts for inclusion/exclusion
• Screen and appraise full text for relevance, methods, 
• Resolve disagreements by consensus

Covidence is a web-based tool that enables you to work with a team to screen titles/abstracts and full text for inclusion in your review, as well as extract data from the included studies.

• Covidence Support
• Critical Appraisal Tools
• Data Extraction Tools

7. Create Flow Diagram

The PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flow diagram is a visual representation of the flow of records through different phases of a systematic review.  It depicts the number of records identified, included and excluded.  It is best used in conjunction with the PRISMA checklist .

Example from: Stotz, S. A., McNealy, K., Begay, R. L., DeSanto, K., Manson, S. M., & Moore, K. R. (2021). Multi-level diabetes prevention and treatment interventions for Native people in the USA and Canada: A scoping review. Current Diabetes Reports, 2 (11), 46. https://doi.org/10.1007/s11892-021-01414-3

• PRISMA Flow Diagram Generator (ShinyApp.io, Haddaway et al. )
• PRISMA Diagram Templates  (Word and PDF)
• Make a copy of the file to fill out the template
• Image can be downloaded as PDF, PNG, JPG, or SVG
• Covidence generates a PRISMA diagram that is automatically updated as records move through the review phases

8. Synthesize & Report Results

There are a number of reporting guideline available to guide the synthesis and reporting of results in systematic literature reviews.

It is common to organize findings in a matrix, also known as a Table of Evidence (ToE).

• Reporting Guidelines for Systematic Reviews
• Download a sample template of a health sciences review matrix  (GoogleSheets)

Steps modified from: 

Cook, D. A., & West, C. P. (2012). Conducting systematic reviews in medical education: a stepwise approach.   Medical Education , 46 (10), 943–952.

• << Previous: Citation Management and Formatting Tools
• Next: What are Literature Reviews? >>
• Last Updated: Mar 28, 2024 2:07 PM
• URL: https://guides.nyu.edu/healthwriting

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker
• APA Citation Generator
• MLA Citation Generator
• Chicago Citation Generator
• Vancouver Citation Generator
• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Make a Literature Review in Research (RRL Example)

What is an RRL in a research paper?

A relevant review of the literature (RRL) is an objective, concise, critical summary of published research literature relevant to a topic being researched in an article. In an RRL, you discuss knowledge and findings from existing literature relevant to your study topic. If there are conflicts or gaps in existing literature, you can also discuss these in your review, as well as how you will confront these missing elements or resolve these issues in your study.

To complete an RRL, you first need to collect relevant literature; this can include online and offline sources. Save all of your applicable resources as you will need to include them in your paper. When looking through these sources, take notes and identify concepts of each source to describe in the review of the literature.

A good RRL does NOT:

A literature review does not simply reference and list all of the material you have cited in your paper.

• Presenting material that is not directly relevant to your study will distract and frustrate the reader and make them lose sight of the purpose of your study.
• Starting a literature review with “A number of scholars have studied the relationship between X and Y” and simply listing who has studied the topic and what each scholar concluded is not going to strengthen your paper.

A good RRL DOES:

• Present a brief typology that orders articles and books into groups to help readers focus on unresolved debates, inconsistencies, tensions, and new questions about a research topic.
• Summarize the most relevant and important aspects of the scientific literature related to your area of research
• Synthesize what has been done in this area of research and by whom, highlight what previous research indicates about a topic, and identify potential gaps and areas of disagreement in the field
• Give the reader an understanding of the background of the field and show which studies are important—and highlight errors in previous studies

How long is a review of the literature for a research paper?

The length of a review of the literature depends on its purpose and target readership and can vary significantly in scope and depth. In a dissertation, thesis, or standalone review of literature, it is usually a full chapter of the text (at least 20 pages). Whereas, a standard research article or school assignment literature review section could only be a few paragraphs in the Introduction section .

Building Your Literature Review Bookshelf

One way to conceive of a literature review is to think about writing it as you would build a bookshelf. You don’t need to cut each piece by yourself from scratch. Rather, you can take the pieces that other researchers have cut out and put them together to build a framework on which to hang your own “books”—that is, your own study methods, results, and conclusions.

What Makes a Good Literature Review?

The contents of a literature review (RRL) are determined by many factors, including its precise purpose in the article, the degree of consensus with a given theory or tension between competing theories, the length of the article, the number of previous studies existing in the given field, etc. The following are some of the most important elements that a literature review provides.

Historical background for your research

Analyze what has been written about your field of research to highlight what is new and significant in your study—or how the analysis itself contributes to the understanding of this field, even in a small way. Providing a historical background also demonstrates to other researchers and journal editors your competency in discussing theoretical concepts. You should also make sure to understand how to paraphrase scientific literature to avoid plagiarism in your work.

The current context of your research

Discuss central (or peripheral) questions, issues, and debates in the field. Because a field is constantly being updated by new work, you can show where your research fits into this context and explain developments and trends in research.

A discussion of relevant theories and concepts

Theories and concepts should provide the foundation for your research. For example, if you are researching the relationship between ecological environments and human populations, provide models and theories that focus on specific aspects of this connection to contextualize your study. If your study asks a question concerning sustainability, mention a theory or model that underpins this concept. If it concerns invasive species, choose material that is focused in this direction.

Definitions of relevant terminology

In the natural sciences, the meaning of terms is relatively straightforward and consistent. But if you present a term that is obscure or context-specific, you should define the meaning of the term in the Introduction section (if you are introducing a study) or in the summary of the literature being reviewed.

Description of related relevant research

Include a description of related research that shows how your work expands or challenges earlier studies or fills in gaps in previous work. You can use your literature review as evidence of what works, what doesn’t, and what is missing in the field.

Supporting evidence for a practical problem or issue your research is addressing that demonstrates its importance: Referencing related research establishes your area of research as reputable and shows you are building upon previous work that other researchers have deemed significant.

Types of Literature Reviews

Literature reviews can differ in structure, length, amount, and breadth of content included. They can range from selective (a very narrow area of research or only a single work) to comprehensive (a larger amount or range of works). They can also be part of a larger work or stand on their own.

• A course assignment is an example of a selective, stand-alone work. It focuses on a small segment of the literature on a topic and makes up an entire work on its own.
• The literature review in a dissertation or thesis is both comprehensive and helps make up a larger work.
• A majority of journal articles start with a selective literature review to provide context for the research reported in the study; such a literature review is usually included in the Introduction section (but it can also follow the presentation of the results in the Discussion section ).
• Some literature reviews are both comprehensive and stand as a separate work—in this case, the entire article analyzes the literature on a given topic.

Literature Reviews Found in Academic Journals

The two types of literature reviews commonly found in journals are those introducing research articles (studies and surveys) and stand-alone literature analyses. They can differ in their scope, length, and specific purpose.

Literature reviews introducing research articles

The literature review found at the beginning of a journal article is used to introduce research related to the specific study and is found in the Introduction section, usually near the end. It is shorter than a stand-alone review because it must be limited to very specific studies and theories that are directly relevant to the current study. Its purpose is to set research precedence and provide support for the study’s theory, methods, results, and/or conclusions. Not all research articles contain an explicit review of the literature, but most do, whether it is a discrete section or indistinguishable from the rest of the Introduction.

How to structure a literature review for an article

When writing a literature review as part of an introduction to a study, simply follow the structure of the Introduction and move from the general to the specific—presenting the broadest background information about a topic first and then moving to specific studies that support your rationale , finally leading to your hypothesis statement. Such a literature review is often indistinguishable from the Introduction itself—the literature is INTRODUCING the background and defining the gaps your study aims to fill.

The stand-alone literature review

The literature review published as a stand-alone article presents and analyzes as many of the important publications in an area of study as possible to provide background information and context for a current area of research or a study. Stand-alone reviews are an excellent resource for researchers when they are first searching for the most relevant information on an area of study.

Such literature reviews are generally a bit broader in scope and can extend further back in time. This means that sometimes a scientific literature review can be highly theoretical, in addition to focusing on specific methods and outcomes of previous studies. In addition, all sections of such a “review article” refer to existing literature rather than describing the results of the authors’ own study.

In addition, this type of literature review is usually much longer than the literature review introducing a study. At the end of the review follows a conclusion that once again explicitly ties all of the cited works together to show how this analysis is itself a contribution to the literature. While not absolutely necessary, such articles often include the terms “Literature Review” or “Review of the Literature” in the title. Whether or not that is necessary or appropriate can also depend on the specific author instructions of the target journal. Have a look at this article for more input on how to compile a stand-alone review article that is insightful and helpful for other researchers in your field.

How to Write a Literature Review in 6 Steps

So how do authors turn a network of articles into a coherent review of relevant literature?

Writing a literature review is not usually a linear process—authors often go back and check the literature while reformulating their ideas or making adjustments to their study. Sometimes new findings are published before a study is completed and need to be incorporated into the current work. This also means you will not be writing the literature review at any one time, but constantly working on it before, during, and after your study is complete.

Here are some steps that will help you begin and follow through on your literature review.

Step 1: Choose a topic to write about—focus on and explore this topic.

Choose a topic that you are familiar with and highly interested in analyzing; a topic your intended readers and researchers will find interesting and useful; and a topic that is current, well-established in the field, and about which there has been sufficient research conducted for a review. This will help you find the “sweet spot” for what to focus on.

Step 2: Research and collect all the scholarly information on the topic that might be pertinent to your study.

This includes scholarly articles, books, conventions, conferences, dissertations, and theses—these and any other academic work related to your area of study is called “the literature.”

Step 3: Analyze the network of information that extends or responds to the major works in your area; select the material that is most useful.

Use thought maps and charts to identify intersections in the research and to outline important categories; select the material that will be most useful to your review.

Step 4: Describe and summarize each article—provide the essential information of the article that pertains to your study.

Determine 2-3 important concepts (depending on the length of your article) that are discussed in the literature; take notes about all of the important aspects of this study relevant to the topic being reviewed.

For example, in a given study, perhaps some of the main concepts are X, Y, and Z. Note these concepts and then write a brief summary about how the article incorporates them. In reviews that introduce a study, these can be relatively short. In stand-alone reviews, there may be significantly more texts and more concepts.

Step 5: Demonstrate how these concepts in the literature relate to what you discovered in your study or how the literature connects the concepts or topics being discussed.

In a literature review intro for an article, this information might include a summary of the results or methods of previous studies that correspond to and/or confirm those sections in your own study. For a stand-alone literature review, this may mean highlighting the concepts in each article and showing how they strengthen a hypothesis or show a pattern.

Discuss unaddressed issues in previous studies. These studies that are missing something you address are important to include in your literature review. In addition, those works whose theories and conclusions directly support your findings will be valuable to review here.

Step 6: Identify relationships in the literature and develop and connect your own ideas to them.

This is essentially the same as step 5 but focused on the connections between the literature and the current study or guiding concepts or arguments of the paper, not only on the connections between the works themselves.

Your hypothesis, argument, or guiding concept is the “golden thread” that will ultimately tie the works together and provide readers with specific insights they didn’t have before reading your literature review. Make sure you know where to put the research question , hypothesis, or statement of the problem in your research paper so that you guide your readers logically and naturally from your introduction of earlier work and evidence to the conclusions you want them to draw from the bigger picture.

Your literature review will not only cover publications on your topics but will include your own ideas and contributions. By following these steps you will be telling the specific story that sets the background and shows the significance of your research and you can turn a network of related works into a focused review of the literature.

Literature Review (RRL) Examples

Because creating sample literature reviews would take too long and not properly capture the nuances and detailed information needed for a good review, we have included some links to different types of literature reviews below. You can find links to more literature reviews in these categories by visiting the TUS Library’s website . Sample literature reviews as part of an article, dissertation, or thesis:

• Critical Thinking and Transferability: A Review of the Literature (Gwendolyn Reece)
• Building Customer Loyalty: A Customer Experience Based Approach in a Tourism Context (Martina Donnelly)

Sample stand-alone literature reviews

• Literature Review on Attitudes towards Disability (National Disability Authority)
• The Effects of Communication Styles on Marital Satisfaction (Hannah Yager)

Additional Literature Review Format Guidelines

In addition to the content guidelines above, authors also need to check which style guidelines to use ( APA , Chicago, MLA, etc.) and what specific rules the target journal might have for how to structure such articles or how many studies to include—such information can usually be found on the journals’ “Guide for Authors” pages. Additionally, use one of the four Wordvice citation generators below, choosing the citation style needed for your paper:

Wordvice Writing and Academic Editing Resources

Finally, after you have finished drafting your literature review, be sure to receive professional proofreading services , including paper editing for your academic work. A competent proofreader who understands academic writing conventions and the specific style guides used by academic journals will ensure that your paper is ready for publication in your target journal.

See our academic resources for further advice on references in your paper , how to write an abstract , how to write a research paper title, how to impress the editor of your target journal with a perfect cover letter , and dozens of other research writing and publication topics.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Synthesizing Sources

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

When you look for areas where your sources agree or disagree and try to draw broader conclusions about your topic based on what your sources say, you are engaging in synthesis. Writing a research paper usually requires synthesizing the available sources in order to provide new insight or a different perspective into your particular topic (as opposed to simply restating what each individual source says about your research topic).

Note that synthesizing is not the same as summarizing.  

• A summary restates the information in one or more sources without providing new insight or reaching new conclusions.
• A synthesis draws on multiple sources to reach a broader conclusion.

There are two types of syntheses: explanatory syntheses and argumentative syntheses . Explanatory syntheses seek to bring sources together to explain a perspective and the reasoning behind it. Argumentative syntheses seek to bring sources together to make an argument. Both types of synthesis involve looking for relationships between sources and drawing conclusions.

In order to successfully synthesize your sources, you might begin by grouping your sources by topic and looking for connections. For example, if you were researching the pros and cons of encouraging healthy eating in children, you would want to separate your sources to find which ones agree with each other and which ones disagree.

After you have a good idea of what your sources are saying, you want to construct your body paragraphs in a way that acknowledges different sources and highlights where you can draw new conclusions.

As you continue synthesizing, here are a few points to remember:

• Don’t force a relationship between sources if there isn’t one. Not all of your sources have to complement one another.
• Do your best to highlight the relationships between sources in very clear ways.
• Don’t ignore any outliers in your research. It’s important to take note of every perspective (even those that disagree with your broader conclusions).

Example Syntheses

Below are two examples of synthesis: one where synthesis is NOT utilized well, and one where it is.

Parents are always trying to find ways to encourage healthy eating in their children. Elena Pearl Ben-Joseph, a doctor and writer for KidsHealth , encourages parents to be role models for their children by not dieting or vocalizing concerns about their body image. The first popular diet began in 1863. William Banting named it the “Banting” diet after himself, and it consisted of eating fruits, vegetables, meat, and dry wine. Despite the fact that dieting has been around for over a hundred and fifty years, parents should not diet because it hinders children’s understanding of healthy eating.

In this sample paragraph, the paragraph begins with one idea then drastically shifts to another. Rather than comparing the sources, the author simply describes their content. This leads the paragraph to veer in an different direction at the end, and it prevents the paragraph from expressing any strong arguments or conclusions.

An example of a stronger synthesis can be found below.

Parents are always trying to find ways to encourage healthy eating in their children. Different scientists and educators have different strategies for promoting a well-rounded diet while still encouraging body positivity in children. David R. Just and Joseph Price suggest in their article “Using Incentives to Encourage Healthy Eating in Children” that children are more likely to eat fruits and vegetables if they are given a reward (855-856). Similarly, Elena Pearl Ben-Joseph, a doctor and writer for Kids Health , encourages parents to be role models for their children. She states that “parents who are always dieting or complaining about their bodies may foster these same negative feelings in their kids. Try to keep a positive approach about food” (Ben-Joseph). Martha J. Nepper and Weiwen Chai support Ben-Joseph’s suggestions in their article “Parents’ Barriers and Strategies to Promote Healthy Eating among School-age Children.” Nepper and Chai note, “Parents felt that patience, consistency, educating themselves on proper nutrition, and having more healthy foods available in the home were important strategies when developing healthy eating habits for their children.” By following some of these ideas, parents can help their children develop healthy eating habits while still maintaining body positivity.

In this example, the author puts different sources in conversation with one another. Rather than simply describing the content of the sources in order, the author uses transitions (like "similarly") and makes the relationship between the sources evident.