literature review on model

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 1, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• UWF Libraries

Literature Review: Conducting & Writing

• Sample Literature Reviews
• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• APA Style This link opens in a new window
• Chicago: Notes Bibliography This link opens in a new window
• MLA Style This link opens in a new window

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

• << Previous: MLA Style
• Next: Get Help! >>
• Last Updated: Mar 22, 2024 9:37 AM
• URL: https://libguides.uwf.edu/litreview

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

Prevent plagiarism, run a free check.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 1 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy pp 1–15 Cite as

Methodological Approaches to Literature Review

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  
• Living reference work entry
• First Online: 09 May 2023

436 Accesses

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

• Literature review
• Systematic review
• Meta-analysis
• Scoping review
• Research methodology

This is a preview of subscription content, log in via an institution .

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Demystifying the Literature Review: Overview

• Getting Started
• Searching the Literature
• Writing the Review
• Getting Help

Why A Literature Review?

• Summarize the literature in a given field
• Evaluate the quality of evidence and strength of existing claims
• Show how published works relate to one another
• Place your work in the context of other research in a field

The Literature Review Model

Webinar recording: Demystifying the Literature Review

Webinar Recording of “Demystifying the Literature Review: A Hands-on Workshop for Taking a Structured Approach”,

Friday, April 24, 2020 by Cornell University Librarians Christine Fournier, Sarah Kennedy, and Sarah J. Wright.

Intended as a hands-on workshop to start or accelerate the literature review for your thesis or dissertation, the 1.5 hour workshop covers the steps of conducting a literature review, a checklist for drafting your topic and search terms, citation management software for organizing your results, and database searching.

Literature Reviews: An Overview for Graduate Students

• Literature Reviews: An Overview for Graduate Students Check out this short video from NC State University that describes what a literature review is, what purpose it serves in your research, and what you should expect when writing one.

Useful Books

• Next: Getting Started >>
• Last Updated: Jan 27, 2023 1:19 PM
• URL: https://guides.library.cornell.edu/literature_review

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Find This link opens in a new window
• Evaluate This link opens in a new window

What Will You Do Differently?

Please help your librarians by filling out this two-minute survey of today's class session..

Professor, this one's for you .

Introduction

Literature reviews take time. here is some general information to know before you start.  .

•  VIDEO -- This video is a great overview of the entire process.  (2020; North Carolina State University Libraries) --The transcript is included --This is for everyone; ignore the mention of "graduate students" --9.5 minutes, and every second is important  
• OVERVIEW -- Read this page from Purdue's OWL. It's not long, and gives some tips to fill in what you just learned from the video.  
• NOT A RESEARCH ARTICLE -- A literature review follows a different style, format, and structure from a research article.  

Steps to Completing a Literature Review

• Next: Find >>
• Last Updated: Sep 26, 2023 10:25 AM
• URL: https://guides.library.jhu.edu/lit-review

• Reference Manager
• Simple TEXT file

People also looked at

Review article, a review of feedback models and theories: descriptions, definitions, and conclusions.

• 1 Queens College and the Graduate Center, City University of New York, New York, NY, United States
• 2 Ikerbasque, Basque Foundation for Science and Universidad de Deusto, Bilbao, Spain

The positive effect of feedback on students’ performance and learning is no longer disputed. For this reason, scholars have been working on developing models and theories that explain how feedback works and which variables may contribute to student engagement with it. Our aim with this review was to describe the most prominent models and theories, identified using a systematic, three-step approach. We selected 14 publications and described definitions, models, their background, and specific underlying mechanisms of feedback processes. We concluded the review with eight main points reached from our analysis of the models. The goal of this paper is to inform the field and to help both scholars and educators to select appropriate models to frame their research and intervention development. In our complementary review ( Panadero and Lipnevich, 2021 ) we further analyzed and compared the fourteen models with the goal to classify and integrate shared elements into a new comprehensive model.

A Review of Feedback Models and Theories: Descriptions, Definitions, Empirical Evidence, and Conclusions

For many decades, researchers and practitioners alike have been examining how information presented to students about their performance on a task may affect their learning ( Black and Wiliam, 1998 ; Lipnevich and Smith, 2018 ). The term feedback was appropriated into instructional contexts from the industry ( Wiliam, 2018 ), and the original definitions referred to feedback as information from an output that was looped back into the system. Over the years, definitions and theories of feedback evolved, and scholars in the field continue to accumulate evidence attesting to feedback’s key role in student learning.

To provide the reader with a brief historical overview, we will start from the early 20th century and consider Thorndike’s Law of Effect to be at the very inception of feedback research ( Thorndike, 1927 ; Kluger and DeNisi, 1996 ). Skinner and behaviorism with positive and negative reinforcements and punishments can also be considered as precursors to what the field currently views as instructional feedback ( Wiliam, 2018 ). Further, the value of formative assessment, as it is now known, was first explicated by Benjamin Bloom in his seminal 1968 article, in which he described the benefits of offering students regular feedback on their learning through classroom formative assessments. Bloom described specific strategies teachers could use to implement formative assessments as part of regular classroom instruction, both to improve student learning, to reduce gaps in the achievement of different subgroups of students, and to help teachers to adjust their instruction ( Bloom, 1971 ). Hence, extending ideas of Scriven (1967) who proposed the dichotomy of formative and summative evaluation, Bloom deserves the credit for introducing the concept of formative assessment ( Guskey, 2018 ).

Importantly, the arrival of cognitive and constructivist theories started to change the general approach to feedback ( Panadero et al., 2018 ), with researchers moving from a monolithic idea of feedback as “it is done to the students to change their behavior” to “it should give information to the students to process and construct knowledge.” So, in the late seventies and most of the eighties, there was a push to investigate the type of feedback that would be most beneficial to students’ learning. Even though the first publications were heavily influenced by behaviorism (e.g. Kulhavy, 1977 ), by the end of the 80s there was a pedagogical push to turn feedback into opportunities for learning (e.g., Sadler, 1989 ).

It was in the nineties when the “new” learning theories gained major traction in psychological and educational literature on feedback. Around that time, cognitive models of feedback were developed such as the ones by Butler and Winne (1995) and Kluger and DeNisi (1996) . These theories focused on cognitive processes that were central to the processing of feedback, and the mechanisms, through which feedback affected cognitive processes and students’ subsequent behavior, were also explored. The key point in the development of the field, however, was Black and Wiliam (1998) publication of their thematic review that expedited and reshaped the field of formative assessment. The main message of their review still stands: across instructional settings assessment should be used to provide information to both the learner and the teacher (or other instructional agent) about how to improve learning and teaching, with feedback being the main vehicle to achieve it. This idea may seem simple but it is neither fully implemented nor sufficiently integrated within the summative functions of assessment ( Panadero et al., 2018 ). After two decades following Black and Wiliam’s (1998) publication, a lot of progress has been made, with classroom assessment literature being fused with learning theories such as self-regulated learning ( Panadero et al., 2018 ), cognitive load ( Sweller et al., 1998 ), and control-value theory of achievement emotions ( Goetz et al., 2018 ; Pekrun, 2007). In the current review we do not intend to delve into these voluminous strands of research, but we encourage the reader to conduct future exploration to expand on these.

As the field of formative assessment and feedback was evolving, scholars were devising feedback models to describe processes and mechanisms of feedback. In general, these models have gotten both more comprehensive and focused, depicting more specific cognitive processes ( Narciss and Huth, 2004 ), student responses to feedback ( Lipnevich et al., 2016 ), the context ( Evans, 2013 ), and pedagogical aspects of feedback ( Carless and Boud, 2018 ; Hattie and Timperley, 2007 ; Nicol and Macfarlane-Dick, 2006 ). In contrast to the earlier conception where “feedback was done” to the student, in the most current models the learner is not only at the center of the feedback process, but is now an active agent that does not only process feedback, but responds to it, can generate it, and acquires feedback expertise to engage with it in more advanced ways ( Shute, 2008 ; Stobart, 2018). Additionally, a lot is known about how to involve students in the creation of feedback either as self-feedback ( Andrade, 2018 ; Boud, 2000 ) or peer feedback ( Panadero et al., 2018 ; van Zundert et al., 2010 ), and what key elements influence students’ use of feedback ( Winstone et al., 2017 ; Jonsson and Panadero, 2018 ). Due to the ongoing proliferation of models, theories, and strands of research, now may be a critical moment to examine the most influential models and theories currently utilized by researchers and educators. It will help us to consider how the models have evolved and what the main developments in our conceptions of feedback are after decades of research. In the current review we did just that. Through a rigorous multi-step process we selected, described, and compared 14 prominent models and theories currently discussed and utilized by researchers in the field. Our aim was to provide a guide for researchers for selecting the most suitable model for framing their studies, as well as to provide the newcomers to the field with a starting point to the key theoretical approaches and descriptions of feedback mechanisms. We worked on two reviews simultaneously. In the current one we focus on the description of the fourteen included models, drawing conclusions about definitions and supporting evidence. In the second review ( Panadero and Lipnevich, 2021 ) we compare typologies of feedback and discuss elements of the included models, proposing an integrative model of feedback elements: the MISCA model (Message, Implementation, Students, Context, and Agents). To get the complete picture we strongly recommend that the reader engages with both reviews, starting with the present one.

Selection of Relevant Publications

To select models for review we searched PsycINFO, ERIC, and Google Scholar databases. Unfortunately, this approach was not fruitful as the combination of “feedback + model” and “feedback + model + education” in either title, abstract, or document offered a mix of results from different disciplines in the first combination and focus on variables other than feedback in the second (e.g. self-assessment, peer assessment). Therefore, we established a three-way search method. First, we used our own reference libraries to locate the feedback models we had previously identified. This exercise resulted in ten models. Second, we consulted colleagues to get their opinion on models that they thought had an impact in feedback literature. We sent emails to 38 colleagues asking the following questions: what were the feedback models they knew of, have they developed any model themselves, and would they be available for further consultation. Our instructions intentionally did not include either a strict definition or operationalization of what constituted a feedback model; this was done purposefully to allow for individual interpretation. To select the 38 colleagues, we first contacted all the first authors of the models we had already identified, the authors of the two existing reviews, and internationally reputed feedback researchers. We obtained 29 responses, but two of them did not provide enough information and were rejected. Through this procedure we identified 65 models, including the ten models that were already selected from our reference libraries. Finally, we used two previous feedback model reviews ( Mory, 2004 ; Shute, 2008 ) to identify models that might not have been included in the two previous steps, selecting two more. Importantly, one of the models included in Mory (2004) that had not been identified earlier, had a very limited impact in the field. Clariana (1999) had only been cited 39 times since 1999, thus this model was no longer considered for inclusion. The resulting number of feedback models was 67.

Acknowledging that so many models could not have been meaningfully represented in our review, we tasked ourselves with further reduction of the database. We used a combination of two inclusion criteria. First, we selected all but one model that had been included in the two previous feedback reviews ( Mory, 2004 ; Shute, 2008 ). Application of this criterion produced a total of six models. Second, in addition to the previous criterion, we established a minimum threshold of three votes by the consulted colleagues to add new models, which resulted in ten models. After deleting duplicates, a total of 14 models were included in this review ( see Figure 1 ). The final list is: Ramaprasad (1983) ; Kulhavy and Stock (1989) ; Sadler (1989) ; Bangert-Drowns et al. (1991) ; Butler and Winne (1995) ; Kluger and DeNisi (1996) ; Tunstall and Gipps (1996) ; Mason and Bruning (2001) ; Narciss (2044, 2008) ; Nicol and Macfarlane-Dick (2006) ; Hattie and Timperley (2007) ; Evans (2013) ; Lipnevich et al. (2016) ; and Carless and Boud (2018) .

FIGURE 1 . Search and selection criteria process.

The fourteen selected publications are different in their nature. Some of them are meta-analytic reviews, others are narrative reviews, in addition to several empirical papers. Importantly, two of the papers present theoretical work without any empirical evidence and without describing any links among variables related to feedback ( Ramaprasad, 1983 ; Sadler, 1989 ). Hence, we would like to alert the reader that some of the articles discussed in this review do not present anything that would fit with the definition of a model. Traditionally, a model explains existing relations among components and is typically pictorially represented ( Gall, Borg, and Gall, 1996 ). Some of the manuscripts that we included neither described relations nor offered graphical representations of feedback mechanisms. Our decision in favor of inclusion was based exclusively on the criteria that we described above. For the purposes of simplicity, we will refer to the fourteen scholarly contributions as models, but will explain the type of each contribution in the upcoming sections.

Extracting, Coding, Analyzing Data and Consulting Models’ Authors

The data were extracted from the included publications for the following categories: 1 ) Reference, 2 ) Definition of feedback, 3 ) Theoretical framework, 4 ) Citations of previous models, 5 ) Areas of feedback covered, 6 ) Model description, 7 ) Empirical evidence supporting the model, 8 ) Pictorial representation, 9 ) References to formative assessment, 10 ) Notes first author, 11 ) Notes second author, and 12 ) Summaries of the model from other publications. We coded the sources in a descriptive manner for most categories except for the definition of feedback category, where we copied them directly from the original source. Both authors coded the articles and there was a total agreement in the assignment of separate components to the aforementioned categories.

Finally, we contacted authors of all the included publications except for one (the two authors were in retirement and not accessible). Twelve authors replied (two from the same publication) but only seven agreed and scheduled an interview (Boud, Carless, Kluger, Narciss, Ramaprasad, Sadler, and Winne). The interviews were recorded and helped us to better understand the models. Also, to ensure adequate interpretation of the models we shared our descriptions with the authors, which they returned with their approval and, in some cases, suggestions for revisions.

We would like to alert the reader that the Method section is shared with the second part of this review ( Panadero and Lipnevich, 2021 ).

Models and Theories: Descriptions

In this section we present descriptions of the models and theories included in the current systematic review. We do not claim to provide a comprehensive overview of each model or theory. Rather, we give the reader a flavor of what they represent. Due to the fact that most summaries were checked by the corresponding authors, we believe that our account is fairly accurate and unbiased, and in this section we try to withhold our own interpretation or opinions about the utility of each model. We will follow this section with a detailed analysis of definitions, summary of existing empirical evidence, as well as conclusions and recommendations. We would also like to direct the reader to the second review, wherein we integrate the fourteen models into a comprehensive taxonomy ( Panadero and Lipnevich, 2021 ).

Ramaprasad (1983) : Clarifying the Purpose of Feedback From Outside the Field

Ramaprasad’s work has been crucial for the current conceptualization of feedback in educational settings with his paper on the definition of feedback being cited over 1500 times, more frequently from educational psychology and education.

Theoretical Framework

Ramaprasad’s seminal article was published in Behavioral Science (now called “Systems Research and Behavioral Science”), which is the official journal of the Society for General Systems Research (SGSR/ISSS). The paper does not contain a single educational source, drawing upon management, behavioral, I/O, and social psychology.

Description

Ramaprasad’s article does not describe a model in its traditional sense. Rather, the author presents a theoretical overview of feedback, focusing on mechanisms, valence, and consequences of feedback for subsequent performance. His work is influential in education because of his definition of feedback, but it contains other concepts that have been somewhat overlooked by educators. We will concentrate on three main aspects of the paper.

1) The definition. Ramaprasad’s definition of feedback is very well-known and is referenced extensively. According to the author: “Feedback is information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way” (p. 4). There are three key points that need to be emphasized: 1 ) the focus of feedback may be any system parameter, 2 ) the necessary conditions for feedback are the existence of data on the reference level of the parameter, data on the actual level of the parameter, and a mechanism for comparing the two to generate information about the gap between the two levels, and 3 ) the information on the gap between the actual level and the reference level is considered to be feedback only when it is used to alter the gap.

2) Three key ideas. Due to the fact that Ramaprasad comes from a different academic field, it is important to clarify the main ideas that are present in his definition. The first one concerns the reference level and is a representation of a position that can be used for assessing a product or performance. According to Ramaprasad, it can vary along two continua bounded by explicit and implicit, and quantitative and qualitative. “When reference levels are implicit and/or qualitative, comparison and consequent feedback is rendered difficult. Despite the above fact, implicit and qualitative reference levels are extremely important in management and cannot be ignored.” (p. 6). In educational settings this would refer to the importance of explicit criteria and standards.

The second key idea is the comparison of reference and actual levels. Here, Ramaprasad states that “irrespective of the unit performing the comparison, a basic and obvious requirement is that the unit should have data on the reference level and the actual level of the system parameter it is comparing. In the absence of either comparison it is impossible” (p. 7). This aspect emphasizes the critical role of the two key questions that Hattie and Timperley (2007) discussed in their work: “where am I going?” and “where am I supposed to be?”

Further, using information about the gap to alter the gap is an idea that is critical in instructional contexts. Ramaprasad maintains that only if the information about the gap is used (and the decision can be to not do anything about it) it can be considered feedback. If it is just information that gets stored but nothing happens with it, then it is just information, and, hence, cannot be considered feedback. It is important to remember that Ramaprasad viewed feedback through the lens of the systems and management perspective. From the learning sciences perspective, if information gets stored in the long term memory, a change has already occurred and feedback has had an effect, even if no external changes had been observed. On the other hand, if the process and the outcome of storing the information is interpreted as ignoring the feedback, then it has not had any effect.

3) Positive and negative feedback. Ramaprasad’s description of positive and negative feedback differs from a more traditional interpretation in terms of its affective valence. From Ramaprasad’s perspective: “if the action triggered by feedback widens the gap between the reference and the actual levels of the system parameter...is called positive feedback; ...if the action reduces the gap between the two levels...is negative” (p. 9). Ramaprasad did acknowledge alternative interpretations of positive and negative feedback, with the valence being determined by the emotions triggered in the feedback receiver (e.g., positive for enjoyment and pride, negative for disappointment and anxiety), as well as within the parameters of positive and negative reinforcement from Skinner. In the context of education, we usually describe feedback as being positive or negative depending on the emotions it elicits.

Kulhavy and Stock (1989) : A Model From Information Processing

Kulhavy and Stock’s (1989) seminal work introduced the idea of multiple feedback cycles, considered types, form, and content of feedback, and explicitly equated feedback with general information. The authors consider feedback from the information processing perspective, juxtaposing it with earlier studies that used the behaviorist approach as their foundation.

According to the authors, many of the early studies conducted within the behaviorist perspective viewed feedback as corrective information that strengthened correct responses through reinforcement, and weakened incorrect responses through punishment. This somewhat mechanistic perspective stressed the importance of minimizing errors, but no description of error correction, nor the means for it were presented. Feedback following an instructional response was viewed as fitting the sequence of events of the Thorndike’s Law of Effect ( Thorndike, 1927 , 1997 ), and was construed as the driving force of human learning. The fact that a learner 1 ) received a task, 2 ) produced a response, and 3 ) received feedback indicating whether the answer was correct or not (punishment or reinforcement) provided a superficial parallel to the familiar sequence of the 1 ) stimulus, 2 ) response, and 3 ) reinforcement. However, as Kulhavy and Stock (1989) noted, people involved in instructional tasks were not under the powerful stimulus control found in the laboratory, which, along with constantly changing stimuli and responses, carried very little resemblance to the typical operant setting. Hence, presentation of corrective feedback following an incorrect response may carry no effect on the learner. Thus, errors that students made were ignored, and instructors’ attention was directed to students’ correct responding only.

From the information-processing perspective, on the contrary, errors were of central importance, as this approach described the exact mechanisms through which external feedback helped to correct mistakes in the products of a learning activity. Both direct and mediated feedback could be distinguished according to its content on two vectors of verification and elaboration. Verification represents students’ evaluation of whether a particular feedback message matches their response, whereas elaboration can be classified according to load, form, and type of information . Load is represented by the amount of information provided in the feedback message that can range from a letter grade to a detailed narrative account of students’ performance. Type of information is reflected in the dichotomy of process-related, or descriptive feedback, and outcome-related, or evaluative feedback. Form is defined as changes in stimulus structure between instruction and the feedback message that a learner receives.

Although Kulhavy and Stock (1989) did not provide a clear definition of feedback, they adhered to the one offered by Kulhavy (1977) . Kulhavy (1977) defined feedback as “any of the numerous procedures that are used to tell a learner if an instructional response is right or wrong” ( Kulhavy, 1977 , p. 211). In its simple form, this would imply simply indicating whether a learner’s response to an instructional prompt was correct or not whereas more complex forms of feedback included messages that provided the learner with additional information on what needed improvement (“correctional review”). Interestingly, the authors also noted that with increasing complexity, feedback would inevitably become indistinguishable from instruction.

The central idea of Kulhavy and Stock’s model is that of response certitude ( Figure 2 ). The authors defined it as a degree to which the learner expected his or her response to be a correct one. This central tenet of the model is only indirectly related to the classification of the response as right or wrong, so the model is not limited to a basic error analysis. In other words, it seeks to make predictions about student performance. In addition to response certitude, the authors discuss response durability, which is the likelihood that an instructional response will be available for the learner’s use at some later point in time. Thus, the key underlying premise of Kulhavy and Stock (1989) model is that certitude estimates and response durability are positively related, so in situations where feedback is unavailable, the magnitude of certitude increases, and the probability of selecting the same response (often incorrect) increases also.

FIGURE 2 . Kulhavy and Stock (1989) feedback components.

The model itself comprises three cycles, each of which includes an iteration loop ( Figure 3 ). The first cycle depicts instructional task demands, the second represents feedback message, and the third is a criterion task demand. In the first cycle the perceived task demand is compared to the set of existing cognitive referents available to the learner. In the second cycle, the feedback message is compared to the cognitive referents retained from the initial cycle. These two cycles are followed by cycle three, where the perceived stimulus is again the original task demand that is compared to the cognitive referents which have been modified by the feedback message.

FIGURE 3 . Adapted from Kulhavy and Stock (1989) three-cycle feedback model.

Kulhavy and Stock (1989) also suggested that an introduction of a small delay between responses and feedback helped to eliminate proactive interference and thus increased the impact of error-correcting feedback.

Sadler (1989) : Seminal Work for Formative Assessment

Sadler’s work has been foundational for the conceptualization of feedback, assessment criteria, and assessment philosophy. His seminal paper outlines a theory of formative assessment. Sadler does not describe an actual model of feedback but focuses on feedback’s formative features. However, a significant portion of his paper is dedicated to feedback.

Before developing and extending his own definition of feedback, Sadler referred to earlier writings on formative assessment and feedback, specifically those by Kulhavy (1977) , Kulik and Kulik (1988) , and the much earlier work, Thorndike’s Law of Effect ( Thorndike, 1913 ; Thorndike, 1927 ). These authors equated feedback with learners’ knowledge of results, whereas Sadler adopted a much broader view. His theoretical exploration built upon a definition of feedback offered by Ramaprasad (1983) , which referred to feedback as it is found in many different contexts – not specifically in education. Sadler brought Ramaprasad’s definition into education and extended it into such areas as writing assessment or qualitative judgment, which are characterized by multidimensional criteria that cannot be evaluated as correct or incorrect (Sadler and Ramaprasad, October 2019, personal communication).

Sadler referred to feedback as: “a key element in formative assessment… usually defined in terms of information about how successfully something has been or is being done.” (p. 120). Sadler applied Ramaprasad’s conceptualization of feedback to situations, in which the teacher provides feedback, and learners are the main actors who have to understand the feedback in order to improve their work. If information from the teacher is too complex, if students do not possess knowledge or opportunity to use the information, then this feedback is no more than “dangling data” and, hence, highly ineffective. For the feedback to be effective students should be familiar with assessment criteria, should be able to monitor the quality of their work, and should have a wide arsenal of strategies from which they can draw to improve their work. Sadler emphasized the importance of continuous self-assessment, which he described as judgments of quality of one’s own work at any given time.

Sadler also discussed three conditions that had to be satisfied for the feedback to be effective. According to Sadler, the first condition had to do with a standard, toward which students aimed as they worked on a task at hand. The second condition required students to compare their actual levels of performance with the standard, and the third emphasized student engagement in actions that eventually closed the gap. Sadler stressed that all three conditions had to be satisfied for any feedback to be effective.

Sadler made an interesting distinction that we did not encounter in any other writings of scholars included in the current review. He juxtaposed feedback and self-monitoring, with the former defined as information that came from an external source, and the latter being self-generated by a learner. He further suggested that one of the key instructional goals was to move learners away from feedback and have them fully rely on self-monitoring.

Sadler also proposed that it was difficult to evaluate students’ work on a dichotomous scale of correct or incorrect. Effective evaluations resulted from “direct qualitative human judgments.” Consequently, Sadler broadened the definition of feedback as information about the quality of performance to include: “…knowledge of the standard or goal, skills in making multicriterion comparisons, and the development of ways and means for reducing the discrepancy between what is produced and what is aimed for.” (p. 142).

Sadler discussed a variety of tools and approaches that should help students with effective self-monitoring. He addressed such topics as peer assessment, the use of exemplars, continuous assessment, Bloom’s taxonomy, grading on the curve, and curriculum structure. In his account, Sadler’s primary focus was on the need to help students to develop effective evaluative skills, so they could transition from their complete reliance on teacher-delivered evaluations to students’ own self-monitoring.

Bangert-Drowns et al. (1991) : The First Attempt to Meta-Analyzing (CAP) the Effects of Feedback

Bangert-Drowns et al. (1991) are well-known for a series of a meta-analyses and empirical reviews that they published throughout the 80s and mid 90s on a range of topics (e.g. coaching aptitude tests, computer-based education, frequent classroom testing). This particular meta-analysis to our knowledge, was the first to aggregate results of empirical studies on the effects of feedback on meaningful educational outcomes.

This meta-analysis is grounded in early feedback research, especially research conducted by Kulhavy and Stock (1989) . The paper also draws upon behaviorist ( Thorndike, 1913 ) and cognitive psychology ( Shuell, 1986 ) ideas on how feedback may affect learning.

In their paper the authors did not present a clear definition of feedback. They discussed previous research on feedback, going back to the first decade of the 19th century, without ever operationalizing feedback. Nevertheless, they presented a typology of feedback that included three main categories that characterized feedback:

1. Intentionality: feedback can be intentional, that is, delivered via interpersonal action or through intervening agents such as computer, or informal, which is more incidental in nature.

2. Target: feedback can influence affective dimensions, for example, motivation, it can scaffold self-regulated learning, and it can signal whether the student has correctly applied concepts, procedures, and retrieved the correct information.

3. Content: characterized by load , which is the total amount of information given in the feedback message, ranging from simple yes/no statements to extended explanations; form , defined as the structural similarity between information as presented in feedback compared to the instructional presentation; and type of information indicating whether feedback restated information from the original task, referred to information given elsewhere in the instruction, or provided new information.

Additionally, the authors presented a five-stage model describing the feedback process. The five stages were: 1 ) Learner’s initial state defined by four elements of interest, goal orientation, self-efficacy, and prior knowledge; 2 ) a question (or task) that activated the search and retrieval strategies, 3 ) the learner’s response to the question, 4 ) followed by the learner’s evaluation of the response and its comparison to the information offered in the feedback, and 5 ) learners’ subsequent adjustments from this evaluation to their knowledge, self-efficacy, and interest.

Finally, in their meta-analytic review the authors used additional moderators that included the “type of feedback” and the “timing of feedback.” Regarding the former, the researchers found that just indicating the correctness of responses was less powerful than providing an explanation. In regards to the latter, the authors found superior effects of delayed feedback. All of the studies included in this meta-analysis had been published before 1990.

Butler and Winne (1995) : Answers From Self-Regulated Learning Theory (SRL)

Butler and Winne (1995) model served a twofold purpose: it explained differential effects of feedback at the cognitive processing level and, at the same time, it represented one of the widely cited SRL models (for a comparison of SRL models see Panadero, 2017 ).

In their model, Butler and Winne (1995) attempted to explain how internal and external feedback influenced students’ learning. Their main theoretical lens was information processing, with their focus expanding to include motivational factors in later years ( Winne and Hadwin, 2008 ). Most of the references the authors used came from the domain of cognitive psychology (e.g. Balzer et al., 1989 ; Borkowski, 1992 ), SRL theory (e.g. Zimmerman, 1989 ), and attribution theory (e.g. Schunk, 1982 ).

The authors did not include an explicit definition of feedback, but in the section “Four views on feedback…” the authors provided a broad description of the processes related to feedback. The model depicted mental processes that students activated when self-regulating during their execution of a task. Figure 4 is the original version, whereas Figure 5 represents the modified version of the model. Our subsequent discussion is based on the more current model depicted in Figure 5 .

FIGURE 4 . First version of Winne’s SRL model (extracted from Winne, 1996 ).

FIGURE 5 . Modified version of Winne’s SRL model (extracted from Panadero et al., 2019 ).

According to this model, there is a number of antecedent variables that affect students’ later performance. These are task conditions that are processed through the learner’s cognitive conditions such as “domain knowledge” or “knowledge of study tactics and strategies” along with motivational conditions. When students begin their performance, there are four different phases that take place: 1 ) defining the task, 2 ) establishing goals and plans, 3 ) applying study tactics and strategies by searching, monitoring, assembling, rehearsing, and translating (SMART) and 4 ) adapting. Throughout the whole process students monitor and control their progress. When they receive an external evaluation and feedback that comes with it, their initial conditions get updated.

Butler and Winne (1995) , drawing upon Carver and Scheier’s (1990) ideas, viewed feedback as an internal source, with students undergoing loops of feedback through monitoring and control. The loop related their interpretation about the product of monitoring (e.g. successful/unsuccessful, slow/fast, satisfying/disappointing) to the learner’s decision to maintain or adapt their thinking or actions in light of the product of monitoring. Feedback was then the information learners perceived about aspects of thought (e.g., accuracy of beliefs or calibration) and performance (e.g., comparison to a standard or a norm). Feedback could come from external sources when additional information was provided by an external agent. At any given point of the performance, students generated internal feedback comparing the profile of their current state to their ideal profile of the goal. According to Butler and Winne (1995) it happened via self-assessment by comparing different features of a task and through learners’ active engagement with the task. With both internal and external feedback learners could undergo small-scale adaptations represented by basic modifications in their current performance, or large-scale adaptations that would subsequently affect their future performance on the task. This model, with some modifications, has been used in at least two other publications to anchor self-regulated learning and different assessment practices ( Nicol and MacFarlane-Dick, 2006 ; Panadero et al., 2018 ).

Kluger and DeNisi (1996) : An Ambitious Meta-Analysis Exploring Moderators of Feedback Interventions

Kluger and DeNisi (1996) paper is regarded as a seminal piece in feedback research literature. It is frequently referenced to support a somewhat counter-intuitive finding – the fact that in 1/3 of cases feedback may have negative effect on performance. The authors did a thorough job reviewing 3000 publications on feedback to reduce it to the final set of 131. The number of considered moderators is also quite impressive and supersedes those examined in other meta-analyses. The paper quantitatively synthesized research into feedback interventions and proposed a new Feedback Intervention Theory with the goal to integrate multiple theoretical perspectives.

This review is one of the most thorough syntheses of the psychological feedback literature. Kluger and DeNisi (1996) carefully summarized work into knowledge of results and knowledge of performance, and stressed the key relevance of Thorndike’s Law of Effect, cybernetics ( Annett, 1969 ), goal setting theory ( Locke and Latham, 1990 ), social cognitive theory ( Bandura, 1991 ), learned helplessness ( Mikulincer, 1994 ), and multiple-cue probability learning paradigm ( Balzer, Doherty, and O’Connor, 1989 ). Interestingly, this review did not reference Kulhavy and Stock (1989) , Bangert-Drowns et al. (1991) , or Sadler (1989) Ramaprasad — all of whom came from the field of educational assessment. It shows that until recently, feedback research in psychology and education was conducted largely in parallel, despite a range of commonly shared ideas. For example, Kulhavy and Stock (1989) idea of “response certitude” has a clear overlap with “discrepancy,” which is a foundational idea of the Feedback Intervention Theory model.

Kluger and DeNisi (1996) did not offer a clear definition of feedback but did define feedback intervention as: “…actions taken by (an) external agent (s) to provide information regarding some aspect(s) of one’s task performance.” (p. 225). The model has several pictorial representations ( see Figure 6 ). Additional diagrams represented the effects of feedback intervention for more specific processes, such as attention and task-motivation processes.

FIGURE 6 . A schematic overview of feedback intervention theory by Kluger and DeNisi (1996) .

Their model focused on feedback that provided information about the discrepancy between the individual’s current level of performance and goals or standards. Kluger and DeNisi further proposed that individuals may have varying goals activated at the same time. For example, they could be comparing their performance to an external standard, to their own prior performance, performance of other reference groups, and their ideal goals. These discrepancies may be averaged or summed into an overall evaluation of feedback. The Feedback Intervention Theory also suggested that when the discrepancy between current and desired performance was established, the individual could: 1 ) choose to work harder, 2 ) lower the standard, 3 ) reject the feedback altogether, or 4 ) abandon their efforts to achieve the standard. Option selection depends upon how committed individuals are to the goal, whether the goal is clear, and how likely success will be if more effort is applied.

In the Feedback Intervention Theory, when an individual received feedback indicating that a goal had not been met, individuals’ attention could be focused on one of three levels: 1 ) the details of how to do the task, 2 ) the task as a whole, and 3 ) processes that the individual engages in doing the task (meta-task processes). Kluger and DeNisi (1996) argued that individuals typically processed feedback at the task level, but that the feedback could influence the level at which the feedback was received and attended to. Similarly to many educational researchers, Kluger and DeNisi claimed that if a task was clear to the individual, receiving feedback containing too many task-specific details could be detrimental to performance.

The impact of Kluger and DeNisi’s work could be seen in much of the theoretical work that followed it (e.g., Hattie and Timperley, (2007) model). The Feedback Intervention Theory model generally focused on feedback that communicated to individuals whether they were doing a particular task at an expected or desired level, thus assuming that individuals knew how to do the task. This is not often the case in educational settings, wherein the development of new skills is often the main goal. The authors acknowledged limitations of the model in that its breadth made the theory hardly falsifiable.

The main and frequently cited finding of their meta-analysis was that feedback interventions increased individuals’ performance by 0.4 standard deviations. At the same time, there was a great deal of variability of results, with 1/3 of studies showing a negative influence on performance. Based on the results of their meta-analysis and close examination of moderators, Kluger and DeNisi demonstrated the utility of the Feedback Intervention Theory.

Tunstall and Gipps (1996) : A Typology for Elementary School Students

This publication was included based on three votes, cast by the consulted experts. Unlike other models, its contribution may be more limited in scope. First, it is not a model that describes links and interactions but a typology. The authors’ primary goal was to categorize different types of feedback that they observed in classrooms. Second, its theoretical framework and links to the literature are rather limited. And third, the typology was developed based on a sample of 6 and 7 year old students, hence, it is only applicable to early primary grades. We decided to keep this publication in the current review to be consistent with our specified inclusion criteria. It may be beneficial for those researchers who are interested in the downward extension of feedback studies, as primary school samples are generally scarce in the field of instructional feedback and assessment research ( Lipnevich and Smith, 2018 ).

The publication has a short theoretical introduction, with the authors referencing studies in early childhood education (e.g. Bennett and Kell, 1989 ), feedback (e.g. Brophy, 1981 ), and educational psychology (e.g. Dweck, 1986 ). They authors do not frame their study within any specific theoretical approach.

Despite the fact that this paper described a typology of feedback, the authors did not present a definition of feedback. Tunstall and Gipps (1996) conducted their study in six schools and selected 49 students for detailed examination. Based on document analyses, recordings of dialogues and observations, they derived a typology that included five different types of feedback and their valence ( Figure 7 ). These types were organized around two more general aims: Feedback and socialization (i.e. Type S) and Feedback in relationship to assessment. The former included four types that differed according their purpose: classroom/individual management, performance orientation, mastery orientation, and learning orientation. Feedback in relationship to assessment was differentiated into feedback that is rewarding, approving, specifying attainment, and constructing achievement. The authors also included dimensions of positive and negative feedback, as well as achievement and improvement feedback. The typology appears to be very descriptive with multiple overlapping categories.

FIGURE 7 . Feedback typology by Tunstall and Gipps (1996) .

Mason and Bruning (2001) : Considering Individual Differences in a Model for Computer Based Instruction

Mason and Bruning’s (2001) contribution is unique in that it is the first one introducing the role of individual differences in the context of the Computer Based Instruction. The authors present a framework for decision making about feedback options in computerized instruction.

The authors review literature on feedback in traditional instructional contexts, summarize research into computer based education (e.g. Cohen, 1985 ), and discuss publications combining studies of both regular and computer-based instructional settings (e.g. Mory, 1994 ).

The authors provided the following definition of feedback: “In general terms, feedback is any message generated in response to a learner’s action.” (p. 3). They started off by describing eight types of feedback that came from the literature, differentiated based on two main vectors of verification and elaboration: 1 ) No Feedback, which presents only a performance score; 2 ) Knowledge of response, which communicates whether the answer was correct or incorrect; 3 ) Answer until correct, which provides verification but no elaboration; 4 ) Knowledge of correct response, which provides verification and knowledge of correct answer; 5 ) Topic contingent, which delivers verification and elaboration regarding the topic; 6 ) Response contingent, which includes both verification and item specific elaboration; 7 ) Bug related, which presents verification and addresses errors and 8 ) Attribute isolation, which focuses learner on key components. In this categorization, the authors considered the instructional context, and some of these types are more common in computer based instruction than others (e.g., “answer until correct”).

This paper’s main contribution is the model that differentiates among types of feedback based on learners’ characteristics, prior knowledge, and the timing of feedback. The pictorial representation includes a flowchart starting at the student achievement level and going down to the complexity of the task, and the type of feedback ( Figure 8 ). The model offers clear guidelines on how to deliver better feedback based on previous empirical research, while considering a range of key variables, such as student level of achievement and prior knowledge, as well as the timing of feedback. Interestingly, “attitude towards feedback” and “learner control,” two additional individual student characteristics that the authors explored in their introduction, were not incorporated into the model.

FIGURE 8 . Mason and Bruning (2001) model.

Narciss and Huth (2004 , 2008 ): An Ambitious Model Created for Computer Supported Learning

This model is probably the most ambitious out of those described in this review. It explores both the reception and processing of feedback. There are connections between this model and Butler and Winne’s, however, this model has a range of unique contributions. This model was presented first in two publications: one from 2004 and an updated and much more specific version from 2008. In later publications Narciss introduced minor changes in the figures to make them easier to understand.

The model is based upon the cybernetic paradigm from systems theory, at the same time having aspects of “notions of competencies and models of self-regulated learning” ( Narciss, 2017 ). The Interactive Tutoring Feedback model, also known as interactive-two-feedback-loops model, is heavily steeped in vast research base of general feedback literature. The model represents interacting processes and factors of the two feedback loops that may account for a large variety of feedback. The model also focuses on computer supported learning with a strong emphasis on tutoring systems that adapt feedback to students’ needs. Despite its strong focus on tutoring systems, the contentions of this model can be applied to face-to-face learning situations (2017).

Narciss describes feedback as follows: “In instructional contexts the term feedback refers to all post-response information which informs the learner on his/her actual state of learning or performance in order to regulate the further process of learning in the direction of the learning standards strived for (e.g., Narciss, 2008 ; Shute, 2008 ). This notion of feedback can be traced back to early cybernetic views of feedback (e.g., Wiener, 1954 ) and emphasizes that a core aim of feedback in instructional contexts is to reduce gaps between current and desired states of learning ( see also Ramaprasad, 1983 ; Sadler, 1989 ; Hattie, 2009 ).” ( Narciss, 2017 p. 174). As it can be seen, it is an ambitious definition that includes aspects from multiple theories.

The model presents factors and processes of both the external and internal loop and how their potential interactions may influence the effects of feedback. When a student receives feedback it is not just the characteristics of the feedback message that will explain student responses. Rather, it is an interactive process in which the students and instructional characteristics create a particular type of feedback processing. Narciss presented three main components that had to be considered when designing feedback strategies ( Figure 9 ): 1 ) characteristics of the feedback strategy (e.g., function, content, and presentation); 2 ) learner’s individual factors (e.g. goals, motivation); and 3 ) instructional factors (e.g. goals, type of task). Hence, the model integrates multiple factors that influence if and how feedback from an external source is processed effectively. Additionally, in 2013 and 2017, Narciss elaborated upon the individual and instructional factors and added specific conditions of the feedback source useful for designing efficient feedback strategies.

FIGURE 9 . Narciss (2008) model of factors and effects of external feedback.

The model is represented in more detail in Figure 10 . As one can see, learners’ engagement with feedback is influenced by both their characteristics and teacher, peer, and instructional medium. Narciss juxtaposes internal and external standards, competencies, and task requirements, as well as internal and external reference values. So, for example, an external controller compares external standards to feedback and communicates this information to the learner’ internal controller, which, in turn, generates internal feedback via self-assessment. This leads to different actions such as control actuator and controlled variables. If we go back to Butler and Winne’s model, the interactive processes described in Narciss’ model explain in a similar way the adaptation (small and large scale) proposed by Butler and Winne.

FIGURE 10 . Narciss (2013) feedback model explaining interactions.

Additionally, Narciss (2008) provided what is probably the most specific taxonomy of feedback from all the models herein included, based on a multidimensional approach to describing the many ways feedback can be designed and provided. According to her typology, feedback can have three functions: 1 ) cognitive (informative, completion, corrective, differentiation, restructuring); 2 ) metacognitive (informative, specification, corrective, guiding); and 3 ) motivational (incentive, task facilitation, self-efficacy enhancing, and reattribution). Additionally, feedback can be classified by its content with an evaluative component or an informative component, with eight different categories ( Narciss, 2008 . Table 11.2tbl112 p. 135). And finally, the presentation of feedback can vary in timing, schedule, and adaptivity. This multidimensional classification, which also represents ways of designing feedback, is extremely detailed and stems from Narciss’ extensive work in the subfield of feedback.

Nicol and McFarlane-Dick (2006) : Connecting Formative Assessment With Self-Regulated Learning

This article has become one of the most important readings in the formative assessment literature. The article connects self-regulated learning theory, more specifically the model developed by Winne (2011) , and seven principles that are introduced as “good feedback practices.” Nicol and MacFarlane-Dick were among the first authors to provide specific connections between the two fields of self-regulated learning and formative assessment ( Panadero et al., 2018 ).

The theoretical framework of the paper draws upon the two fields of self-regulated learning and formative assessment, combining assessment literature ( Sadler, 1998 ; Boud, 2000 ) with studies coming from self-regulated learning scholars (e.g. Pintrich, 1995 ; Zimmerman and Schunk, 2001 ).

According to Nicol and MacFarlane-Dick, “Feedback is information about how the student’s present state (of learning and performance) relates to goals and standards. Students generate internal feedback as they monitor their engagement with learning activities and tasks and assess progress towards goals. Those more effective at self-regulation, however, produce better feedback or are abler to use the feedback they generate to achieve their desired goals ( Butler and Winne, 1995 )” (p. 200).

Additionally, they referred to the seminal work of Sadler (1989) and Black and Wiliam (1998) and emphasized the importance of the three conditions that must be explicated for students to benefit from feedback: 1 ) the desired performance; 2 ) the current performance; 3 ) how to close the gap between the two.

Their model is largely based on Winne’s model of self-regulated learning, and it describes how feedback interacts within each of the components of the model. For example, the authors suggested that comparisons of goals to outcomes generated internal feedback at cognitive, motivational, and behavioral levels, and this information prompted the student to change the process or continue as it was. They emphasized that self-generated feedback about the potential discrepancy between the goal and the performance may result in revisions of the task, changes in internal goals or strategies. The model also presented variable sources of feedback, which could be provided by the teacher, peer, or by other means (e.g. a computer). Just like Sadler (1989) , Nicol and MacFarlane-Dick emphasized the importance of active engagement with feedback.

Their model can be categorized as instructional and pedagogical as it presented seven feedback principles that influenced self-regulated learning. According to the authors, good feedback that may influence self-regulated learning:

1. helps clarify what good performance is (goals, criteria, expected standards);

2. facilitates the development of self-assessment (reflection) in learning;

3. delivers high quality information to students about their learning;

4. encourages teacher and peer dialogue around learning;

5. encourages positive motivational beliefs and self-esteem;

6. provides opportunities to close the gap between current and desired performance;

7. provides information to teachers that can be used to help shape teaching.

These principles are among the main instructional practices that the formative assessment literature has been emphasizing for years (e.g. Lipnevich and Smith, 2018 ; Black and Wiliam, 1998 ; Black et al., 2003 ; Dochy and McDowell, 1997 ). However, the clarity of the presentation of the feedback practices in relationship to self-regulated learning ( Figure 11 ) turns this model into a very accessible one. Additionally, each principle is presented in detail describing empirical support and instructional recommendations.

FIGURE 11 . Nicol and MacFarlane-Dick (2006) model.

Hattie and Timperley (2007) : A Typology Model Supported by Meta-Analytic (CAP) Evidence

This is by far the most cited model of feedback not only in terms of the number of citations (+14000) but also in terms of expert selections (all consulted experts identified this model). It also both a model and a typology because it established instructional recommendations while linking them to four different types of feedback.

The theoretical framework of this paper builds upon previous feedback reviews and meta-analyses, ideas presented in Hattie (1999) , and general educational psychology literature (e.g. Deci and Ryan, 1985 ).

The authors presented a simple definition that is applicable to a wide range of behaviors, contexts, and instructional situations: “... feedback is conceptualized as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding.” (p. 81).

The model is based on the following proposition: Feedback should serve the purpose of reducing the gap between the desired goal and the current performance. To this end, Hattie and Timperley (2007) proposed different ways, in which the students and teachers can reduce this gap ( Figure 12 ). For the feedback to be more effective, it should answer three questions, each of them representing a type of feedback: where am I going? = feed up; how am I going? = feed back; and where to next? = feed forward. The authors claimed that the last type was the least frequently delivered and it was the one having the greatest impact, and when the authors asked students what they meant by feedback this was the one the students overwhelmingly desired (Hattie, personal communication, 30/11/2019). This, in itself, could be considered a typology differentiating feedback based on the context and the content of it. However, the typology that resonated the most with the field places feedback into four levels of task, process, self-regulation, and self. Most of the feedback given in an instructional setting is at the task level (i.e., specific comments relating to the task itself) and the self level (i.e., personal comments), despite the fact the process (i.e., comments on processes needed to perform the task) and self-regulation (i.e., higher-order comments relating to self-monitoring and regulation of actions and affect) are the ones with more potential for improvement. The authors also noted that the self level feedback (e.g., generic, person-level praise) is almost never conducive to enhancing performance regardless of its valence. Self-level feedback may interfere with the task-, process-, or self-regulation feedback by taking individuals’ attention away from those other types. This review tackles a range of additional topics, describing feedback timing, effects of positive and negative feedback, teacher role in feedback, and feedback as part of a larger scheme of assessment.

FIGURE 12 . Hattie and Timperley (2007) feedback model.

Importantly, through personal communication with the authors (Hattie, personal communication, 30/11/2019), Hattie stated: “The BIG idea we missed in the earlier review was that we needed to conceptualize feedback more in terms of what is received as opposed to what is given.” This line of reasoning is prominently featured in Hattie’s recent work ( Hattie and Clarke, 2019 ).

Evans (2013) : Reviewing the Literature on Assessment Feedback in Higher Education

Evans (2013) publication presents a compelling review of literature on feedback in higher education settings. The author showed an excellent understanding and pedagogical reading of the field covering different areas of formative assessment ranging from lecturers’ instructional activities, peer, and self-assessment.

The primary purpose of this article was to review current literature focusing on feedback in the context of higher education. Evans described feedback from socio-constructivist, co-constructivist, and cognitivist perspectives, to name a few, and reviewed characteristics of feedback that were most pertinent in the context of higher education.

Evans spent substantial amount of time reviewing definitions of feedback. She proposed that “Assessment feedback therefore includes all feedback exchanges generated within assessment design, occurring within and beyond the immediate learning context, being overt or covert (actively and/or passively sought and/or received), and importantly, drawing from a range of sources.” (p. 71). Evans systematically reviewed principles of effective feedback and provided an excellent overview of methodological approaches employed in feedback research.

Towards the end of her review, Evans presented a model entitled “The feedback landscape.” The pictorial representation of the model is presented in Figure 13 . The underlying idea of the model is in the close interaction between students and lecturers. Evans suggested that feedback was moderated by 12 variables shared by feedback receivers and givers (e.g. ability, personality, etc.), with three additional mediators selected for lecturers (i.e., awareness of other contexts, alignment of other modules, and knowledge of students). Surrounding this interaction there is the academic learning community (e.g. resources, academic peers, etc.) and an emphasis of temporal and special variability of mediators. Interestingly, most of these variables are not explored in detail in the review but are presented to the reader as part of the model. In general, presentation of the model was not among the articulated goals of the manuscript and its description is somewhat cursory.

FIGURE 13 . Evans (2013) Feedback landscape model.

Nevertheless, the publication provided a remarkable amount of information about instructional applications of feedback. For example, the author presented a table with a list of key principles of effective feedback practice. For each principle, Evans provided a significant number of references. Additionally, she summarized these practices into “12 pragmatic actions”:

1. ensuring an appropriate range and choice of assessment opportunities throughout a program of study;

2. ensuring guidance about assessment is integrated into all teaching sessions;

3. ensuring all resources are available to students via virtual learning environments and other sources from the start of a program to enable students to take responsibility for organizing their own learning;

4. clarifying with students how all elements of assessment fit together and why they are relevant and valuable;

5. providing explicit guidance to students on the requirements of assessment;

6. clarifying with students the different forms and sources of feedback available including e-learning opportunities;

7. ensuring early opportunities for students to undertake assessment and obtain feedback;

8. clarifying the role of the student in the feedback process as an active participant and not as purely receiver of feedback and with sufficient knowledge to engage in feedback;

9. providing opportunities for students to work with assessment criteria and to work with examples of good work;

10. giving clear and focused feedback on how students can improve their work including signposting the most important areas to address;

11. ensuring support is in place to help students develop self-assessment skills including training in peer feedback possibilities including peer support groups;

12. ensuring training opportunities for staff to enhance shared understanding of assessment requirements.

She delivered multiple lists and tables containing recommendations for peer feedback (p. 92), the basics about the feedback landscape (p. 100), and a list of potential avenues for future research (p. 107). These instructional recommendations are arguably more relevant than the model itself as the model is not sufficiently developed in the text.

Lipnevich, Berg, and Smith (2016) : Describing Students – Feedback Interaction

Lipnevich et al. (2016) model is one of the more recent models and it has been first presented in a chapter of a Handbook. The model, however, was selected by five experts as one of the models with which they are most familiar. The model has been recently revised and expanded to incorporate empirical findings and recent developments in the field of feedback.

The authors ground their model in the literature on feedback, reporting a thorough review of the field. The definition that Lipnevich et al. (2016) used came from Shute (2008) who defined feedback as “information communicated to the learner that is intended to modify his or her thinking or behavior for the purpose of improving learning” (p. 154). Lipnevich et al. (2016) emphasized students’ affective responses and used Pekrun’s Control-Value theory of achievement emotions to frame their discussion. In their revision of the model, Lipnevich et al. (2013) proposed the following definition of feedback: “Instructional feedback is any information about a performance that learners can use to improve their performance or learning. Feedback might come from teachers, peers, or the task itself. It may include information on where the learner is, where the learner is going, or what steps should be taken and strategies employed to get there.”

In their walk through the model the authors emphasized that feedback was always received in context ( Figure 14 ). The same type of feedback would be processed differentially depending on a class, academic domain, or consequential nature of the task. Within the context, the feedback is delivered to the student. Students will inevitably vary on their personality, general cognitive ability, receptivity to feedback, prior knowledge, and motivation. The feedback itself may be detailed or sparse, aligned with the students’ level of knowledge or not. It may be direct but delivered in a supportive fashion or may be unpleasantly critical. It may match what the student is expecting or be highly below or above those expectations. All these characteristics will contribute to students’ differential processing of feedback. In their new version of the model Lipnevich et al. (2013) included the source of feedback as a separate variable. The authors reported evidence that the feedback from the teacher, computer, peer, and the task itself may be perceived very differently, and may variably interact with student characteristics.

FIGURE 14 . Lipnevich et al. (2016) Feedback – student interaction Model.

When students receive the feedback message, they produce cognitive and affective responses that are often tightly interdependent. The student may cognitively appraise the situation, deciding whether the task is of interest and importance and whether they have control over the outcome, and make a judgment of whether the feedback is clear and understandable. That is, in reading through the feedback, students might be baffled by the comments, or may fully comprehend them. These appraisals result in a range of emotions, which, in turn, lead to some sort of behavioral responses. Students may engage in adaptive or maladaptive behavioral responses which will have a bearing on performance on a task and, possibly, learning. In the revision of the model the authors differentiated between learning and performance discussing potential effects of feedback on short-term changes on a task and long-term transfer to subsequent tasks. The authors also showed that both the response to the feedback and the actions that the student takes reflect on who the student was, what the student knew and could do in this area, and how the student would respond in the next cycle of feedback. Lipnevich et al. (2016) described feedback as a conversation between the teacher and a student, and cautioned scholars and practitioners that utterance that each party uses would be highly consequential for future student-teacher interactions and student learning progress.

The authors recently revised their model to further emphasize the three types of student processing: cognitive, affective, and behavioral 1 . Figure 15 of the revised model shows that message, student characteristics, and cognitive, affective, and behavioral responses contribute to an action that may alter student performance and learning 1 . Emphasized three questions, important for student receptivity of feedback: Do I understand feedback? How do I feel about feedback? What am I going to do about feedback? Importantly, the authors suggest that all feedback that comes from any external source will have to be internalized and converted into self- or inner feedback ( Nicol, 2021 ; Panadero et al., 2019; Andrade, 2018 ). The efficiency of this internal feedback would vary depending on a variety of factors.

FIGURE 15 . Student-feedback interaction Model (Revised).

Carless and Boud (2018) : A Proposal for Students’ Feedback Literacy

Carless’ work was voted in favor of inclusion by three experts but it was difficult to decide on a specific model, because he had proposed four ( Carless et al., 2011 ; Yang and Carless, 2013 ; Carless, 2018 ; Carless and Boud, 2018 ). The feedback literacy model included into the current review was suggested by the author as being both influential and matching with the goals of the current paper. In its short time since publication this model has achieved a significant number of citations.

The theoretical framework is based on social constructivist learning principles (e.g. Palincsar, 1998 ; Rust, O’Donovan and Price, 2005 ) and previous assessment research literature (e.g., Sadler, 1989 ; Price et al., 2011 ), with a clear grounding in higher education (31 out of the 53 cites come from journals with “higher education” in the title, with 18 of them from Assessment and Evaluation in Higher Education) and a few references to general feedback literature (e.g. Hattie and Timperley, 2007 ; Lipnevich et al., 2016 ).

Carless and Boud provided the following definition of feedback: “Building on previous definitions ( Boud and Molloy, 2013 ; Carless, 2015 ), feedback is defined as a process through which learners make sense of information from various sources and use it to enhance their work or learning strategies. This definition goes beyond notions that feedback is principally about teachers informing students about strengths, weaknesses and how to improve, and highlights the centrality of the student role in sense-making and using comments to improve subsequent work.” (p. 1). The authors then continued by defining student feedback literacy “…as the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies.”

The model and the main propositions are straightforward ( Figure 16 ). The model is composed of four inter-related elements. For students to develop feedback literacy they need to 1 ) appreciate the value and processes of feedback, 2 ) make judgments about their work and that of others, 3 ) manage the affect feedback can trigger in them, and all of this leads towards 4 ) taking action in response to such feedback. However, how these elements are operationalized could have been explicated in further detail as strategies are not explicit in the model. Carless and Boud provided a few illustrative examples of activities needed to develop feedback literacy that included peer feedback and analyzing exemplars, and described teachers’ role in the development of student feedback literacy. Some parts remain at a general level of description and the paper is concisely packaged, with some concepts and practices being mentioned but not described in detail.

FIGURE 16 . Carless and Boud (2018) Feedback literacy model.

Feedback Models Comparison

In the upcoming sections we will compare the models, focusing on definitions and empirical evidence behind them. Importantly, the comparison continues in Panadero and Lipnevich (2021) where we synthesize typologies and models, and propose a new integrative feedback model. Page limitations prevented us from having both articles in one, but we are hopeful that the reader will find them useful.

To offer the reader an idea about the historical and temporal continuity of the publications, we developed Table 1 that shows cross-citations among the fourteen publications discussed in this review. There is a clear evidence of cross-citations among the included publications. This is reassuring because a number of fields suffer from isolated pockets of research that do not inform each other ( Lipnevich and Roberts, 2014 ). An evidence to that is the model by Kluger and DeNisi (1996) that came to us from the field of industrial-organizational psychology: The authors did not reference any of the models published before them, most of which were situated within the field of education. Also, the only publication that was voted to be included but had not been cited was that of Tunstall and Gipps (1996) , possibly due to its limited demographic focus (i.e., early elementary students). However, for most included models the cross-citation is high and especially evident in the latest ones (e.g. Evans, 2013 , cited seven previous models).

TABLE 1 . Cross-citations of articles included in the current review.

When it comes to the number of citations of the models ( Table 2 ), it is clear that feedback models generate significant attention in the field of educational research. Interestingly, it does not matter whether the models are definitional (e.g. Ramaprasad, Sadler), are based on meta-analyses (e.g. Bangert-Drowns et al., Hattie and Timperley) or are linking multiple subfields (e.g. Butler and Winne, Nicol and McFarlane-Dick). Furthermore, it is clear that the field is constantly evolving with new models being developed that reflect the current focus of feedback research. We would like to encourage researchers to frame their studies within models that are currently available to avoid proliferation of redundant depictions of the feedback phenomena. For example, the field of psychosocial skills research currently has 136 models and taxonomies discussed by researchers and practitioners ( Berg et al., 2017 ). Obviously, it is not humanly possible to make sense of all of them, so the utility of proposing new models is very limited. Rather, validating existing models would be a more fruitful investment of researchers’ time. In Panadero and Lipnevich (2021) we integrate the fourteen models, selecting the most prominent elements of Message, Implementation, Student, Context, and Agents (MISCA). We hope the reader will find it instrumental.

TABLE 2 . Models characteristics.

The Definitions of Feedback

The problem of defining feedback has occupied minds of many feedback scholars and it has been a contested area, especially in the confluence of educational psychology and education. There are hundreds of definitions of educational feedback. This is not unique to the field of feedback and is common for other psychological and educational constructs, where lacking agreement on definitions stifles scientific developments.

When it comes to feedback, there appear to be opposing camps with some researchers arguing that feedback is information that is presented to a learner, whereas others viewing feedback as an interactive process of exchange between a student and an agent. There are also more extreme positions that describe feedback as the process where students put the information to use. Hence, according to this view, if not utilized, information delivered to students cannot be regarded as feedback (e.g. Boud and Molloy, 2013 ). However, there seems to be some common ground with some educational psychologists emphasizing the importance of the receptivity of feedback yet acknowledging that not acting upon feedback may be a valid situational response to it ( Lipnevich et al., 2016 ; Winstone et al., 2016 ; Hattie and Clarke, 2019 ; Jonson and Panadero, 2018 ). This tension shows that how researchers define feedback directly influence how they operationalize research. Therefore, it is crucial to explore how the models actually define feedback.

To achieve such goal, we looked in the included publications for sentences or paragraphs that were clearly indicative of a definition (e.g. “feedback is…”, “our definition of feedback is…”). We intentionally tried not to borrow definitions from subsequent or previous work of the authors and focused exclusively on what was presented in publications included in this review. Table 3 contains all definitions and Table 4 a comparison which we further develop below. Four publications did not include definitions ( Kulhavy and Stock, 1989 ; Bangert-Drowns et al., 1991 ; Tunstall and Gipps, 1996 ; Butler and Winne, 1995 ).

TABLE 3 . Definitions of feedback used in the different models.

TABLE 4 . Elements of the definitions of feedback used in the different models.

To offer the authors of the included models a chance at presenting their evolved ideas, in our interviews or email communication we asked whether they still agreed with their definitions or wanted to present one if it was missing in the original manuscript. This way we obtained a definition from Winne. Additionally, Lipnevich et al. (2016) , who used the unaltered Shute’s (2008) definition of feedback, put forward a new definition in their model revision ( Lipnevich et al., 2013 ). After analyzing the definitions, we identified five elements that are present in multiple definitions across the included publications ( Table 4 ). Our conclusions will be anchored to them.

The first conclusion is that the definitions of feedback are getting more comprehensive as more recent definitions include more elements than older definitions. For example, whereas the first chronological definition included two of the six elements we identified in the definitions ( Ramaprasad, 1983 ) the latest included six of the elements (Winne, 2019 via personal communication with authors). Table 4 shows the increase in the number of elements that occurs with the more recent models. This is an important reflection of the evolution and maturity of the feedback field where we are now looking at such aspects as feedback sources or the degree of student involvement in the feedback process. It appears that the expansion of the formative assessment field after the publication of Black and Wiliam (1998) played a major role in the development of definitions, and the models that followed this publication were more likely to include references to formative assessment. This, of course, makes sense. For formative assessment and assessment for learning theories ( Wiliam, 2011 ) feedback has to aim at improving students’ learning, to help students to process such feedback, and to become active agents in the process. Hence, within the realm of formative assessment, the field moved from a static understanding where feedback is “done” to the students (e.g., just indicating their level of performance) to our current understanding of the complex process that feedback involves.

A second conclusion takes us to the individual analysis of the definitional elements. So, all of the definitions discuss feedback as information that is exchanged or produced. This is a crucial component of feedback as without information there is nothing to process and, thus, it simply cannot be successful. This information may range from detailed qualitative commentary to a score, or from being delivered face to face to notes scribbled on the students’ work. Information is the essence of feedback, or, as many would maintain, is feedback.

One of the definitions presents a notable exception: Carless and Boud (2018) , building upon Boud and Molloy (2013) definition, argue that information is not feedback in itself but feedback is what students do with that information. From our perspective, although this pedagogical premise is powerful and instructors need to increase the likelihood of students using such information (e.g. allowing for resubmission of work), we do not fully agree with this position. According to Boud and Molloy’s definition if a student decides not to use the information then the information would not be considered feedback. The latter view conflicts with learner’s autonomy as the student might decide not to react to the feedback and not to improve a piece of work because she is happy with the score and does not want to invest more effort. For us, the information has been delivered and feedback has reached the student. All in all, the element of information is shared by the definitions.

Regarding all the other elements, seven models discuss the gap as the distance between the goal or the standard and student current performance. Feedback is thus construed as the information that is intended to close this gap (e.g. Hattie and Timperley, 2007 ). It is a powerful image for the teachers reminding them of the importance of the final goal and analyzing the students’ performance in relationship to it.

Another element is the process, explicitly mentioned by four of the models, which refers to the understanding that processing feedback involves cognitive, affective, and regulatory steps. Further, seven of the models describe the involvement of different educational agents that provide feedback (e.g. teacher, peer, computer, etc.). Hence, current definitions acknowledge that feedback can be delivered not only by teachers but by other agents too emphasizing the broadening of the definition. Finally, students’ active processing, which refers to the student as an active recipient of feedback, is also a key component of several definitions. This idea was brought into focus by the work of Sadler (1989) and was substantially expanded by Butler and Winne (1995) , who introduced the idea of internal feedback. Internal feedback is defined as feedback produced by the learner. These two elements, mentioned explicitly in four of the models, are very interrelated but can be differentiated. Interestingly, although some papers do not focus on student active processing in their definitions, this idea is very central to their model (e.g., Lipnevich et al., 2016 ). In the upcoming sections of this review we will devote more attention to these two elements.

A third conclusion is that there appears to be a higher level of consensus than could have been expected, with more recent definitions including different agents and stressing the active role of students. Although there are more extreme pedagogical positions (e.g. Boud and Molloy, 2013 ), the latest definitions are generally well aligned. The discussion in the field seems to be moving towards how to help students to use the feedback ( Winstone et al., 2017 ; Jonsson and Panadero, 2018 ).

Following this overview, we would like to aggregate definitions that were presented in the reviewed publications. We propose that: feedback is information that includes all or several components: students’ current state, information about where they are, where they are headed and how to get there, and can be presented by different agents (i.e., peer, teacher, self, task itself, computer). This information is expected to have a stronger effect on performance and learning if it encourages students to engage in active processing .

The Empirical Evidence Supporting the Models

It is important to analyze the models by looking at empirical support behind them. Theoretical explorations have their utility and serve as a starting point for subsequent studies. However, models that are derived based on data or have support after they have been presented have more value to the field. Among our selected models, there are three that are theoretical in nature without empirical evidence behind them: 1 ) Ramaprasad (1983) is a theoretical exploration of feedback definitions; 2 ) Sadler’s (1989) work is purely theoretical also, although Sadler claims that “Empirically, they [formative assessment and feedback practices] are known to produce results.” (p. 143); and 3 ) Carless and Boud (2018) present their first attempt at describing the model, so there are no studies known to date that attempted to validate this model.

The second group of models is also theoretical but these authors make explicit links to prior research. First, Mason and Bruning (2001) derived their model from previous theoretical and empirical work, so there was some support for links among variables depicted in the model. This model serves as a guide for designers of computer based instruction and explicates variables that would matter in the process. Second, Nicol and McFarlane-Dick (2006) model was based on prior empirical research that supported the seven described principles. It is, therefore, a theoretical contribution framed in previous empirical research. And, third, Evans (2013) presented a high number of relevant references to empirical studies to support instructional recommendations. The model itself represents an organizing framework, but it may be useful to devise studies that would examine, for example, the roles of mediators within corresponding buffer zones for lecturers and students. There is a third group of models that is based on authors’ previous work to derive their models. So, Kulhavy and Stock (1989) reported three previous studies, in which students were asked to report their response certitude (or make confidence judgments) following each response to various tasks. The researchers hypothesized that when students were certain that their answer was correct, they would spend little time analyzing feedback, and when students were certain their answer was incorrect, they would spend more time interacting with feedback. Practical implications of their findings are rather simple. That is, educators are to provide elaborated feedback for students who are more certain that their answer is wrong and deliver more limited feedback for those with high certitude of correct answers. Although their own studies supported their hypotheses, other studies did not replicate these findings. For example, Mory (1994) tried to replicate Kulhavy and Stock’s (1989) results and found that although there were differences in the amount of time students studied feedback, there was no significant effect for feedback tailored to response certitude and correctness in terms of student learning.

Similarly, Tunstall and Gipps (1996) derived their typology from their empirical data reported. Their method description is not detailed and it is not clear which data sources were used to arrive at specific categories. The information that we do have suggests that the sample was limited, and, to our knowledge, no studies have attempted to validate this typology.

Narciss also derived her models based on the extensive overview of theoretical and empirical studies conducted by others and herself. Due to the inherent complexity of these models, studies would need to systematically select specific components and test them separately. Modeling all included variables may not be feasible. In a personal communication with the author (3/12/2019), Narciss mentioned that her 2013 paper summarized empirical evidence her team had found so far using the model as a framework for designing and evaluating feedback strategies for digital learning environments. Additionally, many feedback studies conducted by others provided some empirical evidence for the model; yet, so far only a few of them had been explicitly linked to the model.

Finally, there is a number of studies that examined different aspects of the Lipnevich et al. (2016) model. So, Lipnevich and Smith (2009a , 2009b) demonstrated variable effects of differential feedback on student individual characteristics and subsequent responses. Further, Lipnevich et al. (2021) examined mediational role of emotions in the link between different types of feedback and student responses. Hence, there is initial evidence suggesting viability of the model, and due to the recency of it, more studies will be coming out soon.

The fourth group comprises models that are based on meta-analytic data. First, Bangert-Drowns et al. (1991) synthesized a number of relevant studies and emphasized the idea of mindfulness as a key approach to the effective receptivity of feedback. Further, Kluger and DeNisi (1996) meta-analysis “provided partial support to Feedback Intervention Theory” (p. 275). The researchers tested the propositions of the FIT and there was a reasonable support for the Feedback Intervention Theory. Future empirical work in the field of instructional feedback can be used to support the validity of the FIT. Finally, Hattie and Timperley (2007) model emanates from existing empirical research into what constitutes the most useful feedback characteristics. Additionally, there have been studies such as the ones by Harris et al. (2014) with teacher feedback, Harris et al. (2015) with peer and self-assessment, and Lipneviche et al. (2013) with principal’s perception of feedback that found the evidence that Hattie and Timperley’s model typology can be used to effectively categorize feedback in classroom settings.

Finally, Butler and Winne (1995) are in their own special category, in which other scholars reviewed the empirical evidence behind the original model. Greene and Azevedo (2007) published what they considered a theoretical review, where they reviewed 113 studies providing empirical evidence for this model. The researchers found evidence supporting most of the processes from the model, as can be seen in the original Table 2 .

In sum, the general conclusion is that the empirical support for the fourteen included models is variable. More empirical studies are needed, and we implore researchers in the field to invest time into model validation. The and into conducting studies that investigate propositions of existing models.

In this section we will discuss the main conclusions of our review.

1. The models have different aims and focus . Although the models share the purpose of explaining the process of feedback and its effects, they are quite different from each other in their aim or purpose. The focus of these models varies from being descriptive to presenting the specifics of processing of feedback or offering detailed pedagogical recommendations. In our work, which builds upon this review ( Panadero and Lipnevich, 2021 ), we provided a framework organized around five elements: (feedback) message, implementation, (instructional) context, (feedback) agents and student (characteristics). We hope the reader will find it useful.

2. Choosing the “right” model and theory . Some models are better suited for guiding empirical or theoretical investigations. Therefore, feedback researchers may want to choose the model that better suits their aims. For example, if a researcher is interested in exploring the mechanisms of feedback receptivity, it would be better to anchor the work in the interactional models, that describe specifics of student feedback processing. If instructional interventions are the goal, then pedagogical models will be of higher value. Another aspect to consider is that some models are more general and might be easier for teachers to apply to a range of instructional scenarios (e.g. Hattie and Timperley, 2007 ) whereas others might be better suited for specific contexts and more specific interventions (e.g. computer, Narciss, 2013 ).

3. Typologies: wide variety but more research is needed ( Panadero and Lipnevich, 2021 ) . There seems to be a consensus in terms of possible functions of feedback, whereas the content of feedback is a more contested category. Jointly, these two categories of feedback have received considerable attention, whereas the importance of presentation seemed to be less emphasized. Presentation influences students’ receptivity and use of feedback, and, hence, future theoretical explorations may devote more attention to it (e.g., Jonsson, 2013 ). Additionally, the source of feedback represents a useful variable to consider in future typologies.

4. Focus on feedback receptivity . The actual efficiency of any feedback message depends on what the student does next with this information. After all, if instructors or peers prepare the best kind of feedback and students simply dismiss it, the effort will be wasted and no benefit will follow. Many of the models, starting with Ramaprasad (1983) and Sadler (1989) , and ending with the more recent ones of Hattie and Timperley (2007) , Lipnevich et al. (2016) , and Carless and Boud (2018) all stress the role of the student and emphasize the recursive nature of student-teacher (or other agent) interaction. Bringing the role of the student into the focus is critical for the field, and examining the ways in which learners make sense and use feedback to inform their progress is of key importance also. Thoughtful application of the reviewed models may help us to better understand how feedback leads to learning, where it might hinder learning, and which characteristics and contexts of feedback would be more likely to encourage students to actively engage with it.

5. The need for more empirical evidence . Some of the models included in this review have some empirical evidence supporting their validity (e.g. Hattie and Timperley), whereas others do not. Some models are inherently not testable because they do not describe relations and are purely definitional (e.g., Ramaprasad and Sadler), whereas others include multiple components and relations that are virtually impossible to define and investigate in a single study ( Narciss, 2013 ). It is our hope that future research tasks itself with providing empirical evidence for models included in this review. Instructional feedback does not exist in a vacuum, so a host of variables described by the reviewed models needs to be taken into consideration. Conducting such investigations lies at the crux of art and science of research. We should be able to specify and validate models in classroom settings, conducting studies that are not laboratory-sterile, but those that can be applied and replicated in typical instructional settings. At the same time, we should be able to disambiguate relations so that we can make clear attributions of causes and make conclusions about feedback effects with some degree of certainty. The review showed that theoretical developments have been impressive—now more good quality empirical work is needed. At the same time, examining practical applications of models and theories herein reported is of key importance to the field.

6. Lack of an output of the feedback effects: performance versus learning . A number of models does not explicitly describe outcome variables, focusing exclusively on characteristics of feedback itself. In education, there has been a large debate about whether what we measure in our classrooms and in most educational research is learning or performance ( Soderstrom and Bjork, 2015 ). Most commonly, it is the performance that gets measured. Unfortunately, measuring learning is a complex enterprise that implies designing studies that capture transfer from one task to the next, and such investigations are rather costly. Nevertheless, it is crucial that when considering the effects of feedback and student receptivity, researchers start evaluating effects on learning and not just on their performance on the immediate task. Unless we as a field commit to this goal there will be a multitude of unanswered questions about the utility and the general promise of feedback. In other words, we need to measure the impact of feedback interventions on academic achievement.

7. A final remark: do we need more models? This review included fourteen models after leaving a number of important ones because they did not fulfill our selection criteria. If we would just consider the number of models, then the probable answer would be “we have enough.” Nevertheless, the situation is more complex than that. The newer models cover aspects that the previous ones had not or offer new perspectives about already known aspects. Take for example, Lipnevich et al. (2016) that brings to the forefront the mechanisms of how the feedback and students characteristics may jointly influence responses and actions that students perform, while also considering the context. The rule of thumb for the creation of new models could be: “Does my model cover an area in need of an explanation? Does my model explain or clarify aspects the existing models do not?.” If the answer is yes, then it might be worth giving it a try and letting the research community decide.

Although researchers agree that feedback is essential for improved performance and can contribute to enhanced achievement on the task (reported effect sizes are as high as 0.73), we also know that 1 ) learners often dread it and dismiss it, and 2 ) the effectiveness of feedback varies depending on specific characteristic of feedback messages that learners receive ( see , e.g., Hattie and Timperley, 2007 ; Lipnevich and Smith, 2018 ; Shute, 2008 ). Many studies have attempted to identify what constitutes good feedback, which characteristics are most critical for students’ receptivity, and how to encourage students to effectively utilize it – often with inconsistent results (e.g., Lipnevich and Smith, 2018 ). Part of the reason for such inconsistency may be attributed to studies coming from different methodological perspectives and using disparate terminology to label relevant student-, feedback- or context-level factors that link feedback to improved performance (i.e., the “jingle–jangle fallacy,” see Block, 1995 ). To bring more clarity into the field, many researchers have attempted to propose models and theories that describe feedback, student interaction with it, along with specific conditions that make feedback effective. At this time, however, the models are proliferating and the field is missing clarity on what feedback models are available and how can they be used for the development of instructional activities, assessments, and interventions. With this review we attempted to describe the most prominent models in the field and summarize main conclusions along with recommendations for future research and Panadero and Lipnevich (2021) extended this discussion with an attempt to integrate the fourteen included models and theories. We hope this review will be a good resource to both experts and novices who work or are considering joining the exciting field of feedback research.

Author Contributions

AL and EP both contributed to conceptualization, preparation, and writing up of the study.

Review funded by UAM-SANTANDER collaboration with the USA (Reference: 2017/EEUU/12).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

To our feedback experts: Rola Ajjawi, Heidi Andrade, David Boud, Susan Brookhart, Gavin Brown, David Carless, Roy Clariana, Siv Gamlem, Mark Gan, Steve Graham, John Hattie, Avraham Kluger, Angela Lui, Jackie Murray, Susanne Narciss, David Nicol, Ron Pat-El, Reinhard Pekrun, Elizabeth Peterson, Joan Sargeant, Mien Segers, Valerie Shute, Jan-Willem Strijbos, Jeroen Van Merriënboer, Dylan Wiliam, Phil Winne, and Naomi Winstone. Thanks to our interviewees: David Boud, David Carless, Avraham Kluger, Susanne Narciss, Arkalgud Ramaprasad, Royce Sadler and Phil Winne. Thanks to the authors who were not available for an interview but with whom we exchanged emails: Robert Bangert-Drowns, Carol Evans, Caroline Gipps, John Hattie and David Nicol. Special thanks to Dylan Wiliam for providing comments on earlier versions of the manuscript and thanks to Emilian Lipnevich for helping us to redraw some of the figures.

1 Lipnevich, A. A., and Smith, J. K. (2021). Student -Feedback Interaction Model: Revised. Under review.

Andrade, H. (2018). “Feedback in the Context of Self-Assessment,” in Lipnevich Cambridge Handbook of Instructional Feedback . Editors A. A. Lipnevich, and J. K. Smith (Cambridge: Cambridge University Press ), 376–408.

Google Scholar

Annett, J. (1969). Feedback and Human Behaviour . Harmondsworth, UK: Penguin Books .

Balzer, W. K., Doherty, M. E., and O'Connor, R. (1989). Effects of Cognitive Feedback on Performance. Psychol. Bull. 106, 41. doi:10.1037/0033-2909.106.3.410

CrossRef Full Text | Google Scholar

Bandura, A. (1991). Social Cognitive Theory of Self-Regulation. Organizational Behav. Hum. Decis. Process. 50, 248–287. doi:10.1016/0749-5978(91)90022-l

Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., and Morgan, M. (1991). The Instructional Effect of Feedback in Test-like Events. Rev. Educ. Res. 61, 213–238. doi:10.3102/00346543061002213

Bennett, S. N., and Kell, J. (1989). A Good Start? Four Year Olds in Infant Schools . Oxford, UK: Blackwell .

Berg, J., Osher, D., Same, M. R., Nolan, E., Benson, D., and Jacobs, N. (2017). Identifying, Defining, and Measuring Social and Emotional Competencies . Washington, DC: American Institutes for Research .

Black, P., Harrison, C., Lee, C., Marshal, B., and Wiliam, D. (2003). Assessment for Learning: Putting it into Practice . Maidenhead, UK: Open University Press .

Black, P., and Wiliam, D. (1998). Assessment and Classroom Learning. Assess. Educ. Principles, Pol. Pract. 5 (1), 7–74. doi:10.1080/0969595980050102

Block, J. (1995). A Contrarian View of the Five-Factor Approach to Personality Description. Psychol. Bull. 117, 187–215. doi:10.1037/0033-2909.117.2.187

PubMed Abstract | CrossRef Full Text | Google Scholar

Bloom, B. S. (1971). Handbook on Formative and Summative Evaluation of Student Learning . New York, NY: McGraw-Hill .

Bloom, B. S. (1968). Toward a Theory of Testing Which Includes Measurement-Evaluation-Assessment (No. 9) . University of California .

Borkowski, J. G. (1992). Metacognitive Theory: A Framework for Teaching Literacy, Writing, and Math Skills. J. Learn. Disabil. 25, 253–257. doi:10.1177/002221949202500406

Boud, D., and Molloy, E. (2013). Rethinking Models of Feedback for Learning: The Challenge of Design. Assess. Eval. Higher Education 38 (6), 698–712. doi:10.1080/02602938.2012.691462

Boud, D. (2000). Sustainable Assessment: Rethinking Assessment for the Learning Society. Stud. Continuing Education 22 (2), 151–167. doi:10.1080/713695728

Brophy, J. (1981). Teacher Praise: A Functional Analysis. Rev. Educ. Res. 5–32. doi:10.3102/00346543051001005

Butler, D. L., and Winne, P. H. (1995). Feedback and Self-Regulated Learning: A Theoretical Synthesis. Rev. Educ. Res. 65 (3), 245–281. doi:10.3102/00346543065003245

Carless, D., and Boud, D. (2018). The Development of Student Feedback Literacy: Enabling Uptake of Feedback. Assess. Eval. Higher Education 43 (8), 1315–1325. doi:10.1080/02602938.2018.1463354

Carless, D. (2015). Excellence in University Assessment: Learning from Award-Winning Practice . London, UK: Routledge .

Carless, D. (2018). Feedback Loops and the Longer-Term: Towards Feedback Spirals. Assess. Eval. Higher Education 44, 705–714. doi:10.1080/02602938.2018.1531108

Carless, D., Salter, D., Yang, M., and Lam, J. (2011). Developing Sustainable Feedback Practices. Stud. Higher Educ. 36 (4), 395–407. doi:10.1080/03075071003642449

Carver, C. S., and Scheier, M. F. (1990). Origins and Functions of Positive and Negative Affect: A Control-Process View. Psychol. Rev. 97 (1), 19–35. doi:10.1037/0033-295x.97.1.19

Clariana, R. B. (1999). “Differential Memory Effects for Immediate and Delayed Feedback: A Delta Rule Explanation of Feedback Timing Effects,” in Paper presented at the Association for Educational Communications and Technology Annual Convention , Houston, TX , February 10–14, 1999 .

Cohen, V. B. (1985). A Reexamination of Feedback in Computer-Based Instruction: Implications for Instructional Design. Educ. Technology 25 (1), 33–37.

Deci, E. L., and Ryan, R. (1985). Intrinsic Motivation and Self-Determination in Human Behavior . New York, NY: Plenum .

Dochy, F. J. R. C., and McDowell, L. (1997). Assessment as a Tool for Learning. Stud. Educ. Eval. 23 (4), 279–298. doi:10.1016/s0191-491x(97)86211-6

Dweck, C. S. (1986). Motivational Processes Affecting Learning. Am. Psychol. 41, 1040–1048. doi:10.1037/0003-066x.41.10.1040

Evans, C. (2013). Making Sense of Assessment Feedback in Higher Education. Rev. Educ. Res. 83 (1), 70–120. doi:10.3102/0034654312474350

Gall, M. D., Borg, W. R., and Gall, J. P. (1996). Educational Research: An Introduction . Harlow, UK: Longman Publishing .

Goetz, T., Lipnevich, A. A., Krannich, M., and Gogol, K. (2018). “Performance Feedback and Emotions,” in The Cambridge Handbook of Instructional Feedback . Editors A. A. Lipnevich, and J. K. Smith (Cambridge, UK: Cambridge University Press ).

Greene, J. A., and Azevedo, R. (2007). A Theoretical Review of Winne and Hadwin's Model of Self-Regulated Learning: New Perspectives and Directions. Rev. Educ. Res. 77 (3), 334–372. doi:10.3102/003465430303953

Guskey, T. R. (2018). “Feedback, Correctives, and the Use of Pre-assessments,” in The Cambridge Handbook of Instructional Feedback . Cambridge University Press , 432–450.

Harris, L. R., Brown, G. T. L., and Harnett, J. A. (2015). Analysis of New Zealand Primary and Secondary Student Peer- and Self-Assessment Comments: Applying Hattie and Timperley's Feedback Model. Assess. Educ. Principles, Pol. Pract. 22 (2), 265–281. doi:10.1080/0969594x.2014.976541

Hattie, J. A. (1999). Influences OAn Student Learning (Inaugural Professorial Address) . Auckland: University of Auckland . Available at: http://www.arts.auckland.ac.nz/staff/index.cfm?P=8650 .

Hattie, J. A. (2009). Visible Learning. A Synthesis of over 800 Meta-Analyses Relating to Achievement . New York, NY: Routledge .

Hattie, J., and Clarke, S. (2019). Visible Learning: Feedback . New York, NY: Routledge .

Hattie, J., and Timperley, H. (2007). The Power of Feedback. Rev. Educ. Res. 77 (1), 81–112. doi:10.3102/003465430298487

Jonsson, A. (2013). Facilitating Productive Use of Feedback in Higher Education. Active Learn. Higher Educ. 14 (1), 63–76. doi:10.1177/1469787412467125

Jonsson, A., and Panadero, E. (2018). “Facilitating Students’ Active Engagement with Feedback,” in The Cambridge Handbook of Instructional Feedback . Editors A. A. Lipnevich, and J. K. Smith (Cambridge, UK: Cambridge University Press ).

Kluger, A. N., and DeNisi, A. (1996). The Effects of Feedback Interventions on Performance: A Historical Review, A Meta-Analysis, and A Preliminary Feedback Intervention Theory. Psychol. Bull. 119 (2), 254–284. doi:10.1037/0033-2909.119.2.254

Kulhavy, R. W. (1977). Feedback in Written Instruction. Rev. Educ. Res. 47 (2), 211–232. doi:10.3102/00346543047002211

Kulhavy, R. W., and Stock, W. A. (1989). Feedback in Written Instruction: The Place of Response Certitude. Educ. Psychol. Rev. 1 (4), 279–308. doi:10.1007/bf01320096

Kulik, J. A., and Kulik, C.-L. C. (1988). Timing of Feedback and Verbal Learning. Rev. Educ. Res. 58 (1), 79–97. doi:10.3102/00346543058001079

Lipnevich, A. A., and Smith, J. K. (2009a). Effects of Differential Feedback on Students' Examination Performance. J. Exp. Psychol. Appl. 15 (4), 319–333. doi:10.1037/a0017841

Lipnevich, A. A., and Smith, J. K. (2009b). “I Really Need Feedback to Learn:” Students' Perspectives on the Effectiveness of the Differential Feedback Messages. Educ. Asse Eval. Acc. 21 (4), 347–367. doi:10.1007/s11092-009-9082-2

Lipnevich, A. A., McCallen, L. N., and Smith, J. K. (2013). Perceptions of the Effectiveness of Feedback: School Leaders' Perspectives. Am 5, 74–93. doi:10.18296/am.0109

Lipnevich, A. A., and Roberts, R. D. (2014). Noncognitive Skills in Education: Emerging Research and Applications in a Variety of International Contexts. Learn. Individual Differences 22, 173–177.

Lipnevich, A. A., Berg, D., and Smith, J. K. (2016). “Toward a Model of Student Response to Feedback,” in Human Factors And Social Conditions In Assessment, 169–185 . Editors G. T. L. Brown, and L. Harris (New York, NY: Routledge ).

Lipnevich, A. A., and Smith, J. K. (Editors) (2018). The Cambridge Handbook of Instructional Feedback . Cambridge, United Kingdom: Cambridge University Press .

Lipnevich, A. A., Murano, D., Krannich, M., and Goetz, T. (2021). Should I Grade or Should I Comment: Links Among Feedback, Emotions, and Performance. Learn. Individual Differences 89, 102020. doi:10.1016/j.lindif.2021.102020

Locke, E. A., and Latham, G. P. (1990). A Theory of Goal Setting and Task Performance . Englewood Cliffs, NJ: Prentice-Hall .

Mason, B. J., and Bruning, R. (2001). Providing Feedback in Computer-Based Instruction: What the Research Tells Us. Center for Instructional Innovation , 14. Lincoln, NE: University of Nebraska–Lincoln . Available at: http://dwb.unl.edu/Edit/MB/ MasonBruning.html (Accessed April 15, 2020).

Mikulincer, M. (1994). Human Learned Helplessness: A Coping Perspective . New York, NY: Plenum Press .

Mory, E. H. (2004). “Feedback Research Revisited,” in Handbook of Research on Educational Communications and Technology . Editor D. H. Jonassen (Washington, DC: Lawrence Erlbaum Associates Publishers ), 2, 745–783.

Mory, E. H. (1994). The Use of Response Certitude in Adaptive Feedback: Effects on Student Performance, Feedback Study Time, and Efficiency. J. Educ. Comput. Res. Development 40 (3), 5–20.

Narciss, S. (2017). “Conditions and Effects of Feedback Viewed through the Lens of the Interactive Tutoring Feedback Model,” in Scaling up Assessment for Learning in Higher Education . Editors D. Carless, S. M. Bridges, C. K. Y. Chan, and R. Glofcheski (Singapore: Springer Singapore ), 173–189. doi:10.1007/978-981-10-3045-1_12

Narciss, S. (2013). Designing and Evaluating Tutoring Feedback Strategies for Digital Learning Environments on the Basis of the Interactive Tutoring Feedback Model. Digital Education Rev. 23 (1), 7–26.

Narciss, S. (2008). “Feedback Strategies for Interactive Learning Tasks,” in Handbook of Research on Educational Communications and Technology . Editors J. M. Spector, M. D. Merrill, J. J. G. Van Merrienboer, and M. P. Driscoll. 3rd Edn. (Mahwah, NJ: Lawrence Erlbaum Associates ), 125–143.

Narciss, S., and Huth, K. (2004). “How to Design Informative Tutoring Feedback for Multi-media Learning,” in Instructional Design for Multimedia Learning . Editors H. M. Niegemann, D. Leutner, and R. Brünken (Münster: Waxmann ), 181–195.

Nicol, D. J., and Macfarlane‐Dick, D. (2006). Formative Assessment and Self‐regulated Learning: A Model and Seven Principles of Good Feedback Practice. Stud. Higher Educ. 31 (2), 199–218. doi:10.1080/03075070600572090

Nicol, D. (2021). The Power of Internal Feedback: Exploiting Natural Comparison Processes. Assess. Eval. Higher Education 46 (5), 756–778. doi:10.1080/02602938.2020.1823314

Palincsar, A. S. (1998). Social Constructivist Perspectives on Teaching and Learning. Annu. Rev. Psychol. 49, 345–375. doi:10.1146/annurev.psych.49.1.345

Panadero, E. (2017). A Review of Self-Regulated Learning: Six Models and Four Directions for Research. Front. Psychol. 8, 422. doi:10.3389/fpsyg.2017.00422

Panadero, E., Andrade, H., and Brookhart, S. (2018). Fusing Self-Regulated Learning and Formative Assessment: A Roadmap of where We Are, How We Got Here, and where We Are Going. Aust. Educ. Res. 45 (1), 13–31. doi:10.1007/s13384-018-0258-y

Panadero, E., Broadbent, J., Boud, D., and Lodge, J. M. (2019). Using Formative Assessment To Influence Self- And Co-Regulated Learning: The Role Of Evaluative Judgement. J. Psychol. Edu. 34 (3), 535–557. doi:10.1007/s10212-018-0407-8

Panadero, E., and Lipnevich, A. A. (2021). A Review of Feedback Models and Typologies: Towards an Integrative Model of Feedback Elements. Educ. Res. Rev. , 100416. doi:10.1016/j.edurev.2021.100416

Pintrich, P. R. (1995). Understanding Self-Regulated Learning . San Francisco, CA: Jossey-Bass .

Price, M., Handley, K., and Millar, J. (2011). Feedback: Focusing Attention on Engagement. Stud. Higher Education 36 (8), 879–896. doi:10.1080/03075079.2010.483513

Ramaprasad, A. (1983). On the Definition of Feedback. Syst. Res. 28 (1), 4–13. doi:10.1002/bs.3830280103

Rust, C., O’Donovan, B., and Price, M. (2005). A Social Constructivist Assessment Process Model: How the Research Literature Shows Us This Could Be Best Practice. Assess. Eval. Higher Education 30 (3), 231–240. doi:10.1080/02602930500063819

Sadler, D. R. (1989). Formative Assessment and the Design of Instructional Systems. Instr. Sci. 18 (2), 119–144. doi:10.1007/bf00117714

Sadler, D. R. (1998). Formative Assessment: Revisiting the Territory. Assess. Educ. Principles, Pol. Pract. 5 (1), 77–84. doi:10.1080/0969595980050104

Schunk, D. H. (1982). Effects of Effort Attributional Feedback on Children's Perceived Self-Efficacy and Achievement. J. Educ. Psychol. 74, 548–556. doi:10.1037/0022-0663.74.4.548

Scriven, M. S. (1967). The Methodology of Evaluation. Perspect. Curriculum Eval. AERA Monogr. Ser. Curriculum Eval. , 1.

Shuell, T. J. (1986). Cognitive Conceptions of Learning. Rev. Educ. Res. 56, 411–436. doi:10.3102/00346543056004411

Shute, V. J. (2008). Focus on Formative Feedback. Rev. Educ. Res. 78 (1), 153–189. doi:10.3102/0034654307313795

Soderstrom, N. C., and Bjork, R. A. (2015). Learning versus Performance: An Integrative Review. Perspect. Psychol. Sci. 10 (2), 176–199. doi:10.1177/1745691615569000

Sweller, J., van Merrienboer, J. J. G., and Paas, F. G. W. C.(1998). Cognitive Architecture and Instructional Design. Educ. Psychol. Rev. 10 (3), 251–296. doi:10.1023/a.1022193728205

Thorndike, E. L. (1913). Educational Psychology . in The Original Nature of Man , l. New York, NY: Teachers College, Columbia University .

Thorndike, E. L. (1927). The Law of Effect. Am. J. Psychol. 39 (1/4), 212–222. doi:10.2307/1415413

Thorndike, R. M. (1997). “The Early History of Intelligence Testing,” in Contemporary Intellectual Assessment: Theories, Tests and Issues . Editors D. P. Flanagan, J. L. Genshaft, and P. L. Harrison (New York, NY: Guilford Press ).

Tunstall, P., and Gsipps, C. (1996). Teacher Feedback to Young Children in Formative Assessment: A Typology. Br. Educ. Res. J. 22 (4), 389–404. doi:10.1080/0141192960220402

van Zundert, M., Sluijsmans, D., and van Merriënboer, J. (2010). Effective Peer Assessment Processes: Research Findings and Future Directions. Learn. Instruction 20 (4), 270–279. doi:10.1016/j.learninstruc.2009.08.004

Wiener, N. (1954). The Human Use of Human Beings: Cybernetics and Society . Oxford, UK: Houghton Mifflin .

Wiliam, D. (2011). What is Assessment for Learning?. Stud. Educ. Eval. 37 (1), 3–14. doi:10.1016/j.stueduc.2011.03.001

Wiliam, D. (2018). “Feedback: At the Heart Of—But Definitely Not All of—Formative Assessment,” in Cambridge Handbook of Instructional Feedback . Editors A. A. Lipnevich, and J. K Smith (Cambridge: Cambridge University Press ), 376–408.

Winne, P. H. (2011). “A Cognitive and Metacognitive Analysis of Self-Regulated Learning,” in Handbook of Self-Regulation of Learning and Performance . Editors B. J. Zimmerman, and D. H. Schunk (New York, NY: Routledge ), 15–32.

Winne, P. H. (1996). A Metacognitive View of Individual Differences in Self-Regulated Learning. Learn. Individual Differences 8 (4), 327–353. doi:10.1016/S1041-6080(96)90022-9

Winne, P. H., and Hadwin, A. F. (2008). “The Weave of Motivation and Self- Regulated Learning,” in Motivation and Self-Regulated Learning: Theory, Research, and Applications . Editors D. H. Schunk, and B. J. Zimmerman (Mahwah, NJ: Erlbaum ), 297–314.

Winstone, N. E., Nash, R. A., Parker, M., and Rowntree, J. (2017). Supporting Learners' Agentic Engagement with Feedback: A Systematic Review and a Taxonomy of Recipience Processes. Educ. Psychol. 52 (1), 17–37. doi:10.1080/00461520.2016.1207538

Winstone, N. E., Nash, R. A., Rowntree, J., and Menezes, R. (2016). What Do Students Want Most from Written Feedback Information? Distinguishing Necessities from Luxuries Using a Budgeting Methodology. Assess. Eval. Higher Education 41 (8), 1237–1253. doi:10.1080/02602938.2015.1075956

Yang, M., and Carless, D. (2013). The Feedback Triangle and the Enhancement of Dialogic Feedback Processes. Teach. Higher Education 18 (3), 285–297. doi:10.1080/13562517.2012.719154

Zimmerman, B. J. (1989). A Social Cognitive View of Self-Regulated Academic Learning. J. Educ. Psychol. 81, 329–339. doi:10.1037/0022-0663.81.3.329

Zimmerman, B. J., and Schunk, D. H. (2001). Self-regulated Learning and Academic Achievement: Theoretical Perspectives . Mahwah, NJ: Lawrence Erlbaum Associates .

Keywords: feedback, review, definition, feedback model, feedback theory, feedback mechanism

Citation: Lipnevich AA and Panadero E (2021) A Review of Feedback Models and Theories: Descriptions, Definitions, and Conclusions. Front. Educ. 6:720195. doi: 10.3389/feduc.2021.720195

Received: 03 June 2021; Accepted: 02 November 2021; Published: 31 December 2021.

Reviewed by:

Copyright © 2021 Lipnevich and Panadero. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Anastasiya A. Lipnevich, [email protected]

What’s Included: Literature Review Template

This template is structure is based on the tried and trusted best-practice format for formal academic research projects such as dissertations and theses. The literature review template includes the following sections:

• Before you start – essential groundwork to ensure you’re ready
• The introduction section
• The core/body section
• The conclusion /summary
• Extra free resources

Each section is explained in plain, straightforward language , followed by an overview of the key elements that you need to cover. We’ve also included practical examples and links to more free videos and guides to help you understand exactly what’s required in each section.

The cleanly-formatted Google Doc can be downloaded as a fully editable MS Word Document (DOCX format), so you can use it as-is or convert it to LaTeX.

PS – if you’d like a high-level template for the entire thesis, you can we’ve got that too .

FAQs: Literature Review Template

What format is the template (doc, pdf, ppt, etc.).

The literature review chapter template is provided as a Google Doc. You can download it in MS Word format or make a copy to your Google Drive. You’re also welcome to convert it to whatever format works best for you, such as LaTeX or PDF.

What types of literature reviews can this template be used for?

The template follows the standard format for academic literature reviews, which means it will be suitable for the vast majority of academic research projects (especially those within the sciences), whether they are qualitative or quantitative in terms of design.

Keep in mind that the exact requirements for the literature review chapter will vary between universities and degree programs. These are typically minor, but it’s always a good idea to double-check your university’s requirements before you finalize your structure.

Is this template for an undergrad, Master or PhD-level thesis?

This template can be used for a literature review at any level of study. Doctoral-level projects typically require the literature review to be more extensive/comprehensive, but the structure will typically remain the same.

Can I modify the template to suit my topic/area?

Absolutely. While the template provides a general structure, you should adapt it to fit the specific requirements and focus of your literature review.

What structural style does this literature review template use?

The template assumes a thematic structure (as opposed to a chronological or methodological structure), as this is the most common approach. However, this is only one dimension of the template, so it will still be useful if you are adopting a different structure.

Does this template include the Excel literature catalog?

No, that is a separate template, which you can download for free here . This template is for the write-up of the actual literature review chapter, whereas the catalog is for use during the literature sourcing and sorting phase.

How long should the literature review chapter be?

This depends on your university’s specific requirements, so it’s best to check with them. As a general ballpark, literature reviews for Masters-level projects are usually 2,000 – 3,000 words in length, while Doctoral-level projects can reach multiples of this.

Can I include literature that contradicts my hypothesis?

Yes, it’s important to acknowledge and discuss literature that presents different viewpoints or contradicts your hypothesis. So, don’t shy away from existing research that takes an opposing view to yours.

How do I avoid plagiarism in my literature review?

Always cite your sources correctly and paraphrase ideas in your own words while maintaining the original meaning. You can always check our plagiarism score before submitting your work to help ease your mind. 

Do you have an example of a populated template?

We provide a walkthrough of the template and review an example of a high-quality literature research chapter here .

Can I share this literature review template with my friends/colleagues?

Yes, you’re welcome to share this template in its original format (no editing allowed). If you want to post about it on your blog or social media, all we ask is that you reference this page as your source.

Do you have templates for the other dissertation/thesis chapters?

Yes, we do. You can find our full collection of templates here .

Can Grad Coach help me with my literature review?

Yes, you’re welcome to get in touch with us to discuss our private coaching services , where we can help you work through the literature review chapter (and any other chapters).

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• The C.A.R.S. Model
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Introduction

The Creating a Research Space [C.A.R.S.] Model was developed by John Swales based upon his analysis of journal articles representing a variety of discipline-based writing practices. His model attempts to explain and describe the organizational pattern of writing the introduction to scholarly research studies. Following the C.A.R.S. Model can be useful approach because it can help you to: 1) begin the writing process [getting started is often the most difficult task]; 2) understand the way in which an introduction sets the stage for the rest of your paper; and, 3) assess how the introduction fits within the larger scope of your study. The model assumes that writers follow a general organizational pattern in response to two types of challenges [“competitions”] relating to establishing a presence within a particular domain of research: 1) the competition to create a rhetorical space and, 2) the competition to attract readers into that space. The model proposes three actions [Swales calls them “moves”], accompanied by specific steps, that reflect the development of an effective introduction for a research paper. These “moves” and steps can be used as a template for writing the introduction to your own social sciences research papers.

"Introductions." The Writing Lab and The OWL. Purdue University; Coffin, Caroline and Rupert Wegerif. “How to Write a Standard Research Article.” Inspiring Academic Practice at the University of Exeter; Kayfetz, Janet. "Academic Writing Workshop." University of California, Santa Barbara, Fall 2009; Pennington, Ken. "The Introduction Section: Creating a Research Space CARS Model." Language Centre, Helsinki University of Technology, 2005; Swales, John and Christine B. Feak. Academic Writing for Graduate Students: Essential Skills and Tasks. 2nd edition. Ann Arbor, MI: University of Michigan Press, 2004.

Creating a Research Space Move 1: Establishing a Territory [the situation] This is generally accomplished in two ways: by demonstrating that a general area of research is important, critical, interesting, problematic, relevant, or otherwise worthy of investigation and by introducing and reviewing key sources of prior research in that area to show where gaps exist or where prior research has been inadequate in addressing the research problem. The steps taken to achieve this would be:

• Step 1 -- Claiming importance of, and/or  [writing action = describing the research problem and providing evidence to support why the topic is important to study]
• Step 2 -- Making topic generalizations, and/or  [writing action = providing statements about the current state of knowledge, consensus, practice or description of phenomena]
• Step 3 -- Reviewing items of previous research  [writing action = synthesize prior research that further supports the need to study the research problem; this is not a literature review but more a reflection of key studies that have touched upon but perhaps not fully addressed the topic]

Move 2: Establishing a Niche [the problem] This action refers to making a clear and cogent argument that your particular piece of research is important and possesses value. This can be done by indicating a specific gap in previous research, by challenging a broadly accepted assumption, by raising a question, a hypothesis, or need, or by extending previous knowledge in some way. The steps taken to achieve this would be:

• Step 1a -- Counter-claiming, or  [writing action = introduce an opposing viewpoint or perspective or identify a gap in prior research that you believe has weakened or undermined the prevailing argument]
• Step 1b -- Indicating a gap, or  [writing action = develop the research problem around a gap or understudied area of the literature]
• Step 1c -- Question-raising, or  [writing action = similar to gap identification, this involves presenting key questions about the consequences of gaps in prior research that will be addressed by your study. For example, one could state, “Despite prior observations of voter behavior in local elections in urban Detroit, it remains unclear why do some single mothers choose to avoid....”]
• Step 1d -- Continuing a tradition  [writing action = extend prior research to expand upon or clarify a research problem. This is often signaled with logical connecting terminology, such as, “hence,” “therefore,” “consequently,” “thus” or language that indicates a need. For example, one could state, “Consequently, these factors need to examined in more detail....” or “Evidence suggests an interesting correlation, therefore, it is desirable to survey different respondents....”]

Move 3: Occupying the Niche [the solution] The final "move" is to announce the means by which your study will contribute new knowledge or new understanding in contrast to prior research on the topic. This is also where you describe the remaining organizational structure of the paper. The steps taken to achieve this would be:

• Step 1a -- Outlining purposes, or  [writing action = answering the “So What?” question. Explain in clear language the objectives of your study]
• Step 1b -- Announcing present research [writing action = describe the purpose of your study in terms of what the research is going to do or accomplish. In the social sciences, the “So What?” question still needs to addressed]
• Step 2 -- Announcing principle findings  [writing action = present a brief, general summary of key findings written, such as, “The findings indicate a need for...,” or “The research suggests four approaches to....”]
• Step 3 -- Indicating article structure  [writing action = state how the remainder of your paper is organized]

"Introductions." The Writing Lab and The OWL. Purdue University; Atai, Mahmood Reza. “Exploring Subdisciplinary Variations and Generic Structure of Applied Linguistics Research Article Introductions Using CARS Model.” The Journal of Applied Linguistics 2 (Fall 2009): 26-51; Chanel, Dana. "Research Article Introductions in Cultural Studies: A Genre Analysis Explorationn of Rhetorical Structure." The Journal of Teaching English for Specific and Academic Purposes 2 (2014): 1-20; Coffin, Caroline and Rupert Wegerif. “How to Write a Standard Research Article.” Inspiring Academic Practice at the University of Exeter; Kayfetz, Janet. "Academic Writing Workshop." University of California, Santa Barbara, Fall 2009; Pennington, Ken. "The Introduction Section: Creating a Research Space CARS Model." Language Centre, Helsinki University of Technology, 2005; Swales, John and Christine B. Feak. Academic Writing for Graduate Students: Essential Skills and Tasks . 2nd edition. Ann Arbor, MI: University of Michigan Press, 2004; Swales, John M. Genre Analysis: English in Academic and Research Settings . New York: Cambridge University Press, 1990; Chapter 5: Beginning Work. In Writing for Peer Reviewed Journals: Strategies for Getting Published . Pat Thomson and Barbara Kamler. (New York: Routledge, 2013), pp. 93-96.

Writing Tip

Swales showed that establishing a research niche [move 2] is often signaled by specific terminology that expresses a contrasting viewpoint, a critical evaluation of gaps in the literature, or a perceived weakness in prior research. The purpose of using these words is to draw a clear distinction between perceived deficiencies in previous studies and the research you are presenting that is intended to help resolve these deficiencies. Below is a table of common words used by authors.

NOTE : You may prefer not to adopt a negative stance in your writing when placing it within the context of prior research. In such cases, an alternative approach is to utilize a neutral, contrastive statement that expresses a new perspective without giving the appearance of trying to diminish the validity of other people's research. Examples of how to take a more neutral contrasting stance can be achieved in the following ways, with A representing the findings of prior research, B representing your research problem, and X representing one or more variables that have been investigated.

• Prior research has focused primarily on A , rather than on B ...
• Prior research into A can be beneficial but to rectify X , it is important to examine B ...
• These studies have placed an emphasis in the areas of A as opposed to describing B ...
• While prior studies have examined A , it may be preferable to contemplate the impact of B ...
• After consideration of A , it is important to also distinguish B ...
• The study of A has been thorough, but changing circumstances related to X support a need for examining [or revisiting] B ...
• Although research has been devoted to A , less attention has been paid to B ...
• Earlier research offers insights into the need for A , though consideration of B would be particularly helpful to...

In each of these example statements, what follows the ellipsis is the justification for designing a study that approaches the problem in the way that contrasts with prior research but which does not devalue its ongoing contributions to current knowledge and understanding.

Dretske, Fred I. “Contrastive Statements.” The Philosophical Review 81 (October 1972): 411-437; Kayfetz, Janet. "Academic Writing Workshop." University of California, Santa Barbara, Fall 2009; Pennington, Ken. "The Introduction Section: Creating a Research Space CARS Model." Language Centre, Helsinki University of Technology, 2005; Swales, John M. Genre Analysis: English in Academic and Research Settings . New York: Cambridge University Press, 1990

• << Previous: 4. The Introduction
• Next: Background Information >>
• Last Updated: Apr 1, 2024 9:56 AM
• URL: https://libguides.usc.edu/writingguide

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 20 May 2023

A review of glass corrosion: the unique contribution of studying ancient glass to validate glass alteration models

• Roberta Zanini   ORCID: orcid.org/0000-0002-3190-9191 1 , 2   na1 ,
• Giulia Franceschin   ORCID: orcid.org/0000-0003-1817-2962 1   na1 ,
• Elti Cattaruzza   ORCID: orcid.org/0000-0003-0643-0266 2 &
• Arianna Traviglia   ORCID: orcid.org/0000-0002-4508-1540 1  

npj Materials Degradation volume  7 , Article number:  38 ( 2023 ) Cite this article

3713 Accesses

6 Citations

6 Altmetric

Metrics details

• Structure of solids and liquids

Glass has been used in widespread applications within several sectors since ancient times and it has been systematically studied under different perspectives. However, its thermodynamic properties and the variety of its compositions, several aspects related to its durability and its alteration mechanisms remain still open to debate. This literature review presents an overview of the most relevant studies on glass corrosion and the interaction between glass and the environment. The review aims to achieve two objectives. On one hand, it aims to highlight how far research on glass corrosion has come by studying model systems created in the laboratory to simulate different alteration conditions and glass compositions. On the other, it seeks to point out what are the critical aspects that still need to be investigated and how the study of ancient, altered glass can add to the results obtained in laboratory models. The review intends also to demonstrate how advanced analytical techniques commonly used to study modern and technical glass can be applied to investigate corrosion marks on ancient samples.

Similar content being viewed by others

Machine intelligence-accelerated discovery of all-natural plastic substitutes

Tianle Chen, Zhenqian Pang, … Po-Yen Chen

Environmental impact of direct lithium extraction from brines

María L. Vera, Walter R. Torres, … Victoria Flexer

Rapid biodegradation of microplastics generated from bio-based thermoplastic polyurethane

Marco N. Allemann, Marissa Tessman, … Ryan Simkovsky

Introduction

Throughout history, glass has played a central role in the production of goods for widespread use, making it a material that has been extensively studied from various perspectives. However, due to its thermodynamic properties and the vast range of existing compositions, there are still several aspects of its nature that are not fully understood. For instance, evaluating glass durability and alteration mechanisms remains a challenge, as it requires taking into account numerous factors, some of which are difficult to replicate in laboratory experiments. One of the most prominent challenges is the time required for the transformation of the glass structure, which is directly responsible for the kinetics and dynamics of the processes involved.

Technological innovation has recently introduced new tools for studying the durability of glass and shedding light on the deep connections in extremely complex materials between compositions, structure, and the surrounding environment. As an example, a recent review 1 reported about the possibility of employing mechanistic models to study aqueous glass alteration. Thanks to the availability of such novel techniques, glass alteration mechanisms and kinetics can be hypothesised using non-conventional experimental methods (simulations) and without direct experimental validations. In parallel, the long-term perspective provided by studying ancient vitreous objects can inform and underpin the predictions obtained through simulation and laboratory methods 2 , 3 , 4 , 5 , 6 .

Most of the studies performed to understand glass corrosion are based on artificial ageing experiments that are designed to monitor, step by step, the process of glass alteration and the physicochemical evolution of the glass structure 7 , 8 , 9 , 10 . Because of the great variability of the experimental conditions involved in these studies, however, results are not yet conclusive, and the glass transformation process is still only partially described in literature.

Standard approaches to the study of glass corrosion are limited in that they are valid in some specific experimental conditions, but not in others. Indeed, most of the available research focuses on the corrosion mechanisms of glass with specific compositions (lead-rich or borosilicate families), or under conditions of saturated humidity (relative humidity, RH ≥ 100%) 11 , 12 , 13 . In both cases, the atomic diffusion occurring in the material alteration processes is assisted by two key parameters, which are the presence of ionic species with high mobility and the presence of liquid water on the surface, respectively. Only an handful of published works 2 , 7 , 8 –focused on ageing under conditions of RH < 100% and on a wider variety of glass compositions, like those of the silica-soda-lime (SSL) glass family– are available nowadays in literature: they represent valuable experimental approaches to determine and separate the relative influence of different parameters on the chemical and environmental durability of glass.

In view of the above, this review focuses on the literature available on glass alteration and related structural investigations that have contributed to the comprehension of the modifications occurring in the glass network when glass degrades. In addition, an examination of the existing –but limited– literature on the analysis of ancient glass will bring to the attention of the chemistry community a vast amount of unexplored information that is relevant also for the study of contemporary glass types, to validate the prediction of glass behaviour obtained through laboratory experiments. The most advanced techniques to study alteration on heritage glass will be described with the aim of highlighting how deepening the understanding of altered ancient glass properties is key for a full understanding of the phenomenon of natural ageing of both modern and ancient glass objects.

For better clarity, the term a rchaeological glass will be used to refer to glass specimens that have aged in soil or marine environments for centuries, whereas the term historical glass will be used when speaking about glass aged under the effect of environmental moisture fluctuations (in museums or private collections). The term ancient glass is more generally used to refer to both the above mentioned cathegories.

By using a combination of comprehensive characterization of ancient samples and laboratory-based ageing experiments, researchers can obtain valuable information about glass alterations that have occurred over hundreds of years while also monitoring the alteration process step-by-step. This integrated approach offers a unique opportunity to consolidate and supplement current theories of glass corrosion and validate predictive models using real-world examples of long-term alteration. Overall, this approach provides a more complete understanding of the complex process of glass corrosion and can help guide the development of more durable glass materials.

Glass structure and its dissolution prediction: key highlights

Glass is an amorphous material that shares structural features at the atomic level with a supercooled liquid, while exhibiting mechanical properties typical of solids.

Zachariasen 14 was the first to describe the amorphous structure of glass in 1932. His studies led him to conclude that there is no long-range atomic periodicity in glass structure. He also defined the requirements for a particular oxide to exist in the vitreous state. Silicon dioxide, also known as silica, is the primary network former in both ancient as well as in modern glass.

In silica-based glass, the bonding forces involved in the formation of the glass network are those of crystalline SiO 2 , which has a tetrahedral structure with 4 oxygen atoms located at the corners of a tetrahedron and a Si atom placed in the middle of the structure 15 (Fig. 1 ). The glass network can be figured as a built-up of such tetrahedral elements, which share each corner with neighbouring tetrahedra, one per corner. The remaining corners are available to form other chemical bonds. SiO 2 is considered a primary network former, but other oxide types exist with these same properties, such as B 2 O 3 or P 2 O 5 . The addition of alkali oxides, such as Na 2 O, K 2 O, Li 2 O, to the glass network former is responsible for breaking some of the Si-O-Si bonds bridging silicon atoms and for including the alkaline cations into the glass structure. These network modifiers generate the formation of non-bridging oxygens (NBOs) in addition to the bridging oxygens (BOs) of the silica network. NBOs hold a negative charge that is locally compensated by one highly mobile alkali cation with a positive charge (i.e., Na + ). The main effect of such network modifiers is to decrease the viscosity of the glass melt, thus facilitating its workability at lower temperatures. Alkaline earth oxides like CaO are added to the glass batch as network stabilisers 16 . They connect to two NBOs and are commonly considered as lower mobility ions (i.e., Ca 2+ ) that are possibly effective in inhibiting the diffusion of other cations across the silica network, hence improving the chemical resistance of the glass (Fig. 2 ). It is now clear why 90% of commercial glass is made of a mixture of silica (as network former), sodium and calcium oxide (as network modifiers). Nevertheless, the positive influence of alkaline earths in increasing glass stability against alteration has not been fully demonstrated yet, and several works indicated that Ca 2+ and Mg 2+ ions are at least as mobile as alkalis within the hydrated layer in atmospheric conditions 17 , 18 .

Schematic representation of a tetrahedral silica unit (not to scale).

For clarity’s sake, only the BOs and NBOs helping to identify the Q n configuration in the silica network are coloured in red for BO and in yellow for NBO.

The structure of glass is not in thermodynamic equilibrium. The durability of glass depends on both the kinetic and thermodynamic stability of its oxide components. In a state of thermodynamic equilibrium, the chemical potential of the species on the glass surface and those in solution are equal, and as a result, no net mass transfer occurs. However, glass does not exist in a state of thermodynamic equilibrium and is therefore prone to undergo chemical reactions that can result in degradation over time. The understanding of the kinetic and thermodynamic stability of glass components is crucial for developing more durable and long-lasting glass materials.

An essential factor to be taken into consideration to predict glass dissolution is the knowledge of the relative concentration of bridging and non-bridging oxygen atoms 19 . The latter are bonded to only one silicon atom and their quantity within the glass network is proportional to the concentration of modifier ions.

Considering the two possible configurations for an oxygen atom (BO and NBO), the silicon atom may be found in five different tetrahedral arrangements: Q 0 , Q 1 , Q 2 , Q 3 , and Q 4 , where the subscript indicates the number of bridging oxygens (Fig. 2 ). The structure of the glass network is the result of the distribution of the rings and voids regulated by the interconnection between these different silicate tetrahedra. The size of the voids in the network controls the rate of water diffusion, which is kinetically favoured when the dimension of the voids is comparable to the diameter of the water molecule (0.28 nm). In complex glass (mixed alkali glass), modifier cations can totally or partially fill the voids, but when the material is exposed to high relative humidity conditions, these alkali ions are leached from the glass surface and replaced by hydrogen ions as part of molecular water. The ion-exchange reaction drives the hydrolysis of the glass network with kinetics depending on both the distribution of local structural units (Q n ) and the modifier content. In addition, the exchange of high-radius cations as K + from the bulk is considered to leave a bigger void in the glass network compared to the exchange of smaller cation as Na + , thus facilitating the entrance of water molecules into the deeper areas. In general, it can be said that the higher the concentration of NBO, the higher the number of ion exchange sites available and the rate of ion-exchange and network hydrolysis, following the reaction trend: Q 1  > Q 2  > Q 3  > Q 4 19 , 20 . For this reason, in the discussion about the kinetics of the processes of ionic exchange and hydrolysis reactions it is essential to take into consideration the chemical composition of glass.

Taking into account what has been said above, knowing the chemical composition of complex glass and its Q n concentration and distribution is fundamental for the appreciation of its chemical stability and leaching resistance, so as to adopt an adequate preventive conservation strategy. Nevertheless, the glass reactivity does not depend on Q n species only. Understanding the correlation between local structural features of the glass and the activation energies of individual bonds is also crucial to predict the dissolution mechanism of the glass network. Potential Mean Force (PMF) calculations estimated the activation barrier for Si dissolution in presence of aluminium (Al). They revealed that Al is easily dissociated from glass network, but Si dissociation is hindered when Al is present as a second neighbour 21 . As a result, Al causes opposing effects on glass durability if added at low and high concentrations: the addition of Al in small concentration increases the durability by reinforcing the strength of Si and increasing the polymerisation of the glass network, while at high Al concentration, the preferential release of Al results in the weakening of the silicate network. This predicting method can be extended to understand the role of Na, Mg, or B in more complex glass compositions.

Since the prediction of dissolution of glass network is a complicated topic of research, many recent works 22 , 23 , 24 reported the use of machine learning based approaches to account for the percentage of bridging oxygen species, network connectivity, average ring size, as well as the composition modification due to the preferential release of modifier cations during the incongruent dissolution.

Several glass studies demonstrated the usefulness of Raman spectroscopy as an analytical technique to discriminate the characteristic vibrational modes of each Q n configuration 25 , 26 , 27 , 28 . Through the deconvolution of the Raman bands typically associated to the glass network it is possible to determine the single Q n distribution and associate the variation of the area of the related Q n band with the chemical composition of the sample analysed 29 , 30 . When using this analytical approach, Raman spectroscopy can be adopted as a technique to distinguish a stable glass from an unstable one by means of the rigorous deconvolution of the vibrational bands of the glass network 29 , 31 , 32 .

Nowadays, an analytical protocol that combines the potentialities of Raman spectroscopy described above with the advantages of using portable instrumentation to evaluate the chemical stability of glass, to predict the glass network dissolution for preventive conservation purposes, or even to establish the suitability of glass as storage material for nuclear waste is not available. Moreover, the in-depth spatial resolution of Raman spectroscopy is inadequate if one wants to determine the layered structures on an altered surface, whose features, in fact, sometimes vary on the nanometre scale. The association of other complementary analytical techniques would be ideal to optimise the reliability of Raman spectroscopy results.

X-ray absorption near edge structure (XANES) spectroscopy at the Si K-edge is another interesting technique that has been used to study the polymerisation degree of SiO 4 tetrahedra silica glass. In XANES spectra, position and structure of the absorption edge are largely determined by the charge of the absorber atom and by geometry of the first coordination shell, which depends on the coordination of the nearest neighbour atoms, the degree of polymerisation, and the presence of network modifiers and network substitutes 33 . A study conducted on silicate glass reported that, with the increase in the polymerisation degree of the silica tetrahedra, the Si K-edge shifts towards higher energy, while it shifts towards lower energy when Si is substituted for another network former (Al) 34 .

The chemical changes occurring at the surface of a corroding glass often cause an alteration of the local environment of metal atoms, especially of the metal-oxygen pair distribution. These characteristics have been measured using conventional and glancing angle extended X-ray absorption fine structure (EXAFS) techniques, able to give information about number and distance of the atoms surrounding the absorber one. Examining the modifiers distribution in the vicinity of the surface gives important hints on how the surface is modified as corrosion advances 35 .

As transition metal cations are generally of interest for the application of X-ray absorption techniques, their study can be easily exploited to monitor the decay of ancient glass. The modifications of the chemical environment of chromophore species (i.e., transition metal cations) can be recorded using both XANES and EXAFS techniques by monitoring the spectra of a selected metal species in an altered glass sample. By means of this method, it was possible to establish a relationship between the oxidation state of Fe and Cu cations during glass decay. Whereas, the Mn oxidation state was not directly correlated with the glass decay of the samples studied 36 .

XANES and EXAFS analysis are not common and straightforward techniques, since they require the preparation of tailored samples from analysed objects (which is not always possible when dealing with cultural items) and the access to a synchrotron facility. However, this type of analysis provides important information to help the identification of structural and chemical changes in altered glass samples when investigating the molecular changes around cations with high field strength and well-defined short-range order.

In the following section, the main models of glass corrosion will be explored: they are based on the observation of the interactions between glass structure and external environment and on the different mechanisms in place during the process of glass alteration.

Exploring the mechanisms of glass alteration through the different existing models

The terminology used in the published works is imprecise. The terms corrosion , alteration , degradation , and deterioration are often found to be frequently used interchangeably as synonyms, despite having different shades of meaning. For the sake of clarity, all these terms will be used here as synonyms when talking about the phenomena that induce a change in glass physicochemical properties, regardless of the intrinsic or environmental factors that have determined such change. The term dissolution will be, instead, used to refer only to the rupture of the Si-O covalent bonds and the breakup of the structural silica network, and the term leaching will be used to identify the step of degradation that consists in the loss of alkali and alkaline earth metals ionically bonded to the silica network that precedes the network dissolution 37 . Even if leaching is often associated with the initial steps of glass degradation, the loss of alkalis can be a secondary phenomenon in specific conditions of high temperature (RH < 100%) and unstable glass compositions 16 . In other particular conditions, the degradation process occurs without any loss of alkalis, because the water penetrated into the glass network is unable to solvate them 8 , 9 , 18 .

The interaction between water and glass can activate two different degradation phenomena, i.e., leaching and network brakeage, mainly depending on the pH of the solution in contact with the surface. During leaching, the aqueous solution in contact with the surface typically has pH<9 38 , 39 . In this condition (pH<9), ion exchange that involves alkalis (Na, K) and alkaline earth metals (Ca, Mg) occurs, forming ionic bonds with the oxygen of the glass network and the H + ion from the aqueous solution 16 . This is a diffusive phenomenon and the thickness of the glass region involved in the reaction (indicatively a few microns) depends on the glass composition and on the time and temperature of exposition. The altered layers that form on the surface can act as a diffusion barrier to further extraction, even if hazardous cracks that allow the penetration of water molecules into the pristine glass may form. The alteration due to leaching does not affect the Si atoms: the network distribution does not change, only the Si-O-M bonds do.

On the other hand, the ions interdiffusion during leaching leads to a pH increase (eventually above 9) due to the formation of Na + OH - species in solution and Si-OH acid from the reaction between Si-O-Na and H 2 O. An alkaline environment results in more aggressive attacks to the glass network, since it promotes the dissolution of the Si–O bonds 40 . The reaction with the hydroxyl ions (OH – ) breaks the Si-O–Si bonds and silanol groups Si–OH are formed.

The sites left free by the leaching of cations from the glass surface can be easily filled by hydrogen ions, which have small ionic radius. The hydrolysis process induces the introduction of H 2 O molecules and OH - ions into the opened silica structure, thus increasing the rate of the hydration process and the ion exchange.

The interactions and reactions that occur between aqueous solution and glass have been over the years the subject matter of extensive research focusing on the mechanism involved in this process at different scales of observation, research that has been key to support the formulation of the classic theory of alteration and Interfacial Dissolution Precipitation model for dissolution of vitreous materials.

The Classic Inter-Diffusion (CID) model of glass corrosion is based on diffusion-controlled hydrolysis and ion exchange reactions 41 , which lead to the formation of structurally and chemically distinct zones (Fig. 3 ).

The latter consists of a hydrated, cation-depleted layer resulting from a selective cation release. Courtesy of Gin et al. 48 .

The classic theory of glass corrosion is underpinned by different formulated models of ion-exchange that take into consideration the effects of the preferential dissolution of more soluble cations during the initial part of the leaching process 42 , 43 .

Many experimental and theoretical results obtained from observation in liquid conditions (and not in unsaturated humidity ones) have reported that the leaching mechanism involves the preferential release of alkali and alkali-earth ions rather than that of network formers, such as Si or Al ions, with a consequent formation of alumina/silicate-rich layer on the glass surface 38 , 43 .

In general, the concept of preferential leaching is based on the thermodynamic and kinetic stability of the different glass modifiers. At lower temperatures and for ions with the same charge, the diffusion of larger ions (for example Ba 2+ or Ca 2+ ) becomes energetically unfavoured, while the smaller ions (for example Mg 2+ ) can move more easily through the glass network 44 . In any case, double-charged ions usually show less diffusivity in SSL glass than single-charged ones, mainly because of the marked effect of the very intense local electric fields acting on them. If the leaching mechanism proceeds, the solution becomes richer and richer in OH – and its pH increases favouring the dissolution of silica through the break of O-Si-O network. Preferential leaching supports the selective removal of specific cations (non-stoichiometric release) and designs a theory about incongruent dissolution of glass that explains the formation of altered surface layer.

The technological developments of the last decades have made it possible to reveal increasingly more detailed evidence on the process that controls glass corrosion 11 . The chemical reactions proposed in the general mechanism of glass corrosion (hydration, hydrolysis, and interdiffusion) are still considered valid in the most recent studies. However, over the last years the attention of the scientific community has been focusing more on understanding how these reactions evolve kinetically and thermodynamically during the alteration process, and how they influence the structural and microstructural properties of the alteration layer at the atomic scale 1 , 45 .

In 2015, a nanometre-scale study 46 of glass corrosion was performed using a combination of high mass and spatial resolution techniques, proposing a revised theory of glass corrosion called the Interfacial Dissolution-Reprecipitation (IDP) model. The IDP model is based on the congruent dissolution of silicate glass coupled in space and time with the reprecipitation of amorphous spherical silica aggregates of variable size. In opposition to the traditional glass alteration model, this recent theory supports the stoichiometric dissolution of glass without interdiffusion-controlled ion-exchange mechanisms at the glass reaction front 46 .

Hellmann et al. validated this model through the study of artificially aged borosilicate glass altered at 50 °C in deionised water, using a unique combination of techniques with high spatial and mass resolution 46 . By following the mobility of the major constituent elements of complex borosilicate glass, an identical release behaviour was noticed for modifier and former ions, regardless of their charge. These results validate two processes at the basis of this novel corrosion mechanism, i.e., the stoichiometric release of all the glass elements and the precipitation of amorphous silica with the formation of an altered surface layer. Furthermore, the interface between pristine glass and altered zone was demonstrated to be chemically and structurally well defined, with elemental gradients in the nano- to sub-nano-metric range.

A schematic representation of the IDP model is presented in Fig. 4 according to the results of oxygen and silicon isotope tracer experiments in ternary borosilicate glass 47 .

a Initial congruent dissolution of glass is the first step occurring at the glass-water interface. This stage continues until the amorphous silica solution is supersaturated and etching pits are formed on the surface; ( b ) Si-rich interfacial solution layer is formed depending on the ratio of silica in solution to silica released during glass dissolution. Under this condition, the localised saturation of silica in solution promotes condensation and nucleation reactions that lead to the polymerisation of monomeric silica to form dimers and oligomers; ( c ) Precipitation of silica in the form of spheres on the dissolving glass surface occurs after silica supersaturation and nucleation in the solution; ( d ) Formation of SAL (Surface Alteration Layer) composed of altered amorphous silica proceeds along with congruent glass dissolution and further diffusion of dissolved species through the developing SAL; ( e ) Diffusive transport of water and dissolved species through the corrosion rim continues depending on the porosity of the SAL, i.e., higher porosity values correspond to higher diffusion rates; ( f ) Precipitation of secondary minerals like zeolites and clays sometimes occurs within and at the silica surface.

As described above, the IDP model entails the formation of alteration layer through the precipitation of hydrated species from the thin film of water to the hydrolysis front, with high degree of liberty to reorganise (Fig. 5 ). This leads to a sharp concentration profile of highly soluble cations and an interstitial water layer that would allow an easy separation of the altered layer from the pristine glass 48 .

Courtesy of Gin et al. 48 .

The rate of the reactions involved in the glass corrosion (ion-exchange, hydration, and dissolution) depends on factors such as glass composition, temperature, and pH. All of them may occur simultaneously during process of alteration in contact with liquid water and can be rate-limiting in function of the experimental conditions. In conditions of unsaturated humidity, studies demonstrating the existence of a rate drop followed by a residual regime are lacking. For these reasons, both the two models presented in literature (CID and IDP) are not able to describe universally the distribution of mobile ions and hydrous species inside the alteration layer.

A work published in 2017 49 reported the in-depth characterisation using atom probe tomography (ATP), transmission electron microscopy (TEM), and time-of-flight secondary ion mass spectrometry (ToF-SIMS) of the alteration layer formed under close-to-saturation conditions. The results revealed the presence of an alteration layer with a more complex structure, made of three different sub-layers (Fig. 6 ): (i) close to the pristine glass, a thin hydrated layer containing all the glass components, (ii) moving towards the surface, a passivating layer with constant concentration of glass formers (Si, Al) and decreasing concentration of modifiers (Na, Ca) which is delimitated by a rough interface where alkaline and alkaline earths are preferentially leached out, and (iii) an external nanometric layer where Si undergoes hydrolysis and condensation reactions 49 . These results contradict the IDP model of glass corrosion recently designed, which highlights that many gaps are still present in the explanation of the mechanism of glass alteration.

Courtesy of Gin et al. 49 .

The authors of this review would like to emphasize that the universal application of the IDP model and other intermediate models to all silicate glass types is still an open question. This is because most recent research has focused solely on the artificial alteration of borosilicate glass, and the applicability of these models to other types of silicate glass remains uncertain. The characterisation of synthetic and geological glass altered over long time periods can be key to understand both the complex mechanisms responsible of the long-term transformation of glass network and the physicochemical features of the material transformed by the alteration. Multi analytical and high spatial and mass resolution investigation performed on surface of ancient glass provide clues about the characteristics of the altered layers and about how they reorganised their internal structure during long time periods. Instead, looking at the interface between the alteration layer and the pristine glass provides information about the mechanisms of glass alteration 2 . In this way, this approach promises to be efficient in validating the above mentioned debated kinetic models. The knowledge of the long-term behaviour of glass structure is also pivotal to predict the network dissolution in burial conditions in nuclear waste management studies 4 , 50 .

Evidence of degradation on ancient glass

Archaeological glass shows multiple and clearly visible symptoms of deterioration that can help identify well-distinguished classes of glass alteration 51 depending on the conditions it aged in (i.e., in soil, underwater, under extreme environmental conditions) for several centuries. Dulling, iridescence, weeping, pitting, discolouration and cracking of the surface affecting the specimens are common phenomena that can be observed on glass that has been recovered from archaeological excavation 52 . The formation of one or the other of them depends both on the physicochemical properties of the glass and on the environmental factors it has been exposed to. More than one of the manifestations of alteration can be found in a single glass object, thus making it sometimes difficult to identify the most appropriate strategy for conservation and/or stabilisation.

The atmospheric deterioration of glass appears markedly different from the degradation that occurs when glass objects have been buried for centuries, for example under soil, and it is rarely observed on this type of glass that has remained interred up to discovery and recovery. Crizzling , also known as glass sickness or glass disease, has been identified as the major alteration symptom for glass objects stored in museums and in private collections.

Alteration of ancient glass in archaeological stratigraphic contexts

The term dulling is used to refer to the loss of clarity and transparency typically observed in ancient glass and caused by the formation of layers of alteration products on the glass surface 52 .

As discussed above in the section dedicated to the general mechanism of glass corrosion (Section “Exploring the mechanisms of glass alteration through the different existing models”), in presence of neutral or acidic conditions, elements like alkalis are typically leached out from the first glass layers onto the surface. These leached species, reacting with humidity and moisture of the environment, tend to form corrosion products (like salts) that build up on the object’s surface and determine at first the loss of the original clarity 53 . In addition, at advanced stage of alteration, also hydrated silica (silica-gel) particles can also reprecipitate on the glass surface, leading to the formation of thicker alteration layers and causing an additional loss of glass transparency and the appearance of translucency 54 . This phenomenon is due to a combination of effects occurring between the local presence of water and the composition of glass, which determines the diffusion of ionic species from the first atomic layers under the surface to the environment and the consequent reprecipitation of hydrated silica and other alkali-derived compounds. The extent of the visual effect is much more considerable as the ion exchange proceeds and the thickness of the deposited layers on surface increases.

In more advanced stages, dulling can lead to the formation of iridescence patinas (Fig. 7 ) that may eventually detach in the form of crusts from the original glass substrate. In 1863 Brewster 55 demonstrated that this iridescent effect is due to the diffraction of incident light from layers of weathering products containing metal oxides formed after ion leaching. The rays of light are reflected from thin alternating layers of air and weathered glass crusts. These densely overlapping layers gradually penetrate deeper into the glass and they eventually change in colour towards darker hues 56 . The cationic species leached from the glass are often prone to reacting with the anionic species derived from reaction between OH - (especially in basic conditions) and atmospheric acid gases, thus forming salts with hygroscopic properties on the surface. This generates a phenomenon called weeping , which was first described by Organ in 1956. Weeping can lead to the formation of crystals or solutions of salts, depending on their deliquescence relative humidity 57 .

Images collected with Olympus BX43F optical microscope, x10 magnification. A detail of the indented rim where the overlapping of thin layers of altered patina is clearly visible (right panel).

The alteration phenomenon called pitting , is described as micro, small or large based on pit size, and it can occur simultaneously at different individual sites that later merge into interconnected complex structures producing an altered top layer, which causes the loss of glass transparency 52 .

Contrary to dulling, pitting is a visible mark of the weathering process, which occurs in alkaline solution 58 and commonly found on excavated glass. As described in Section “Exploring the mechanisms of glass alteration through the different existing models”, during the alteration process in alkaline solution, the prevailing deterioration mechanism is the dissolution of the silica network through the breaking of Si-O-Si bonds. Subsequently, the prolonged presence of a layer of moisture on the surface of glass causes an increase in the pH of the attacking solution, and ultimately pits are formed as a result of local dissolution of the silica network 52 .

A model that explains the formation of altered pits was recently developed by observing the decay process of different silicate glasses in river and marine aquatic environments 59 . The experimental results showed that the alteration of SSL glass is characterised by a two-step mechanism. The first step, called “hydration period” , is short and causes the formation of isolate fissures, while the second step, called “pit development period” , involves the creation of basic species (OH - ) during the dealkalinisation process that progressively break the silica network, thus widening the fissures to form pits. According to the results of this work 59 , the formation of pits is correlated to a dynamic loss of mass, i.e., the slow rate of the first step of hydration allows the diffusion of solution and the consequent basic attack inside the fissures, causing local network dissolution.

Figure 8 shows the surface of a Roman archaeological sample affected by pitting.

Image collected with Olympus BX43F optical microscope, x10 magnification.

Discolouration can be often seen on archaeological glass surface in combination with other types of weathering phenomena described above. It is closely related to darkening, which occurs when the oxidation of specific leached ions, such as iron, manganese, and copper, changes the colour of the weathering crusts, or to the production of hydrogen sulphide by sulphur-reducing bacteria in anaerobic environment and the formation of lead sulphide 53 . The latter case occurs only when the glass contains a high concentration of Pb oxide, and it is buried under anaerobic condition. In other cases, the presence of manganese and iron causes the darkening of glass with the formation of brownish pits 60 , as those visible in Fig. 9 . Ancient glass contains these elements in the form of impurities present in raw materials (sand and wood ash) or as a result of their deliberate addition as chromophores or/and decolourant agents in the form of minerals (i.e., pyrolusite) 61 .

In the case of archaeological glass that has been interred for centuries, cracks may be present on the surface of the glass fragments: these are caused by the shrinkage of the alkali-deficient layer due to temperature and humidity changes. Typically, these cracks are filled with mineralised material, likely originating from the soil in which it was buried. The mechanism of their formation is currently being studied since it has not yet been fully understood.

Alteration of glass in atmospheric conditions

Crizzling has been identified as the appearance of minute cracks on the surface of glass (Fig. 10 ) developing over time. These cracks penetrate deeper in the body of the object, ultimately resulting in its physical collapse. This phenomenon is due to two main factors: the unstable composition and the storage in fluctuating humidity environments 62 . Better storage conditions can slow down, but not stop the deterioration, because the role of glass composition remains a key factor in the evolution of the crizzling alteration 63 .

Object exposed at the Museo Civico di Modena (Italy). Picture by Renaud Bernadet.

Early publications reported the chemical effects causing crizzling through experiments that reproduced the condition of glass alteration in the laboratory. The results showed that crizzling is mostly associated with glass compositions characterised by high alkali and low CaO contents, and/or a high K/Ca ratio 64 .

In 1975 Brill 63 first used the term crizzled to describe glass with a decrease in its transparency due to the formation of fine cracks on the surface. He noted that certain glass objects that were in contact with water for centuries do not exhibit a high degree of degradation; however, once they are exposed to museum storage conditions (light, low RH and temperature), they display the formation of crizzling. This alteration mechanism is due to the dehydration of the glass surface, i.e., the low RH in museum display cases (15-20 %) causes a loss (up to twenty percent in weight) of the water that penetrated the gel layer of altered glass, bringing on a significant loss of volume in the gel layer itself which ultimately results in the cracking of the glass surface 62 , 63 . In general, the cracking of the hydrated gel layer formed on the surface of unstable glass can be attributed to several factors, including the dehydration of the gel layer itself - as mentioned above - the leaching process, which can lead to network contraction after the replacement of larger alkali ions (Na + ) by smaller hydrogen ions (H + ), and the different coefficient of expansion of the bulk glass and the gel layer 16 .

The guidelines of the Corning Museum of Glass 52 describes the process of crizzling indicating five stages. In the first stage (Initial stage) the glass has a blurred appearance due to the presence of leached alkali on the surface. During this phase, it is still possible to wash the surface and the glass can return to its original appearance. Conversely, in the second stage (Incipient crizzling) the haziness remains also after washing and the glass surface exhibits fine cracks like tiny silvery lines. Cracking progresses in stages three (Full-blow crizzling) and four (Advanced crizzling) until it gets to the deepest regions, leading to the loss of small fragments. Eventually, crizzling is so deep that the glass loses its structural integrity, even without any external contribution (Fragmentation stage).

Often crizzled glass has a pinkish hue. When alkaline leaching occurs and the glass structure is open, the manganese ions present in the surface cracks oxidise, yielding a pink colour 62 . This phenomenon is more evident in ancient glass which contains manganese as a decolourant.

To limit the evolution of the crizzling process, preventive conservation is an essential strategy for the safety of museum glass objects. The Corning guidelines set the optimal RH range for glass conservation between 35 to 65%, however crizzled glass or glass with a particularly fragile composition require specific individually controlled cases with a stable RH condition in the range of 40–50% inside 65 .

In addition to preventive conservation, the development of an innovative and compatible consolidation treatment for the protection or repair of cracked glass is an open challenge for the scientific community of cultural heritage conservation.

Intrinsic and extrinsic parameters influencing glass deterioration

As already mentioned, atmospheric conditions, such as temperature, the pH of the environment, salts and ions concentration, relative pressure under burial or marine conditions, and the presence of water in liquid form (RH ≥ 100%) or vapour (RH < 100%), strongly influence the kinetic of the glass surface alteration and its chemical transformation.

Many published papers of archaeological interest use the term weathering to refer to the typical degradation process that affects archaeological glass that has been exposed to particularly unfavourable environmental conditions (especially in burial and underwater contexts) 52 . Weathering is a degradation process occurring through contact with water in the environment, both in the vapour and in the liquid state, whereas the term atmospheric deterioration is used to describe glass that aged under the effect of water in the form of moisture (especially in protected environments like museum display cases), which means in the form of vapour state interaction.

The study of extrinsic factors, that are closely linked to the environmental conditions acting during the alteration process over centuries requires the development of appropriate artificial ageing protocols, which allow the modelling of the phenomenon as a function of these parameters. Real cases of glass degradation are the result of the combined actions of the intrinsic and extrinsic factors mentioned above, which generate entangled mechanisms of ionic interdiffusion from the glass network to the environment and vice versa. From an experimental point of view, studying this interconnected process of ionic interdiffusion and the formation and growth of novel phases implies the need of relying on simplified model systems to investigate the effect of specific variables to the detriment of others, which are kept constant.

The study of the effect of the various factors on glass degradation has attracted the curiosity of researchers since the beginning of the XX century. Already in 1925, G. W. Morey 66 stated that the subject was still in an empirical state, despite the considerable number of works carried out. At that time, to understand the effect of water on the alteration process many experiments were conducted by varying temperature, the pH of the environment and glass composition.

The following paragraphs describe the effect of various parameters that have a primary influence on the process of glass alteration. The studies there cited use ancient glass samples as evidence of the long-term effect of these parameters.

Effect of glass composition

Many ancient finds of SSL glass are macroscopically preserved intact in their shape, despite the physicochemical alterations caused by the burial environment over the centuries. Currently, 90% of globally manufactured glass is still based on the SSL composition, which has been kept largely consistent over the centuries except for a few modifications introduced at the beginning of the 20 th century to enhance chemical durability and resistance to devitrification 67 . Besides SSL glass, many other types of ancient glass exist, such as potash lime glass (K 2 O – CaO – SiO 2 ), lead silicate glass (PbO – SiO 2 ), or potash lead silica glass (K 2 O – PbO – SiO 2 ) 68 . The optimal preservation of certain ancient samples and the complete collapse of others is the result of a complex interplay between their intrinsic material properties and the extrinsic factors acting on them.

Intrinsically, glass physicochemical properties play a significant role in determining its degradation behaviour. Such properties typically correspond to the chemical composition of glass, the nature of its surface, the presence of impurities, inclusions, inhomogeneity, and phase separations. In particular, the concentration of silica, alkali (soda, potash), stabiliser (lime, lead), as well as the inclusions of trace elements and additives like metal oxides, added into glass as chromophores, opacifiers and decolourants all strongly affect material durability 53 . Small variations in the concentration of these components determine strong variations in glass durability. Diffusion through the leached layer is more likely to occur for smaller ions, such as Na, Mg, Li, rather than for larger ones, i.e., Ca or Ba. Although all types of alkaline silicate glass are susceptible to weathering degradation, from a thermodynamic point of view stability increases as in the following: K 2 SiO 3  < Na 2 SiO 3  < Li 2 SiO 3 38 .

Silica rich glass, such as Roman SSL glass, is more durable than poor silica glass, like medieval glass 69 . In addition, K + , which is contained in medieval glass as a monovalent cation modifier, is more susceptible to leaching out from the deeper region of the glass network during weathering alteration than bivalent cations such as Ca 2+ , which are present in Roman glass 70 , 71 . This behaviour is due to the bivalent cations forming stronger bonds with non-bridging oxygen, as previously mentioned. The preferential leaching of K + over Ca 2+ during the weathering process was also confirmed by an experimental study that investigated the weathering phenomena on naturally weathered potash-lime-silica-glass 72 . This behaviour of K + cations was also observed during leaching experiments in aqueous acidic solution.

The study of the varieties of composition of ancient glass and their resistance in RH > 100% have made it possible to identify the presence of compositions that are more chemically stable than others and to define general conditions to discriminate between stable and unstable glass 73 .

In 1978 74 , Hench systematically studied the surface of glass exposed to pure water and distinguished between 6 main types of surfaces with increasing inclination to deterioration in relation to their composition. The surface layers of the different glass types may have protective or non-protective properties for the glass substrate, depending on the capacity of reducing ion leaching and glass dissolution. In this work Hench distinguished stable and unstable glass based on the different compositions of the alteration products that form the first surface layers when glass is altered in liquid conditions. However, the protective character of hydrated layer has not been demonstrated in unsaturated humidity conditions yet; and, moreover, a work of Sessegolo et al . 75 studied medieval stained-glass windows in unsaturated conditions with isotopic water experiment and demonstrated that the alteration layer is not protective against vapour transport and interdiffusion.

A further way to discriminate a stable glass from an unstable one could be to use the ternary diagram of Fig. 11 , which was formulated in 1975 by Newton et al . 76 with a view to help predict the weathering behaviour of different types of glass. Plotting the concentration (mol. %) of network stabilisers (RO), network modifiers (R 2 O), and silica (SiO 2 ) determines the chemical stability of a given glass composition, i.e., highly durable glasses are placed near the centre. This diagnostic model may work well when binary or ternary glass is considered, however complications may occur when classifying ancient glass, which has a more complex composition, because it may be necessary to consider the combination of multiple formers (SiO 2 ad Al 2 O 3 ), monovalent oxides (R 2 O) and divalent oxides (RO).

R 2 O represent the content of network stabilisers (monovalent oxides), RO the content of network modifiers (divalent oxides), and SiO 2 the content of silica.

By using electron microprobe analysis (EMPA) and hydration-dehydration experiments on ancient glass Brill observed that a deficit of stabiliser (CaO content less than 4 wt%) combined with an excess of alkali (over 20 wt%) in SSL glass easily leads to an extensive surface deterioration known as crizzling 63 . He defined as unstable glass the one with total alkali oxide concentration over 20%; this composition range determines silica network configurations that are open enough to facilitate the migration of monovalent cations. In line with these findings, other studies reported that an increase in the CaO to SiO 2 ratio increases glass stability, while a concentration of calcium oxide over 15 wt% entails rapid glass instability 38 . Recently, a comparison in terms of glass chemical stability was made using artificial mock-ups with different glass compositions which were exposed to high humidity environment and different levels of formic acid 77 . The results showed that glass with a higher content of stabilisers exhibits greater stability, especially glass with calcium.

The presence of potash as a modifier and the low silica content in the composition make glass particularly fragile and susceptible to chemical alteration. Numerous works reported the considerable damage that affects medieval glass windows with Si-K-Ca-based composition, which is considered one of the most unstable 78 , 79 , 80 . Typical alteration marks are mainly pitting 81 or the formation of a corrosion crust 71 on the surface due to the combined attack of water and increased pollution in the air. The weathering crust is often very heterogeneous and fractured because of the wetting-drying cycle conditions and generally consists of calcite, gypsum and/or syngenite formed by the reaction between the alkaline and alkaline-earth elements released during ion-exchange and sulphur dioxide in the atmosphere 6 .

Effect of minor components in glass composition

In addition to the major elements discussed above, other minor elements may be present in the composition of ancient glass as decolorising, colorising or impurities of the raw materials used for their production. These minor elements can in turn contribute to the alteration process of the glass itself by giving rise to distinctive corrosion marks.

Several authors have remarked that manganese that accumulated between the altered layers may have originated from the burial environment 82 , 83 . Dark deposits have been detected inside the dealkalinisation layer on the surface of Roman glass samples 84 . Secondary electron images have uncovered that these deposits are formed from the interconnection of spherical particles of about 2 µm in size, with iron and manganese oxides as their major components. These studies clarify that during the leaching process Fe (II) and Mn (II) ions are hydrated and oxidised, giving rise to the formation of dark amorphous products that precipitate into the pores of the leached silica film.

It is well-known that glass technologists used to add Mn, as well as Sb, to the glass melt as a decolorising agent, but it is also known that, in this type of glass, the appearance of brownish areas on the altered surface is due to an oxidation process of Mn(II) to higher oxidation states. It is generally observed that in the dark areas Mn is present in +IV oxidation state 85 , 86 . Nevertheless, a recent work performed using synchrotron radiation X-ray absorption spectroscopy (XAS) analysis on historical stained-glass windows demonstrated that the most extended brown altered areas mainly contain Mn mainly under a +III oxidation state 87 .

Through the analysis of 14th–17th century window glass, Schalm et al. 82 concluded that the formation of Mn-rich inclusions takes place simultaneously with the growing of leached layers settling along their interfaces and their concentration is mainly caused by the environment (soil) in which the glass was buried for several centuries.

Although brown/black staining has mostly been attributed to manganese compounds, iron (and titanium) compounds were often reported in association with the manganese 53 . Other elements which are often present in historical glass network as impurities can improve the chemical stability of glass. For example, low amounts of ZrO 2 (2 %wt) increase the acid and alkaline durability of glass 88 , and a hydrated ZrO 2 surface can act as a barrier to further dissolution of other ionic species. More recently, it was demonstrated that the substituting an insoluble oxides like zirconia to a fraction of silica slows the glass dissolution kinetics, but prevents the alteration gel reorganisation inhibiting the pore closure mechanisms and leading to greater degree of corrosion 89 .

These results show that it is essential to know the exact composition of an ancient object to determine a range of the aging behaviour that it is prone to. To do that, it is important to also consider the presence of elements in lower or trace concentrations to fully describe the corrosion phenomenon and formulate models that are closer to real cases of ancient glass alteration.

Effect of liquid vs atmospheric water

Water was observed to be the primary environmental ageing factor that causes glass deterioration by Lavoisier already in the early 1770s. Lavoisier also indicated two different mechanisms of water penetration into the glass network, i.e., through network voids between oxygen atoms in molecular form, and by hydrolysis and condensation reactions with the metal-oxygen bonds 19 .

The alteration of silicate glass differs when it occurs in liquid-phase or in vapour-phase regime. In contact with water the glass surface undergoes chemical attack through ion-interdiffusion and hydrolysis of metal-oxygen bonds 6 . In vapour conditions (RH < 100%) the hydration process does not release elements into the fluid, but it involves a redistribution of elements in the alteration layer (expect for volatised elements) 90 . The formed hydrated layer has chemical composition and porosity that are different from those obtained in liquid conditions, thus resulting in the glass durability properties specific for liquid or vapour-phase regimes. Even if the molecular process is the same, the interplay between the intrinsic and extrinsic variable changes affecting the macroscopic transformation of the material 45 .

During the phenomenon of water diffusion, no matter in which RH condition, the glass surface is covered by different monolayers of water, depending on the water partial pressure in the atmosphere. As soon as the silica network of a freshly made glass encounters moisture from the environment the chemisorption of water molecules starts from the first layers of the surface, which is consequentially more subjected to physical and chemical transformation. Water is chemisorbed on the glass surface and a thin film of water of few molecular layers builds up onto the chemisorbed layer, depending on RH condition.

One of the first characterisations of altered glass surface was conducted by Hench in 1975 56 , using a combination of analytical techniques that enables to obtain physicochemical information from different depths of the altered glass surface. He systematically studied the glass surface altered in liquid water, using techniques like Auger electron spectroscopy (AES), infrared reflection spectroscopy (IRRS), and electron microprobe (EMP), and distinguished 6 main types of surfaces with increasing inclination to deterioration (Table 1 ). Of these, only Type I is stable and shows no main differences with bulk glass. The other types are considered unstable and prone to deterioration over time. Type II and type IV are of interest for historical deterioration. Type II is formed when the glass contains a high level of network formers, and it involves the formation of a silica-rich layer on the surface that acts as a protective film, preventing the leaching of alkali ions and the rupture of the glass network (Si-O-Si). On the other hand, no protective film is formed for Type IV and so the leaching of alkali can proceed. This latter situation is typical of unstable glass.

According to Hench’s classification, ancient glass has a surface that corresponds to Type II and Type IV surfaces, which are characterised respectively by the presence of silica-rich surface protective layers (when the concentration of network formers is high enough) and the presence of non-protective surface layers that allow the alkali leaching to proceed (when the soda to silica ratio is high enough) 56 .

In the case of discontinuous contact with aqueous media, which is typical of the atmospheric alteration, the formation of laminated structure on the altered glass surface has been explained by the intermittent water supply 6 . Through the observation of ancient glass surface by TEM, a sequence of bands of different thickness has been recently distinguished 91 , i.e., thinner bands called lamellae (20-50 nm) and thicker ones called laminations (0.1-4 µm), which consist in groups of laminae with the same orientation. The amorphous laminae, which are depleted of alkaline ions, are formed from the local rearrangement of glass elements resulting from the repetition of several cycles of interdiffusion and glass dissolution processes. Moreover, cracks are developed perpendicularly and parallelly to the laminae and laminations. Based on this study, two main processes take place within the cracks, i.e., the migration of atmospheric solutions deeper into the bulk glass, thus moving the alteration front further, and the precipitation of secondary mineral phases which favours the mechanic separation and the loss of glass fragments.

In 2016, a study of the altered layers observed in ancient glass proposed a model to explain the formation process of this laminated degradation 92 . Thanks to optical microscope, field emission-scanning electron microscopy (FE-SEM) and energy dispersive X-ray spectroscopy (EDX) analyses, it was possible to describe surface lamellae as an alternation of random amorphous silica nanoparticles with different packing densities and with thickness between 0.1 and 10 µm. Moreover, the growth of nanosized silica particles on the surface of the altered glass was observed to be one of the by-products of the leaching process of glass components that occurs under alkaline conditions.

While glass alteration in contact with water in a liquid state was widely studied during the 1900s, testing approaches in unsaturated atmosphere have been seldom explored until more recently 93 . In 2020, Majerus et al. 7 published an overview of the experimental protocols for glass alteration tests in unsaturated atmosphere. All the tests reported were conducted under RH < 100% condition and, when a saline solution was present, their configuration ensured that the samples were never in contact with it. These results obtained under RH < 100% suggested that the glass alteration mechanism was different from the one obtained in contact with liquid water (RH > 100%). In contrast to saturated humidity conditions, in unsaturated environments, the hierarchical order of glass alteration processes involves network hydrolysis first and interdiffusion second, particularly for glass with compositions classified as unstable.

Alloteau et al. recently demonstrated that in water vapour conditions, the process of glass hydration is independent from dealkalinisation: at higher temperature (80 °C) hydrolysis predominates over diffusion processes and solvation, whilst at lower temperature the two processes proceed together in parallel. To state that, they performed ageing test in static conditions (no T and RH cycling) at controlled temperature and humidity (RH 85%) on three different types of unstable historical glass (soda-lime silicate from antiquity, mixed alkali silicate from middle-age/renaissance and potassium silicate from XVI-XVIII century). The conditions were set to avoid any liquid water flow on the samples during tests.

Sessegolo et al. 75 evaluated the respective contribution of rain, wet periods and unsaturated humidity on the kinetics of formation of the alteration layer of potash-lime-silica glass. The results demonstrated that the fluctuation of dry and wet periods (non-static condition) leads to the formation of an altered layer rich of cracks, pits, and scales, which constitute a network of pores and fractures behaving as a major vector for interdiffusion of liquid and vapour water. The characterisation of this fracture network is particularly important in case of prolonged alteration in atmospheric conditions, where liquid and vapour water phases are systematically combined in the associated diffusion processes. In another subsequent work 94 the authors proposed a model to estimate the thickness of alteration of stained glass windows subjected to liquid water (such as rain), comparing the expected value of alteration thickness extrapolated linearly over 650 year with those observed on ancient stained glass. The values predicted by the developed model match well with the alteration layer thickness observed on real ancient medieval samples 6 , 95 , 96 .

Effect of saline and burial environments

In the last two decades, several works have been done to explore the effect of ageing in burial or marine environments on the alteration mechanisms of glass and on the formation of new alteration phenomena. A considerable number of works focused on the prediction of the durability of high-level nuclear waste glass and the migration of radioactive and non-radioactive elements into the burial soil 9 , 97 , 98 .

Verney-Carron et al. 5 studied ancient Roman glass blocks naturally aged in seawater obtaining the kinetic parameters to develop a geochemical model to simulate the alteration of archaeological glass. Comparing the results obtained from the characterisation of ancient glass with those from the computer simulation, this work demonstrated the importance of archaeological glass for validating the predictive capacity of geochemical long-term model, thus bridging the gap between the results obtained from short-term experiments and long-term alteration of complex systems.

Palomar devoted many studies to weathering and to the comprehension of the environmental effect on the stages of glass alteration 59 , 99 , 100 . In particular, she reported the effect of coastal atmosphere on glass degradation 101 , which is a scarcely investigated subject. The alteration of glass surface exposed to coastal environment is mainly caused by the high presence of liquid water that covers the glass surface, thus inducing a hydrolytic attack and the dealkalisation process, and by the high wind speed, which favours the transportation and deposition of sodium and chlorine ions on the glass surface. Marine aerosol in elevated concentrations represents a hazardous agent for the chemical stability of glass, since its action could increase the hygroscopicity of the glass surface and open the glass structure, allowing the alteration to proceed deeper into the glass.

Another visible mark of degradation observed on the surface of glass altered in saline environment is the formation of peculiar chemical ring-like patterns that have been explained according to the Liesegang theory (Fig. 12 ). Diffusion, reaction, nucleation and crystal growth are all phenomena that have been used to formulate models that explain the Liesegang rings formation 102 , 103 . Dal Bianco et al . observed these weathering rings present on glass fragments from the Roman ship Iulia Felix found on the Grado lagoon, in North-East Italy, and dated back to the 2 nd century AD 104 , 105 . Their characterisation study showed a maximum diameter of the rings of about 1 mm and the interconnection of interface lines during the simultaneous growth of adjacent rings. This study did not report an exhaustive theory about the formation process of rings, but the authors observed that the structure was similar to the descriptions of Liesegang kinetic evolution of precipitates in gel found in other research works 106 , 107 . This assumption is acceptable since hardly corroded glass structure can be assumed as a gel where weakly soluble salts periodically precipitate due to the reaction between two soluble substances, one of which is dissolved in the gel medium. The final appearance of the precipitates depends on their solubility and on the initial concentration ratio of the reagents, but generally their aspect is concentric around the centre in which precipitation starts 108 .

Image collected with an Olympus BX43F optical microscope, x10 magnification. Interconnection between adjacent rings can be observed.

Nowadays, the formation of this type of rings on archaeological glass has been observed on the surface of samples recovered in submarine environment 104 , 109 . The authors of this review suggest that a saline-rich environment, whether seawater or wet soil, play a role in the formation of the rings. The soluble substances present in these environments may react with soluble substances from the aged glass forming salt precipitates on the surface of glass that act as centre of nucleation for the growth of the concentric structure.

The mechanisms that generate the formation of rings on altered glass surface have not been analytically confirmed, and the study of the evolution of the kinetics that controls ring growth on archaeological glass remains incomplete, despite the availability of modern techniques. Although the layers observed and experimentally described by Schalm 110 can be compared to the bands structure obtained from Liesegang kinetics, there is a lack of systematic experimental work in the literature to confirm whether this theory can be applied to the dissolution-precipitation mechanisms of glass.

Chemical studies on the composition of the soil where archaeological glass aged for centuries might validate the hypothesis of ionic exchange between the elements of the soil and the elements of the glass network, providing the opportunity to understand how the interaction with the external environment can drive the process of alteration in different types of glass.

Many published research studies mention that also for glass samples that altered buried in soil the most common pathology observed is the formation of dealkalinisation layers 111 , 112 , 113 . One of these performed a stratigraphical analysis using a non-invasive technique (laser induced breakdown spectroscopy, LIBS) to observe the progressive dealkalinisation of glass bulk composition, reporting an increase in the calcium and sodium intensity signals on the glass surface 84 .

In the field of ancient glass, experimental studies simulating ageing in soil are less common 114 , 115 , 116 . In a recent work, Palomar et al . tried to replicate ageing in soil in a natural burial environment to understand the corrosion mechanisms acting on different ancient glass types (Roman, medieval, lead crystal glass, common window glass) 117 . The burial tests were set up to last 300 days and were carried out at 60 °C, to accelerate the alteration processes. The pH of the burial soil and the glass composition showed to have a key role in controlling the reactions between the constituent glass elements and those from the environment.

When considering burial artificial alteration of mixed alkali glass compositions with low silica concentration, acidic or neutral conditions (pH 6.5 – 7.5) lead to the formation of micro pits and cracks; an iridescent and translucent layer is formed, instead, on the surface of glass under alkaline soil condition (8.0 – 9.0) 118 . Silica-soda-lime glass shows a different behaviour. In fact, in acidic soil the formation of both isolated and interconnected fissures may be observed. However, under neutral and alkaline soil conditions an increase in the number and depth of pits, whose rate of accretion depends on the content of alkaline oxides in the glass, can be detected, with a considerable increase in the diffusion of surface degradation under the alkaline condition 117 . This experimental evidence is a clear example of the necessity to consider glass composition and environmental factors simultaneously when approaching the understanding of the glass corrosion mechanism.

Effect of the environmental pH

A pivotal factor in determining the rate of glass corrosion is the pH of the attacking solution both in case of vapour and liquid conditions 10 . Under conditions of low pH (acidic solution), the deterioration mechanism predominantly involves the ion-exchange process due to the abundance of hydronium ions in solution and the formation of silanol groups [Si-OH], generating a hydrated gel on the surface which slows down degradation 117 . Differently, at high pH, the interaction between the glass surface and the alkaline solution leads to the dissolution of the silica network through the rupture of the Si-O-Si bonds, which implicates a more aggressive condition.

In works performed in 2016 92 and 2021 110 , Schalm et al. observed that glass surface is transformed in both acid and alkaline condition, but different morphologies can be developed depending on the pH of the solution. At pH<7 the transformed glass has a homogeneous morphology determined by the dissolution of leached silicate network and precipitation of dissolved compounds as amorphous silica forming linear and randomly branched chains of alteration product that can be described as a silica gel. At pH between 7 and 10 of the local solution the transformation process leads to the formation of silica nanoparticles packed into alternating density lamellae. The authors successfully reproduced the lamination in laboratory experiment performed around pH 10, where silica solubility is highly variable and induces the cyclical dissolution-precipitation of silica. The mechanism of formation of consecutive lamellae with alternating packing density is proposed to be dependent on the pH oscillation in the local solution during the alteration process. In extreme alkaline condition (pH>10) the fast dissolution of silica network occurs without any material precipitation.

A recent work 94 highlighted how the pH-dependency is linked to the specific composition of glass and consequently to the solubility of different glass constituents. The results show that the pH-dependency of Ca-Mg-silicates highly differs from the behaviour of aluminosilicates, and that Si-K-Ca medieval glasses have a very low pH dependency at alkaline conditions.

Effect of pollutants

Air pollution was identified as a particularly dangerous agent that enables to speed up and enhance alteration processes. In the museum context, the presence of carboxylate acids pollutants is generally the main cause of glass corrosion and of the formation on the glass surface of efflorescence salts as deterioration products 119 . With regards to historical stained glass, an experimental work carried out over a six-year period to quantify the influence of various air pollutants from different local environments (Europe and North America) on the degradation of potash-lime-silica glass, which has a similar composition to that of medieval stained glass, showed the formation, after exposure to rain and solar radiation, of crystalline carbon-rich products unlike those of samples aged under sheltered conditions 120 .

Melcher and Steiner 120 performed a 6-year experiment to evaluate the effect of acid gas and pollutant on replicas of stained glass, comparing the leaching depth formed on potash-lime-silica glass replicas to former leaching studies performed on medieval stained glass. This work demonstrated that the hypothetic leaching depth cannot be directly related to pollution data because, while the leaching depth increases with time, the leaching rate decreases. Even a paper of Robinet et al . 30 highlighted the role of the organic pollutants formic acid, acetic acid and formaldehyde in the alteration of unstable soda silicate glass. He confirmed that museum wooden cabinets emitting organic pollutants must be avoided since they foster the progressive alteration with an estimated rate of ~2 nm/day.

Advancements in monitoring ancient glass conservation state

The kinetics of glass corrosion, as well as the sequence of events involved and the prevalence of one interconnected mechanism over another, are dependent on both the chemical composition and structure of the glassy material and the environmental conditions to which it is exposed. These conditions include the amount of water that reacts, the chemistry and pH of the solution, and the duration of exposure. Given the complexity of these factors, the alteration of glass is a complex phenomenon that requires careful study. Therefore, particular attention should be paid to the analysis of ancient glass samples that have undergone modifications over time scales that cannot be replicated in the laboratory. These samples provide valuable insights into the long-term alteration mechanisms of glass and can aid in the development of more accurate predictive models for understanding the complex behaviour of glass materials in various environmental conditions.

The characterisation of altered ancient glass provides concrete evidence of the transformation of the vitreous structure, the nature of the dissolution products, and the way in which a specific glass composition reacts to a particular environment 3 , 4 , 7 , 70 , 121 , 122 . Over the course of history, the evolution of glass manufacturing has led to the production of various types of glass, such as Roman SSL glass, medieval stained glass, or Venetian crystal glass ( Cristallo ). Artefacts (and their fragments) made from all of these types of glass inherited from the past represent an exceptional opportunity to better understand the processes involved in glass corrosion on a long-term scale. This is due to their specific chemical stability, resulting from their unique chemical composition, and their aging in soil, underwater, or confined spaces with specific microclimates for centuries.

The analysis of glass surfaces is currently considered the most effective scientific approach for studying chemical and physical variations that occur at the interface between the atmosphere and the glass surface. Advanced surface analysis techniques, such as X-ray photoelectron spectroscopy (XPS) and ToF and/or dynamic SIMS, provide high-resolution data that enable the investigation of the chemical composition of the glass surface’s first nanometres and facilitate monitoring of its modification during the alteration process 67 , 123 . SIMS is widely considered one of the most suitable techniques for studying corroded glass 124 , due to its ability to detect hydrogen and investigate the glass surface up to a depth of a few microns. Another technique commonly reported in the literature for studying the weathered surface of ancient glass is laser ablation inductively coupled plasma mass spectrometry (LA-ICP-MS), which allows for the acquisition of 2D and 3D elemental distribution with spatial resolution as high as the spot size used for the analysis. Only one work, published in 2013 125 , presents the procedure to obtain elemental maps to investigate surface layer phenomena on pitted ancient glass. The results of this application showed that the dealkalinisation of glass sample occurred on surface, as the direct or indirect consequence of the hydration process, resulting in the final formation of pits and of the so-called Liesegang rings on the surface of the sample. Despite the novelty of this research, the spot size of the laser beam used in this work (diameter of 80 µm) is higher than the average size of the corrosion marks present on ancient samples. The spatial resolution on the reconstructed maps can be improved by using smaller spot size of the laser beam (down to 10 µm) making it possible to obtain a resolution suitable for appreciating the chemical variability of extremely heterogeneous samples. This can potentially increase the understanding of the mechanisms of formation of these altered phases.

Critically speaking, the extensive study of glass surface composition using these advanced analytical techniques is still seldom considered when investigating the corrosion mechanism of glass both on ancient samples and through an artificial ageing approach in the laboratory 30 , 126 , 127 . Each surface technique has its own peculiar applications and limitations 128 , but a combination of different techniques allows a complementary and more comprehensive characterisation of corroded glass.

Collecting morphological, structural, and chemical information through a multi-scale analytical approach is necessary to gain a comprehensive understanding of the evolution and kinetics of glass alteration, including the modification of glass composition. Well-designed experiments are crucial due to the co-operation and mutual influence of different mechanisms involved in glass corrosion. This approach allows for investigation from a macroscopic to a nanometre scale observation. The information gathered can help determine how and when to intervene for conservation and preservation of glass objects, whether they are of industrial or historical significance.

The evaluation of storage parameters such as relative humidity, temperature, and light, as well as the condition of the glass, are crucial for preventing degradation of glass objects in museum collections or other sites of cultural interest. This approach is particularly important for understanding the influence of intrinsic and extrinsic factors in the processes of alteration and finding the best conditions for conservation. Surveys 129 , 130 , 131 conducted in various museums in Europe have shown that a significant number of glass objects are in critical condition due to extensive degradation, highlighting the need for preventive actions. For example, at the Victoria and Albert Museum, out of 6,500 glass vessels optically investigated, more than 400 showed clear signs of glass deterioration 54 . Similarly, the glass storage conditions of the Royal Palace of Madrid and the Technological Museum of Glass (Segovia) were evaluated during the period from September 2019 to November 2020, detecting a high concentration of formic acid in the display cabinets and wardrobes, mainly due to the presence of wood, which results in a higher glass surface hygroscopicity (pH=8) 132 .

Several methods have been used to categorise unstable glass from museum collections based on its appearance 65 or by analytical techniques –like X-ray fluorescence (XRF) 133 , ion beam techniques (particle -induced X-ray emission, PIXE, and particle-induced gamma ray emission, PIGE) 134 or spectroscopic techniques (Raman and Fourier-transform infrared, FTIR) 135 , 136 – in order to determine glass composition. Using these techniques to analyse cultural heritage objects can be complex due to many operative limitations, as for instance the need of carrying out micro-sampling (which is often not possible) or moving the objects to the specialised laboratories where the instruments are hosted. With the aim to distinguish stable from unstable glass in museum collection, the ideal analytical technique should provide the chemical composition of the sample, be highly sensitive, and have a very fast time of analysis to characterise as many samples as possible in a short period of time, while operating in a non-invasive way. Consequently, it can be safely stated that no scientific and straightforward approach for understanding the chemical nature and composition of unstable glass in a non-invasive way and for large museum collections of glass objects has been developed yet.

In the light of the above, it is evident that the most effective approach to study glass degradation involves the integration of two main distinct but interconnected methods:

the evaluation of the surface of ancient glass artefacts;

the evaluation of intrinsic and extrinsic causes for glass alteration.

Advanced analytical techniques can be used to characterize ancient glass surfaces and observe the products of the long-term transformation of the glass structure resulting from the alteration process. In parallel, laboratory-based aging experiments can be used to evaluate intrinsic and extrinsic causes of glass alteration, distinguishing the effects of different parameters and identifying the most relevant factors that influence glass alteration kinetics. Parameters such as the content of alkalis and stabilizers in the glass composition, as well as temperature and humidity, strongly impact the chemical durability of glass by affecting the concentration of alkalis and hydroxyl or non-bridging oxygens in the hydrated layer. It is difficult to determine which factor has the most significant influence on the process of glass alteration because the final degradation symptoms are the result of a mutual effect between all the aforementioned.

As evidenced, over the years many experiments aimed at understanding the phenomenon have been reported in literature and almost as many methods have been proposed. This abundance of results represents an outstanding opportunity for future research studies in this field, which will be able to rely on advanced analytical approaches and more accessible high-resolution techniques even for the domain of cultural heritage science. The results obtained from the study of ancient materials are pivotal to validate the long-term capacity of kinetic models -which are based on experimental data only - and to strengthen current theories of glass corrosion.

Moreover, comparing the characteristics of artificial replicas with those of ancient glass enables a better understanding of the glass corrosion phenomena, that in turn can underpin both the formulation of new protective solutions to preserve and protect glass artefacts in the long-term and the design of technologies exploiting glass properties in a variety of applications.

Deng, L. et al. Ion-exchange mechanisms and interfacial reaction kinetics during aqueous corrosion of sodium silicate glasses. Npj Mater. Degrad. 5 , 1–13 (2021).

Article   CAS   Google Scholar  

Sessegolo, L. et al. Alteration of potash-lime silicate glass in atmospheric medium: study of mechanisms and kinetics using 18O and D isotopes. J. Non-Cryst. Solids 570 , 121020 (2021).

Verney-Carron, A., Gin, S. & Libourel, G. A fractured roman glass block altered for 1800 years in seawater: Analogy with nuclear waste glass in a deep geological repository. Geochim. Cosmochim. Acta 72 , 5372–5385 (2008).

Libourel, G. et al. The use of natural and archeological analogues for understanding the long-term behavior of nuclear glasses. Comptes Rendus Geosci. 343 , 237–245 (2011).

Verney-Carron, A., Gin, S. & Libourel, G. Archaeological analogs and the future of nuclear waste glass. J. Nucl. Mater. 406 , 365–370 (2010).

Lombardo, T. et al. Long term assessment of atmospheric decay of stained glass windows. Corros. Eng. Sci. Technol. 45 , 420–424 (2010).

Majérus, O. et al. Glass alteration in atmospheric conditions: crossing perspectives from cultural heritage, glass industry, and nuclear waste management. Npj Mater. Degrad. 4 , 27 (2020).

Article   Google Scholar  

Alloteau, F. et al. Temperature-dependent mechanisms of the atmospheric alteration of a mixed-alkali lime silicate glass. Corros. Sci. 159 , 108129 (2019).

Alloteau, F. et al. New insight into atmospheric alteration of alkali-lime silicate glasses. Corros. Sci. 122 , 12–25 (2017).

Abrajano, T. A., Bates, J. K. & Byers, C. D. Aqueous corrosion of natural and nuclear waste glasses I. Comparative rates of hydration in liquid and vapor environments at elevated temperatures. J. Non-Cryst. Solids 84 , 251–257 (1986).

Geisler, T. et al. The mechanism of borosilicate glass corrosion revisited. Geochim. Cosmochim. Acta 158 , 112–129 (2015).

Priest, D. K. & Levy, A. S. Effect of Water Content on Corrosion of Borosilicate Glass. J. Am. Ceram. Soc. 43 , 356–358 (1960).

Conradt, R. Chemical Durability of Oxide Glasses in Aqueous Solutions: A Review. J. Am. Ceram. Soc. 91 , 728–735 (2008).

Zachariasen, W. H. The atomic arrangement in glass. J. Am. Chem. Soc. 54 , 3841–3851 (1932).

Henderson, G. The structure of silicate melts: A glass perspective. Can. Mineral. 43 , 1921–1958 (1921).

Kunicki-Goldfinger, J. J. Unstable historic glass: symptoms, causes, mechanisms and conservation. Stud. Conserv. 53 , 47–60 (2008).

Bange, K. et al. Multi-method characterization of soda-lime glass corrosion. Part 2. Corros. humidity. Glass Sci. Technol. Glastech. Ber. 75 , 20–33 (2002).

CAS   Google Scholar  

Alloteau, F. et al. Evidence for different behaviors of atmospheric glass alteration as a function of glass composition. Npj Mater. Degrad. 4 , 36 (2020).

Bunker, B. C. Molecular mechanisms for corrosion of silica and silicate glasses. J. Non-Cryst. Solids 179 , 300–308 (1994).

Jantzen, C. M. & Plodinec, M. J. Thermodynamic model of natural, medieval and nuclear waste glass durability. J. Non-Cryst. Solids 67 , 207–223 (1984).

Damodaran, K., Delaye, J.-M., Kalinichev, A. G. & Gin, S. Deciphering the non-linear impact of Al on chemical durability of silicate glass. Acta Mater. 225 , 117478 (2022).

Du, J. et al. Predicting the dissolution rate of borosilicate glasses using QSPR analysis based on molecular dynamics simulations. J. Am. Ceram. Soc. 104 , 4445–4458 (2021).

Lillington, J. N. P., Goût, T. L., Harrison, M. T. & Farnan, I. Assessing static glass leaching predictions from large datasets using machine learning. J. Non-Cryst. Solids 546 , 120276 (2020).

Anoop Krishnan, N. M. et al. Predicting the dissolution kinetics of silicate glasses using machine learning. J. Non-Cryst. Solids 487 , 37–45 (2018).

Cesaratto, A. et al. Characterization of archeological glasses by micro-Raman spectroscopy. J. Raman Spectrosc. 41 , 1682–1687 (2010).

Colomban, P. On-site Raman identification and dating of ancient glasses: A review of procedures and tools. J. Cult. Herit. 9 , 55–60 (2008).

Hruška, B. et al. Raman spectroscopic study of corroded historical glass. Int. J. Appl. Glass Sci. 12 , 613–620 (2021).

Quaranta, A. et al. Spectroscopic Investigation of Structural Rearrangements in Silver Ion-Exchanged Silicate Glasses. J. Phys. Chem. C. 116 , 3757 (2012).

Colomban, P., Etcheverry, M.-P., Asquier, M., Bounichou, M. & Tournié, A. Raman identification of ancient stained glasses and their degree of deterioration. J. Raman Spectrosc. 37 , 614–626 (2006).

Robinet, L., Hall, C., Eremin, K., Fearn, S. & Tate, J. Alteration of soda silicate glasses by organic pollutants in museums: Mechanisms and kinetics. J. Non-Cryst. Solids 355 , 1479 (2009).

Robinet, L., Coupry, C., Eremin, K. & Hall, C. The use of Raman spectrometry to predict the stability of historic glasses. J. Raman Spectrosc. 37 , 789–797 (2006).

Colomban, P. Polymerization degree and Raman identification of ancient glasses used for jewelry, ceramic enamels and mosaics. J. Non-Cryst. Solids 323 , 180–187 (2003).

Li, D., Bancroft, G. M., Fleet, M. E. & Feng, X. H. Silicon K-edge XANES spectra of silicate minerals. Phys. Chem. Miner. 22 , 115–122 (1995).

Li, D., Bancroft, G. M. & Fleet, M. E. Coordination and local structure of Si and Al in silicate glasses: Si and Al K-edge XANES spectroscopy. Miner. Spectros. 5 , 153–163 (1996).

Google Scholar  

Greaves, G. N. Corrosion studies of glass using conventional and glacing angle EXAFS. J. Phys. Colloq. 47 , C8–824 (1986).

Abuín, M., Serrano, A., Chaboy, J., García, M. A. & Carmona, N. XAS study of Mn, Fe and Cu as indicators of historical glass decay. J. Anal. Spectrom. 28 , 1118–1124 (2013).

Chabas, A. Modern Silica-Soda-Lime Glass in Polluted Atmosphere. in The Materials of Cultural Heritage in Their Environment vol. B, R.-A. Lefèvre (2004).

Paul, A. Chemical durability of glasses; a thermodynamic approach. J. Mater. Sci. 12 , 2246–2268 (1977).

El-Shamy, T. M., Lewins, J. & Douglas, R. W. The dependence on the pH of the decomposition of glasses by aqueous solutions. Glass Technol. 13 , 81–87 (1972).

Do Remus, R. H., Mehrotra, Y., Lanford, W. A. & Burman, C. Reaction of water with glass: influence of a transformed surface layer. J. Mater. Sci. 18 , 612–622 (1983).

Doremus, R. H. Interdiffusion of hydrogen and alkali ions in a glass surface. J. Non-Cryst. Solids 19 , 137–144 (1975).

White, B. Theory of Corrosion of Glass and Ceramics. In Corrosion of Glass, Ceramics and Ceramic Superconductors vol. 2, Noyes Publications (1992).

Tait, J. C. & Mandolesi, D. L. The chemical durability of alkali aluminosilicate glasses . (1983).

Soules, T. F. A molecular dynamic calculation of the structure of sodium silicate glasses. J. Chem. Phys. 71 , 4570–4578 (1979).

Gin, S., Delaye, J.-M., Angeli, F. & Schuller, S. Aqueous alteration of silicate glass: state of knowledge and perspectives. Npj Mater. Degrad. 5 , 42 (2021).

Hellmann, R. et al. Nanometre-scale evidence for interfacial dissolution–reprecipitation control of silicate glass corrosion. Nat. Mater. 14 , 307–311 (2015).

Lenting, C. et al. Towards a unifying mechanistic model for silicate glass corrosion. Npj Mater. Degrad. 2 , 28 (2018).

Gin, S. et al. The controversial role of inter-diffusion in glass alteration. Chem. Geol. 440 , 115–123 (2016).

Gin, S. et al. Atom-Probe Tomography, TEM and ToF-SIMS study of borosilicate glass alteration rim: A multiscale approach to investigating rate-limiting mechanisms. Geochim. Cosmochim. Acta 202 , 57–76 (2017).

Parruzot, B., Jollivet, P., Rébiscoul, D. & Gin, S. Long-term alteration of basaltic glass: Mechanisms and rates. Geochim. Cosmochim. Acta 154 , 28–48 (2015).

Gueli, A. M. et al. Weathering and deterioration of archeological glasses from late Roman Sicily. Int. J. Appl. Glass Sci. 11 , 215–225 (2020).

van Giffen, N. A. R. & Koob, S. P. Deterioration of vitreous materials. The Encyclopedia of Archaeological Science. 1–4 (2018).

Newton, R. G. & Davison, S. Conservation and restoration of glass . (Routledge, 2011).

Oakley, V. Vessel glass deterioration at the Victoria and Albert museum: Surveying the collection. Conservator 14 , 30–36 (1990).

Brewster, D. XIX.—On the structure and optical phenomena of ancient decomposed glass. Earth Environ. Sci. Trans. R. Soc. Edinb. 23 , 193–204 (1863).

Hench, L. L. Characterization of glass corrosion and durability. J. Non-Cryst. Solids 19 , 27–39 (1975).

Organ, R. M. The safe storage of unstable glass. Mus. J. 56 , 265–272 (1957).

Fisher, A. J., Hyatt, N. C., Hand, R. J. & Corkhill, C. L. The formation of pitted features on the international simple glass during dynamic experiments at alkaline pH. MRS Adv. 4 , 993–999 (2019).

Palomar, T. & Llorente, I. Decay processes of silicate glasses in river and marine aquatic environments. J. Non-Cryst. Solids 449 , 20–28 (2016).

Navarro, J. M. F. Alteration processes of medieval stained glass windows. Study and protection treatments. Mater. Constr. 46 , 5–25 (1996).

Henderson, J. Ancient Glass: An Interdisciplinary Exploration . (Cambridge University Press, 2016).

Koob, S. P. Crizzling glasses: problems and solutions. Glass Technol. - Eur. J. Glass Sci. Technol. Part A 53 , 225–227 (2012).

Brill, R. H. Crizzling – a problem in glass conservation. Stud. Conserv. 20 , 121–134 (1975).

Kunicki-Goldfinger, J., Kierzek, J., Ma, B. & Kasprzak, A. J. Some observations on crizzled glass (preliminary results of a survey of 18th century central European tableware). Glass Technol. 43 , 364–368 (2002).

Koob, S. A. Conservation and Care of Glass Objects . (Archetype Publications / The Corning Museum of Glass, 2006).

Morey, G. W. The corrosion of glass surfaces. Ind. Eng. Chem. 17 , 389–392 (1925).

Sprenger, D., Bach, H., Meisel, W. & Gütlich, P. XPS study of leached glass surfaces. J. Non-Cryst. Solids 126 , 111–129 (1990).

Rehren, T. H. & Freestone, I. C. Ancient glass: from kaleidoscope to crystal ball. J. Archaeol. Sci. 56 , 233–241 (2015).

Heimann, R. B. Weathering of ancient and medieval glasses—potential proxy for nuclear fuel waste glasses. A perennial challenge revisited. Int. J. Appl. Glass Sci. 9 , 29–41 (2018).

Melcher, M. & Schreiner, M. Leaching studies on naturally weathered potash-lime–silica glasses. J. Non-Cryst. Solids 352 , 368–379 (2006).

Melcher, M. & Schreiner, M. Statistical evaluation of potash-lime-silica glass weathering. Anal. Bioanal. Chem. 379 , 628–639 (2004).

Woisetschläger, G., Dutz, M., Paul, S. & Schreiner, M. Weathering phenomena on naturally weathered potash-lime-silica-glass with medieval composition studied by secondary electron microscopy and energy dispersive microanalysis. Microchim. Acta 135 , 121–130 (2000).

Verhaar, G. Glass sickness: Detection and prevention: Investigating unstable glass in museum collections . (University of Amsterdam, 2018).

Hench, L. L. & Clark, D. E. Physical chemistry of glass surfaces. J. Non-Cryst. Solids 28 , 83–105 (1978).

Sessegolo, L. et al. Long-term weathering rate of stained-glass windows using H and O isotopes. Npj Mater. Degrad. 2 , 1–9 (2018).

Newton, R. The weathering of medieval window glass. J. Glass Stud. 17 , 161–168 (1975).

Arévalo, R., Mosa, J., Aparicio, M. & Palomar, T. The stability of the Ravenscroft’s glass. Influence of the composition and the environment. J. Non-Cryst. Solids 565 , 120854 (2021).

Gentaz, L., Lombardo, T., Chabas, A., Loisel, C. & Verney-Carron, A. Impact of neocrystallisations on the SiO2–K2O–CaO glass degradation due to atmospheric dry depositions. Atmos. Environ. 55 , 459–466 (2012).

De Bardi, M., Hutter, H., Schreiner, M. & Bertoncello, R. Potash-lime-silica glass: protection from weathering. Herit. Sci. 3 , 22 (2015).

Brostoff, L. B. et al. Glass at risk: A new approach for the study of 19th century vessel glass. J. Cult. Herit. 54 , 155–166 (2022).

Frenzel, G. The restoration of medieval stained glass. Sci. Am. 252 , 126–135 (1985).

Schalm, O. et al. Manganese staining of archaeological glass: the characterization of Mn-rich inclusions in leached layers and a hypothesis of its formation. Archaeometry 53 , 103–122 (2011).

Palomar, T. Chemical composition and alteration processes of glasses from the Cathedral of León (Spain). Bol. Soc. Esp. Cerámica Vidr. 57 , 101–111 (2018).

Palomar, T., Oujja, M., García-Heras, M., Villegas, M. A. & Castillejo, M. Laser induced breakdown spectroscopy for analysis and characterization of degradation pathologies of Roman glasses. Spectrochim. Acta Part B . Spectrosc. 87 , 114–120 (2013).

Cooper, G. I., Cox, G. A. & Perutz, R. N. Infra-red microspectroscopy as a complementary technique to electron-probe microanalysis for the investigation of natural corrosion on potash glasses. J. Microsc. 170 , 111–118 (1993).

Doménech-Carbó, M.-T., Doménech-Carbó, A., Osete-Cortina, L. & Saurí-Peris, M.-C. A Study on Corrosion Processes of Archaeological Glass from the Valencian Region (Spain) and its Consolidation Treatment. Microchim. Acta 154 , 123–142 (2006).

Ferrand, J. et al. Browning Phenomenon of Medieval Stained Glass Windows. Anal. Chem. 87 , 3662–3669 (2015).

Kennedy, C., Addyman, T., Murdoch, K. & Young, M. 18th- and 19th- Century Scottish Laboratory Glass: Assessment of Chemical Composition in Relation to Form and Function. J. Glass Stud. 60 , 253–267 (2018).

Cailleteau, C. et al. Insight into silicate-glass corrosion mechanisms. Nat. Mater. 7 , 978–983 (2008).

Zhang, H., Suzuki-Muresan, T., Morizet, Y., Gin, S. & Abdelouas, A. Investigation on boron and iodine behavior during nuclear glass vapor hydration. Npj Mater. Degrad. 5 , 1–9 (2021).

Lombardo, T. et al. Characterisation of complex alteration layers in medieval glasses. Corros. Sci. 72 , 10–19 (2013).

Schalm, O. & Anaf, W. Laminated altered layers in historical glass: Density variations of silica nanoparticle random packings as explanation for the observed lamellae. J. Non-Cryst. Solids 442 , 1–16 (2016).

Turner, W. E. S. The use of boric oxide in glass-making. J. Am. Ceram. Soc. 7 , 313–317 (1924).

Sessegolo, L. et al. Alteration rate of medieval potash-lime silicate glass as a function of pH and temperature: A low pH-dependent dissolution. Chem. Geol. 550 , 119704 (2020).

Carmona, N., Villegas, M. A. & Navarro, J. M. F. Characterisation of an intermediate decay phenomenon of historical glasses. J. Mater. Sci. 41 , 2339–2346 (2006).

Sterpenich, J. & Libourel, G. Using stained glass windows to understand the durability of toxic waste matrices. Chem. Geol. 174 , 181–193 (2001).

Ojovan, M. I., Hand, R. J., Ojovan, N. V. & Lee, W. E. Corrosion of alkali–borosilicate waste glass K-26 in non-saturated conditions. J. Nucl. Mater. 340 , 12–24 (2005).

Fournier, M., Gin, S., Frugier, P. & Mercado-Depierre, S. Contribution of zeolite-seeded experiments to the understanding of resumption of glass alteration. Npj Mater. Degrad. 1 , 1–13 (2017).

Palomar, T., Chabas, A., Bastidas, D. M., de la Fuente, D. & Verney-Carron, A. Effect of marine aerosols on the alteration of silicate glasses. J. Non-Cryst. Solids 471 , 328–337 (2017).

Rodrigues, A., Fearn, S., Palomar, T. & Vilarigues, M. Early stages of surface alteration of soda-rich-silicate glasses in the museum environment. Corros. Sci. 143 , 362–375 (2018).

Palomar, T., de la Fuente, D., Morcillo, M., Alvarez de Buergo, M. & Vilarigues, M. Early stages of glass alteration in the coastal atmosphere. Build. Environ. 147 , 305–313 (2019).

Duley, J. M., Fowler, A. C., Moyles, I. R. & O’Brien, S. B. G. On the Keller–Rubinow model for Liesegang ring formation. Proc. R. Soc. Math. Phys. Eng. Sci. 473 , 20170128 (2017).

George, J., Nair, S. & Varghese, G. Role of colloid dynamics in the formation of Liesegang rings in multi-component systems. J. Mater. Sci. 39 , 311–313 (2004).

Dal Bianco, B., Bertoncello, R., Milanese, L. & Barison, S. Surface study of water influence on chemical corrosion of Roman glass. Surf. Eng. 21 , 393–396 (2005).

Barbana, F., Bertoncello, R., Milanese, L. & Sada, C. Alteration and corrosion phenomena in Roman submerged glass fragments. J. Non-Cryst. Solids 337 , 136–141 (2004).

Eltantawy, M. M. et al. Self-assembled liesegang rings of hydroxyapatite for cell culturing. Adv. NanoBiomed. Res. 1 , 2000048 (2021).

Manley, D. R. & Stern, K. H. Liesegang rings in inhomogeneous media. Powdered glass J. Colloid Sci. 10 , 409–412 (1955).

Pask, J. A. & Parmelbe, C. W. Study of diffusion in glass*. J. Am. Ceram. Soc. 26 , 267–277 (1943).

Dal Bianco, B., Bertoncello, R., Milanese, L. & Barison, S. Glasses on the seabed: surface study of chemical corrosion in sunken Roman glasses. J. Non-Cryst. Solids 343 , 91–100 (2004).

Schalm, O., Nuyts, G. & Janssens, K. Some critical observations about the degradation of glass: The formation of lamellae explained. J. Non-Cryst. Solids 569 , 120984 (2021).

Palomar, T., Garcia, M. & Villegas, M.-A. Model historical glasses under simulated burial conditions. Coalition 23 , 2–6 (2012).

Huisman, D. J., Pols, S., Joosten, I., van Os, B. J. H. & Smit, A. Degradation processes in colourless Roman glass: cases from the Bocholtz burial. J. Archaeol. Sci. 35 , 398–411 (2008).

Friedrich, K. T. & Degryse, P. Soil vs. glass: an integrated approach towards the characterization of soil as a burial environment for the glassware of Cucagna Castle (Friuli, Italy). STAR Sci. Technol. Archaeol. Res. 5 , 138–156 (2019).

Evans, J. G. & Limbrey, S. The Experimental Earthwork on Morden Bog, Wareham, Dorset, England: 1963 to 1972: Report of the Experimental Earthworks Committee of the British Association for the Advancement of Science. Proc. Prehist. Soc. 40 , 170–202 (1974).

Fletcher, W. W. The chemical durability of glass. a burial experiment at ballidon in derbyshire. J. Glass Stud. 14 , 149–151 (1972).

Hand, R. J. The Ballidon Glass Burial Experiment – 35 years on. Glass Technol. 46 , 237–242 (2005).

Palomar, T. Effect of soil pH on the degradation of silicate glasses. Int. J. Appl. Glass Sci. 8 , 177–187 (2017).

Roemich, H. et al. Results from burial experiments with simulated medieval glasses. MRS Proc. 757 , II2.3 (2002).

Gibson, L. T., Cooksey, B. G., Littlejohn, D. & Tennent, N. H. Characterisation of an unusual crystalline efflorescence on an Egyptian limestone relief. Anal. Chim. Acta 337 , 151–164 (1997).

Melcher, M. & Schreiner, M. Evaluation procedure for leaching studies on naturally weathered potash-lime-silica glasses with medieval composition by scanning electron microscopy. J. Non-Cryst. Solids 351 , 1210 (2005).

Greiner-Wronowa, E. & Stoch, L. Influence of environment on surface of the ancient glasses. J. Non-Cryst. Solids 196 , 118–127 (1996).

Schreiner, M., Woisetschläger, G., Schmitz, I. & Wadsak, M. Characterisation of surface layers formed under natural environmental conditions on medieval stained glass and ancient copper alloys using SEM, SIMS and atomic force microscopy. J. Anal. Spectrom. 14 , 395–403 (1999).

Fearn, S., McPhail, D. S. & Oakley, V. Moisture attack on museum glass measured by SIMS. Phys. Chem. Glasses 46 , 505–511 (2005).

Clegg, J. B. Secondary Ion Mass Spectrometry—a Practical Handbook for Depth Profiling and Bulk Impurity Analysis Wiley, New York, 1989. Surf. Interface Anal. 17 , 221–221 (1991).

van Elteren, J. T. et al. 3D laser ablation-ICP-mass spectrometry mapping for the study of surface layer phenomena – a case study for weathered glass. J. Anal. Spectrom. 28 , 994 (2013).

De Bardi, M., Hutter, H. & Schreiner, M. ToF-SIMS analysis for leaching studies of potash–lime–silica glass. Appl. Surf. Sci. 282 , 195–201 (2013).

Robinet, L., Pulham, C., Hall, C., Eremin, K. & Fearn, S. Understanding glass deterioration in museum collections through Raman spectroscopy and SIMS analysis. MRS Online Proc. Libr. 852 , 247–254 (2004).

Gong, Y., Xu, J. & Buchanan, R. C. The aqueous corrosion of nuclear waste glasses revisited: Probing the surface and interfacial phenomena. Corros. Sci. 143 , 65–75 (2018).

Cobo del Arco, B. Survey of the National Museum of Scotland Glass Collection. in The Conservation of Glass and Ceramics: Research, Practice and Training 229–238 (1999).

Eremin, K., Cobo del Arco, B., Robinet, L. & Gibson, L. Deteriorating Nineteenth and Twentieth - Century British Glass in the National Museums of Scotland. in ‘Deteriorating Nineteenth and Twentieth Century British Glass in the National Museums of Scotland’ 380–385 (AIHV, 2005).

Burghout, F. & Slager, M. Cloudy Patches and Misty Glass: Early Signs of Glass Disease? 327–329 (H. Roemich and K. Van Lookeren Campagne, 2013).

Palomar, T., García-Patrón, N. & Pastor, P. Spanish Royal glasses with crizzling in historical buildings. The importance of environmental monitoring for their conservation. Build. Environ. 202 , 108054 (2021).

Ulitzka, S. & Touchard, V. Corrosion phenomena of bohemian and potsdam glasses from i7th and 18th century. in Science, Technology and European Cultural Heritage (eds. Baer, N. S., Sabbioni, C. & Sors, A. I.) 872–875 (Butterworth-Heinemann, 1991).

Mäder, C. N. & M. Endangered glass objects identified by ion beam analysis. in Cultural Heritage Conservation and Environmental Impact Assessment by Non-Destructive Testing and Micro-Analysis (CRC Press, 2005).

Robinet, L., Coupry, C., Eremin, K. & Hall, C. Raman investigation of the structural changes during alteration of historic glasses by organic pollutants. J. Raman Spectrosc. 37 , 1278–1286 (2006).

Earl, N. The Investigation of Glass Deterioration as a Result of Storage Systems for Waterlogged Archaeological Glass. in The Conservation of Glass and Ceramics: Research, Practice and Training 96–113 (1999).

Download references

Acknowledgements

The authors would like to express their gratitude to the anonymous reviewers: their insightful comments and critiques helped to significantly improve the quality and clarity of this paper. The authors are deeply grateful for their time, effort, and dedication.

Author information

These authors contributed equally: Roberta Zanini, Giulia Franceschin.

Authors and Affiliations

Istituto Italiano di Tecnologia, Center for Cultural Heritage Technology (CCHT), via Torino 155, 30172, Venezia-Mestre, Italy

Roberta Zanini, Giulia Franceschin & Arianna Traviglia

Department of Molecular Sciences and Nanosystems, Ca’ Foscari University of Venice, via Torino 155, 30172, Venezia-Mestre, Italy

Roberta Zanini & Elti Cattaruzza

You can also search for this author in PubMed   Google Scholar

Contributions

Both G.F. and R.Z. contributed equally to the concept, drafting and revision of this review. E.C. and A.T. revised and critically reviewed this work.

Corresponding authors

Correspondence to Roberta Zanini or Giulia Franceschin .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Zanini, R., Franceschin, G., Cattaruzza, E. et al. A review of glass corrosion: the unique contribution of studying ancient glass to validate glass alteration models. npj Mater Degrad 7 , 38 (2023). https://doi.org/10.1038/s41529-023-00355-4

Download citation

Received : 13 June 2022

Accepted : 25 April 2023

Published : 20 May 2023

DOI : https://doi.org/10.1038/s41529-023-00355-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Mechanisms and energetics of calcium aluminosilicate glass dissolution through ab initio molecular dynamics-metadynamics simulations.

• Luis Ruiz Pestana

npj Materials Degradation (2024)

Borate speciation and structural studies of vanadium ion doping in borate bioglass

• E. G. Elbasuoni
• A. M. Abdelghany
• G. El-Damrawi

Applied Physics A (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.