funnel approach to literature review

Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy pp 1–15 Cite as

Methodological Approaches to Literature Review

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  
• Living reference work entry
• First Online: 09 May 2023

447 Accesses

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

This is a preview of subscription content, log in via an institution .

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Through the Paradigm Funnel: A Conceptual Tool for Literature Analysis

2003, Marketing Education Review

Related Papers

European Journal of Marketing

Adrian Carr

mark tadajewski

IOSR Journals

This paper makes a conceptual clarification of some research elements-paradigm, methodology, design and method which have proved confusing to early career researchers, postgraduate supervisors and authors. This confusion has often been created and perpetuated by many research textbooks and journals over the years. By using a literature review and author's experience, this paper provides an exposition of the distinction and relationship between these concepts with a view to better the understanding and application of the concepts, for early career researchers, especially Master's and PhD students and postgraduate supervisors.

Ana María Munar

saeed ahmed

The Qualitative Report

Godswill Makombe

It is crucial that any research inquiry be guided by a paradigm. However, many early career researchers do not mention the research paradigm guiding their inquiry. Furthermore, qualitative and quantitative methods are sometimes erroneously referred to as research paradigms or research designs. Experienced researchers often use the terms research paradigm, research methods and research design in a loose and confusing manner. Although it is reasonable to assume that experienced researchers do understand the distinction and relationship between the three concepts, the loose use of the concepts leads to confusion among early career researchers, especially Master’s and PhD students. By using a literature review, this paper provides an expose of the relationship between these three concepts and highlights the sources of confusion from the literature. A qualitative approach, using a sample of 11 students from different South African universities, is used to provide an understanding of thes...

Communicare: Journal for Communication Studies in Africa

Benita Steyn

JIMMA UNIVERSITY College Of Education and Behavioral Sciences Department of Educational Planning and Management

Chala Ayele

A research philosophy is what the researcher perceives to be truth, reality and knowledge. The research process involves generating a hypothesis, collecting data to test that hypothesis, then analyzing and interpretation of the results. It outlines the theory, beliefs and values that guide the design, the collection, and analysis of the research study. A research paradigm is a set of beliefs, values, theories, models, and techniques that are used by research workers to legitimize what they are doing or to give direction to their inquiries. Research is influenced by paradigms which are lenses that guide the types of questions that should be identified by a specific investigation, the methods that should be used in addressing the research questions, and how data should be interpreted (Aliyu, Bello, Kasim, & Martin, 2014; Bryman, 2012) cited in (Aberra, 2016). There are four major types of paradigms; the positivist (knowledge being empirical and objective), the interpretivist (knowledge being socially constructed and subjective), and the critical (knowledge based on many truths of societies and are influenced by their own perceptions and experiences,) as it discussed in (Ryan, G. , 2018). All these paradigms have their ontological, epistemological and methodological assumptions. Ontology refers to the nature of reality whereas epistemology refers to acceptable knowledge, and methodological premises concerned with procedures for making knowledge and how the knower goes about obtaining the desired knowledge and understanding the valuable and worthwhile types of data are to be collected and organized. Therefore, understanding the research paradigms and their assumptions are very important to conduct any type of research. Accordingly, this paper contains an explanation of the basic philosophical major types of paradigms and its elements in research with their associated assumptions.

Jaana M Tähtinen

This article develops and applies a conceptual analysis method (CAM). The CAM is a critical reflection on multiple definitions and descriptions of concepts and terms all used to refer to a phenomenon or the experiences of it. The method particularly helps researchers working in emerging research fields to discover any conceptual confusion and elucidate multiple terms and concepts. We demonstrate the utility of the CAM by discovering conceptual confusion on an example field: business relationship uncoupling, and elucidating its terms and concepts. This article adds to the discussion on the importance of conscious conceptual language for theory development, on the level of a research field.

Zachary Sapienza , Aaron S Veenstra

Utilizing an implicit general semantics framework, this article explicates Thomas Kuhn’s conceptions of paradigms with specific attention paid to their context and application within the field of mass communication. In doing so, this paper will highlight three potential problems with the concept of paradigms ranging from multiple and diverse definitions to Kuhn’s insistence that it was not applicable to the social sciences. Building off the work of Rosengren (1983) and Renckstorf & McQuail (1996), this paper will then examine the potential of research quadrants as an alternative to paradigms and make a case for their use in the field of mass communication.

RELATED PAPERS

Hikmet Secim

Boris Zaltzman

Romanica Olomucensia

Eva Klímová

María Paula Gago

Baharudin Arus

Jurnal Studi Komunikasi (Indonesian Journal of Communications Studies)

Maula Jannah

Buhari Fakkah

ANIAV - Revista de Investigación en Artes Visuales

Laura Benítez Valero

Journal of Physics: Conference Series

GENARO SANCHEZ

Les Cahiers Nantais

Patrick Chaumette

Información tecnológica

LAURA JUDITH HUILLCA AQUINO

Journal of Physical Oceanography

Giulio Boccaletti

Victor Hugo Velez Palacios

Spinal Cord Series and Cases

Haleluya Moshi

Jurnal Penelitian Hutan dan Konservasi Alam

sofian iskandar

Revista Brasileira de Fruticultura

Geraldo marcondes Nogueira

ACTA ZOOLÓGICA MEXICANA (N.S.)

Jose samuel Azcorra Euan

Saranyaphat Boonmee

Languages and Compilers For Parallel Computing

OSCAR ALBERTO ESPINOZA PLATA

The Biblical Annals

Henryk Drawnel

Regional Studies Association …

nese kumral

Manoel Oriosvaldo de Moura

BMJ case reports

Nilanchali Singh

Oceanography

Shenfu Dong

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

3 Planning Your Research: Reviewing the Literature and Developing Questions

ESSENTIAL QUESTIONS

• What is relevant literature? What are the best ways to find it?
• What are the best ways to organize your relevant literature?
• What are the intended outcomes of reviewing your relevant literature?

Nearly all research begins with a review of literature that is relevant to the topic of research, even if it is only a casual review. Reviewing the available literature on your topic is a vital step in the research process. The literature review process provides an anchor for your inquiry. O’Leary (2004, p. 66) states, the “production of new knowledge is fundamentally dependent on past knowledge” because “it is virtually impossible for researchers to add to a body of literature if they are not conversant with it.” By reviewing the literature in the initial stages of the inquiry process, researchers are better able to:

• Understand their topic;
• Develop and focus a topic;
• Provide a clear rationale for, or better situate, their topic;
• Fine-tune their research questions.

In terms of thinking about methodology and the actual research process, reviewing the literature can help researchers:

• Identify well-vetted data collection and analysis methods on their topic;
• Determine whether to replicate a previous study, or develop a completely new study;
• Add rigor and validity to the research by validating the topic, methods, and significance.

Lastly, reviewing the literature also helps the researcher make sense of their findings, in both their field of study and in their educational context, by:

• Assessing whether the findings correlate with findings from another study;
• Determining which of the findings are different than previous studies;
• Determining which of the findings are unique to the researcher’s educational context. [1]

As you may see, the literature review is the backbone, anchor, or foundation of your research study. Overall the review of literature helps you answer three important questions that are the result of the bullet points outlined above. The literature review helps you answer the following:

• What do we know about your topic?
• What do we not know about your topic?
• How does your research address the gap between what we know and what we don’t about your topic?

After reviewing the literature, if you are able to answer those three questions, you will have a very clear and well-rationalized justification for your inquiry. If you cannot answer those questions, then you should probably keep reviewing the literature by looking for related topics or synonyms of major concepts.

While an extensive review of the literature about your topic of study is expected, you should also be realistic as to what you are able to manage. For topics that have a lot of research literature available, make sure you establish parameters for your research, such as:

• Temporal (e.g., only articles in the last 5 or 10 years)
• Content Area (e.g., only in science and math classrooms)
• Age or grade (e.g., only middle school classrooms)
• Research Subject (e.g., girls only, teachers, struggling readers)

These categories provide only a few examples, but parameters like these can make your review of literature much more manageable and your study much more focused.

What Types of Literature Should You Consider in Your Review?

It is helpful to consider the characteristics, purposes, and outcomes of different types of literature. Below are four broad categories I identify within educational research literature. I want to emphasize that my categories are in no way definitive, and only represent my own understanding.

Policy-Based Literature

Policy-based literature includes official documents that outline education policy with which the practitioner needs to be familiar. For example, the Common Core Standards or Content Standards are often refenced in articles to situate the need for research in relation to the standards; if my topic was on place-based education with middle school social studies students, I might have to look at national social studies standards. There also may be initiatives launched by organizations or researchers that become accepted practice. The documents that launch these initiatives (e.g., reports, articles, speeches) would also be useful to review. An example would be the Report of the National Reading Panel: Teaching Children to Read report from the National Reading Panel. These documents may provide rationale, based on the theories and concepts they utilize, and they may provide new ways of thinking about your topic. Similarly, if your topic is based on the local context, recent newspaper articles could also provide policy-type insights. All of these policy-based insights will be useful in providing the landscape or background for your work.

Theoretical Literature

Once you have identified your theoretical perspective, it is also important to locate your research within the appropriate theoretical literature. Many of you may be engaged in highly practice-based or small-scale research and wonder if you need a theoretical basis in your literature review. Regardless of the extent of your project, theoretical literature will help with the rigor and validity of your study and will help identify any theoretical views that underlie your topic. For example, if your study focuses on the place-based education in enhancing social studies students’ learning, it is highly probable that you would cite Kolb’s (1984/2014) work on Experiential Learning. By using Kolb’s work, you situate your research theoretically in the area of experiential learning.

Applicable Literature

Applicable literature will account for the bulk of your literature review. The previous two types of literature provide indication as to where your research is rationalized professionally and situated theoretically. Applicable literature will mainly come from journals related to your specific field of study. If I was doing a study in a social studies classroom, I would look at the journals The Social Studies, Social Education, and Social Studies Research and Practice . Use Google Scholar or your university library databases to examine literature in your specific area. When using these search engines and databases, start as specific as possible with your topic and related concepts. Using the example of place-based learning from above, I would search for “place-based learning” and “social studies” and “middle school” and “historic sites”. If I did not find many articles with this first search, then I would remove “historic sites” and search again. Books or handbooks on research may also have some useful studies to support your literature review section.

Methodological Literature

When sharing or reporting your work, you will want to review and cite research methodology literature to justify the methods you chose. When reading other research articles, pay attention to the research methods used by researchers. It is especially important to find articles that use and cite action research methodology. This type of literature will provide further support of your data gathering and analysis methods. Again, your methods should fit within your theoretical and epistemological stances. In addition, you’ll want to review data collection methods and potentially borrow or adapt rubrics or surveys from other studies.

Sources of Relevant Literature

When searching for these four types of literature, there are two ways to think about possible sources:

• primary sources include government publications, policy documents, research papers, dissertations, conference presentations and institutional occasional papers with accounts of research;
• secondary sources use primary sources as references, such as papers written for professional conferences and journals, books written for practicing professionals and book reviews. This is often called “reference mining” as you look through the reference lists of other studies and then return to the primary source that was cited.

Secondary sources are often just as valuable as primary sources, or potentially more valuable. When beginning your search, secondary sources can provide links to a wealth of primary sources that the secondary source author has already vetted for you, and likely with similar intentions. This is especially true of research handbooks. You will come across both types of literature wherever you search, and they both provide a landscape for your topic and add value to your literature review.

Regardless of being a primary or secondary source, you want to make sure the literature you review is peer-reviewed. Peer-reviewed simply means that the article was reviewed by two to three scholars in the field before it was published. Books, or edited books, would have also gone through a peer-review process. We often recommend teachers to look at professional books from reputable publishing companies and professional organizations, such as ASCD, NCTE, NCTM, or NCSS. This is a way for scholars to objectively review each other’s work to maintain a high level of quality and ethics in the publication of research. Most databases have mostly peer-reviewed journals, and often provide a filter to sort out the non-peer-reviewed journals.

Using the Internet

The internet is a valuable research tool and is becoming increasingly efficient and reliable in providing peer-reviewed literature. Sites like Google Scholar are especially useful. Often, and depending on the topic, the downside of internet-based searches is that it will generate thousands or millions of sources. This can be overwhelming, especially for new researchers, and you will have to develop ways to narrow down the results.

Professional organization websites will also have resources or links to sources that have typically been vetted. With all internet sources, you should evaluate the information for credibility and authority.

Evaluating sources from the Internet

Evaluating internet sources is a whole field of study and research within itself, and an in-depth discussion would take away from the focus of this book. However, O’ Dochartaigh (2007) provides a chapter to help guide the internet source evaluation process. Here is a brief summary, based on O’ Dochartaigh’s book, to give you a general idea of the task of evaluating sources:

• Examine if the material belongs to an advocacy group. Many times, these sources are fine, however, they require extra examination for bias or funding interests.
• As mentioned above, many academic papers are published in refereed journals which are subject to peer-review. Papers found on academic or university websites are typically refereed in some manner; however, some papers are posted by academics on their personal sites and have not been reviewed by other academics. Papers published solely by academics or other experts require further scrutiny before citing.
• When you are reviewing newspaper and magazine articles from the internet be weary of potential conflicts of interest based on the political stance of that periodical.

Therefore, it is wise to consider the objectivity of any source you find on the internet before you accept the literature.

We always recommend that students consider a few questions in their evaluation of sources, which are similar to the formal questions outlined by O’ Dochartaigh (2007):

• Is it clear who is responsible for the document?
• Is there any information about the person or organization responsible for the page?
• Is there a copyright statement?
• Does it have other publications that reinforce its authority?
• Are the sources clearly listed so they can be verified?
• Is there an editorial involvement?
• Are the spelling and grammar correct?
• Are biases and affiliations clearly stated?
• Are there dates for when the document was last updated or revised?

Organizing your Literature

When you begin, here are some things to think about as your read the literature. Again, these are not definitive, but merely provided for guidance. These questions are especially focused on other action research literature:

Questions to Think about as You Examine the Literature

• What was the context of their research?
• Who was involved? Was it a collaborative project?
• Was the choice of using action research as a method justified? Are any models discussed?
• What ‘actions’ actually took place?
• How was data gathered?
• How was data analyzed?
• Were ethical considerations addressed? How?
• What were the conclusions? Were they justified using appropriate evidence?
• Was the report accessible? Useful?
• Is it possible to replicate the study?

Regardless of the amount of literature you review, your challenge will be to organize the literature in way that is manageable and easy to reference. It is important to keep a record of what you read and how it relates conceptually to your topic. Some researchers even use the questions above to organize their literature. It is easy to read and think about the content of an article by making brief notes, however, this is often not enough to initially begin to develop your study or write about your findings. I will state the obvious here: organizing your literature search efficiently from the start is vital!

No matter how you choose to record or document the articles you read (e.g., paper, computer, photo), I would suggest thinking about the format in terms of index cards. Index cards are a very practical and simple model because the space limits you to be precise in recording vital information about each article. I typically create a document on my computer, allow each article the space of an index card, and focus on recording the following information:

• Journal/Book Chapter Title
• Main Arguments/Key findings
• Pertinent Quote(s)
• Implications
• Connective Points (how does it relate to my work and/or other articles)

I find that these aspects provide the information I need to be refreshed on the article and to be able to use it upon review.

There are also a lot of computer applications that are very useful and efficient in managing your literature. For example, Mendeley © provides comprehensive support for reviewing literature, even allowing you to store the article itself and make comments or highlights in text. There are also many citation apps that are helpful if you continue this research agenda and use roughly the same literature for each project.

Using the Literature

Think ahead to when you have collected and read a good amount of literature on your topic. You are now ready to use the literature to think about your topic, your research question, and the methods you plan to use. It might be helpful to peek ahead to Chapter 7 where I discuss writing the literature review for a report to give you an idea of the end goal. The primary purpose of engaging in a literature review is to provide knowledge to construct a framework for understanding the landscape of your topic. I often suggest for students to think of it as constructing an argument for your research decisions, or as if you are telling a story of how we got to this point in researching your topic. Either way you are situating your research in what we know and don’t know about your topic.

Naturally we tend to think about, and potentially write about, the literature in relation to the article’s author (e.g. Clark and Porath (2016) found that…). However, more commonly today in educational research you will find that literature reviews are organized by themes. It can be a little more organic to think about literature in terms of themes because they emerge or become more defined as you read. Also thinking thematically allows the articles to naturally connect and build on each other, whereas thinking in terms of authors can fragment thinking about the topic. In terms of thinking thematically, here are some guidelines:

• Identify the significant themes that have emerged organically from the literature review. These themes would be concepts or ideas that you typed or wrote down in your note-taking or management system.
• Introduce the common concepts or ideas by themes, instead of by authors’ disjointed viewpoints. Paragraphs in a thematic literature review begin like: The research on teacher self-efficacy has identified several key factors that contribute to strong self-efficacy… .
• Lastly, once you have introduced each theme and explained it, then present evidence from your readings to demonstrate the parameters of the knowledge on the theme, including areas of agreement and disagreement among researchers. Using the evidence, explain what the evidence for the theme means to your topic and any of your own relational or critical commentary.

Another way to think about structuring a literature review is a funnel model. A funnel model goes from broad topic, to sub-topics, to link to the study being undertaken. You can think of a literature review as a broad argument using mini-arguments. To use the funnel model, list your topic and the related subtopics, then design questions to answer with the literature. For example if our topic was discussion-based online learning, we might ask the following questions before reading the literature:

• Why is discussion important in learning?
• How does discussion support the development of social, cognitive, and teacher presence in an online course?
• What does research say about the use of traditional discussion boards ?
• What does the research say about asynchronous, video-based discussion ?
• How have other researchers compared written and video responses?
• How does this literature review link to my study?

O’Leary (2004) provides an interesting representation and model of the purpose for the literature review in the research process, in Figure 3.1. We will leave this for you to think about before moving on to Chapter 4.

• We will talk about this aspect of literature reviews further in Chapters 6 and 7. ↵

Action Research Copyright © by J. Spencer Clark; Suzanne Porath; Julie Thiele; and Morgan Jobe is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

Funnel Plot in a Systematic Review

Automate every stage of your literature review to produce evidence-based research faster and more accurately.

Systematic reviews are considered the gold standard in evidence-based research. These intensive “studies of studies” involve taking a systematic approach to collect, assess, and synthesize relevant literature on a specific subject, including peer-reviewed journal articles, gray literature, and other sources, to answer a well-defined research question. Systematic reviews, like other reviews, can also be prone to bias. The bias types that are most evaluated in systematic reviews are selection bias, reporting bias, detection bias, and attrition bias. But this can be prevented through several checks before you start the review. For example, ensuring that the eligibility criteria in a systematic review is well defined when designing the systematic review protocol of a systematic review article , will reduce the risk of selection bias during study selection. An effective tool called a funnel plot is designed to examine the existence of a publication bias, among studies included in a systematic review, or the tendency of authors to only publish studies with significant results, other reporting biases, and small study effects.

What Is A Funnel Plot In Systematic Reviews?

A funnel plot is a simple scatter plot of the treatment effects estimated from individual studies against a measure of the study size. The x-axis (horizontal axis) shows the results of the study, expressed as an odds or risk ratio or a percent difference, while the y-axis (vertical axis) displays the sample size or an index of precision. Other measures could also be plotted, such as reciprocals, or variances.

The scale of the y-axis is reversed; studies with higher precision are placed at the top while studies with lower precision are placed at the bottom. At the bottom where the low precision studies are placed, the points that represent the mean value of effect in each study are widely spread. The spread of these points begins to reduce, as you move upwards in the y-axis. This effect creates a plot that resembles a pyramid or an inverted funnel.

What Is The Purpose Of Funnel Plots?

A funnel plot is designed to check for the existence of publication bias, other reporting biases, and systematic heterogeneity in a systematic review. These are biases caused by the absence of information from unpublished sources (missing studies), or selective outcome reporting of a study’s result (missing outcomes). For example, the study authors may omit information that they may feel does not agree with their findings. In the absence of publication bias, the funnel plot, as its name suggests, should create a symmetrical funnel-shaped distribution. Deviations from this, like an asymmetrical plot, may indicate that there is a bias.

That said, it’s important to note that publication bias is only one of the issues examined by funnel plots. These plots can also assess small study effects, or the tendency for smaller studies to show larger treatment effects.

How to Interpret a Funnel Plot?

Funnel plots, typically, are symmetrical or asymmetrical. Here’s how to interpret them:

Symmetrical Funnel Plot

A “well-behaved” data set, one where the precision of the estimated intervention effect increases as the size of the study increases will yield a symmetric inverted funnel shape. This signifies the unlikeliness of bias.

Asymmetrical Funnel Plot

An asymmetrical funnel plot can indicate the presence of a bias, suggesting a relationship between treatment effect estimate and study precision. With these deviant shapes, you can assume the possibility of publication bias, small-study effects, or study heterogeneity. Asymmetry can also be caused by the use of an inappropriate effect measure. Whatever the cause, an asymmetric funnel plot leads to doubts about the systematic review. In this case, an investigation must be done to get to the bottom of the possible cause and correct the mistake.

Systematic reviews are one of the most rigorous research processes. But they’re not without risks as they can be prone to biases. Researchers can use tools to prevent these challenges—a funnel plot is one way to do it as it’s designed to check for publication bias, other reporting biases, and small study effects. Another great way to ensure that your systematic review is accurate, objective, and comprehensive is to use a literature review software like DistillerSR, which helps you automate each stage of your review to securely produce evidence-based research faster, and more precisely. This gives you the time and energy to focus on evaluating other information to ensure that your protocol is free from any biases.

3 Reasons to Connect

Skip to Content

Massey University

• Search OWLL
• Handouts (Printable)
• Pre-reading Service
• StudyUp Recordings
• StudyUp Postgraduate
• Academic writing
• Intro to academic writing
• What is academic writing?
• Writing objectively
• Writing concisely
• 1st vs. 3rd person
• Inclusive language
• Te Reo Māori
• Assignment planning
• Assignment planning calculator
• Interpreting the assignment question
• Command words
• Organising points
• Researching
• Identifying academic sources
• Evaluating source quality
• Editing & proofreading
• Apostrophes
• Other punctuation
• Active voice
• American vs. British spelling
• Conditionals
• Prepositions
• Pronoun Reference
• Sentence fragments
• Sentence Structure
• Subject-verb agreement
• Formatting and layout
• Word limits and assignment length
• Commonly confused words
• How assignments are marked
• Marking guides
• Getting an A
• Levels of assessment
• Using feedback
• Professional emails
• Forum posts
• Forum netiquette guidelines
• Sharing personal information
• Writing about personal experiences
• Assignment types
• What is an essay?
• Essay planning and structure
• Introduction
• Thesis statement
• Body paragraphs
• Essay revision
• Essay writing resources
• What is a report?
• Report structure
• Analysing issues for a report
• Business report
• What is a business report?
• Business report structure
• Inductive vs. deductive reports
• Other kinds of business communication
• Business report format and layout
• What is a lab report?
• Lab report structure
• Science lab report writing resources
• Psychology lab report writing resources
• Lab report body paragraphs
• Literature review
• What is a literature review?
• Writing a literature review

Literature review structure

• Literature review writing resources
• Research proposal
• Writing a research proposal
• Research proposal structure
• Other types
• Article critique
• Book review
• Annotated bibliography
• Reflective writing
• Oral presentation
• Thesis / dissertation
• Article / conference paper
• Shorter responses
• Computer skills
• Microsoft Word
• Basic formatting
• Images, tables, & figures
• Long documents
• Microsoft Excel
• Basic spreadsheets
• Navigating & printing spreadsheets
• Charts / graphs & formulas
• Microsoft PowerPoint
• Basic skills
• Advanced skills
• Distance study
• Getting started
• How to study
• Online study techniques
• Distance support
• Reading & writing
• Reading strategies
• Writing strategies
• Grammar resources
• Listening & speaking
• Listening strategies
• Speaking strategies
• Maths & statistics
• Trigonometry
• Finance formulas
• Postgraduate study
• Intro to postgrad study
• Planning postgrad study
• Postgrad resources
• Postgrad assignment types
• Referencing
• Intro to referencing
• What is referencing?
• Why reference?
• Common knowledge
• Referencing styles
• What type of source is this?
• Reference list vs. bibliography
• Referencing software
• Quoting & paraphrasing
• Paraphrasing & summarising
• Paraphrasing techniques
• APA Interactive
• In-text citation
• Reference list
• Online material
• Other material
• Headings in APA
• Tables and Figures
• Referencing elements
• 5th vs. 6th edition
• 6th vs. 7th edition
• Chicago style
• Chicago Interactive
• About notes system
• Notes referencing elements
• Quoting and paraphrasing
• Author-date system
• MLA Interactive
• Abbreviations
• List of works cited
• Captions for images
• 8th vs 9th edition
• Oxford style
• Other styles
• Harvard style
• Vancouver style
• Legal citations
• Visual material
• Sample assignments
• Sample essay 1
• Sample essay 2
• Sample annotated bibliography
• Sample book review
• Study skills
• Time management
• Intro to time management
• Procrastination & perfectionism
• Goals & motivation
• Time management for internal students
• Time management for distance students
• Memory skills
• Principles of good memory
• Memory strategies
• Note-taking
• Note-taking methods
• Note-taking in lectures
• Note-taking while reading
• Digital note-taking
• Reading styles
• In-depth reading
• Reading comprehension
• Reading academic material
• Reading a journal article
• Reading an academic book
• Critical thinking
• What is critical thinking?
• Constructing an argument
• Critical reading
• Logical fallacies
• Tests & exams
• Exam & test study
• Planning exam study
• Gathering & sorting information
• Reviewing past exams
• Phases of revision
• Last-minute study strategies
• Question types
• Short answer
• Multi-choice
• Problem / computational
• Case-study / scenario
• Open book exam
• Open web exam or test
• Take home test
• In the exam
• Online exam
• Physical exam

The main attribute of a good literature review is that it is well structured.

A good place to start is to look at other theses, but make sure that you are looking at a good example of a thesis (your supervisor may recommend a few for you to look at). When examining these pieces of work, try to identify the structure and see how they have linked their ideas together.

Before you emulate what you have read, be aware that plagiarism extends to copying the structure of others' work, so please acknowledge any influences appropriately. In addition, to assist you in organising your review, see your supervisor or department to identify the format of your thesis, as this can impact the organisation of the literature review (Mauch & Birch, 1998).

The information in the literature review is synthesised, or brought together to form a cohesive whole. Those who read the review should clearly understand the reasons for selecting your research area or question, its relationship to past work, and the central procedures that have been employed by past investigations. In addition, they should know the weaknesses of past studies and how your research contributes to this field in the advancement of knowledge.

Beginning your review

As you review books and journals, write down the topic words that you have selected. It is suggested that you create a mind map of the all of the terms that apply to your topic before conducting a literature search. This mind map can then be used to guide your literature search as well as making sure that you discuss pertinent concepts in the review itself. See an example mind map .

This mind map and its sections can also be the subsections that you use for storing the results of your research e.g. in Endnote .

The following is a guide to structuring your literature review based on Newman et al. (1997). It is recommended that the literature review forms one separate chapter of your thesis. This is most common when the research problem is defined early on and remains relatively unchanged. However, if the direction of the study changes due to new research findings, then new literature may need to be included in subsequent sections or chapters. Each sub-section of the suggested review structure will be expanded upon separately.

This section is most likely the longest section of the thesis. It includes:

• An introduction
• A review of the past and present literature in relation to your research purpose
• Clarification of the purpose of study
• General hypotheses to be tested.

The review begins with an introduction that discusses the topic, key concepts and terms, and describes the scope and organisation of the review. You can use the two-topic format or the funnel format (see Alternative methods below). Both formats include the identification of key topics that will be covered in the review. In addition, these formats guide the way the review is structured, which makes the writing of the task easier because you can focus on writing one section at a time and keep on track with your review topics.

This section outlines relevant theories that impact your study.

You may find it difficult to find information for this section, especially in new fields of research. Still, even in ground-breaking research, there should be some theoretical foundation upon which your work rests. There may not be a strong link in this case, but it does help if there is some basis for your work, albeit indirect. Whatever topics you include in your review, they must bear some relationship to your focus. Though you may not find literature that specifically relates to your topic you should integrate key points from related studies that to allow you to make inferences and indicate what you expect to happen in your study.

This section is a review of the literature on the instruments or measures you will use as part of your study.

You need to present evidence that supports your choice of instrument over those not chosen. This section should be focused on relevant literature specific to the study. One suggestion is to examine the most current instruments first and work back from there. You need to include reliability and validity estimates and a description of the samples that have received the instrument. When dealing with many variables, it is useful to write a separate section on each variable in the review (Cone & Foster, 1993; Newman et al., 1997).

The summary is a precis of what has been written about in the chapter. It should not be verbose, or a repetition of the entire contents of that chapter, but rather a succinct account of the current state of knowledge on your topic and the instruments used in the study. In addition, there should be a sense that you have explained the background to your study that endorses the decision you have made to study your topic.

Sequence of the review structure

Your review will not only synthesise the literature, but it must present the literature in a logical sequence or order. Your aim is to indicate to the reader an understanding of the problem under investigation. However you organise it, your review should highlight important aspects of the literature; especially areas that you wish to address or improve on. You can organise your review by:

• moving from general concepts to the more specific concepts
• type of research
• or any method that makes sense (Cone & Foster, 1993; Newman et al., 1997)
• Cover studies that examine related independent variables . e.g. “What strategies enhance organisational effectiveness?” You can organise the review according to the strategy type.
• Cover studies that examine related dependent variables . e.g. “Characteristics of adult children of alcoholics”. You could organise the review according to personality studies, drinking patterns, relationship skills, and so on.
• Organise by type of design . The order usually moves from weaker to stronger designs, for example correlational before experimental designs.
• Organise by chronology . Particular theories may develop according to strict chronological changes perhaps due to technology, expansion of theories, or social changes, etc.
• Organise by theoretical premises . This is useful if you are dealing with competing explanations or when different theories contribute to your research question.
• Organise by findings . You may wish to use findings to develop a rationale for your studies. This can be the most difficult way to organise your review.

There is no single best way to organise your chapter, so do not waste precious time looking for it; just write it.

Alternative methods

There are other way to organise your review if you find the above suggestions unhelpful. If your thesis is a two-topic thesis , you can use these two variables as your guide for the organisation of the literature review. If your thesis examines many variables, then the funnel-format is the other way to organise your review.

The two-topic format thesis is a thesis that examines the relationship between two variables. This thesis tends to be easier to organise simply because of the lack of variables you need to discuss. This does not mean the review is less extensive or necessarily easier. Examples of two-topic formats are: Does depression relate to quality of life? Does anxiety relate to learning? And so on. The organisation of your review will look something like this: Firstly, an introductory paragraph, a literature review on topic one, a literature review on topic two, a review of the literature linking topics one and two, and finally, a statement of the purpose of the proposed study. In some cases, this may be included in the summary.

Alternatively, the funnel format, which is commonly used in essay writing, may be used to structure your literature review. In this approach you begin by discussing the topic in the most general of terms, and then gradually narrow the focus of the discussion to become closer and closer to the topic or purpose of the present study.

References and further reading

Cone, J. D., & Foster, S. L. (2006). Dissertations and theses from start to finish: Psychology and related fields (2nd ed.). American Psychological Association. [Massey Library link]

Mauch, J. E., & Park, N. (2003). Guide to the successful thesis and dissertation: A handbook for students and faculty (5th ed.). Marcel Dekker. [Massey Library link]

Newman, I., Benz, C. R., Weis, D., & McNeil, K. (1997). Theses and dissertations: A guide to writing in the social and physical sciences. University Press of America. [Massey Library link]

Page authorised by Director - Centre for Learner Success Last updated on 26 February, 2020

• Academic Q+A

Have a study or assignment writing question? Ask an expert at Academic Q+A

Live online workshops

• StudyUp (undergraduate)
• Campus workshops
• Albany (undergraduate)
• Albany (postgraduate)
• Albany (distance)
• Manawatu (undergraduate)
• Manawatu (postgraduate)

Upcoming events

• All upcoming events
• Academic writing and learning support
• 0800 MASSEY | (+64 6 350 5701)
• [email protected]
• Online form

• About LiveInnovation.org
• Prof. Dr. Francisco Tigre Moura
• Academic Research
• Live AM: Artist Monitor
• Live FM: Fan Monitor
• Books and e-Books
• Media/Events
• Consumer Behavior
• Marketing Research
• Statistics Support
• Thesis Writing

• Research Support

How To Write an Excellent LITERATURE REVIEW For Your Research

Let me guess:  You know what you will write about, you have mastered Chapter 1 (Introduction) but you don’t know how to move forward with your Chapter 2 (literature review)?

Alright, alright… I feel your pain. But that’s why I’ve written down some recommendations that hopefully will be useful for you!

INITIAL WARNING

Look, I need to be honest with you here. Writing your theoretical background will not be the most entertaining thing you have done in your life.

Writing a theoretical background is definitely not like sitting under the sun on a tropical beach with the sea water washing your feet exactly when the waiter comes to serve you with a dish of fresh fried fish and a bucket filled with extremely cold beer while your gorgeous smiley girlfriend (wearing a bikini) is singing beautifully “God Only Knows” (Best Beach Boys song). If you don’t know the band or the song, shame on you!) and this is only the first day of vacation.

Writing a literature review is more like going to your City Hall at 7.30am on a hot summer Monday and standing in line for 2 hours at an old building with no AC to pay your taxes to an impolite government employee.

Is it fun? NO . Do you have any choice? NO . And is it important? VERY .

Summing up: Head down, take a deep breath and get it done with!

So how do you do it? SIMPLE. I suggest you do the following:

1. FIRST OF ALL: In this chapter you will discuss only LITERATURE (theoretical concepts) related to your topic.

What does that mean? Assume you want to understand the buying criteria of consumers in regards to Smartphones. Assume this is the title of your Thesis: “ Understanding Consumers’ Evaluation of Cellphone Attributes during Their Purchase Decision Making Process ”.

In this chapter you will NOT discuss cellphones in specific! (e.g. Describe types, brands, etc). This is simply your CONTEXT . In this chapter you will discuss THEORY ( applied to your context! ).

FOR EXAMPLE: In this case, you could discuss the following topics:

• Decision Making Process (BROAD TOPIC)
• Product Value Perception
• Consumer Motivation and Involvement

3.1 Consumer Involvement with Electronic Products

• Attribute Evaluation

4.1 Fischbein’s Theory: Multi-attribute Model

• Smartphone Attribute Evaluation (SPECIFIC TOPIC)
• Research Gap

6.1 Research Aim/Question

With this structure did we discuss types of cellphones or their brands? NO ! Again, NO ! There are only THEORETICAL concepts listed and related to the context of the study!

Also, did you notice that the list goes from the BROADEST (most general) topic to the MOST SPECIFIC topic ?

Exactly! That’s the second recommendation!

Video Support: Literature Review

In case you are enjoying the article, do not forget to watch the video with further support on how to write the literature review in your thesis.

 2. SECOND OF ALL: Structure the SCOPE of your study.

Think of your theoretical background (or Literature review) as a FUNNEL . Yup, you heard it. A FUNNEL!   You must structure it so that you can cover it from the broadest topic until the most specific !

Now here it is difficult for me to help because this is very topic specific. So use your imagination and please try to envision the figure below as if it was a FUNNEL.

 (I know it has nothing to do with a funnel but that’s the maximum I could manage, so don’t judge my funnel development skills!).

But HOW do you know how to structure your concepts from the broadest to the most specific? This depends on EACH project. So, sorry but I can’t help you with this.

Discuss it with your SUPERVISOR! I’m just a friend trying to help!

3. THIRD OF ALL: Conclude the chapter by presenting your RESEARCH GAP and RESEARCH QUESTION (or Aim).

The theoretical background should BROADLY enable the reader to do the following:

• Become familiar with the main topics related to your study (So that he/she can understand what you have developed!)
• Understand the logic of how you got to your research question or research aim (depends how you framed it!).

Thus, I recommend you to CONCLUDE your theoretical background by discussing your research GAP and your RESEARCH question.

• RESEARCH GAP

In this section you will generally summarize the topics discussed before and highlight what needs to be researched in your field ( the RESEARCH GAP you have identified! )

• RESEARCH AIM/QUESTION

In this sub-section you will remind the reader what your research aim is. Or you may have framed the aim into a research question and will present it here.

Basically , the reader has to read your RESEARCH GAP section and think: “ OHHH… Now I get it how all concepts CONNECT to each other and what you are trying to investigate ”!

4. FOURTH OF ALL: Use RELIABLE sources and WISELY!

A) Use almost ONLY scientific articles to describe concepts and previous studies! And why

• Scientific articles are peer reviewed, thus credible sources! (Generally speaking, I know).
• Check HERE to see their ranking and use HIGH ranked journals!

B) DO NOT use commercial websites to describe concepts!

• BBC, CNN and others are not scientific sources! But what if they mention a study? Then remember: ALWAYS LOOK FOR THE PRIMARY SOURCE OF SECONDARY DATA!

C) Avoid OVERUSING the same author to explain a concept.

Have you used the same source three times or more on the same page or section?

• If so, basically you are telling your reader: I am too LAZY to look for other sources so I’m just going to cite this same authors as much as possible. And of course, that is NOT a good sign!

In general you must have done the following in this chapter:

• In this chapter you have discussed only LITERATURE (theoretical concepts) related to your topic.
• You structured the SCOPE of your study.
• You concluded the chapter by presenting your RESEARCH GAP and RESEARCH QUESTION (or Aim).
• You used only RELIABLE sources and WISELY!

  So now that you are DONE with Chapter Two, you can focus on the methodology of the thesis!

(In case you want to thank me later: I like cappuccino and Formula 1).

DOWNLOAD THE RECOMMENDATIONS

Did you like this article? Would like to have these recommendations with you while studying?

GREAT! Simply download the file here with all details: LiveInnovation.org - Developing a Theoretical Background Literature Review of a Thesis.pdf

In case you would like to have more research suggestions, check our research resources section .

RELATED ARTICLES MORE FROM AUTHOR

SOUNDS LIKE A THESIS is now available on Spotify!

Download Our e-Book: “Sounds Like A Thesis”

SPSS Tutorial Series on YouTube: Learn Quickly and Easily

Privacy overview.

This is an necessary category.

This is an non-necessary category.

Literature reviews

• Book a session

What is a literature review?

Finding your sources, structuring your review, critical writing, literature reviews, critiques and annotated bibliographies.

• Science and Health-based reviews This link opens in a new window
• Quick resources (5-10 mins)
• e-learning and books (30 mins+)
• SkillsCheck This link opens in a new window
• ⬅ Back to Skills Centre This link opens in a new window

Looking for sessions and tutorials on this topic? Find out more about our session types and how to register to book for sessions. You can view our full and up-to-date availability in UniHub  Appointments and Events . 

Not sure where to start developing your academic skills? Take the  SkillsCheck for personalised recommendations on how to build your academic writing and study skills alongside your course.

Literature reviews take on many forms at university: you could be asked to write a literature review as a stand-alone document or as part of a dissertation or thesis. You may also be asked to write an annotated bibliography or a critical review - both of these assignments are closely related to literature reviews, and follow many of the same conventions.

A literature review is an extended piece of writing that should collate, link and evaluate key sources related to a chosen topic or research question. Rather than simply summarising the existing research on your chosen topic, you should aim to show which papers can be clustered around a similar theme or topic - they may have a shared methodology, or have been carried out in the same context. You will be looking for strengths and weaknesses in the research, questioning the relevance and significance of the results in relation to your topic, and looking for any gaps or under researched areas. Your writing should make these thoughts and evaluations clear to the reader, so that they have a good understanding and overview of the body of research you have chosen to investigate.

Here is a short video that explains a literature review from the perspective of the reader:  

Salter, J. [Dr. Jodie Salter]. (2016, March 14). Writing the Literature Review: A Banquet Hall Analogy [Video file]. Retrieved from: https://www.youtube.com/watch?v=QE_Us8UjS6

Using Library Search

The more you read around your subject, the more familiar you will become with the current literature, and you will start to build a map of the sources you already have, and the information you are missing.  A clear search strategy can help fill these gaps in your knowledge, and your themes or topics of interest can be used as key search terms when looking for further resources.   

Top tips for searching the Library Gateway

• Use Boolean terms to help the search engine recognise which words should be treated as a phrase. For example, if you search “costume design” , the search engine will know to treat “costume design” as a phrase, not two separate words.  
• You can then add AND and OR to add in additional terms and synonyms. For example, “costume design” AND “film” will only find articles or sources that include both of these terms together, helping you to narrow your search. To go wider, think about adding in synonyms using the OR function: “costume design” AND “film” or “cinema” or “movies”.  
• You can also use an asterisk (*) to search for a word stem to help widen your search. For example, if you search teach*, this will find articles that include the word teach, teacher, teachers, teaching and so on.  
• Decide at the start on what your inclusion and exclusion criteria  will be. These might include limits on: - Date of publication - Language - Type/group of participants - Peer-reviewed journals - Keywords and synonyms - Type of study, ie. systematic review, case study etc.  

Search strategies

Explore the following resources for more information on search strategy models:

• PICO (Population, Intervention, Comparison, Outcome) - Used primarily in Health Sciences but offers a clear step-by-step approach to literature searching that could be adapted for use in other subjects.
• SPIDER - Developed from the PICK method, SPIDER searching is used mainly in qualitative research to identify a phenomenon or behaviour, rather than a specific intervention (more quantitative).

Finding the gap

You might have heard about  finding the gap  in your literature review - but what does this mean?

Looking for the gap in the literature means finding an aspect of your topic that hasn't been fully explored by researchers. This might be because you are researching a new technique or technology, or that your method or approach hasn't been used before in your field of study. You don't always need to find a gap, but it is a good way of demonstrating your literature searching skills and ability to compare a wide range of different sources. If you are able to find one, introduce the gap towards the end of the literature review, so that the reader can trace your path through the evidence first.

Literature review structure: A three-tier model

Imagine you are explaining your dissertation topic to a friend for the first time. Even for someone on the same degree course, they would need some context on the topic before you introduced more detail and complex examples.

A literature review follows the same  ‘funnel’ narrative , moving from general themes to more specific detail:

• Appraisal grid A template for taking notes from reading that enables you to find clear links and similarities for discussion in your literature review.
• PEP tables A template for organising notes from your reading by themes, theories and perspectives.

Paragraph structure

Each paragraph of your literature review should bring together or synthesise two or more pieces of reading (these could be articles, book chapters, reports, videos, policy documents etc.) 

Synthesis is the term we use in academic writing to describe the process of creating an opinion or argument based on a trend you find in the literature. If you are able to synthesis evidence, you are not only creating a robust argument (by avoiding relying too heavily on just one piece of writing) but you are also showing that you are a critical writer that can make conclusions based on a diverse range of evidence. Bingo!

As in other forms of academic writing, the paragraphs in your literature review should have four key sections:

Compare the following paragraphs against this four-part structure - which version is more critical?  

Although both paragraphs use the TIED structure, we can see that the discussion in paragraph B is much more developed, and gives a specific suggestion about how future research could be conducted. We can also see that the evidence in paragraph B is clearly linked together , and that the conclusions or critical features of the papers are explained to the reader. Although drawing on the same evidence, paragraph A summarises and describes the research papers , rather than giving an evaluation or clear comparison of the different sources.

Focusing on the discussion sections (in bold), we can see that paragraph B is more critical , as it answers a key questions to keep in mind when writing critically: 'so what?'  What conclusion or take home message do you want the reader to get from the evidence you have presented?  ‘Therefore’, ‘Consequently’ and ‘As a result’ are all good terms to use here, as they prompt you to be clear and explicitly explain on interpretation of the source you have included.

Other  types of literature review

Not all literature reviews form part of a dissertation. Use the tabs below for guidance on different assignment formats related to literature reviews:

• What are the emotional and behavioral impacts of therapy dogs for autistic children?
• How might aptitude be tested and measured in puppies selected for guide dog training?
• What are the key success factors for dogs as social media influencers on Instagram and Facebook?

Your initial reading will help you to identify trends or themes in the literature that might help to focus your search. You can then follow the standard structure for writing a  literature review, using the  funnel structure  from this guide.

A critical review, or ‘critique’, involves breaking a journal article down into its key sections so that you can evaluate the strengths and weaknesses of each part. Making notes on each of the following headings is a useful way to kickstart your analysis of any article:

• Research aim
• Research approach (ie. quantitative)
• Ethical issues
• Data collection method
• Data analysis method
• Generalisability/transferability

This list is not exhaustive and depending on your discipline, there may be other relevant categories to focus on in the article, such as theoretical models or implications for practice. The subheadings from the article will also provide an overview of the key sections to include in your review, and you may already have an idea from your wider reading of what sections often appear in articles in your field of study. Breaking down the article in this way allows you to focus your critique and evaluation, highlighting significant or relevant aspects of the article to the reader . Your assessment criteria will help you to identify which elements of the article to include in your critique: for example, if you needed to include a reflection on how the article links to your professional practice, it would make sense to include your thoughts on the articles key findings and transferability in your critique. For examples of sentence starters and academic language to use in your critical review, take a look at the following resources:

• Writing a critical review, UCL
• Academic Phrasebank, University of Manchester

An annotated bibliography combines a correctly formatted list of references (APA) with a short paragraph that gives:

• a short summary of the source, that picks out the key points of the article, such as context and setting, participants and conclusions;
• a brief evaluation of the strengths and weaknesses of the article;
• a sentence or two on the relevance of the source to your question or topic – what does it contribute to your knowledge of the subject, and in what ways might its relevance be limited?

​ ​ Sources are not discussed together in the same paragraph, but the document itself will have a key theme or topic that ties the different sources together – almost like a module reading list: Brym, R., Godbout, M., Hoffbauer, A., Menard, G. & Huiquan Zhang, T. (2014) Social media in the 2011 Egyptian uprising. The British Journal of Sociology , 65 : 266-271. https://doi.org/10.1111/1468-4446.12080 This article conducts a comparative analysis of quantitative data on social media usage and political engagement during the 2011 Egyptian uprising, using new bit.ly and Gallup survey results. The study generates a large amount of data on the key differences in social media usage between active demonstrators and sympathetic onlookers. Most significantly, the study explores the key drivers of participating in social unrest, such as a lack of confidence in the government, and how these are facilitated by social media. However, by only gathering quantitative data, the study is limited in its ability to provide an insight into how protestors narrate and explain their involvement in the protests in their own words. Overall, this article offers significant evidence to support a study of the importance of social media in contemporary political movements, and is particularly useful as one of few studies to focus on events outside of Europe and North America. Be sure to check your assessment criteria for tips on how you should evaluate your sources: for example, you might be asked to include specific methodology types or to link your sources to professional practice.

Two key principles apply to every literature review, whether it is part of a dissertation or an individual assignment:

 1. A literature review is more than just a list of sources. The articles and evidence you include must be linked together around shared themes and characteristics, or highlight significant disagreements and contrast. Map your reading using keywords or themes that occur in multiple articles - these can be used as subheadings in your draft literature review.

2. While it is important to show that you are familiar with research in your field, you also need to show that you can evaluate and offer interpretations of the evidence you present to the reader. Remember to keep answering the 'so what?' question as you write.  

• << Previous: Book a session
• Next: Science and Health-based reviews >>
• Last Updated: Feb 29, 2024 9:46 AM
• URL: https://libguides.shu.ac.uk/literaturereviews

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Literature reviews: Reviewing for research

• Reviewing for research
• Stand-alone review
• Scoping and planning
• Screening and appraising
• The process of reviewing
• Planning a search strategy

On this page:

“The researcher first addresses the current state of knowledge about the study question. Then, based on these findings, the researcher proposes a thesis defining an issue for further study” Lawrence A Machi, The Literature Review: Six Steps to Success

This page concentrates on undertaking a literature review as the first stage of a research-based dissertation or independent project. Such a review needs to both position your study within the wider literature and justify the research question that you are asking. This page is looking at a traditional literature review rather than a systematic review which has its own page in this guide.

What a literature review for a research project should do

Position your research within the wider body of literature on the topic.

Demonstrate an in-depth understanding of your topic area.

Identify who the major thinkers are.

Identify what research has already been done in that area.

Find gaps or new directions to help you formulate your own question.

Identify the main research methodologies in your area.

Identify the main areas of agreement or controversy.

Convince the reader that your research questions are significant , important and interesting.

Convince the reader that your research will make an original contribution.

What a literature review is, and isn't

• Critical analysis
• Evaluation of previous research on a topic
• Addressing a clearly articulated question (or questions)

• A descriptive list
• Summaries of books/articles
• Exhausted bibliography of everything written on the topic
• Your arguments and ideas

The structure of a literature review

Funnel structure.

The most common structure is one where you start your literature review looking at the bigger picture and then increasingly focus onto the specific aspects you are interested in. This is known as a funnel structure.

• As can be seen, the review starts by looking at a fair number of papers but not in great detail - these position your research  within the wider literature to show how your topic fits in to the bigger picture.
• The review then moves to consider a number of papers (less than the previous part) in a bit more detail. These papers cover more narrow categories that are closer to your topic but not matched directly. 
• The review should then focus on a few papers that are the most relevant to your work. You are likely to look at these in considerably more detail.
• You finish the review by confirming  how the literature has led you to your specific question.

Jigsaw structure

Sometimes, if your topic area has clear sub-areas it can be more appropriate to use a jigsaw structure.

• Give a proportion of your review to each sub-area.
• Discuss the links between each of the sub-areas.
• Make sure your conclusion pulls these together and shows where your research will fit into this picture.

Chronological structure

This structure is not particularly common but can be useful for some reviews - specifically when you need to show how ideas have changed through time . For example, in medicine you could look at how treatments for a particular condition have progressed from early treatments to the present day.

• You will begin with the earliest papers, grouping them together by publication date. For example, papers from 1990-1999 then papers from 2000-2009, then 2010-2019, finishing with the very latest papers from 2020 onwards. 
• You focus on how the research (ideas, theories or methods) has changed over the period and emphasise the key changes that happened.
• You finish by showing how this led you to choose the direction that your own research will take.

Top Tips Video

• << Previous: Home
• Next: Stand-alone review >>
• Last Updated: Apr 9, 2024 3:41 PM
• URL: https://libguides.hull.ac.uk/literaturereviews
• Login to LibApps
• Library websites Privacy Policy
• University of Hull privacy policy & cookies
• Website terms and conditions
• Accessibility
• Report a problem

Skip navigation

• Log in to UX Certification

World Leaders in Research-Based User Experience

The funnel technique in qualitative user research.

July 24, 2022 2022-07-24

• Email article
• Share on LinkedIn
• Share on Twitter

The funnel technique has been around since qualitative interviews emerged as a research method. This technique involves asking broad  open-ended questions  before gradually introducing more narrowly-scoped open-ended questions, as well as  closed questions . 

In This Article:

Why is it called the funnel technique, the funnel technique in user interviews, the funnel technique in qualitative usability tests.

This idea of  starting broad before getting more specific  is valuable in other types of studies besides user interviews. This technique can help you organize:

• Interview questions
• Followup questions
• Usability tasks
• Research in a multimethod study

This article discusses how the funnel technique can be used in  user interviews  and in  moderated usability tests . 

A funnel is broad at the top and narrow at the bottom, which makes it easy to pour a substance (for example, oil or rice) into a container with a narrow opening (for example, a bottle or jar). 

Similarly, the funnel technique in user research involves moving from broad to narrow — in other words, from general to specific. The funnel is a fitting metaphor because a user interview or usability-test session should start with broad, exploratory questions or tasks before introducing specific, narrowly scoped questions or tasks. 

The purpose of the funnel technique is to avoid influencing user behavior or perceptions as much as possible. When we ask specific questions or give specific tasks too early in a research session, we risk introducing bias and missing important data.

An  interview guide  for a user interview will usually consist of 5–8  open-ended questions  that get participants to share relevant stories or experiences. These are followed by followup questions, which can be open-ended or closed.

An interview should start with broad, open-ended questions like  Tell me about the last time you ordered movie tickets .

Starting with broad open-ended questions:

• Gets the participant comfortable with talking
• Allows the participant to begin sharing stories
• Generates lots of new, unanticipated information
• Avoids the researcher  priming  the participant

After the participant responds, the interviewer should ask open-ended followup questions (known as  probing questions ) to dig into areas of the participant’s response that the researcher wants to learn more about. These include questions like:

• Can you expand on that?
• What do you mean by that?
• How do you feel about that?
• Why do you think that?

The interviewer may also want to gather additional information and details that the participant omitted, by introducing closed questions, such as: 

• When did you see this movie?
• Did you go alone?
• How long was the movie?

For each main question prepared in the interview guide,  the interviewer is moving from broad open-ended questions to closed questions . With each new question in the guide, the funneling process repeats.

When we make use of the funnel technique, we allow space for us to  learn new things before gathering detail.  This approach to interviewing is the best way to build rapport and learn what users really think without priming them.

In qualitative usability testing, funneling can be used in: 

• Creating and ordering tasks
• Devising the sequence of followup questions after a participant completes a task

Funneling Tasks

The same process of moving from broad to specific can apply to constructing and ordering tasks for a qualitative usability test. Broad exploratory tasks — which allow us to learn how people do things on their own — should be given before specific, directed tasks that ask users to find or do something that they might not otherwise do.

Another metaphor for the funnel technique is a mouse in a maze. Imagine a researcher in a lab wants to see if a mouse can find a piece of cheese in a large maze. They might start off by giving the mouse the entire maze and seeing whether it finds the cheese. After some time, if the mouse hasn’t arrived at the cheese, they could slowly start closing off incorrect pathways, increasing the chances that the mouse will find the cheese.

In this metaphor, the mouse is the participant, and the cheese is the aspect of the design that we’re interested in studying. It’s better to first see if our participants get there on their own, but we might have to nudge them a bit.

If we started with task 3, and moved in the reverse order, we would prime participants on how to behave in task 1 and task 2. It’s likely that we would miss out on learning how they would do things on their own without prompting. Funneling tasks, by starting with broad exploratory tasks before introducing directed tasks, ensures you get valid data about behavior.

The funnel technique is the idea behind  stepped tasks . Stepped tasks are multistep tasks that start broad and then provide more specific instructions as necessary. 

For example, let’s imagine that we’re interested in studying the  comparison feature  on an ecommerce site. 

If we get to Task 4.2 in the stepped task example above, we might conclude that there are some discoverability or desirability issues with the feature. However, providing this level of specificity would allow us to make those conclusions while also getting a chance to see if there are any interaction-design problems with the feature itself.

Funneling in Followup Questions

In  qualitative usability tests , the facilitator will make use of the funnel technique when asking followup questions. For example, we might ask the following questions after the participant has finished a task:

In the above example, we are starting with broad, open-ended questions before narrowing the scope to specific, open-ended questions that relate to a UI element that we want more feedback on. 

We shouldn’t start with Question 3 and move in the reverse direction, because it’s always better if participants volunteer information unprompted. We’re hoping the participant will organically provide feedback without our specific direction, because unsolicited feedback is more likely to be the participant’s true opinion. Asking participants for feedback on specific elements can be risky — there’s always a chance that the participant will simply make up an opinion for the researcher. For example, if the participant said the filters were really helpful in response to question 1, we can be more confident that they really think so than if they gave us a similar response to question 3.

The funnel technique can be used when administering questions or tasks in user interviews or usability tests. Start with broad open-ended questions or tasks before introducing more specific questions or tasks. This approach helps to ensure that you don’t miss out on important information and you avoid prompting or priming participants too early.

Related Courses

User interviews.

Uncover in-depth, accurate insights about your users

Usability Testing

Learn how to plan, conduct, and analyze your own studies, whether in person or remote

Assessing UX Designs Using Proven Principles

Generate insights, product scorecards and competitive analyses, even when you don’t have access to user data

Related Topics

• User Testing User Testing
• Research Methods

Learn More:

Please accept marketing cookies to view the embedded video. https://www.youtube.com/watch?v=KoEiX0oX7no

Using the Funnel Technique in Usability Testing

How to Use the Zeigarnik Effect in UX

Feifei Liu · 5 min

Using the Funnel Technique in User Interviews

Maria Rosala · 3 min

Probing in User Interviews

Related Articles:

User Interviews 101

Maria Rosala and Kara Pernice · 7 min

6 Mistakes When Crafting Interview Questions

Maria Rosala · 5 min

When to Use Which User-Experience Research Methods

Christian Rohrer · 9 min

Focus Groups 101

Therese Fessenden · 7 min

Obtaining Consent for User Research

Therese Fessenden · 8 min

Remote Usability Testing: Study Guide

Kate Moran · 2 min