article using qualitative research methods

Criteria for Good Qualitative Research: A Comprehensive Review

• Regular Article
• Open access
• Published: 18 September 2021
• Volume 31 , pages 679–689, ( 2022 )

Cite this article

You have full access to this open access article

• Drishti Yadav   ORCID: orcid.org/0000-0002-2974-0323 1  

76k Accesses

28 Citations

72 Altmetric

Explore all metrics

This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then, references of relevant articles were surveyed to find noteworthy, distinct, and well-defined pointers to good qualitative research. This review presents an investigative assessment of the pivotal features in qualitative research that can permit the readers to pass judgment on its quality and to condemn it as good research when objectively and adequately utilized. Overall, this review underlines the crux of qualitative research and accentuates the necessity to evaluate such research by the very tenets of its being. It also offers some prospects and recommendations to improve the quality of qualitative research. Based on the findings of this review, it is concluded that quality criteria are the aftereffect of socio-institutional procedures and existing paradigmatic conducts. Owing to the paradigmatic diversity of qualitative research, a single and specific set of quality criteria is neither feasible nor anticipated. Since qualitative research is not a cohesive discipline, researchers need to educate and familiarize themselves with applicable norms and decisive factors to evaluate qualitative research from within its theoretical and methodological framework of origin.

Similar content being viewed by others

What is Qualitative in Qualitative Research

Patrik Aspers & Ugo Corte

How to use and assess qualitative research methods

Loraine Busetto, Wolfgang Wick & Christoph Gumbinger

Different uses of Bronfenbrenner’s ecological theory in public mental health research: what is their value for guiding public mental health policy and practice?

Malin Eriksson, Mehdi Ghazinour & Anne Hammarström

Avoid common mistakes on your manuscript.

Introduction

“… It is important to regularly dialogue about what makes for good qualitative research” (Tracy, 2010 , p. 837)

To decide what represents good qualitative research is highly debatable. There are numerous methods that are contained within qualitative research and that are established on diverse philosophical perspectives. Bryman et al., ( 2008 , p. 262) suggest that “It is widely assumed that whereas quality criteria for quantitative research are well‐known and widely agreed, this is not the case for qualitative research.” Hence, the question “how to evaluate the quality of qualitative research” has been continuously debated. There are many areas of science and technology wherein these debates on the assessment of qualitative research have taken place. Examples include various areas of psychology: general psychology (Madill et al., 2000 ); counseling psychology (Morrow, 2005 ); and clinical psychology (Barker & Pistrang, 2005 ), and other disciplines of social sciences: social policy (Bryman et al., 2008 ); health research (Sparkes, 2001 ); business and management research (Johnson et al., 2006 ); information systems (Klein & Myers, 1999 ); and environmental studies (Reid & Gough, 2000 ). In the literature, these debates are enthused by the impression that the blanket application of criteria for good qualitative research developed around the positivist paradigm is improper. Such debates are based on the wide range of philosophical backgrounds within which qualitative research is conducted (e.g., Sandberg, 2000 ; Schwandt, 1996 ). The existence of methodological diversity led to the formulation of different sets of criteria applicable to qualitative research.

Among qualitative researchers, the dilemma of governing the measures to assess the quality of research is not a new phenomenon, especially when the virtuous triad of objectivity, reliability, and validity (Spencer et al., 2004 ) are not adequate. Occasionally, the criteria of quantitative research are used to evaluate qualitative research (Cohen & Crabtree, 2008 ; Lather, 2004 ). Indeed, Howe ( 2004 ) claims that the prevailing paradigm in educational research is scientifically based experimental research. Hypotheses and conjectures about the preeminence of quantitative research can weaken the worth and usefulness of qualitative research by neglecting the prominence of harmonizing match for purpose on research paradigm, the epistemological stance of the researcher, and the choice of methodology. Researchers have been reprimanded concerning this in “paradigmatic controversies, contradictions, and emerging confluences” (Lincoln & Guba, 2000 ).

In general, qualitative research tends to come from a very different paradigmatic stance and intrinsically demands distinctive and out-of-the-ordinary criteria for evaluating good research and varieties of research contributions that can be made. This review attempts to present a series of evaluative criteria for qualitative researchers, arguing that their choice of criteria needs to be compatible with the unique nature of the research in question (its methodology, aims, and assumptions). This review aims to assist researchers in identifying some of the indispensable features or markers of high-quality qualitative research. In a nutshell, the purpose of this systematic literature review is to analyze the existing knowledge on high-quality qualitative research and to verify the existence of research studies dealing with the critical assessment of qualitative research based on the concept of diverse paradigmatic stances. Contrary to the existing reviews, this review also suggests some critical directions to follow to improve the quality of qualitative research in different epistemological and ontological perspectives. This review is also intended to provide guidelines for the acceleration of future developments and dialogues among qualitative researchers in the context of assessing the qualitative research.

The rest of this review article is structured in the following fashion: Sect.  Methods describes the method followed for performing this review. Section Criteria for Evaluating Qualitative Studies provides a comprehensive description of the criteria for evaluating qualitative studies. This section is followed by a summary of the strategies to improve the quality of qualitative research in Sect.  Improving Quality: Strategies . Section  How to Assess the Quality of the Research Findings? provides details on how to assess the quality of the research findings. After that, some of the quality checklists (as tools to evaluate quality) are discussed in Sect.  Quality Checklists: Tools for Assessing the Quality . At last, the review ends with the concluding remarks presented in Sect.  Conclusions, Future Directions and Outlook . Some prospects in qualitative research for enhancing its quality and usefulness in the social and techno-scientific research community are also presented in Sect.  Conclusions, Future Directions and Outlook .

For this review, a comprehensive literature search was performed from many databases using generic search terms such as Qualitative Research , Criteria , etc . The following databases were chosen for the literature search based on the high number of results: IEEE Explore, ScienceDirect, PubMed, Google Scholar, and Web of Science. The following keywords (and their combinations using Boolean connectives OR/AND) were adopted for the literature search: qualitative research, criteria, quality, assessment, and validity. The synonyms for these keywords were collected and arranged in a logical structure (see Table 1 ). All publications in journals and conference proceedings later than 1950 till 2021 were considered for the search. Other articles extracted from the references of the papers identified in the electronic search were also included. A large number of publications on qualitative research were retrieved during the initial screening. Hence, to include the searches with the main focus on criteria for good qualitative research, an inclusion criterion was utilized in the search string.

From the selected databases, the search retrieved a total of 765 publications. Then, the duplicate records were removed. After that, based on the title and abstract, the remaining 426 publications were screened for their relevance by using the following inclusion and exclusion criteria (see Table 2 ). Publications focusing on evaluation criteria for good qualitative research were included, whereas those works which delivered theoretical concepts on qualitative research were excluded. Based on the screening and eligibility, 45 research articles were identified that offered explicit criteria for evaluating the quality of qualitative research and were found to be relevant to this review.

Figure  1 illustrates the complete review process in the form of PRISMA flow diagram. PRISMA, i.e., “preferred reporting items for systematic reviews and meta-analyses” is employed in systematic reviews to refine the quality of reporting.

PRISMA flow diagram illustrating the search and inclusion process. N represents the number of records

Criteria for Evaluating Qualitative Studies

Fundamental criteria: general research quality.

Various researchers have put forward criteria for evaluating qualitative research, which have been summarized in Table 3 . Also, the criteria outlined in Table 4 effectively deliver the various approaches to evaluate and assess the quality of qualitative work. The entries in Table 4 are based on Tracy’s “Eight big‐tent criteria for excellent qualitative research” (Tracy, 2010 ). Tracy argues that high-quality qualitative work should formulate criteria focusing on the worthiness, relevance, timeliness, significance, morality, and practicality of the research topic, and the ethical stance of the research itself. Researchers have also suggested a series of questions as guiding principles to assess the quality of a qualitative study (Mays & Pope, 2020 ). Nassaji ( 2020 ) argues that good qualitative research should be robust, well informed, and thoroughly documented.

Qualitative Research: Interpretive Paradigms

All qualitative researchers follow highly abstract principles which bring together beliefs about ontology, epistemology, and methodology. These beliefs govern how the researcher perceives and acts. The net, which encompasses the researcher’s epistemological, ontological, and methodological premises, is referred to as a paradigm, or an interpretive structure, a “Basic set of beliefs that guides action” (Guba, 1990 ). Four major interpretive paradigms structure the qualitative research: positivist and postpositivist, constructivist interpretive, critical (Marxist, emancipatory), and feminist poststructural. The complexity of these four abstract paradigms increases at the level of concrete, specific interpretive communities. Table 5 presents these paradigms and their assumptions, including their criteria for evaluating research, and the typical form that an interpretive or theoretical statement assumes in each paradigm. Moreover, for evaluating qualitative research, quantitative conceptualizations of reliability and validity are proven to be incompatible (Horsburgh, 2003 ). In addition, a series of questions have been put forward in the literature to assist a reviewer (who is proficient in qualitative methods) for meticulous assessment and endorsement of qualitative research (Morse, 2003 ). Hammersley ( 2007 ) also suggests that guiding principles for qualitative research are advantageous, but methodological pluralism should not be simply acknowledged for all qualitative approaches. Seale ( 1999 ) also points out the significance of methodological cognizance in research studies.

Table 5 reflects that criteria for assessing the quality of qualitative research are the aftermath of socio-institutional practices and existing paradigmatic standpoints. Owing to the paradigmatic diversity of qualitative research, a single set of quality criteria is neither possible nor desirable. Hence, the researchers must be reflexive about the criteria they use in the various roles they play within their research community.

Improving Quality: Strategies

Another critical question is “How can the qualitative researchers ensure that the abovementioned quality criteria can be met?” Lincoln and Guba ( 1986 ) delineated several strategies to intensify each criteria of trustworthiness. Other researchers (Merriam & Tisdell, 2016 ; Shenton, 2004 ) also presented such strategies. A brief description of these strategies is shown in Table 6 .

It is worth mentioning that generalizability is also an integral part of qualitative research (Hays & McKibben, 2021 ). In general, the guiding principle pertaining to generalizability speaks about inducing and comprehending knowledge to synthesize interpretive components of an underlying context. Table 7 summarizes the main metasynthesis steps required to ascertain generalizability in qualitative research.

Figure  2 reflects the crucial components of a conceptual framework and their contribution to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice (Johnson et al., 2020 ). The synergy and interrelationship of these components signifies their role to different stances of a qualitative research study.

Essential elements of a conceptual framework

In a nutshell, to assess the rationale of a study, its conceptual framework and research question(s), quality criteria must take account of the following: lucid context for the problem statement in the introduction; well-articulated research problems and questions; precise conceptual framework; distinct research purpose; and clear presentation and investigation of the paradigms. These criteria would expedite the quality of qualitative research.

How to Assess the Quality of the Research Findings?

The inclusion of quotes or similar research data enhances the confirmability in the write-up of the findings. The use of expressions (for instance, “80% of all respondents agreed that” or “only one of the interviewees mentioned that”) may also quantify qualitative findings (Stenfors et al., 2020 ). On the other hand, the persuasive reason for “why this may not help in intensifying the research” has also been provided (Monrouxe & Rees, 2020 ). Further, the Discussion and Conclusion sections of an article also prove robust markers of high-quality qualitative research, as elucidated in Table 8 .

Quality Checklists: Tools for Assessing the Quality

Numerous checklists are available to speed up the assessment of the quality of qualitative research. However, if used uncritically and recklessly concerning the research context, these checklists may be counterproductive. I recommend that such lists and guiding principles may assist in pinpointing the markers of high-quality qualitative research. However, considering enormous variations in the authors’ theoretical and philosophical contexts, I would emphasize that high dependability on such checklists may say little about whether the findings can be applied in your setting. A combination of such checklists might be appropriate for novice researchers. Some of these checklists are listed below:

The most commonly used framework is Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ). This framework is recommended by some journals to be followed by the authors during article submission.

Standards for Reporting Qualitative Research (SRQR) is another checklist that has been created particularly for medical education (O’Brien et al., 2014 ).

Also, Tracy ( 2010 ) and Critical Appraisal Skills Programme (CASP, 2021 ) offer criteria for qualitative research relevant across methods and approaches.

Further, researchers have also outlined different criteria as hallmarks of high-quality qualitative research. For instance, the “Road Trip Checklist” (Epp & Otnes, 2021 ) provides a quick reference to specific questions to address different elements of high-quality qualitative research.

Conclusions, Future Directions, and Outlook

This work presents a broad review of the criteria for good qualitative research. In addition, this article presents an exploratory analysis of the essential elements in qualitative research that can enable the readers of qualitative work to judge it as good research when objectively and adequately utilized. In this review, some of the essential markers that indicate high-quality qualitative research have been highlighted. I scope them narrowly to achieve rigor in qualitative research and note that they do not completely cover the broader considerations necessary for high-quality research. This review points out that a universal and versatile one-size-fits-all guideline for evaluating the quality of qualitative research does not exist. In other words, this review also emphasizes the non-existence of a set of common guidelines among qualitative researchers. In unison, this review reinforces that each qualitative approach should be treated uniquely on account of its own distinctive features for different epistemological and disciplinary positions. Owing to the sensitivity of the worth of qualitative research towards the specific context and the type of paradigmatic stance, researchers should themselves analyze what approaches can be and must be tailored to ensemble the distinct characteristics of the phenomenon under investigation. Although this article does not assert to put forward a magic bullet and to provide a one-stop solution for dealing with dilemmas about how, why, or whether to evaluate the “goodness” of qualitative research, it offers a platform to assist the researchers in improving their qualitative studies. This work provides an assembly of concerns to reflect on, a series of questions to ask, and multiple sets of criteria to look at, when attempting to determine the quality of qualitative research. Overall, this review underlines the crux of qualitative research and accentuates the need to evaluate such research by the very tenets of its being. Bringing together the vital arguments and delineating the requirements that good qualitative research should satisfy, this review strives to equip the researchers as well as reviewers to make well-versed judgment about the worth and significance of the qualitative research under scrutiny. In a nutshell, a comprehensive portrayal of the research process (from the context of research to the research objectives, research questions and design, speculative foundations, and from approaches of collecting data to analyzing the results, to deriving inferences) frequently proliferates the quality of a qualitative research.

Prospects : A Road Ahead for Qualitative Research

Irrefutably, qualitative research is a vivacious and evolving discipline wherein different epistemological and disciplinary positions have their own characteristics and importance. In addition, not surprisingly, owing to the sprouting and varied features of qualitative research, no consensus has been pulled off till date. Researchers have reflected various concerns and proposed several recommendations for editors and reviewers on conducting reviews of critical qualitative research (Levitt et al., 2021 ; McGinley et al., 2021 ). Following are some prospects and a few recommendations put forward towards the maturation of qualitative research and its quality evaluation:

In general, most of the manuscript and grant reviewers are not qualitative experts. Hence, it is more likely that they would prefer to adopt a broad set of criteria. However, researchers and reviewers need to keep in mind that it is inappropriate to utilize the same approaches and conducts among all qualitative research. Therefore, future work needs to focus on educating researchers and reviewers about the criteria to evaluate qualitative research from within the suitable theoretical and methodological context.

There is an urgent need to refurbish and augment critical assessment of some well-known and widely accepted tools (including checklists such as COREQ, SRQR) to interrogate their applicability on different aspects (along with their epistemological ramifications).

Efforts should be made towards creating more space for creativity, experimentation, and a dialogue between the diverse traditions of qualitative research. This would potentially help to avoid the enforcement of one's own set of quality criteria on the work carried out by others.

Moreover, journal reviewers need to be aware of various methodological practices and philosophical debates.

It is pivotal to highlight the expressions and considerations of qualitative researchers and bring them into a more open and transparent dialogue about assessing qualitative research in techno-scientific, academic, sociocultural, and political rooms.

Frequent debates on the use of evaluative criteria are required to solve some potentially resolved issues (including the applicability of a single set of criteria in multi-disciplinary aspects). Such debates would not only benefit the group of qualitative researchers themselves, but primarily assist in augmenting the well-being and vivacity of the entire discipline.

To conclude, I speculate that the criteria, and my perspective, may transfer to other methods, approaches, and contexts. I hope that they spark dialog and debate – about criteria for excellent qualitative research and the underpinnings of the discipline more broadly – and, therefore, help improve the quality of a qualitative study. Further, I anticipate that this review will assist the researchers to contemplate on the quality of their own research, to substantiate research design and help the reviewers to review qualitative research for journals. On a final note, I pinpoint the need to formulate a framework (encompassing the prerequisites of a qualitative study) by the cohesive efforts of qualitative researchers of different disciplines with different theoretic-paradigmatic origins. I believe that tailoring such a framework (of guiding principles) paves the way for qualitative researchers to consolidate the status of qualitative research in the wide-ranging open science debate. Dialogue on this issue across different approaches is crucial for the impending prospects of socio-techno-educational research.

Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16 (10), 1472–1482.

Article   Google Scholar  

Barker, C., & Pistrang, N. (2005). Quality criteria under methodological pluralism: Implications for conducting and evaluating research. American Journal of Community Psychology, 35 (3–4), 201–212.

Bryman, A., Becker, S., & Sempik, J. (2008). Quality criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal of Social Research Methodology, 11 (4), 261–276.

Caelli, K., Ray, L., & Mill, J. (2003). ‘Clear as mud’: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2 (2), 1–13.

CASP (2021). CASP checklists. Retrieved May 2021 from https://casp-uk.net/casp-tools-checklists/

Cohen, D. J., & Crabtree, B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. The Annals of Family Medicine, 6 (4), 331–339.

Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (pp. 1–32). Sage Publications Ltd.

Google Scholar  

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38 (3), 215–229.

Epp, A. M., & Otnes, C. C. (2021). High-quality qualitative research: Getting into gear. Journal of Service Research . https://doi.org/10.1177/1094670520961445

Guba, E. G. (1990). The paradigm dialog. In Alternative paradigms conference, mar, 1989, Indiana u, school of education, San Francisco, ca, us . Sage Publications, Inc.

Hammersley, M. (2007). The issue of quality in qualitative research. International Journal of Research and Method in Education, 30 (3), 287–305.

Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., & Mokkink, L. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19 , 1609406920976417.

Hays, D. G., & McKibben, W. B. (2021). Promoting rigorous research: Generalizability and qualitative research. Journal of Counseling and Development, 99 (2), 178–188.

Horsburgh, D. (2003). Evaluation of qualitative research. Journal of Clinical Nursing, 12 (2), 307–312.

Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10 (1), 42–46.

Johnson, J. L., Adkins, D., & Chauvin, S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84 (1), 7120.

Johnson, P., Buehring, A., Cassell, C., & Symon, G. (2006). Evaluating qualitative management research: Towards a contingent criteriology. International Journal of Management Reviews, 8 (3), 131–156.

Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23 (1), 67–93.

Lather, P. (2004). This is your father’s paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10 (1), 15–34.

Levitt, H. M., Morrill, Z., Collins, K. M., & Rizo, J. L. (2021). The methodological integrity of critical qualitative research: Principles to support design and research review. Journal of Counseling Psychology, 68 (3), 357.

Lincoln, Y. S., & Guba, E. G. (1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation, 1986 (30), 73–84.

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163–188). Sage Publications.

Madill, A., Jordan, A., & Shirley, C. (2000). Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies. British Journal of Psychology, 91 (1), 1–20.

Mays, N., & Pope, C. (2020). Quality in qualitative research. Qualitative Research in Health Care . https://doi.org/10.1002/9781119410867.ch15

McGinley, S., Wei, W., Zhang, L., & Zheng, Y. (2021). The state of qualitative research in hospitality: A 5-year review 2014 to 2019. Cornell Hospitality Quarterly, 62 (1), 8–20.

Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and implementation. San Francisco, US.

Meyer, M., & Dykes, J. (2019). Criteria for rigor in visualization design study. IEEE Transactions on Visualization and Computer Graphics, 26 (1), 87–97.

Monrouxe, L. V., & Rees, C. E. (2020). When I say… quantification in qualitative research. Medical Education, 54 (3), 186–187.

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52 (2), 250.

Morse, J. M. (2003). A review committee’s guide for evaluating qualitative proposals. Qualitative Health Research, 13 (6), 833–851.

Nassaji, H. (2020). Good qualitative research. Language Teaching Research, 24 (4), 427–431.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89 (9), 1245–1251.

O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19 , 1609406919899220.

Reid, A., & Gough, S. (2000). Guidelines for reporting and evaluating qualitative research: What are the alternatives? Environmental Education Research, 6 (1), 59–91.

Rocco, T. S. (2010). Criteria for evaluating qualitative studies. Human Resource Development International . https://doi.org/10.1080/13678868.2010.501959

Sandberg, J. (2000). Understanding human competence at work: An interpretative approach. Academy of Management Journal, 43 (1), 9–25.

Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2 (1), 58–72.

Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5 (4), 465–478.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 (2), 63–75.

Sparkes, A. C. (2001). Myth 94: Qualitative health researchers will agree about validity. Qualitative Health Research, 11 (4), 538–552.

Spencer, L., Ritchie, J., Lewis, J., & Dillon, L. (2004). Quality in qualitative evaluation: A framework for assessing research evidence.

Stenfors, T., Kajamaa, A., & Bennett, D. (2020). How to assess the quality of qualitative research. The Clinical Teacher, 17 (6), 596–599.

Taylor, E. W., Beck, J., & Ainsworth, E. (2001). Publishing qualitative adult education research: A peer review perspective. Studies in the Education of Adults, 33 (2), 163–179.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19 (6), 349–357.

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16 (10), 837–851.

Download references

Open access funding provided by TU Wien (TUW).

Author information

Authors and affiliations.

Faculty of Informatics, Technische Universität Wien, 1040, Vienna, Austria

Drishti Yadav

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Drishti Yadav .

Ethics declarations

Conflict of interest.

The author declares no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Yadav, D. Criteria for Good Qualitative Research: A Comprehensive Review. Asia-Pacific Edu Res 31 , 679–689 (2022). https://doi.org/10.1007/s40299-021-00619-0

Download citation

Accepted : 28 August 2021

Published : 18 September 2021

Issue Date : December 2022

DOI : https://doi.org/10.1007/s40299-021-00619-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Evaluative criteria
• Find a journal
• Publish with us
• Track your research

Qualitative Study

Affiliations.

• 1 University of Nebraska Medical Center
• 2 GDB Research and Statistical Consulting
• 3 GDB Research and Statistical Consulting/McLaren Macomb Hospital
• PMID: 29262162
• Bookshelf ID: NBK470395

Qualitative research is a type of research that explores and provides deeper insights into real-world problems. Instead of collecting numerical data points or intervene or introduce treatments just like in quantitative research, qualitative research helps generate hypotheses as well as further investigate and understand quantitative data. Qualitative research gathers participants' experiences, perceptions, and behavior. It answers the hows and whys instead of how many or how much. It could be structured as a stand-alone study, purely relying on qualitative data or it could be part of mixed-methods research that combines qualitative and quantitative data. This review introduces the readers to some basic concepts, definitions, terminology, and application of qualitative research.

Qualitative research at its core, ask open-ended questions whose answers are not easily put into numbers such as ‘how’ and ‘why’. Due to the open-ended nature of the research questions at hand, qualitative research design is often not linear in the same way quantitative design is. One of the strengths of qualitative research is its ability to explain processes and patterns of human behavior that can be difficult to quantify. Phenomena such as experiences, attitudes, and behaviors can be difficult to accurately capture quantitatively, whereas a qualitative approach allows participants themselves to explain how, why, or what they were thinking, feeling, and experiencing at a certain time or during an event of interest. Quantifying qualitative data certainly is possible, but at its core, qualitative data is looking for themes and patterns that can be difficult to quantify and it is important to ensure that the context and narrative of qualitative work are not lost by trying to quantify something that is not meant to be quantified.

However, while qualitative research is sometimes placed in opposition to quantitative research, where they are necessarily opposites and therefore ‘compete’ against each other and the philosophical paradigms associated with each, qualitative and quantitative work are not necessarily opposites nor are they incompatible. While qualitative and quantitative approaches are different, they are not necessarily opposites, and they are certainly not mutually exclusive. For instance, qualitative research can help expand and deepen understanding of data or results obtained from quantitative analysis. For example, say a quantitative analysis has determined that there is a correlation between length of stay and level of patient satisfaction, but why does this correlation exist? This dual-focus scenario shows one way in which qualitative and quantitative research could be integrated together.

Examples of Qualitative Research Approaches

Ethnography

Ethnography as a research design has its origins in social and cultural anthropology, and involves the researcher being directly immersed in the participant’s environment. Through this immersion, the ethnographer can use a variety of data collection techniques with the aim of being able to produce a comprehensive account of the social phenomena that occurred during the research period. That is to say, the researcher’s aim with ethnography is to immerse themselves into the research population and come out of it with accounts of actions, behaviors, events, etc. through the eyes of someone involved in the population. Direct involvement of the researcher with the target population is one benefit of ethnographic research because it can then be possible to find data that is otherwise very difficult to extract and record.

Grounded Theory

Grounded Theory is the “generation of a theoretical model through the experience of observing a study population and developing a comparative analysis of their speech and behavior.” As opposed to quantitative research which is deductive and tests or verifies an existing theory, grounded theory research is inductive and therefore lends itself to research that is aiming to study social interactions or experiences. In essence, Grounded Theory’s goal is to explain for example how and why an event occurs or how and why people might behave a certain way. Through observing the population, a researcher using the Grounded Theory approach can then develop a theory to explain the phenomena of interest.

Phenomenology

Phenomenology is defined as the “study of the meaning of phenomena or the study of the particular”. At first glance, it might seem that Grounded Theory and Phenomenology are quite similar, but upon careful examination, the differences can be seen. At its core, phenomenology looks to investigate experiences from the perspective of the individual. Phenomenology is essentially looking into the ‘lived experiences’ of the participants and aims to examine how and why participants behaved a certain way, from their perspective . Herein lies one of the main differences between Grounded Theory and Phenomenology. Grounded Theory aims to develop a theory for social phenomena through an examination of various data sources whereas Phenomenology focuses on describing and explaining an event or phenomena from the perspective of those who have experienced it.

Narrative Research

One of qualitative research’s strengths lies in its ability to tell a story, often from the perspective of those directly involved in it. Reporting on qualitative research involves including details and descriptions of the setting involved and quotes from participants. This detail is called ‘thick’ or ‘rich’ description and is a strength of qualitative research. Narrative research is rife with the possibilities of ‘thick’ description as this approach weaves together a sequence of events, usually from just one or two individuals, in the hopes of creating a cohesive story, or narrative. While it might seem like a waste of time to focus on such a specific, individual level, understanding one or two people’s narratives for an event or phenomenon can help to inform researchers about the influences that helped shape that narrative. The tension or conflict of differing narratives can be “opportunities for innovation”.

Research Paradigm

Research paradigms are the assumptions, norms, and standards that underpin different approaches to research. Essentially, research paradigms are the ‘worldview’ that inform research. It is valuable for researchers, both qualitative and quantitative, to understand what paradigm they are working within because understanding the theoretical basis of research paradigms allows researchers to understand the strengths and weaknesses of the approach being used and adjust accordingly. Different paradigms have different ontology and epistemologies . Ontology is defined as the "assumptions about the nature of reality” whereas epistemology is defined as the “assumptions about the nature of knowledge” that inform the work researchers do. It is important to understand the ontological and epistemological foundations of the research paradigm researchers are working within to allow for a full understanding of the approach being used and the assumptions that underpin the approach as a whole. Further, it is crucial that researchers understand their own ontological and epistemological assumptions about the world in general because their assumptions about the world will necessarily impact how they interact with research. A discussion of the research paradigm is not complete without describing positivist, postpositivist, and constructivist philosophies.

Positivist vs Postpositivist

To further understand qualitative research, we need to discuss positivist and postpositivist frameworks. Positivism is a philosophy that the scientific method can and should be applied to social as well as natural sciences. Essentially, positivist thinking insists that the social sciences should use natural science methods in its research which stems from positivist ontology that there is an objective reality that exists that is fully independent of our perception of the world as individuals. Quantitative research is rooted in positivist philosophy, which can be seen in the value it places on concepts such as causality, generalizability, and replicability.

Conversely, postpositivists argue that social reality can never be one hundred percent explained but it could be approximated. Indeed, qualitative researchers have been insisting that there are “fundamental limits to the extent to which the methods and procedures of the natural sciences could be applied to the social world” and therefore postpositivist philosophy is often associated with qualitative research. An example of positivist versus postpositivist values in research might be that positivist philosophies value hypothesis-testing, whereas postpositivist philosophies value the ability to formulate a substantive theory.

Constructivist

Constructivism is a subcategory of postpositivism. Most researchers invested in postpositivist research are constructivist as well, meaning they think there is no objective external reality that exists but rather that reality is constructed. Constructivism is a theoretical lens that emphasizes the dynamic nature of our world. “Constructivism contends that individuals’ views are directly influenced by their experiences, and it is these individual experiences and views that shape their perspective of reality”. Essentially, Constructivist thought focuses on how ‘reality’ is not a fixed certainty and experiences, interactions, and backgrounds give people a unique view of the world. Constructivism contends, unlike in positivist views, that there is not necessarily an ‘objective’ reality we all experience. This is the ‘relativist’ ontological view that reality and the world we live in are dynamic and socially constructed. Therefore, qualitative scientific knowledge can be inductive as well as deductive.”

So why is it important to understand the differences in assumptions that different philosophies and approaches to research have? Fundamentally, the assumptions underpinning the research tools a researcher selects provide an overall base for the assumptions the rest of the research will have and can even change the role of the researcher themselves. For example, is the researcher an ‘objective’ observer such as in positivist quantitative work? Or is the researcher an active participant in the research itself, as in postpositivist qualitative work? Understanding the philosophical base of the research undertaken allows researchers to fully understand the implications of their work and their role within the research, as well as reflect on their own positionality and bias as it pertains to the research they are conducting.

Data Sampling

The better the sample represents the intended study population, the more likely the researcher is to encompass the varying factors at play. The following are examples of participant sampling and selection:

Purposive sampling- selection based on the researcher’s rationale in terms of being the most informative.

Criterion sampling-selection based on pre-identified factors.

Convenience sampling- selection based on availability.

Snowball sampling- the selection is by referral from other participants or people who know potential participants.

Extreme case sampling- targeted selection of rare cases.

Typical case sampling-selection based on regular or average participants.

Data Collection and Analysis

Qualitative research uses several techniques including interviews, focus groups, and observation. [1] [2] [3] Interviews may be unstructured, with open-ended questions on a topic and the interviewer adapts to the responses. Structured interviews have a predetermined number of questions that every participant is asked. It is usually one on one and is appropriate for sensitive topics or topics needing an in-depth exploration. Focus groups are often held with 8-12 target participants and are used when group dynamics and collective views on a topic are desired. Researchers can be a participant-observer to share the experiences of the subject or a non-participant or detached observer.

While quantitative research design prescribes a controlled environment for data collection, qualitative data collection may be in a central location or in the environment of the participants, depending on the study goals and design. Qualitative research could amount to a large amount of data. Data is transcribed which may then be coded manually or with the use of Computer Assisted Qualitative Data Analysis Software or CAQDAS such as ATLAS.ti or NVivo.

After the coding process, qualitative research results could be in various formats. It could be a synthesis and interpretation presented with excerpts from the data. Results also could be in the form of themes and theory or model development.

Dissemination

To standardize and facilitate the dissemination of qualitative research outcomes, the healthcare team can use two reporting standards. The Consolidated Criteria for Reporting Qualitative Research or COREQ is a 32-item checklist for interviews and focus groups. The Standards for Reporting Qualitative Research (SRQR) is a checklist covering a wider range of qualitative research.

Examples of Application

Many times a research question will start with qualitative research. The qualitative research will help generate the research hypothesis which can be tested with quantitative methods. After the data is collected and analyzed with quantitative methods, a set of qualitative methods can be used to dive deeper into the data for a better understanding of what the numbers truly mean and their implications. The qualitative methods can then help clarify the quantitative data and also help refine the hypothesis for future research. Furthermore, with qualitative research researchers can explore subjects that are poorly studied with quantitative methods. These include opinions, individual's actions, and social science research.

A good qualitative study design starts with a goal or objective. This should be clearly defined or stated. The target population needs to be specified. A method for obtaining information from the study population must be carefully detailed to ensure there are no omissions of part of the target population. A proper collection method should be selected which will help obtain the desired information without overly limiting the collected data because many times, the information sought is not well compartmentalized or obtained. Finally, the design should ensure adequate methods for analyzing the data. An example may help better clarify some of the various aspects of qualitative research.

A researcher wants to decrease the number of teenagers who smoke in their community. The researcher could begin by asking current teen smokers why they started smoking through structured or unstructured interviews (qualitative research). The researcher can also get together a group of current teenage smokers and conduct a focus group to help brainstorm factors that may have prevented them from starting to smoke (qualitative research).

In this example, the researcher has used qualitative research methods (interviews and focus groups) to generate a list of ideas of both why teens start to smoke as well as factors that may have prevented them from starting to smoke. Next, the researcher compiles this data. The research found that, hypothetically, peer pressure, health issues, cost, being considered “cool,” and rebellious behavior all might increase or decrease the likelihood of teens starting to smoke.

The researcher creates a survey asking teen participants to rank how important each of the above factors is in either starting smoking (for current smokers) or not smoking (for current non-smokers). This survey provides specific numbers (ranked importance of each factor) and is thus a quantitative research tool.

The researcher can use the results of the survey to focus efforts on the one or two highest-ranked factors. Let us say the researcher found that health was the major factor that keeps teens from starting to smoke, and peer pressure was the major factor that contributed to teens to start smoking. The researcher can go back to qualitative research methods to dive deeper into each of these for more information. The researcher wants to focus on how to keep teens from starting to smoke, so they focus on the peer pressure aspect.

The researcher can conduct interviews and/or focus groups (qualitative research) about what types and forms of peer pressure are commonly encountered, where the peer pressure comes from, and where smoking first starts. The researcher hypothetically finds that peer pressure often occurs after school at the local teen hangouts, mostly the local park. The researcher also hypothetically finds that peer pressure comes from older, current smokers who provide the cigarettes.

The researcher could further explore this observation made at the local teen hangouts (qualitative research) and take notes regarding who is smoking, who is not, and what observable factors are at play for peer pressure of smoking. The researcher finds a local park where many local teenagers hang out and see that a shady, overgrown area of the park is where the smokers tend to hang out. The researcher notes the smoking teenagers buy their cigarettes from a local convenience store adjacent to the park where the clerk does not check identification before selling cigarettes. These observations fall under qualitative research.

If the researcher returns to the park and counts how many individuals smoke in each region of the park, this numerical data would be quantitative research. Based on the researcher's efforts thus far, they conclude that local teen smoking and teenagers who start to smoke may decrease if there are fewer overgrown areas of the park and the local convenience store does not sell cigarettes to underage individuals.

The researcher could try to have the parks department reassess the shady areas to make them less conducive to the smokers or identify how to limit the sales of cigarettes to underage individuals by the convenience store. The researcher would then cycle back to qualitative methods of asking at-risk population their perceptions of the changes, what factors are still at play, as well as quantitative research that includes teen smoking rates in the community, the incidence of new teen smokers, among others.

Copyright © 2024, StatPearls Publishing LLC.

• Introduction
• Issues of Concern
• Clinical Significance
• Enhancing Healthcare Team Outcomes
• Review Questions

Publication types

• Study Guide

• Search Menu
• Advance articles
• Editor's Choice
• ESHRE Pages
• Mini-reviews
• Author Guidelines
• Submission Site
• Reasons to Publish
• Open Access
• Advertising and Corporate Services
• Advertising
• Reprints and ePrints
• Sponsored Supplements
• Branded Books
• Journals Career Network
• About Human Reproduction
• About the European Society of Human Reproduction and Embryology
• Editorial Board
• Self-Archiving Policy
• Dispatch Dates
• Contact ESHRE
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, when to use qualitative research, how to judge qualitative research, conclusions, authors' roles, conflict of interest.

• < Previous

Qualitative research methods: when to use them and how to judge them

• Article contents
• Figures & tables
• Supplementary Data

K. Hammarberg, M. Kirkman, S. de Lacey, Qualitative research methods: when to use them and how to judge them, Human Reproduction , Volume 31, Issue 3, March 2016, Pages 498–501, https://doi.org/10.1093/humrep/dev334

• Permissions Icon Permissions

In March 2015, an impressive set of guidelines for best practice on how to incorporate psychosocial care in routine infertility care was published by the ESHRE Psychology and Counselling Guideline Development Group ( ESHRE Psychology and Counselling Guideline Development Group, 2015 ). The authors report that the guidelines are based on a comprehensive review of the literature and we congratulate them on their meticulous compilation of evidence into a clinically useful document. However, when we read the methodology section, we were baffled and disappointed to find that evidence from research using qualitative methods was not included in the formulation of the guidelines. Despite stating that ‘qualitative research has significant value to assess the lived experience of infertility and fertility treatment’, the group excluded this body of evidence because qualitative research is ‘not generally hypothesis-driven and not objective/neutral, as the researcher puts him/herself in the position of the participant to understand how the world is from the person's perspective’.

Qualitative and quantitative research methods are often juxtaposed as representing two different world views. In quantitative circles, qualitative research is commonly viewed with suspicion and considered lightweight because it involves small samples which may not be representative of the broader population, it is seen as not objective, and the results are assessed as biased by the researchers' own experiences or opinions. In qualitative circles, quantitative research can be dismissed as over-simplifying individual experience in the cause of generalisation, failing to acknowledge researcher biases and expectations in research design, and requiring guesswork to understand the human meaning of aggregate data.

As social scientists who investigate psychosocial aspects of human reproduction, we use qualitative and quantitative methods, separately or together, depending on the research question. The crucial part is to know when to use what method.

The peer-review process is a pillar of scientific publishing. One of the important roles of reviewers is to assess the scientific rigour of the studies from which authors draw their conclusions. If rigour is lacking, the paper should not be published. As with research using quantitative methods, research using qualitative methods is home to the good, the bad and the ugly. It is essential that reviewers know the difference. Rejection letters are hard to take but more often than not they are based on legitimate critique. However, from time to time it is obvious that the reviewer has little grasp of what constitutes rigour or quality in qualitative research. The first author (K.H.) recently submitted a paper that reported findings from a qualitative study about fertility-related knowledge and information-seeking behaviour among people of reproductive age. In the rejection letter one of the reviewers (not from Human Reproduction ) lamented, ‘Even for a qualitative study, I would expect that some form of confidence interval and paired t-tables analysis, etc. be used to analyse the significance of results'. This comment reveals the reviewer's inappropriate application to qualitative research of criteria relevant only to quantitative research.

In this commentary, we give illustrative examples of questions most appropriately answered using qualitative methods and provide general advice about how to appraise the scientific rigour of qualitative studies. We hope this will help the journal's reviewers and readers appreciate the legitimate place of qualitative research and ensure we do not throw the baby out with the bath water by excluding or rejecting papers simply because they report the results of qualitative studies.

In psychosocial research, ‘quantitative’ research methods are appropriate when ‘factual’ data are required to answer the research question; when general or probability information is sought on opinions, attitudes, views, beliefs or preferences; when variables can be isolated and defined; when variables can be linked to form hypotheses before data collection; and when the question or problem is known, clear and unambiguous. Quantitative methods can reveal, for example, what percentage of the population supports assisted conception, their distribution by age, marital status, residential area and so on, as well as changes from one survey to the next ( Kovacs et al. , 2012 ); the number of donors and donor siblings located by parents of donor-conceived children ( Freeman et al. , 2009 ); and the relationship between the attitude of donor-conceived people to learning of their donor insemination conception and their family ‘type’ (one or two parents, lesbian or heterosexual parents; Beeson et al. , 2011 ).

In contrast, ‘qualitative’ methods are used to answer questions about experience, meaning and perspective, most often from the standpoint of the participant. These data are usually not amenable to counting or measuring. Qualitative research techniques include ‘small-group discussions’ for investigating beliefs, attitudes and concepts of normative behaviour; ‘semi-structured interviews’, to seek views on a focused topic or, with key informants, for background information or an institutional perspective; ‘in-depth interviews’ to understand a condition, experience, or event from a personal perspective; and ‘analysis of texts and documents’, such as government reports, media articles, websites or diaries, to learn about distributed or private knowledge.

Qualitative methods have been used to reveal, for example, potential problems in implementing a proposed trial of elective single embryo transfer, where small-group discussions enabled staff to explain their own resistance, leading to an amended approach ( Porter and Bhattacharya, 2005 ). Small-group discussions among assisted reproductive technology (ART) counsellors were used to investigate how the welfare principle is interpreted and practised by health professionals who must apply it in ART ( de Lacey et al. , 2015 ). When legislative change meant that gamete donors could seek identifying details of people conceived from their gametes, parents needed advice on how best to tell their children. Small-group discussions were convened to ask adolescents (not known to be donor-conceived) to reflect on how they would prefer to be told ( Kirkman et al. , 2007 ).

When a population cannot be identified, such as anonymous sperm donors from the 1980s, a qualitative approach with wide publicity can reach people who do not usually volunteer for research and reveal (for example) their attitudes to proposed legislation to remove anonymity with retrospective effect ( Hammarberg et al. , 2014 ). When researchers invite people to talk about their reflections on experience, they can sometimes learn more than they set out to discover. In describing their responses to proposed legislative change, participants also talked about people conceived as a result of their donations, demonstrating various constructions and expectations of relationships ( Kirkman et al. , 2014 ).

Interviews with parents in lesbian-parented families generated insight into the diverse meanings of the sperm donor in the creation and life of the family ( Wyverkens et al. , 2014 ). Oral and written interviews also revealed the embarrassment and ambivalence surrounding sperm donors evident in participants in donor-assisted conception ( Kirkman, 2004 ). The way in which parents conceptualise unused embryos and why they discard rather than donate was explored and understood via in-depth interviews, showing how and why the meaning of those embryos changed with parenthood ( de Lacey, 2005 ). In-depth interviews were also used to establish the intricate understanding by embryo donors and recipients of the meaning of embryo donation and the families built as a result ( Goedeke et al. , 2015 ).

It is possible to combine quantitative and qualitative methods, although great care should be taken to ensure that the theory behind each method is compatible and that the methods are being used for appropriate reasons. The two methods can be used sequentially (first a quantitative then a qualitative study or vice versa), where the first approach is used to facilitate the design of the second; they can be used in parallel as different approaches to the same question; or a dominant method may be enriched with a small component of an alternative method (such as qualitative interviews ‘nested’ in a large survey). It is important to note that free text in surveys represents qualitative data but does not constitute qualitative research. Qualitative and quantitative methods may be used together for corroboration (hoping for similar outcomes from both methods), elaboration (using qualitative data to explain or interpret quantitative data, or to demonstrate how the quantitative findings apply in particular cases), complementarity (where the qualitative and quantitative results differ but generate complementary insights) or contradiction (where qualitative and quantitative data lead to different conclusions). Each has its advantages and challenges ( Brannen, 2005 ).

Qualitative research is gaining increased momentum in the clinical setting and carries different criteria for evaluating its rigour or quality. Quantitative studies generally involve the systematic collection of data about a phenomenon, using standardized measures and statistical analysis. In contrast, qualitative studies involve the systematic collection, organization, description and interpretation of textual, verbal or visual data. The particular approach taken determines to a certain extent the criteria used for judging the quality of the report. However, research using qualitative methods can be evaluated ( Dixon-Woods et al. , 2006 ; Young et al. , 2014 ) and there are some generic guidelines for assessing qualitative research ( Kitto et al. , 2008 ).

Although the terms ‘reliability’ and ‘validity’ are contentious among qualitative researchers ( Lincoln and Guba, 1985 ) with some preferring ‘verification’, research integrity and robustness are as important in qualitative studies as they are in other forms of research. It is widely accepted that qualitative research should be ethical, important, intelligibly described, and use appropriate and rigorous methods ( Cohen and Crabtree, 2008 ). In research investigating data that can be counted or measured, replicability is essential. When other kinds of data are gathered in order to answer questions of personal or social meaning, we need to be able to capture real-life experiences, which cannot be identical from one person to the next. Furthermore, meaning is culturally determined and subject to evolutionary change. The way of explaining a phenomenon—such as what it means to use donated gametes—will vary, for example, according to the cultural significance of ‘blood’ or genes, interpretations of marital infidelity and religious constructs of sexual relationships and families. Culture may apply to a country, a community, or other actual or virtual group, and a person may be engaged at various levels of culture. In identifying meaning for members of a particular group, consistency may indeed be found from one research project to another. However, individuals within a cultural group may present different experiences and perceptions or transgress cultural expectations. That does not make them ‘wrong’ or invalidate the research. Rather, it offers insight into diversity and adds a piece to the puzzle to which other researchers also contribute.

In qualitative research the objective stance is obsolete, the researcher is the instrument, and ‘subjects’ become ‘participants’ who may contribute to data interpretation and analysis ( Denzin and Lincoln, 1998 ). Qualitative researchers defend the integrity of their work by different means: trustworthiness, credibility, applicability and consistency are the evaluative criteria ( Leininger, 1994 ).

Trustworthiness

A report of a qualitative study should contain the same robust procedural description as any other study. The purpose of the research, how it was conducted, procedural decisions, and details of data generation and management should be transparent and explicit. A reviewer should be able to follow the progression of events and decisions and understand their logic because there is adequate description, explanation and justification of the methodology and methods ( Kitto et al. , 2008 )

Credibility

Credibility is the criterion for evaluating the truth value or internal validity of qualitative research. A qualitative study is credible when its results, presented with adequate descriptions of context, are recognizable to people who share the experience and those who care for or treat them. As the instrument in qualitative research, the researcher defends its credibility through practices such as reflexivity (reflection on the influence of the researcher on the research), triangulation (where appropriate, answering the research question in several ways, such as through interviews, observation and documentary analysis) and substantial description of the interpretation process; verbatim quotations from the data are supplied to illustrate and support their interpretations ( Sandelowski, 1986 ). Where excerpts of data and interpretations are incongruent, the credibility of the study is in doubt.

Applicability

Applicability, or transferability of the research findings, is the criterion for evaluating external validity. A study is considered to meet the criterion of applicability when its findings can fit into contexts outside the study situation and when clinicians and researchers view the findings as meaningful and applicable in their own experiences.

Larger sample sizes do not produce greater applicability. Depth may be sacrificed to breadth or there may be too much data for adequate analysis. Sample sizes in qualitative research are typically small. The term ‘saturation’ is often used in reference to decisions about sample size in research using qualitative methods. Emerging from grounded theory, where filling theoretical categories is considered essential to the robustness of the developing theory, data saturation has been expanded to describe a situation where data tend towards repetition or where data cease to offer new directions and raise new questions ( Charmaz, 2005 ). However, the legitimacy of saturation as a generic marker of sampling adequacy has been questioned ( O'Reilly and Parker, 2013 ). Caution must be exercised to ensure that a commitment to saturation does not assume an ‘essence’ of an experience in which limited diversity is anticipated; each account is likely to be subtly different and each ‘sample’ will contribute to knowledge without telling the whole story. Increasingly, it is expected that researchers will report the kind of saturation they have applied and their criteria for recognising its achievement; an assessor will need to judge whether the choice is appropriate and consistent with the theoretical context within which the research has been conducted.

Sampling strategies are usually purposive, convenient, theoretical or snowballed. Maximum variation sampling may be used to seek representation of diverse perspectives on the topic. Homogeneous sampling may be used to recruit a group of participants with specified criteria. The threat of bias is irrelevant; participants are recruited and selected specifically because they can illuminate the phenomenon being studied. Rather than being predetermined by statistical power analysis, qualitative study samples are dependent on the nature of the data, the availability of participants and where those data take the investigator. Multiple data collections may also take place to obtain maximum insight into sensitive topics. For instance, the question of how decisions are made for embryo disposition may involve sampling within the patient group as well as from scientists, clinicians, counsellors and clinic administrators.

Consistency

Consistency, or dependability of the results, is the criterion for assessing reliability. This does not mean that the same result would necessarily be found in other contexts but that, given the same data, other researchers would find similar patterns. Researchers often seek maximum variation in the experience of a phenomenon, not only to illuminate it but also to discourage fulfilment of limited researcher expectations (for example, negative cases or instances that do not fit the emerging interpretation or theory should be actively sought and explored). Qualitative researchers sometimes describe the processes by which verification of the theoretical findings by another team member takes place ( Morse and Richards, 2002 ).

Research that uses qualitative methods is not, as it seems sometimes to be represented, the easy option, nor is it a collation of anecdotes. It usually involves a complex theoretical or philosophical framework. Rigorous analysis is conducted without the aid of straightforward mathematical rules. Researchers must demonstrate the validity of their analysis and conclusions, resulting in longer papers and occasional frustration with the word limits of appropriate journals. Nevertheless, we need the different kinds of evidence that is generated by qualitative methods. The experience of health, illness and medical intervention cannot always be counted and measured; researchers need to understand what they mean to individuals and groups. Knowledge gained from qualitative research methods can inform clinical practice, indicate how to support people living with chronic conditions and contribute to community education and awareness about people who are (for example) experiencing infertility or using assisted conception.

Each author drafted a section of the manuscript and the manuscript as a whole was reviewed and revised by all authors in consultation.

No external funding was either sought or obtained for this study.

The authors have no conflicts of interest to declare.

Beeson D , Jennings P , Kramer W . Offspring searching for their sperm donors: how family types shape the process . Hum Reprod 2011 ; 26 : 2415 – 2424 .

Google Scholar

Brannen J . Mixing methods: the entry of qualitative and quantitative approaches into the research process . Int J Soc Res Methodol 2005 ; 8 : 173 – 184 .

Charmaz K . Grounded Theory in the 21st century; applications for advancing social justice studies . In: Denzin NK , Lincoln YS (eds). The Sage Handbook of Qualitative Research . California : Sage Publications Inc. , 2005 .

Google Preview

Cohen D , Crabtree B . Evaluative criteria for qualitative research in health care: controversies and recommendations . Ann Fam Med 2008 ; 6 : 331 – 339 .

de Lacey S . Parent identity and ‘virtual’ children: why patients discard rather than donate unused embryos . Hum Reprod 2005 ; 20 : 1661 – 1669 .

de Lacey SL , Peterson K , McMillan J . Child interests in assisted reproductive technology: how is the welfare principle applied in practice? Hum Reprod 2015 ; 30 : 616 – 624 .

Denzin N , Lincoln Y . Entering the field of qualitative research . In: Denzin NK , Lincoln YS (eds). The Landscape of Qualitative Research: Theories and Issues . Thousand Oaks : Sage , 1998 , 1 – 34 .

Dixon-Woods M , Bonas S , Booth A , Jones DR , Miller T , Shaw RL , Smith JA , Young B . How can systematic reviews incorporate qualitative research? A critical perspective . Qual Res 2006 ; 6 : 27 – 44 .

ESHRE Psychology and Counselling Guideline Development Group . Routine Psychosocial Care in Infertility and Medically Assisted Reproduction: A Guide for Fertility Staff , 2015 . http://www.eshre.eu/Guidelines-and-Legal/Guidelines/Psychosocial-care-guideline.aspx .

Freeman T , Jadva V , Kramer W , Golombok S . Gamete donation: parents' experiences of searching for their child's donor siblings or donor . Hum Reprod 2009 ; 24 : 505 – 516 .

Goedeke S , Daniels K , Thorpe M , Du Preez E . Building extended families through embryo donation: the experiences of donors and recipients . Hum Reprod 2015 ; 30 : 2340 – 2350 .

Hammarberg K , Johnson L , Bourne K , Fisher J , Kirkman M . Proposed legislative change mandating retrospective release of identifying information: consultation with donors and Government response . Hum Reprod 2014 ; 29 : 286 – 292 .

Kirkman M . Saviours and satyrs: ambivalence in narrative meanings of sperm provision . Cult Health Sex 2004 ; 6 : 319 – 336 .

Kirkman M , Rosenthal D , Johnson L . Families working it out: adolescents' views on communicating about donor-assisted conception . Hum Reprod 2007 ; 22 : 2318 – 2324 .

Kirkman M , Bourne K , Fisher J , Johnson L , Hammarberg K . Gamete donors' expectations and experiences of contact with their donor offspring . Hum Reprod 2014 ; 29 : 731 – 738 .

Kitto S , Chesters J , Grbich C . Quality in qualitative research . Med J Aust 2008 ; 188 : 243 – 246 .

Kovacs GT , Morgan G , Levine M , McCrann J . The Australian community overwhelmingly approves IVF to treat subfertility, with increasing support over three decades . Aust N Z J Obstetr Gynaecol 2012 ; 52 : 302 – 304 .

Leininger M . Evaluation criteria and critique of qualitative research studies . In: Morse J (ed). Critical Issues in Qualitative Research Methods . Thousand Oaks : Sage , 1994 , 95 – 115 .

Lincoln YS , Guba EG . Naturalistic Inquiry . Newbury Park, CA : Sage Publications , 1985 .

Morse J , Richards L . Readme First for a Users Guide to Qualitative Methods . Thousand Oaks : Sage , 2002 .

O'Reilly M , Parker N . ‘Unsatisfactory saturation’: a critical exploration of the notion of saturated sample sizes in qualitative research . Qual Res 2013 ; 13 : 190 – 197 .

Porter M , Bhattacharya S . Investigation of staff and patients' opinions of a proposed trial of elective single embryo transfer . Hum Reprod 2005 ; 20 : 2523 – 2530 .

Sandelowski M . The problem of rigor in qualitative research . Adv Nurs Sci 1986 ; 8 : 27 – 37 .

Wyverkens E , Provoost V , Ravelingien A , De Sutter P , Pennings G , Buysse A . Beyond sperm cells: a qualitative study on constructed meanings of the sperm donor in lesbian families . Hum Reprod 2014 ; 29 : 1248 – 1254 .

Young K , Fisher J , Kirkman M . Women's experiences of endometriosis: a systematic review of qualitative research . J Fam Plann Reprod Health Care 2014 ; 41 : 225 – 234 .

• conflict of interest
• credibility
• qualitative research
• quantitative methods

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1460-2350
• Copyright © 2024 European Society of Human Reproduction and Embryology
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Open access
• Published: 18 April 2024

Attitudes and perceptions towards developing a health educational video to enhance optimal uptake of malaria preventive therapy among pregnant women in Uganda: a qualitative study involving pregnant women, health workers, and Ministry of health officials

• Rita Nakalega 1 ,
• Ruth Nabisere-Arinaitwe 2 ,
• Nelson Mukiza 3 ,
• Cynthia Ndikuno Kuteesa 3 ,
• Denis Mawanda 2 ,
• Paul Natureeba 1 ,
• Ronnie Kasirye 1 ,
• Clemensia Nakabiito 1 ,
• Jane Nabakooza 4 ,
• Emmie Mulumba 1 ,
• Josephine Nabukeera 1 ,
• Joseph Ggita 1 ,
• Abel Kakuru 5 ,
• Lynn Atuyambe 6 ,
• Philippa Musoke 1 ,
• Mary Glenn Fowler 7 &
• Zubair Lukyamuzi 1  

BMC Health Services Research volume  24 , Article number:  484 ( 2024 ) Cite this article

16 Accesses

1 Altmetric

Metrics details

Malaria in pregnancy remains a major global public health problem. Intermittent prophylaxis treatment of malaria in pregnancy with Sulphadoxine-pyrimethamine and co-trimoxazole is efficacious for prevention of malaria in pregnancy HIV negative and positive women, respectively. However, uptake of the recommended doses of therapies has remained suboptimal in Uganda, majorly due to inadequate knowledge among pregnant women. Therefore, this study aimed to explore attitudes and perceptions towards developing an educational video for malaria preventive therapy.

We conducted an exploratory study with qualitative methods among pregnant women attending antenatal care at Kisenyi Health Center IV (KHCIV), health workers from KHCIV, and officials from the Ministry of Health. The study was conducted at KHCIV from October 2022 to March 2023. Focus group discussions (FGD) were conducted among purposively selected pregnant women and key informant interviews (KII) among health workers and Ministry of Health officials. Data were analyzed using inductive and deductive thematic methods in atlas ti.8.

A total of five FGDs comprising of 7–10 pregnant women were conducted; and KIIs were conducted among four mid-wives, two obstetricians, and two Ministry of Health officials. Generally, all respondents mentioned a need for interventions to improve malaria preventive knowledge among pregnant women; were positive about developing an educative video for malaria preventive therapy in pregnancy; and suggested a short, concise, and edutaining video focusing both the benefits of taking and risks of not taking malaria preventive therapy. They proposed that women may be encouraged to view the video as soon as they conceive and throughout the pregnancy. It also was suggested that the video may be viewed on television sets in maternal and reproductive health clinics and homes, and on smart phones.

Pregnant women, health workers, and Ministry of Health officials were positive about the development of a short edutaining video on malaria preventive therapy that focuses on both benefits of taking and risks of not taking the malaria preventive therapy in pregnancy. This information guided the video development and therefore, in the development of health educative videos, client and stakeholder inputs may always be solicited.

Peer Review reports

Introduction

Malaria in pregnancy remains a major public health concern worldwide; more than 50 million pregnant women live in malaria-endemic areas with 88% in sub-Saharan Africa (SSA) [ 1 , 2 ]. Malaria in pregnancy is associated with undesirable fetal and maternal outcomes including miscarriage, low birth weight, stillbirth, maternal morbidity, and mortality; moreover, malaria is the leading cause of preventable fetal and maternal deaths [ 3 , 4 , 5 ]. These undesirable fetal and maternal outcomes are worsened by coinfection with other diseases particularly HIV [ 6 , 7 , 8 ]; and Uganda remains one of the most affected countries by both HIV and malaria [ 9 , 10 , 11 , 12 ]. The country is malaria endemic with the prevalence malaria in pregnancy ranging from 8.9 to 51.1% [ 13 , 14 , 15 ], and HIV prevalence of 8% among women of reproductive age [ 16 ].

Intermittent prophylaxis treatment of malaria in pregnancy with Sulphadoxine-pyrimethamine (IPTp-SP) and co-trimoxazole (CTx) is efficacious for prevention of malaria in pregnancy in pregnant women without HIV and those with HIV, respectively [ 17 , 18 ]. These medications have shown to reduce malaria parasitemia in pregnancy by approximately 80% [ 18 , 19 ]. In HIV negative women, IPTp-SP is initiated after the 1st trimester and taken on a monthly basis throughout the pregnancy [ 20 , 21 ] whereas, CTx is taken on a daily basis throughout the pregnancy for women living with HIV [ 11 , 22 ]. Although antenatal care (ANC) attendance in Uganda had increased to 95%, uptake of the recommended doses of IPTp-SP and CTx had remained suboptimal especially in the urban areas [ 3 , 23 ]. In the country’s capital, Kampala, optimal uptake of IPTp-SP was as low as 39% in 2018 [ 3 ].

Multi-sectoral factors contribute to sub-optimal uptake of malaria preventive therapy in pregnancy. However, lack of education, ignorance, and inadequate awareness about malaria preventive therapy among pregnant women and health workers remain the major factors [ 3 , 24 , 25 , 26 , 27 ]. This is partly due to persistent low ratio of health workers to patients in Low Resource Settings (LRS) [ 28 ]; resulting in inadequate sensitization and education of women on malaria preventive therapy during ANC visits. Hence, a critical need for interventions to improve knowledge and uptake of malaria preventive therapy during pregnancy.

Brief video-based educational interventions are promising approaches in improving health related knowledge among the target populations [ 29 ]. These videos are highly cost-effective [ 29 , 30 ], have a strong track record in improving knowledge, and fostering behavior change in various health related fields including HIV [ 29 , 31 , 32 , 33 ]. However, the responses and reactions to these interventions among stakeholders in malaria prevention during pregnancy remain unknown.

In this study, we recorded client and stakeholder opinions, attitudes, and perceptions, to the development of an educational video to improve knowledge and uptake of malaria preventive therapy in pregnancy.

Study design and setting

This was a sub-study of a larger two phased sequential cross-sectional study that used a client- centered and stakeholder consultative approach to develop an educational video on malaria preventive therapy in pregnancy. The primary study had two phases; the first phase aimed to solicit inputs from clients and stakeholders in the development of an educative video for improving knowledge of malaria preventive therapy among pregnant women; and the second phase aimed to assess the feasibility and acceptability of the developed video. Therefore, the current study describes the first phase of the primary study. This phenomenological qualitative study was conducted at Kisenyi Health Center IV (KHCIV) from October 2022 to March 2023. KHCIV is a public health facility in Kampala Central division, administered by Kampala Capital City Authority (KCCA). In Uganda, a HCIV provides general health services plus minor surgeries [ 34 , 35 ]. The facility has a catchment area of ≈ 2,000,000 people from Kampala and suburbs, and serves over 200 persons per day [ 36 ]. It offers free HIV/TB Care services, ANC services, and other services. Daily, the facility had ANC attendance of about 100 women and conducted about 20 births.

Study population

The study population included pregnant women at any gestation age attending ANC at KHCIV during the study period and key informants. The key informants were obstetricians and midwives from maternal and child health department at KHCIV, and Ministry of Health (MoH) officials from the malaria control department.

Study outcomes

The study outcomes were opinions, attitudes, and perceptions of the study participants about the development of an educative video on malaria preventive therapy in pregnancy.

Sample size and sampling procedures

Fifty pregnant women attending ANC at KHCIV were purposively selected and enrolled in the study. Women were stratified by age and HIV status to undergo focus group discussions (FGD). A total of five FGDs each comprising of 7–10 women were conducted among HIV negative women aged < 18years, 18-24years, and 25-49years; and women living with HIV aged 18-24years, and 25-49years. Due to the limited number of women living with HIV aged < 18 years during the study period, we were unable to get enough women for a FGD for this age group. We, therefore, instead conducted an in-depth interview (IDI) with a teenager living with HIV. We also conducted key informant interviews (KII) among obstetricians, midwives, and Ministry of Health (MoH) officials. Focus group discussions, IDI, and KIIs were conducted until when saturation was reached [ 37 ] in each group.

Data collection procedures

Pregnant women attending ANC at KHCIV were purposively selected and informed about the study, and those who were interested were consented to participate. Women were selected from the ANC clinic daily and scheduled for FGDs. Demographics and clinical information were obtained from all participants using a questionnaire before participating in the FGDs. The FGDs were based on open discussion, and each lasted typically for about two hours in duration. Using an FGD guide (Appendix 1), the discussions were conducted by a trained social scientist (facilitator) in a calm place where conversations could not be overhead. Similarly, the IDI for a teenager living with HIV and KIIs for key informants were conducted in a calm place where conversations could not overhead. The interviews lasted between 15 and 30 min and were conducted face to face using an IDI or KII guide. The FGDs, IDI, and KIIs were conducted in respondents’ preferred language which were Luganda or English. All the interviews were audiotaped and transcribed by a professional. The study documents such as structured questionnaire, FGD/IDI guides, and consents were translated from English to Luganda, the local language used. These were then back translated to English; and we then compared the new translation with the original text and reconciled any meaningful differences between the two to ensure that the translations were accurate.

Data management

Audio recordings were transcribed verbatim directly into English by the study team within one week of collection. Quality checks were performed for each transcript, with corrections and revisions made to identified errors.

Data analysis

Using Stata version 17 (StataCorp, College Station, TX, USA), descriptive continuous variables, such as age, were summarized using mean and standard deviation (SD) while categorical variables were summarized using frequencies and proportions.

Qualitative data analysis

Using an inductive and deductive content analytic approach, recorded data were transcribed and analyzed by the study team supervised by a qualitative research expert, using Atlas t.8 software. The initial review was done during debriefing process to give a baseline understanding of the data and constant comparison process continued to identify new information until there was no redundancy in the themes and this was part of the steps during inductive approach.The Consolidated Criteria for Reporting Qualitative Studies checklist was used to report study findings [ 38 ]. During the inductive analysis, open coding was carried out to identify specific portions of text corresponding to information required for the video development. Provisional labels were defined and illustrated to become codes, which were assembled into a codebook. Data was coded by two coders (social scientist and corresponding author). After development of the initial codebook, the codebook was reviewed for consistency of text segmentation and code application with continued inter-coder agreement. The coders reached a consensus and grouped identified codes into grouped sub-category, category and then into subthemes and themes. Coded themes from all data were compared to obtain generalized themes after removing inconsistent codes. The choice of thematic headings was guided by both the core concepts emerging from the data [ 39 ] and by theoretical concepts of the health belief model (HBM) [ 40 ]. During the second phase of analysis, specific topics were designated as core categories; axial coding and constant comparison to explore the relationships between the discussion of sensitive data and contextual situation [ 41 ]. Findings and interpretations of the data were discussed until there was group consensus on the dominant themes and meanings contained in the data.

Ethical approval

Approval to conduct this study was obtained from the Makerere University School of Medicine Research Ethics Committee (Mak-SOMREC-2021-279) and Uganda National Council for Science and Technology (NS384ES). Administrative clearance was obtained from the Director of Health Services at KCCA and from KHCIV administration. Written informed consent was obtained from all participants; confidentiality and anonymity were strictly observed. Informed consent for illiterate participants was obtained in the presence of an impartial witness (guardian or other literate person not part of study team). The procedure of obtaining informed consent from illiterate participants was approved by the Makerere University School of Medicine Research Ethics Committee IRB. All methods were performed in accordance with the relevant guidelines and regulations of good clinical practice and human subject protection of ICH-6.

A total of 50 women were enrolled for FGDs and IDI. Eight key informants were also enrolled for key informant interviews. These included two obstetricians, four mid-wives, and two Ministry of Health officials. The mean age of the enrolled women was 23 years (SD ± 6 years); the majority, 26 (53.1%) had completed primary level of education and the median gestation age was 24 weeks with interquartile range (IQR) of 20–32 weeks. The majority 36 (73.5%) were currently living with their partners and most 41 (83.7%) were receiving financial support from their partner. The majority 27(55.1%) used 1–2 h to travel to the ANC facility (KHCIV) and most 26 (53.1%) had had more than two ANC visits, and the average number of ANC visits was 3 as shown in Table  1 .

To thematically categorize opinions, perceptions, and attitudes of respondents, we used the HBM [ 40 ] and Gain/Loss messaging Framework [ 42 ]. The HBM postulates that cues to action and presence of an enabling environment are among the vital constructs in the uptake of a particular behavior [ 40 ]. The Gain/Loss framework describes the packaging of a message focusing on the positive benefits of doing something or the risk of not doing something [ 42 ] as shown in Table  2 .

Four broad themes emerged from the data to describe the use of video-based intervention to educate women about malaria preventive therapy in pregnancy. These themes were: needed actions and strategies to improve knowledge of malaria preventive therapy in pregnancy; positive attitudes towards the use of the video-based intervention to educate women on malaria preventive therapy in pregnancy; Packaging and Framing of the message of malaria preventive therapy in the video; and delivery of the message of malaria preventive therapy in the video.

Strategies and actions needed to improve knowledge of malaria preventive therapy in pregnancy

Respondents acknowledged the need for strategies to improve knowledge about malaria preventive therapy in pregnancy; and they mentioned various strategies to improve knowledge and uptake of IPTp-SP and co-trimoxazole during pregnancy. The mentioned strategies included engagement of male sexual partners or treatment supporters and providing regular health education talks.

Regarding male partner and treatment supporter involvement, r espondents mentioned that the involvement males and treatment supporters could help to enhance women’s understanding of the purpose and benefits of taking malaria preventive therapy. The male partners and treatment supporters can educate further the women and can remind them to swallow the medicines in case the women forget.

“We always counsel women to bring their partners or come with a treatment supporter. This helps because if a woman has not understood the message or information given, the treatment supporter or partner can understand and explain or remind her.“ , midwife III.

For continuous educational talks, respondents mentioned that efforts should be made to provide comprehensive and repeated health education talks during ANC to minimize malaria preventive therapy knowledge deficits among women. Respondents suggested various strategies including education and sensitization of women during ANC visits and the use of communication materials (IEC), village health team (VHT) system, and media-based education such as televisions, radios, and social media.

“Like I told you earlier, providing women with information regarding the use of malaria preventive therapy including how to minimize the associated side effects will improve uptake. Women can be organized in small groups during ANC in addition to one-on-one sessions. We should also ensure that the information given reaches the sexual partners and treatment supporters” , Obstetrician II. “Women need to be adequately educated on why they should take these drugs to improve uptake and adherence”. FGD lady aged _15–17 yrs.

Demonstrations and illustrations were also encouraged during education talks.

“Yes, because with the flip chart, you can easily illustrate the importance of taking malaria preventive therapy. You can also have brochures which can show various actions on different pages. For example, you can show a happy woman taking IPTp-SP or co-trimoxazole on the first page, and on next page you show a healthy born baby” , midwife. III. “…like there various apps on phones, they can also develop for us a malaria prevention app to teach women about malaria. I can have that App on my phone and can learn the schedule for taking malaria preventive therapy”. FGD woman 25-49yrs.

Positive attitudes to developing and designing an educative video for malaria preventive therapy

All respondents agreed that an educative video was one of the best interventions that can improve knowledge and uptake of malaria preventive therapy in pregnancy. This is because videos can be easily understood and always attract and capture women’s attention.

“The video can be good and more receptive in educating women because many women don’t know why should take these drugs. We not only need to have health workers acting in the video but also pregnant women. We can have women giving success stories”. Obstetrician I. “ By viewing, we are entertained but also being educated. Yah, I think it is a good innovation”. MoH official I . “It’s a good innovation because, you know how people, especially those in reproductive age want to watch videos. MoH official II.

Packaging and Framing of the message of malaria preventive therapy in the video

Respondents mentioned how to package and provide (message) information in the video.

Packaging information in the video

Regarding packaging, respondents suggested various ways of packaging the information in the video. They mentioned that the video should be short and concise, focused, engaging or edutaining, and be acted by real human beings.

Short, concise, and focused video

Respondents mentioned that for the video to be effective and efficient, it should be short, concise, and focused. This is because some people don’t concentrate for long.

“The video should have well packaged key information. You know people’s concentration, especially for adults, is very short. So, the video should be short. For example , not be more than 5 minutes “. Obstetrician II.

Other respondents were worried about having a very short video which could result in insufficient information given.

“If the video is very fast and short, people may miss out on some information. So, I suggest the video to be about 10 minutes long”. FGD lady aged_15-17yrs. “It depends on how the information is given in the video. It needs actors who talk fast in about three to four minutes. Because, if it lasts longer, one can walk away before the video ends”. FGD lady aged_15-17yrs.

Engaging or edutaining video

Respondents emphasized that the video should be able engage women. This can be achieved by making the video edutaining.

“ …. but I suggest the developed video to be edutaining. Because people are good at watching videos if the videos are entertaining. So, it should not be a boring video”. MoH official I.

Use of real human beings to act in the video

Respondents suggested that the video should be designed in the form of a play to attract attention and concentration.

“A play would work better, but for talking, some of our speeches bore. It needs something that can attract attention. People can be very attentive to watch a play” Midwife. II.

Despite mixed feelings associated with the use of real human beings or animations like cartoons in the video, most of the respondents preferred real human beings because it could provide an impression of reality and seriousness. Moreover, respondents preferred having an individual acting other than providing a narrative story.

“I think having someone act is the best because it attracts someone’s attention to watch how the story goes. But narrating at times may be boring. Some people don’t take cartoons seriously, they think cartoons or animations are for children. But if it is a human talking and identifying themselves, people usually take it seriously “. Obstetrician I. “…a video should use a person who can explain to the women and understand. That person can explain the benefits of swallowing the medicine for malaria prevention, and the consequences of not swallowing the medicines. And that person should be serious to show what happens if you swallow or not swallow the medicine”. FGD lady 18-24yrs.

Messaging or providing information in the video

When asked about the key messages that can be included or provided in the video, respondents mentioned that the information should include the benefits of taking and risks of not taking malaria preventive therapy in pregnancy, and instructions of taking the therapy.

Including information on the benefits of taking and the risks of not taking malaria preventive therapy in pregnancy

Respondents mentioned that the video should have information emphasizing the purpose and benefits of malaria preventive therapy in pregnancy, and the risks of not taking the therapy.

“The video should show the real importance of taking these medicines because we always communicate the risks of not taking the medicine and forget to emphasize the purpose and benefits”. midwife I.

However, respondents mentioned that the purpose and benefits should be more emphasized than risks of not taking the therapy. This is because over emphasizing the potential risks of not taking IPTp-SP or co-trimoxazole can cause fear in women and deter them from following instructions. Additionally, over focusing on the potential risks of not taking malaria preventive therapy can sometimes be interpreted as coercion or forcing women to take the drugs. Therefore, the video should be directed towards motivating women to take the drugs.

“….and include information that motivates or encourages them to take the drugs. For example, giving them assurance about protection of both the woman and the unborn baby, and being healthier for both the woman and the unborn baby”, Obstetrician II. However, some respondents recommended that adequate information on the potential risks of not taking the drugs should be included as well. “We can also include adverse effects of malaria in pregnancy such as severe anemia, miscarriage, fetal death, and low birth weight” , midwife IV. “Let us also include problems like getting miscarriages, babies dying in womb, and mother herself dying. If a woman hears such problems, she can say; let me go and get drugs so that I don’t get problems”. FGD_18–24 years.

Including information on the instructions of taking malaria preventive therapy

Respondents mentioned that the instructions and schedules of taking these medicines should be included in the video.

“The video should include the drugs to use, how to use them, and their benefits, and then the effects of not taking the drugs”. FGD woman aged _25-49years.

Delivery of the message of malaria preventive therapy in the video

Respondents proposed practices that can be used in delivering the developed video to the target population. They mentioned the appropriate stage when pregnant women should start watching the video, the appropriate time of watching the video, channels of watching the video, and places where to watch the video from.

The stage at which a pregnant woman can start watching the video

Many of the respondents recommended watching such educational videos right from preconception or early pregnancy and through the entire pregnancy period.

“These women should be educated about malaria preventive therapy right before they become pregnant (preconception). And the information should be given to them continuously on a regular basis throughout pregnancy” , midwife III . “As early as we get in touch with the pregnant woman, we should provide them with malaria prevention messages. Even if someone has not reached the time of starting IPTp-SP, they can be looking forward to that time. And remember for co-trimoxazole, it should be started as soon as one becomes pregnant”. Obstetrician I.

The appropriate time, channels, and places of watching the video

Respondents also suggested various channels through which the video should be delivered. Respondents mentioned watching the video on television screens during antenatal care clinics or at home and watching the video on phones (smart phones). However, respondents emphasized that the video should be aired out at all points where pregnant women are found especially the ANC and reproductive health clinics.

“Using media like the televisions, this is the information someone can watch for example before the news or favorable programs, for example, women like soaps. You know that we have a bigger audience before the news, before the soaps and on peak of some programs”. Obstetrician II.

There were no noticeable differences regarding preferences to watch the video between women living with HIV and those without.

“I prefer watching the video on television screen at home because when I come to the health facility, I can be busy looking for health workers to work on me. I may not concentrate to watch the video. But when I am at home, I can even rewind the video several times so that I get to understand the message”. FGD lady aged_18–24 years. …. “You realize that someone can sit somewhere pick up their phone and watch the videos that they have on their phones. And if there is a TV, for example in the clinic or at home, people will watch it. MoH official II.

In this study, we aimed to obtain client and stakeholder opinions, attitudes, and perceptions to develop and design an educative video to improve knowledge and uptake of malaria preventive therapy in pregnancy. This study revealed that strategies and actions are needed to improve knowledge of malaria preventive therapy among pregnant women such as engaging of male sexual partners or treatment supporters and providing effective regular health education talks; pregnant women and key stakeholders were positive towards developing and designing a video-based intervention to educate women on malaria preventive therapy in pregnancy; a need for proper packaging and framing of the message of malaria preventive therapy in the video such as having a video that is short, edutaining, and focused on both benefits of taking and risks of not taking the malaria preventive therapy in pregnancy; and a need for effective means of delivering the video to pregnant women such as use of television sets to display the video at the clinic or at home, use of smart phones and social media, and ensuring that pregnant women access or watch the video as soon as possible after conception. The implications of these findings are as below.

Strategies and actions are needed to improve knowledge of malaria preventive therapy among pregnant women

The current study suggests a need for engaging male sexual partners and treatment supporters when educating women about malaria preventive therapy. This is because these can support the women in understanding the purpose and the benefits of the therapy. The male partner and treatment supporter can also remind the women to take their medicine and, hence increasing adherence. Previous studies have also proposed this strategy [ 43 , 44 ]. It’s reported that male partner involvement in ANC services is associated with better utilization of maternal health service, decreased delays in making the decision to seek medical attention, improves material, emotion, and physical support, and increased likelihood of adherence to medical care advice [ 45 , 46 ]. Therefore, more efforts are needed to involve male partners or treatment supporters in maternal and child health services.

Provision of effective regular health education talks. The study revealed that there is a need for more education talks and these talks should be effective. Many women don’t understand the purpose and the benefits of taking malaria preventive therapy, and don’t understand the associated risks of not taking the therapy. Other women may forget what they learnt previously, and policies and guidelines change overtime. Therefore, there is a need to have regular education talks concerning malaria preventive therapy during pregnancy. The importance of regular education talks to pregnant women has been reported previously [ 47 , 48 ]. However, there is a need to ensure that these talks are effective for all categories of pregnant women. In LRS like Uganda, education talks may be ineffective due to low ratio of health workers to patients [ 28 ]. This study revealed that the low health worker to patient ratio results in inadequate sensitization and education of pregnant women on malaria preventive therapy during ANC visits; and therefore, more scalable, and cost-effective interventions may be needed.

Positive reactions towards developing and designing a video-based intervention to educate women on malaria preventive therapy in pregnancy

This study revealed that pregnant women and key stakeholders were positive about the video based educational strategy in increasing knowledge and uptake of malaria preventive therapy in pregnancy. The strategy was seen as an innovation in the field of maternal and child health services. This finding was consistent with a study done in Ghana which showed that video job-aids were welcomed and effectively supported the delivery of seasonal malaria chemoprevention [ 49 ]. In this Ghanaian study, the conveyed videos enhanced learning and information retention. The videos also reinforced messages due to the fact that they could be viewed at any time and repeatedly [ 49 ]. Kim et al. also reported that brief video-based educational interventions are promising approaches in improving health related knowledge among the target population [ 29 ].

Proper packaging and framing of the message of malaria preventive therapy in the video

The current study emphasized a short, concise, and focused educative video. The study in Ghana also utilized short videos, which were found to be easily distributed on social media and thus reached many people in a short period of time [ 49 ]. Therefore, the current study showed that a short video of about 5 min maybe appropriate to improve knowledge and uptake of malaria preventive therapy in pregnancy.

Regarding the contents of the video, the current study showed that the video should emphasize the purpose and benefits of malaria preventive therapy in pregnancy and be edutaining. This was similar to the study conducted in Guinea which showed that positive messaging of the video content is a consensus of good practice and successful video-based education as reflected in the WHO and national guidelines on malaria prevention [ 50 , 51 ]. However, the current study also revealed that the risks of not taking malaria preventive therapy should equally be emphasized. It was mentioned that pregnant should also be informed about the adverse effect of malaria in pregnancy including miscarriage, low birth weight babies, and maternal or fetal death. A study among diabetic patients showed that patients who received negatively framed message showed significantly more favorable attitudes and perceived control toward diabetes self-care than those who viewed the positively framed message [ 52 ]. Therefore, the video may have both the information about the benefits of taking malaria preventive therapy and information about the risks of not taking the therapy.

The study revealed that the video should consistently and address concerns relevant to malaria etiology and prevention measures. This was similar to the recommendations from the Ugandan national health survey [ 53 ]. Relatedly, studies have also shown that highlighting and identifying the correct health messages in a consistent and repetitive manner boosts audience knowledge and malaria intervention uptake [ 49 , 54 ].

Delivering the video-based information to pregnant women

The study revealed that all women of reproductive age should be a target for this video-based education. This was consistent with previous reports and World Health Organization recommendations [ 55 , 56 ]. However, the findings of the current study emphasized women to receive this video-based education as soon as they become pregnant and throughout pregnancy. This was because malaria preventive medication like cotrimoxazole needs to be taken as soon as one becomes pregnant and throughout the pregnancy. Additionally, all women should know what they are supposed to do such as taking IPTp-SP even if they are not yet in 2nd trimester. This strategy has been previously recommended [ 57 , 58 ]. Therefore, women may be provided with the video-based education for malaria prevention as soon as they conceive.

The study revealed that the video-based education may be provided on television sets in maternal and childcare clinics as well as sexual and reproductive health clinics. A previous study showed that planning health messaging and delivery systematically offers the chance to fulfill people’s actual informational demands in addition to creating effective communication tactics [ 59 ]. It was also revealed that the video can also be viewed at home on a television set or anywhere on smart phone. A systematic review showed that short video education messages can be repeatedly watched from anywhere [ 49 ]. Therefore, the educative video for malaria preventive therapy in pregnancy may be designed to be viewed on television sets at clinics and home and on smart phones.

Strengths and limitations

Our study strengths include using iterative client centered processes to tailor and develop a video to improve knowledge on malaria prevention therapy among pregnant women. The study obtained data through a participatory process from pregnant women and key stakeholders in maternal and child health which ensured triangulation and complementation of information from the respondents. The study included HIV negative and HIV-positive women as well as women of different age groups, which enabled us to obtain varied responses and representation from various categories of pregnant women. Our study was conducted in an urban setting therefore, the findings are limited to the urban population and may not be generalized to rural communities.

Conclusions

In conclusion, pregnant women, health care workers, and ministry of health officials were enthusiastic and positive about developing and designing an educative video for malaria preventive therapy during pregnancy. However, they suggested a short edutaining video with properly packaged and framed information focused on both benefits of taking and risks of not taking the malaria preventive therapy in pregnancy. They also suggested delivering the video to pregnant women as soon as they conceive through television sets at maternal and reproductive health clinics and through smart phones. Data from this study was used to develop and design and educative video for malaria preventive therapy in pregnancy.

Data availability

The dataset used and analyzed during this study is available from the corresponding author on reasonable request.

Abbreviations

Acquired immunodeficiency syndrome

Focus Group Discussion

Human Immunodeficiency Virus

In-Depth Interviews

World Health Organization. A strategic framework for malaria prevention and control during pregnancy in the African region. World Health Organization. Regional Office for Africa; 2004.

World Health Organization. World Malaria Report 2016 (World Health Organization, Geneva) 2016.

Ministry of Health. U.N.M.C.D., Malaria indicator survey , T.D.P.I. Uganda Bureau of Statistics, Editor. 2018–29.

Arinaitwe E, et al. Intermittent preventive therapy with sulfadoxine-pyrimethamine for malaria in pregnancy: a cross-sectional study from Tororo. Uganda PloS One. 2013;8(9):e73073.

Article   CAS   PubMed   Google Scholar  

Saito M, et al. Deleterious effects of malaria in pregnancy on the developing fetus: a review on prevention and treatment with antimalarial drugs. Lancet Child Adolesc Health. 2020;4(10):761–74.

Twabi HS, Manda SO, Small DS. Assessing the effects of maternal HIV infection on pregnancy outcomes using cross-sectional data in Malawi. BMC Public Health. 2020;20:1–15.

Article   Google Scholar  

Enuma JN, et al. Malaria an opportunistic infection in HIV/AIDS patients?-A Nigerian experience. Afr J Lab Med. 2022;11(1):1–6.

Flateau C, Le Loup G, Pialoux G. Consequences of HIV infection on malaria and therapeutic implications: a systematic review. Lancet Infect Dis. 2011;11(7):541–56.

Article   PubMed   Google Scholar  

Vashishtha VM. World Malaria Report 2008: a billion-dollar moment for a centuries old disease. Indian Pediatr. 2008;45:985–6.

Google Scholar  

World Health Organization, World malaria report 2020: 20 years of global progress and challenges. World malaria report 2020: 20 years of global progress and challenges. 2020: p. 299.

World Health Organization. HIV treatment and care: WHO HIV policy adoption and implementation status in countries: factsheet. World Health Organization; 2019.

UNAIDS, t.J.U.N.P.o.H.A. IN DANGER: UNAIDS Global AIDS Update 2022. Geneva: Joint United Nations Programme on HIV/AIDS; 2022. Licence: CC BY-NC-SA 3.0 IGO . 2022.

Mangusho C, et al. High prevalence of malaria in pregnancy among women attending antenatal care at a large referral hospital in northwestern Uganda: a cross-sectional study. PLoS ONE. 2023;18(4):e0283755.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Braun V, et al. Lack of effect of intermittent preventive treatment for malaria in pregnancy and intense drug resistance in western Uganda. Malar J. 2015;14:1–10.

Okiring J, et al. Household and maternal risk factors for malaria in pregnancy in a highly endemic area of Uganda: a prospective cohort study. Malar J. 2019;18:1–9.

UGANDA AIDS COMMISSION SECRETARIAT. U. Factsheet - Facts on HIV and AIDS in Uganda 2021 (Based on Data ending 31st December 2020). Uganda AIDS commission 2021; https://uac.go.ug/media/attachments/2021/09/13/final-2021-hiv-aids-factsheet.pdf .

Gasasira AF, et al. Effect of trimethoprim-sulphamethoxazole on the risk of malaria in HIV-infected Ugandan children living in an area of widespread antifolate resistance. Malar J. 2010;9(1):1–8.

Mermin J, et al. Effect of co-trimoxazole prophylaxis, antiretroviral therapy, and insecticide-treated bednets on the frequency of malaria in HIV-1-infected adults in Uganda: a prospective cohort study. Lancet. 2006;367(9518):1256–61.

Fokam EB, et al. Assessment of the usage and effectiveness of intermittent preventive treatment and insecticide-treated nets on the indicators of malaria among pregnant women attending antenatal care in the Buea Health District, Cameroon. Malar J. 2016;15(1):1–7.

World Health Organization. Guidelines for the treatment of malaria, World Health Organization. Geneva, Switzerland.[Google Scholar]; 2015.

Al Khaja KA, Sequeira RP. Drug treatment and prevention of malaria in pregnancy: a critical review of the guidelines. Malar J. 2021;20:1–13.

Ministry of Health. U., Consolidated guidelines for Prevention and Treatment of HIV and AIDs in Uganda February 2020 2020.

Nekaka R, et al. Malaria preventive practices and delivery outcomes: a cross-sectional study of parturient women in a tertiary hospital in Eastern Uganda. PLoS ONE. 2020;15(8):e0237407.

Okethwangu D, et al. Factors associated with uptake of optimal doses of intermittent preventive treatment for malaria among pregnant women in Uganda: analysis of data from the Uganda Demographic and Health Survey, 2016. Malar J. 2019;18(1):1–8.

Article   CAS   Google Scholar  

Rassi C, et al. Assessing supply-side barriers to uptake of intermittent preventive treatment for malaria in pregnancy: a qualitative study and document and record review in two regions of Uganda. Malar J. 2016;15(1):1–16.

Mutulei ACNe. Factors influencing the uptake of intermittent preventive treatment for malaria in pregnancy: evidence from Bungoma East District, Kenya. Am J Public Health Res. 2013;1(5):110–23.

Wanzira H, et al. The challenge of using intermittent preventive therapy with sulfadoxine/pyrimethamine among pregnant women in Uganda. Malar J. 2016;15(1):1–7.

Bolan N, et al. Human resources for health-related challenges to ensuring quality newborn care in low-and middle-income countries: a scoping review. Global Health: Sci Pract. 2021;9(1):160–76.

Kim MH, et al. The video intervention to Inspire Treatment Adherence for Life (VITAL start): protocol for a multisite randomized controlled trial of a brief video-based intervention to improve antiretroviral adherence and retention among HIV-infected pregnant women in Malawi. Trials. 2020;21(1):1–13.

Sweat M, O’Donnell C, O’Donnell L. Cost-effectiveness of a brief video-based HIV intervention for African American and latino sexually transmitted disease clinic clients. Aids. 2001;15(6):781–7.

Fisher HH, et al. Evaluation of an HIV prevention intervention for African americans and hispanics: findings from the VOICES/VOCES Community-based Organization Behavioral Outcomes Project. AIDS Behav. 2011;15:1691–706.

Wong IY, et al. Development and assessment of an innovative culturally sensitive educational videotape to improve adherence to highly active antiretroviral therapy in Soweto, South Africa. JAIDS J Acquir Immune Defic Syndr. 2006;43:S142–8.

Tuong W, Larsen ER, Armstrong AW. Videos to influence: a systematic review of effectiveness of video-based education in modifying health behaviors. J Behav Med. 2014;37:218–33.

Statistics UBo. The National Population and Housing Census 2014– Main Report, Kampala, Uganda . 2016.

UBOS. UBOS Uganda demographic and health survey 2016: key indicators report. Kampala, Uganda (2017) 2017.

Ministry of Health, U. Annual health sector performamnce report, financial 2017/2018 . 2018.

Walker JL. Research column. The Use of Saturation in qualitative research. Can J Cardiovasc Nurs, 2012. 22(2).

Booth A et al. COREQ (consolidated criteria for reporting qualitative studies) Guidelines for reporting health research: A user’s manual, 2014: pp. 214–226.

Creswell JW, Creswell JD. Research design: qualitative, quantitative, and mixed methods approaches. Sage; 2017.

Strecher VJ, Rosenstock IM. The health belief model. Camb Handb Psychol Health Med. 1997;113:117.

Thorne S. Interpretive description (Vol. 2). Walnut Creek. CA: Left Coast; 2008.

Nan X, Daily K, Qin Y. Relative persuasiveness of gain-vs. loss-framed messages: a review of theoretical perspectives and developing an integrative framework. Rev Communication. 2018;18(4):370–90.

Alemi S, et al. Male participation in antenatal care and its influence on their pregnant partners’ reproductive health care utilization: insight from the 2015 Afghanistan demographic and Health Survey. J Biosoc Sci. 2021;53(3):436–58.

Omolola A, et al. Male participation in Antenatal Care and its effect on pregnancy. J Health Med Nurs. 2022;8(1):54–64.

World Health Organization. Engaging men, addressing harmful masculinities to improve sexual and reproductive health and rights 2019.

Tokhi M, et al. Involving men to improve maternal and newborn health: a systematic review of the effectiveness of interventions. PLoS ONE. 2018;13(1):e0191620.

Article   PubMed   PubMed Central   Google Scholar  

Jibril UN, et al. Health education intervention on knowledge and accessibility of pregnant women to antenatal care services in Edu, Kwara State, Nigeria. Int J Womens Health Reprod Sci. 2018;6(2):154–60.

Raoof AM, Al-Hadithi TS. Antenatal care in Erbil city-Iraq: assessment of information, education and communication strategy. DMJ. 2011;5(1):31–40.

Owusu-Addo E, Owusu-Addo SB. Effectiveness of health education in community-based malaria prevention and control interventions in sub-Saharan Africa: a systematic review 2014.

Scott S, et al. The use of video job-aids to improve the quality of seasonal malaria chemoprevention delivery. PLOS Digit Health. 2022;1(12):e0000165.

Mutanyi JA, et al. Determinants of the uptake of intermittent preventive treatment of malaria in pregnancy with sulphadoxine pyrimethamine in Sabatia Sub County, Western Kenya. Volume 10. Infectious Diseases of Poverty; 2021. p. 106. 1.

Park J, Kim SH, Kim JG. Effects of message framing and health literacy on intention to perform diabetes self-care: a randomized controlled trial. Diabetes Res Clin Pract. 2020;161:108043.

Ameyaw EK. Uptake of intermittent preventive treatment of malaria in pregnancy using sulfadoxine-pyrimethamine (IPTp-SP) in Uganda: a national survey. Malar J. 2022;21(1):1–10.

Darteh EKM, et al. Factors influencing the uptake of intermittent preventive treatment among pregnant women in sub-saharan Africa: a multilevel analysis. Archives Public Health. 2021;79(1):1–9.

Kilfoyle KA, et al. Health literacy and women’s reproductive health: a systematic review. J Women’s Health. 2016;25(12):1237–55.

Organization WH. WHO recommendations on adolescent sexual and reproductive health and rights 2018.

Wylie B, et al. Malaria in pregnancy: prevention and treatment. Waltham, MA: Uptodate; 2020.

Almalik MM, Mosleh SM. Pregnant women: what do they need to know during pregnancy? A descriptive study. Women Birth. 2017;30(2):100–6.

World Health Organization. WHO Strategic Communications Framework for Effective Communication . 2017.

Download references

Acknowledgements

The authors would like to thank the staff at Kisenyi Health Center IV for supporting study recruitment and data collection procedures. We thank study participants for their invaluable time and information. Finally, we thank Kampala Capital City Authority for granting us administrative permission to undertake the study.

This study was funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (grants 1R03HD106185-01A1). This content is entirely the responsibility of the authors and does not necessarily represent the official views of the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

Author information

Authors and affiliations.

Makerere University-Johns Hopkins University (MU-JHU) Research Collaboration, Kampala, Uganda

Rita Nakalega, Paul Natureeba, Ronnie Kasirye, Clemensia Nakabiito, Emmie Mulumba, Josephine Nabukeera, Joseph Ggita, Philippa Musoke & Zubair Lukyamuzi

Infectious Diseases Institute, Makerere University, Kampala, Uganda

Ruth Nabisere-Arinaitwe & Denis Mawanda

RineCynth Advisory, Kampala, Uganda

Nelson Mukiza & Cynthia Ndikuno Kuteesa

Ministry of Health, Kampala, Uganda

Jane Nabakooza

Infectious Diseases Research Collaboration, Kampala, Uganda

Abel Kakuru

Makerere University, Kampala, Uganda

Lynn Atuyambe

John Hopkins University, Baltimore, USA

Mary Glenn Fowler

You can also search for this author in PubMed   Google Scholar

Contributions

RN: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Supervision; Validation; Visualization; Writing– original draft; Writing– review & editing. RNA: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. NM: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. CNK: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. PN: Funding acquisition; Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. DM: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. RK: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. CN: Project administration; Resources; Supervision; Writing– review & editing. JN: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. EM: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. JN: Investigation; Methodology; Writing– original draft; Writing– review & editing. Joseph Ggita: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. AK: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. LA: Data curation; Investigation; Methodology; Writing– original draft; Writing– review & editing. PM: Project administration; Resources; Supervision; Writing– review & editing. MGF: Project administration; Resources; Supervision; Writing– review & editing. ZL: Funding acquisition; Methodology; Project administration; Resources; Software; Supervision; Validation; Visualization; Writing– original draft; Writing– review & editing.

Corresponding author

Correspondence to Rita Nakalega .

Ethics declarations

Ethics approval and consent to participate.

Approval to conduct this study was obtained from the Makerere University School of Medicine Research Ethics Committee (Mak-SOMREC-2021-279) and Uganda National Council for Science and Technology (NS384ES). Administrative clearance was obtained from the Director of Health Services at KCCA and from KHCIV administration. Written informed consent was obtained from all participants; confidentiality and anonymity were strictly observed. Informed consent for illiterate participants was obtained in the presence of an impartial witness (guardian or other literate person not part of study team). The procedure of obtaining informed consent from illiterate participants was approved the Makerere University School of Medicine Research Ethics Committee IRB. All methods were performed in accordance with the relevant guidelines and regulations of good clinical practice and human subject protection of ICH-6.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nakalega, R., Nabisere-Arinaitwe, R., Mukiza, N. et al. Attitudes and perceptions towards developing a health educational video to enhance optimal uptake of malaria preventive therapy among pregnant women in Uganda: a qualitative study involving pregnant women, health workers, and Ministry of health officials. BMC Health Serv Res 24 , 484 (2024). https://doi.org/10.1186/s12913-024-10944-x

Download citation

Received : 07 November 2023

Accepted : 02 April 2024

Published : 18 April 2024

DOI : https://doi.org/10.1186/s12913-024-10944-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Preventive therapy
• Educational video
• Qualitative study

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 17 April 2024

Distinct experiences and care needs of advanced cancer patients with good ECOG performance status: a qualitative phenomenological study

• Ping Chen 1 , 3   na1 ,
• Mingfu Ding 2   na1 ,
• Changlin Li 3 ,
• Yujuan Long 4 ,
• Deng Pan 5 ,
• Taiguo Liu 3 &
• Cheng Yi 1  

BMC Palliative Care volume  23 , Article number:  102 ( 2024 ) Cite this article

108 Accesses

1 Altmetric

Metrics details

Advanced cancer patients with good Eastern Cooperative Oncology Group (ECOG) performance status (score 0–1) are underrepresented in current qualitative reports compared with their dying counterparts.

To explore the experiences and care needs of advanced cancer patients with good ECOG.

A qualitative phenomenological approach using semi-structured interview was employed. Data was analyzed using the Colaizzi’s method.

Setting/Participants

Purposive sample of terminal solid cancer patients on palliative care aged 18–70 years with a 0–1 ECOG score were recruited from a tertiary general hospital.

Sixteen participants were interviewed. Seven themes were generated from the transcripts, including experiencing no or mild symptoms; independence in self-care, decision-making, and financial capacity; prioritization of cancer growth suppression over symptom management; financial concerns; hope for prognosis and life; reluctance to discuss death and after-death arrangements; and use of complementary and alternative medicine (CAM) and religious coping.

Conclusions

Advanced cancer patients with good ECOG have distinct experiences and care needs from their dying counterparts. They tend to experience no or mild symptoms, demonstrate a strong sense of independence, and prioritize cancer suppression over symptom management. Financial concerns were common and impact their care-related decision-making. Though being hopeful for their prognosis and life, many are reluctant to discuss death and after-death arrangements. Many Chinese patients use herbal medicine as a CAM modality but need improved awareness of and accessibility to treatment options. Healthcare professionals and policy-makers should recognize their unique experiences and needs when tailoring care strategies and policies.

Key statements

What is already known about the topic?

• Even in their advanced stage, cancer patients with good ECOG performance status are capable of self-care and less reliant on care provided by other.

• Existing qualitative research mainly focuses on advanced cancer patients with poor ECOG, emphasizing pain management, emotional distress, and palliative care.

What this paper adds?

• Our findings reveal distinct experiences and care needs of advanced cancer patients with good ECOG performance status from their dying counterparts.

Implications for practice, theory or policy .

• Healthcare professionals should recognize and address the patient group’s distinct needs.

• Future research should further investigate their symptom trajectory, influencing factors, and care needs to fill the gap in their cancer journey.

• Policy-makers should develop tailored policies that consider good ECOG performance status.

Peer Review reports

Introduction

Cancer is a major global public health issue, which affects millions of people around the world and in China. There were estimated 19.3 million new cancer cases and 10 million cancer deaths globally in 2020 [ 1 ]. China accounted for 24% of newly diagnosed cases and 30% of the cancer-related deaths [ 2 ]. As treatments and early detection methods continue to advance, the number of people living with cancer are on the rise, whose life expectancy is extended. For instance, the five-year survival rate for cancer patients in China rose from 30.9% in 2003–2005 to 40.5% in 2012–2015 [ 3 ], highlighting the growing population of individuals living with cancer for extended periods.

Cancer staging is crucial in clinical oncology. It provides pivotal information on the extent and spread of the disease to guide treatment decisions and prognosis [ 4 ]. Advanced-stage cancer patients, typically characterized by extensive tumor growth and metastasis, often face more aggressive treatments and poorer prognoses compared with those in an earlier stage. However, advanced cancer patients with good physical performance status, commonly measured as an Eastern Cooperative Oncology Group performance status (ECOG PS) score, are often underrepresented in the current qualitative research. Instead, their counterparts with poorer ECOG PS who are usually in the final stage of disease are the focus of current research interests, who typically experience more severe symptoms, functional decline, and higher risks of adverse events [ 5 , 6 ]. Researchers may opt not to recruit patients with good ECOG PS, who are more likely to continue working and engaging in daily activities and may be less accessible [ 7 ]. Also, this patient population may be neglected in favor of cancer patients nearing death because of limited research funding and resources [ 8 ].

In our clinical practice of palliative and hospice care, we often encounter cancer patients in the advanced stage with good ECOG PS (score 0–1). They impress us by demonstrating distinct physical and psychological experiences as well as varied care needs from those with poorer PS. This led us to hypothesize that despite the advanced stage of disease, cancer patients with good ECOG PS may have different experiences and care needs from their dying counterparts. Therefore, we devised this qualitative phenomenological study, which aimed to explore the experiences and care needs of advanced cancer patients with good ECOG PS. Our findings may shed light on some distinct characteristics of this underrepresented patient group and reveal novel areas potentially meaningful for future research. The evidence may find merit in supporting care-providers and decision-makers to more specifically tailor their strategies when caring for cancer patients in the advanced stage.

Study design

This was a phenomenological qualitative study. Semi-structured, in-depth interviews were conducted face-to-face with eligible participants to gather information about their lived experiences and care needs. The interviews continued until data saturation was achieved [ 9 ], at which point no new insights emerged from further interviews. Colaizzi’s method was employed to analyze the collected data [ 10 ].

Ethical consideration and informed consent

The study was ethically approved by the Ethics Committee of Chengdu Seventh People’s Hospital (reference number AF-SOP-09-2.1). Written informed consents were signed with all participants.

The present study took place at the Oncology Department of Chengdu Seventh People’s Hospital, a tertiary general hospital situated in Chengdu, Sichuan Province, Southwest China. As one of the national pilot centers for palliative and hospice care, this department offers both curative cancer treatments and palliative or hospice care to cancer patients across all stages of tumor progression.

Research team

Our research team consisted of 8 members, including 6 oncologists and 2 nurses. Each researcher had a minimum of 5 years of clinical experience in oncology, with at least 2 years in palliative care. Additionally, all team members were trained in phenomenological qualitative methodology. None of the researchers held religious affiliations or had any known inclined theoretical or ethical perspectives. The principal investigator underwent training in face-to-face semi-structured interview.

Participants

Participants were recruited from patients who received treatment and/or follow-up care at the Oncology Department between July and September 2022. The principal investigator screened potential participants based on the following inclusion and exclusion criteria:

Inclusion criteria: A patient should (1) be 18–70 yrs old with a stage IV cancer diagnosis of solid tumor and a life expectancy > 6 mo; (2) have an ECOG PS score of 0–1; (3) be currently undergoing palliative therapies, including but not limited to palliative chemotherapy and radiotherapy; (4) be aware of their diagnosis; (5) be able to participate in a 30 min in-depth interview; (6) possess sufficient cognitive capacity and verbal communication ability; and (7) provide consent to participate in the study and for the publication of findings.

Exclusion criteria: A patient was ineligible if they (1) were < 18 or > 70 yrs old; (2) had a non-cancer or non-solid tumor diagnosis, a tumor stage below IV, an ECOG PS score > 1, or a life expectancy < 6 mo; (3) were unaware of their diagnosis; (4) were receiving hospice or end-of-life care; (5) were physically unfit for an interview, as assessed by the interviewer; (6) had a known mental illness or exhibited insufficient cognitive and communication capacity; or (7) failed to provide consent.

Sampling and data collection

Purposive sampling was employed to select participants. The primary investigator accessed candidates’ medical records on the hospital’s electronic medical record system, screening them against the inclusion and exclusion criteria. She then approached a potential participant and made a brief casual conversation to visually assess their physical and mental status, cognitive abilities, and communication skills.

If the candidate seemed suitable for further interview, the primary investigator explained the study’s objectives and process, and inquired if they were interested in participating. After obtaining informed consent, the primary investigator either initiated the interview immediately or scheduled it for a later time, typically within the next 72 h. For those with scheduled interviews, the interviewer reassessed the participant’s status at the time of the interview. If the participant was deemed unfit by the primary investigator, the interview would be canceled and the participant excluded.

Participants were interviewed individually in a designated room, without the presence of family members. If a participant displayed signs of physical discomfort or reluctance to continue, the interview would be discontinued and any incomplete interviews discarded.

The sampling process persisted until data saturation was reached [ 9 ], at which point the interviews no longer produced new analytical information.

Semi-structured interview

The primary investigator conducted all semi-structured interviews to ensure consistency across the study. To prevent interviewer burnout and allow for timely verbatim transcription, no more than three participants were interviewed on a single day. Each interview was audio-recorded in its entirety.

Before delving into specific questions, the interviewer posed a grand tour question to guide the participant, typically phrased as “How do you feel today?” or “Could you tell me about how you feel recently?” Probing questions such as “Could you tell me more about the care/doctors/nurses?“, “Has the therapy made you feel better?“, and “What makes you think that?” were utilized to encourage participants to provide more detailed responses. The interviewer employed various techniques, such as rhetorical questioning, repetition, and response, to uncover the participants’ genuine feelings. Field notes were taken throughout the interviews to document the participants’ tone of voice, notable facial expressions, and body gestures. (Supplement 1 )

Immediately following each interview, the audio recordings and notes were cataloged. Another investigator transcribed the recordings verbatim and verified the accuracy of transcriptions within 24 h after the interview. To ensure anonymity, the records and transcriptions were de-identified, with participants assigned numerical identifiers (P1, P2, etc.) in place of personal information. All patient data, recordings, and transcriptions were maintained with strict confidentiality.

To ensure the rigor of this study, several strategies were employed. The interviewer maintained neutrality throughout the interviews by refraining from expressing personal opinions or judgments. When unclear statements or feelings arose, the primary investigator sought clarification from the participants during the interview. In cases of disagreement among researchers, the team referred back to the interview transcriptions and, if necessary, sought further clarification from the participants.

Participants and interviews

In this study, we interviewed 16 participants aged 33–64 yrs (mean age, 55.1 yrs), including 7 females. The interviews had an average duration of 17.7 min (range, 10.5–42.5 min). All participants were medically insured and diagnosed with stage IV solid tumors. Half of the participants (8/16) were asymptomatic, while the remaining experienced mild symptoms. Among the participants, 15 expressed expectations with their current treatment to suppress tumor growth, and 4 of symptomatic patients expected it to alleviate symptoms. Nine participants reported using Chinese herbal medicine (CHM) as a complement to their ongoing treatment. Two participants reported holding religious beliefs, both identifying as Christians. Detailed sociodemographic and clinical information of the participants is in Tables  1 and 2 .

Findings from interviews

We extracted seven overarching themes from the interviews, as follows:

Theme 1: experiencing no or mild symptoms

The majority of participants reported experiencing either no symptoms or only mild symptoms during the interview period, which did not substantially impact their daily lives. However, a few participants recounted instances of severe symptoms, such as debilitating pain, that hindered their ability to engage in routine daily activities.

“(I) feel fine. Almost nothing. Just a little coughing now and then.” - P7 .

“No, I feel not bad… (I) only feel more tired than before every day. Nothing else.” - P8 .

“My shoulder used to hurt really badly. I couldn’t lift my right arm… It hurt so badly that I’d rather die… Couldn’t do anything… (The pain) was relieved again after ascites extraction. Now it hurts a little some times but OK.” - P11 .

The most severe symptom was described by Participant 16, who had experienced abdominal distention, which was managed by ascites extraction:

“My belly felt very full all the time… Better after the ascites was extracted. (I) couldn’t do anything then but can take care of myself again now, at least do some of my own things now.” (Smiled) - P16 .

Theme 2: independence in self-care, decision-making, and financial capacity

The participants exhibited a strong sense of independence in multiple aspects, including self-care, care-related decision-making, and financial capability for care expenses.

Rather than being heavily dependent on family members or other informal caregivers, the majority of participants were either fully or partially self-sufficient in managing their daily living activities:

“I take care of myself. My husband helps sometimes especially when I’m hospitalized but I don’t like his cooking.” - P1 .

“I can take care of myself. They (family members) are busy working every day. I cook my meals. Easier for me to choose what I want to eat.” - P4 .

“My wife looks after me and I try to take care of my own daily living as long as I feel good enough.” - P10 .

Most participants seemed resolute in making their own decisions while having little trouble seeking input from a variety of sources:

“… I make my own decisions… My nephew works for a pharmaceutical company… He suggests me to ask if I can test only some of the gene sequencing tests…” - P1 .

In some cases, they even exhibited resistance to external interference with their decision-making, especial concerning their treatment-related decisions:

“This is my life. (I) should think for myself, whether (I) continue my treatment or stop.” - P3 .

Some participants mentioned that they paid for their treatments with their own savings and were hesitant to spend money from other family members or borrowed money for fear of becoming a burden:

“For now (I) still have enough money (for the treatments). (My medical) insurance reimburse most of the expense… more than 90% reimbursed. I only pay less than 10%.” - P5 .

“I pay for my own treatments… (I) don’t want him (his son) to spend his money. He’s not even married yet.” - P16 .

Some did not even quit working:

“I go back to work sometimes but they (superiors at work) don’t require me. Most of the time I work at home. Only some easy assignments… They pay me the minimal salary… I understand. It’s not easy for them either. Already very kind of them to keep me like this.” - P15 .

Theme 3: prioritization of cancer growth suppression over symptom management

Notably, when asked about their primary expectations with their current care, 15 out of 16 participants expressed a desire for cancer growth suppression. This included all 8 asymptomatic participants and 4 of the mildly symptomatic patients:

“I’ll take the PD-1 treatment (a targeted therapy) if my gene sequencing can find some new biomarker.” - P6 .

“… to control its (tumor) growth. I don’t know how much longer… There might be a new drug or treatment. Who knows.” (Laughed) - P7 .

Only 4 symptomatic participants mentioned alleviating their symptoms:

“My shoulder hurt really badly… After the ascites extraction, it didn’t hurt any more. It came back later and was relieved again after ascites extraction. Now it hurts a little some times but OK.” - P1 .

Theme 4: financial concerns

During the interviews, all participants mentioned financial concerns or burdens at some point, in some cases without any prompting from the interviewer. This was frequently associated with their decision to discontinue their current treatment. Notably, “running out of money” was identified as the primary reason for discontinuing treatment, which was emphasized more strongly than “not respond to therapy”:

“I’ll go on with the therapies as long as (I) have money… My son told me just to keep getting treated. He borrowed 8,000 Yuan when (he) came back from Shanghai. But I don’t want him to borrow money… I don’t want to become a burden (for my family). (I) just stop my treatment when I run out of money.” - P4 .

“… I’ll go on (with the treatment) as long as I can. Have to discontinue if no more money. What else can I do?” - P6 .

The availability of medical insurance reimbursement played a critical role in enabling them to afford ongoing treatment:

“The disease is a heavy burden. I can still afford it now. My medical insurance reimburses most of it but still a heavy burden. I’ll just discontinue therapy when run out of money.” - P2 .

Theme 5: hope for prognosis and life

Despite being aware of their diagnoses, the participants demonstrated a sense of hope for the prognosis of their treatments and their future prospects in life:

“I have no regret in life. My only unfinished business is to see my son getting married. I want to live to see it.” - P1 .

“The targeted therapy worked very well for me. I’m still taking it. Hopefully, it will last long.” - P9 .

Instead of giving up, they tended to seek new potential treatments if a particular therapy had failed:

“The doctor told me to take gene sequencing and see if some new biomarkers can be found after I stopped responding to the last medicine. I did and am waiting for the results now. (I) hope they can find a new one… Is there any other treatment I can use?” - P2 .

“… (the tumor) grew and spread again after my last surgery. The doctor said that more surgeries may not do any more good but there are still other possibilities, like targeted therapy.” - P7 .

Interestingly, none of the participants brought up topics such as the desire for dying with dignity or the consideration of a Do-Not-Resuscitate (DNR) agreement.

Theme 6: reluctance to discuss death and after-death arrangements

Two specific questions were incorporated into the interviews to inquire whether participants had contemplated their own death and discussed after-death arrangements with their families. The responses were varied. Nearly half of the participants (7/16) reported that they had either never or rarely thought about death, nor had they discussed after-death arrangements with their family members:

“I have never thought about it (death). Don’t want to… still a little afraid to talk about it.” - P5 .

“It crosses my mind sometimes but I don’t think about it… (I) have never discussed (the after-death arrangements) with them (family). It’s not time yet.” - P8 .

In contrast, some other participants seemed open to think about and discuss them:

“I did. That’s fine. I know I have it (cancer). They know I have it too. There’s no need to hide or fear. It’s pointless. Doesn’t help with anything. I talked about it (my death) with them (my family) once. They were kinda shy and almost cried. Then I stopped.” (Laughed) - P12 .

“(I) discussed my after-death arrangements with my family already. Better to get prepared earlier. No one knows when the time will come.” - P13 .

Theme 7: use of complementary and alternative medicine and religious coping

Seven participants reported utilizing complementary and alternative medicine (CAM) modalities, which were primarily limited to Chinese herbal medicine, acupuncture, and massage. The main reasons for use of CAM were to assist in suppressing tumor growth and alleviating symptoms:

“I took Chinese herbal medicine after chemotherapy to help with my nausea and vomiting. I also had acupuncture for my headache.” - P2 .

“I went to see a traditional Chinese medicine doctor. He was famous for treating cancer with herbal medicine.” - P9 .

An interesting observation was that two participants cited engaging in religious activities as part of their coping strategies. Notably, one of them adopted her religious practices shortly after receiving her cancer diagnosis:

“I began to believing in Christianity two years again… one month after I was diagnosed. The sisters (fellow believers) said that they would pray for me to heal… I enjoy the peace and joy.” - P2 .

“Yes, I’m a Christian… I pray for my health and healing sometimes.” - P16 .

Main findings

On this qualitative study, we sought to explore the experiences and care needs of advanced cancer patients with good ECOG PS, a population that has not been extensively studied. Understanding their unique perspectives is vital for optimizing care and ensuring that the patients’ needs are met in the continuum of cancer care. We were able to extract seven overarching themes from the narratives of our in-depth interviews with 16 participants, including their symptoms, sense of independence, treatment priorities, financial concerns, hope, reluctance to discuss death, and use of complementary and alternative medicine.

In the current study, we found most of our participants either asymptomatic or mildly symptomatic. This is consistent with previous studies about the relationship between performance status, symptom burden, and disease stage. The ECOG PS is widely used and essential measure of the functional capacity of cancer patient, which is shown to correlate with clinical outcomes, treatment tolerance, and survival [ 11 ]. Patients with good ECOG PS generally have no or limited disease-related restrictions, despite the advanced cancer stage. According to Cormier et al., several factors may contribute to their contrast with the poor PS patients, such as better overall health, effective symptom management, and more favorable tumor characteristics or response to treatment [ 12 ]. This practically differentiates the two sub-groups of advanced cancer patients not only in terms of their experiences of symptoms but their distinct care needs. Compared with their counterparts with poor PS, the better performing patients need less symptom management and psychological support and may have different care priorities, which echoes with our finding that the participants prioritized tumor suppression therapies over symptom management. It is important for care-providers to recognize the variability in symptom burden due to varied ECOG PS, monitor the symptom trajectory of a patient, and devise care strategies accordingly.

The second theme highlights the participants’ strong sense of independence in various aspects of their cancer journey, including self-care, care-related decision-making, and financial capacity. Their good ECOG PS may enable them to maintain functional autonomy and actively engage in daily activities and decision-making processes [ 11 ], who are either fully or partially self-sufficient in managing their daily living activities. This finding is significant as advanced cancer patients with good PS may require less assistance from family members and other informal caregivers. This autonomy in self-care can contribute to their overall quality of life and psychological well-being because maintaining independence is often considered essential in coping with cancer [ 13 ]. The participants’ resoluteness in making their own decisions while seeking input from various sources demonstrates their active engagement in their care process, which may lead to better treatment outcomes and satisfaction with care, as patients who participate in their care decisions often report feeling more empowered and in control of their lives [ 14 ]. It is vital for healthcare professionals to respect and support the patients’ autonomy by providing the necessary information and guidance for them to make informed choices about care [ 15 ]. Another aspect of independence was their financial capacity. According to Lentz and colleagues, having the financial resources to pay for care can alleviate some of the stress and burden associated with managing cancer and its treatments [ 16 ], which can further contribute to a patient’s sense of control and well-being during their care journey.

It is worth noting, however, that the participants in this study may not be representative of the broader population of advanced cancer patients because they were all medically insured, which covered a significant portion of their medical expenses. This reduced their out-of-pocket costs significantly, contributing to their sense of financial independence. The potential selection bias in our sample should be considered when interpreting our findings. The experiences of participants with medical insurance might not accurately reflect those without such coverage. Uninsured or under-insured patients may face substantial financial burden and stress relating to the costs of care [ 16 ]. Furthermore, financial ability to pay for care can play a crucial role in a patient’s decision-making, particularly when it comes to deciding whether to continue or discontinue treatment. Patients who are financially constrained may be more likely to consider discontinuing treatment, even if they are clinically eligible for and may benefit from the treatment [ 16 ], which was evident in our study. The financial burden of cancer treatment can lead to significant distress, prompting patients to weigh the benefits of treatment against the costs [ 17 ]. This may result in decisions not entirely aligned with their medical needs and preferences and potentially compromise their quality of care and clinical outcomes [ 18 ]. It is imperative that healthcare professionals should be attentive to a patient’s financial concerns, even when they perform well physically, and engage in open discussions about the costs of care and potential resources to support their decision-making.

The finding that advanced cancer patients with good ECOG PS prioritize cancer suppression therapies over symptom management was expectable. This preference can be easily understood according to Maslow’s hierarchy of needs where one would strive to fulfill their basic needs before addressing higher-order needs [ 19 ]. In our context, symptom management can be considered a basic need, as it addresses a patient’s physiological and safety concerns. Once the basic need is met, patients tend to shift their focus to higher-order needs, such as achieving the best possible cancer control and prolonging survival. In our case, the participants with good ECOG PS are mostly asymptomatic or mildly symptomatic. Their basic needs in terms of symptom management are relatively well-addressed. Therefore, they are more likely to prioritize such higher-order needs as cancer suppression for better clinical outcomes and maintain their functional status. This finding has significant implications for improving the prognosis of advanced cancer patients with good ECOG PS, who are generally more functional and experience fewer severe symptoms and as a result may be physically more tolerant to therapies, including aggressive or experimental treatments. This increased tolerance can enable clinicians to consider a broader range of treatment options, which may lead to better cancer control and improved their clinical outcomes, including better symptom management and prolonged survival. Additionally, these patients seem more driven to explore new and experimental treatment options. Their willingness to participate in clinical trials or seek innovative therapies may provide them with access to cutting-edge treatments with potentially benefits for their prognosis and survival [ 20 ]. As healthcare professionals, we should recognize this motivation and support them by providing information on clinical trials or innovative therapies.

Hope is known to have significant implications for advanced cancer patients, which may influences their emotional well-being, treatment decisions, and overall quality of life [ 21 , 22 , 23 , 24 ]. We found that the patients with good ECOG PS demonstrate hope for their prognosis and future prospects of life. Though being hopeful is generally beneficial for advanced cancer patients and may reinforce patient’s motivation to seek care positively, it must be noted that it can sometimes lead to unrealistic expectations or misguided decision-making [ 25 ]. This is particularly relevant in advanced cancer patients with good ECOG PS because their hopefulness and desire for aggressive treatment may lead them to misjudge their physical status and pursue therapies with limited efficacy or significant side effects [ 26 , 27 ]. It becomes crucial for healthcare professionals to be aware of this potential pitfall and ensure that patients are properly informed about their prognosis, treatment options, and potential risks and benefits.

It was noteworthy that none of our participants brought up such topics as the desire for dying with dignity or the consideration of a DNR agreement. Their relatively better quality of life and sense of hopefulness could make them more focused on treatment and improving prognosis, rather than considering end-of-life decisions [ 23 , 24 ]. Besides fear for death, another possible attributing factor is culture. Death and after-death arrangements are commonly thought of as negative topics or even taboos in Chinese culture, which are usually avoided especially when an individual is still living [ 28 ]. This resonates with Theme 6 where many participants were found reluctant to contemplate their own death and discuss after-death arrangements with their families. Their needs to be prepared for death, seek dignity in passing, and make arrangements after their deaths are expected to increase as they near the end of life [ 29 ]. Healthcare professionals should monitor their disease progression closely and offer support when needed.

Almost half of our participants (7/16) reported using CAM modalities, which were primarily limited to Chinese herbal medicine, acupuncture, and massage. The main reasons for employing CAM were to aid in suppressing tumor growth and alleviating symptoms. This finding highlights the diverse approaches that patients may take in searching for effective treatments, particularly where conventional therapies have limited efficacy or are associated with substantial side effects [ 11 ]. The limited diversity of CAM modalities suggests that the patients may lack awareness or accessibility to a broader array of CAM options, which is consistent with previous reports [ 30 , 31 ]. Furthermore, despite the clinical trials to investigate the safety and efficacy of Chinese herbal medicine as a complement to mainstream cancer treatments, there is still insufficient evidence to establish CAM modalities for suppressing tumor growth. As a result, use of CAM should be approached with caution and mainly considered for purposes other than tumor treatment.

It is intriguing to find that two of the participants engaged in religious activities as part of their coping strategies because Chinese people are often believed to have a lower prevalence of religious beliefs and studies investigating religious coping in the Chinese population are scarce [ 32 ]. A main impression of the two religious patients was that they seemed to use religion for practical purposes, rather than fully embracing the beliefs. This is consistent with previous studies where cancer patients may seek out religious practices as a way to manage stress, find peace, and maintain a sense of hope during difficult times. Praying for healing, for example, can provide a sense of control and agency in a situation where they may feel powerless [ 33 ]. Future research may continue to pursue the subject among Chinese cancer patients.

Compared with dying advanced cancer patients with poor ECOG PS, those with good ECOG PS display distinct experiences and care needs. They generally have milder symptoms, higher independence in self-care, decision-making, and financial capacity, and prioritize tumor suppression therapies over symptom management. On the other hand, patients with poor ECOG PS grapple with a higher symptom burden, increased reliance on support, and a focus on symptom relief and palliative care. Though sharing concerns about finances, demonstrating hopefulness, utilizing CAM, the two groups have varying degrees and objectives. Recognizing these distinctions is essential for healthcare professionals to provide customized, patient-centered care to address the unique needs of the well-performing cancer patients.

Strengths and limitations

As one of the few qualitative investigations to explore the experiences and care needs of advanced cancer patients with good ECOG PS, our study revealed that this patient group is distinct from their dying counterparts, who are the focus of the current research literature. Our findings highlight some observations characteristic of the patient group such as their asymptomatic or mildly symptomatic experiences, stronger sense of independence in various perspectives, and distinct care prioritization from those nearing death. However, further quantitative studies are needed to determine the association and synergistic dynamics of such characteristics.

Implications for practice

It is vital for healthcare professionals and policy-makers to recognize these unique experiences and care needs of advanced cancer patients with good ECOG PS and respond by providing necessary information, education, treatment options, and care strategies.

Data availability

Due to the sensitive nature of the interview recordings, the original audio recordings are prohibited from sharing. The desensitized transcripts and demographic data are available from the corresponding author on reasonable request.

Sabanciogullari S, Yilmaz FT. The effect of religious coping on hope level of cancer patients receiving chemotherapy. J Relig Health. 2019;60:2756–69.

Article   Google Scholar  

Bai M, Lazenby M. A systematic review of associations between spiritual well-being and quality of life at the scale and factor levels in studies among patients with cancer. J Palliat Med. 2015;18:286–98.

Article   PubMed   PubMed Central   Google Scholar  

Yang G, Zhang H, Gan Z, et al. Discrepant views of oncologists and cancer patients on complementary and alternative medicine in a Chinese General Hospital. Integr Cancer Ther. 2017;17:451–7.

Fan Y, Zhang H, Yang G, et al. China’s cancer patients’ perceptions, attitudes and participation in clinical trials of complementary and alternative medicine: a multi-center cross-sectional study. Eur J Integr Med. 2018;19:115–20.

Balneaves LG, Watling CZ, Hayward EN, et al. Addressing complementary and alternative medicine use among individuals with cancer: an integrative review and clinical practice guideline. JNCI: J Natl Cancer Inst. 2021;114:25–37.

Article   PubMed Central   Google Scholar  

Wright AA. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300:1665.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yu Xu. Death and dying in the Chinese culture: implications for health care practice. Home Health Care Manage Pract. 2007;19:412–4.

Sato T, Soejima K, Fujisawa D, et al. Prognostic understanding at diagnosis and associated factors in patients with advanced lung cancer and their caregivers. Oncologist. 2018;23:1218–29.

Weeks JC, Catalano PJ, Cronin A, et al. Patients’ expectations about effects of chemotherapy for advanced cancer. N Engl J Med. 2012;367:1616–25.

Avishai A, Conner M, Sheeran P. Setting realistic health goals: antecedents and consequences. Ann Behav Med. 2019;53:1020–31.

Article   PubMed   Google Scholar  

Kyota A, Kanda K. The lived experience of persons with terminal cancer: a qualitative study. Nurs Sci Q. 2021;34:189–95.

Benzein E, Norberg A, Saveman B-I. The meaning of the lived experience of hope in patients with cancer in palliative home care. Palliat Med. 2001;15:117–26.

Article   CAS   PubMed   Google Scholar  

Breitbart W, Pessin H, Rosenfeld B, et al. Individual meaning-centered psychotherapy for the treatment of psychological and existential distress: a randomized controlled trial in patients with advanced cancer. Cancer. 2018;124:3231–9.

Rustøen T, Cooper BA, Miaskowski C. A longitudinal study of the effects of a hope intervention on levels of hope and psychological distress in a community-based sample of oncology patients. Eur J Oncol Nurs. 2011;15:351–7.

Unger JM, Cook E, Tai E et al. The role of clinical trial participation in cancer research: barriers, evidence, and strategies. Am Soc Clin Oncol Educational Book. 2016:185–98.

Zalenski RJ, Raspa R. Maslow’s hierarchy of needs: a framework for achieving human potential in hospice. J Palliat Med. 2006;9:1120–7.

Fenn KM, Evans SB, McCorkle R, et al. Impact of financial burden of cancer on survivors’ quality of life. J Oncol Pract. 2014;10:332–8.

Wan C, Williams CP, Nipp RD, et al. Treatment decision making and financial toxicity in women with metastatic breast cancer. Clin Breast Cancer. 2021;21:37–46.

Lentz R, Benson AB, Kircher S. Financial toxicity in cancer care: prevalence, causes, consequences, and reduction strategies. J Surg Oncol. 2019;120:85–92.

Xiao L, Miao J, Peng M, et al. The effect of health literacy on patient’s perceived shared decision-making among Chinese cancer patients. Psycho-oncology. 2021;31:70–7.

Jovanovic N, Doyle PC, Theurer JA. Supporting patient autonomy in shared decision making for individuals with head and neck cancer. Am J Speech-Language Pathol. 2022;31:1588–600.

Ndetei D, Musyimi C, Tele A, et al. Psychological well-being and social functioning across the cancer stages: implications for palliative care. J Soc Work End-of-Life Palliat Care. 2018;14:194–208.

Cormier JN, Askew RL, Mungovan KS, et al. Lymphedema beyond breast cancer. Cancer. 2010;116:5138–49.

Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern cooperative oncology group. Am J Clin Oncol. 1982;5:649–56.

Northall T, Chang E, Hatcher D, et al. The application and tailoring of Colaizzi’s phenomenological approach in a hospital setting. Nurse Res. 2020;28:20–5.

Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS ONE. 2020;15:e0232076.

Levit L, Balogh E, Nass S, et al. editors. Delivering high-quality cancer care. Epub ahead of print December 27, 2013. https://doi.org/10.17226/18359

Cohen EEW, LaMonte SJ, Erb NL et al. American cancer society head and neck cancer survivorship care guideline. CA: A Cancer Journal for Clinicians. 2016;66:203–239.

Peppercorn JM, Smith TJ, Helft PR, et al. American society of clinical oncology statement: toward individualized care for patients with advanced cancer. J Clin Oncol. 2011;29:755–60.

Edge SB, Compton CC. The American joint committee on cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol. 2010;17:1471–4.

Zeng H, Chen W, Zheng R, et al. Changing cancer survival in China during 2003–15: a pooled analysis of 17 population-based cancer registries. Lancet Global Health. 2018;6:e555–67.

Fan X, Zhang B, He Y, et al. Burden of disease due to cancer — China, 2000–2019. China CDC Wkly. 2022;4:306–11.

Sung H, Ferlay J, Siegel RL et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians. 2021;71:209–249.

Download references

Acknowledgements

The authors would like to thank the Cancer Psychology and Health Management Committee of Sichuan Cancer Society (S.C.S.) for their exceptional guidance.

This work was supported by Medical Research Project of Sichuan Medical Association (S20061) and the Joint Research Fund of Chengdu Medical College-Chengdu Seventh People’s Hospital (2020LHJYZD-03). The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Author information

Ping Chen and Mingfu Ding contributed equally to this work.

Authors and Affiliations

Abdominal Oncology Ward, Division of Medical Oncology, Cancer Center, West China Hospital, Sichuan University, No. 37 Guoxue Alley, Wuhou District, Chengdu, 610041, Sichuan Province, China

Ping Chen & Cheng Yi

Rehabilitation Medicine Department, West China Hospital, Sichuan University, No. 37 Guoxue Alley, Wuhou District, Chengdu, 610041, Sichuan Province, China

Mingfu Ding

Department of Oncology, Chengdu Seventh People’s Hospital (Affiliated Cancer Hospital of Chengdu Medical College), No. 1 Shi’er Zhong Street, Wuhou District, Chengdu, 610041, Sichuan Province, China

Ping Chen, Changlin Li, Li Ma & Taiguo Liu

Department of Intensive Care Unit, Chengdu Seventh People’s Hospital (Affiliated Cancer Hospital of Chengdu Medical College), No. 1 Shi’er Zhong Street, Wuhou District, Chengdu, 610041, Sichuan Province, China

Yujuan Long

Department of Medical Oncology, Guangxi Academy of Medical Sciences and the People’s Hospital of Guangxi Zhuang Autonomous Region, Nanning, 530016, Guangxi, China

You can also search for this author in PubMed   Google Scholar

Contributions

PC is the primary investigator and conducted the interviews. PC, MD, TL, and CY conceptualised and designed this study. PC, CL, and YL processed the data and verified accuracy of data against the transcripts. PC, MD, CL, YL, DP, LM, TL, and CY analyzed the data and interpreted the findings. PC, TL, and CY wrote the initial draft. MD, CL, and YL provided critical feedback about the draft. All authors reviewed and approved the final manuscript for submission.

Corresponding authors

Correspondence to Taiguo Liu or Cheng Yi .

Ethics declarations

Ethics approval and consent to participate.

This study was performed according to the appropriate Chinese laws and regulations, the principles outlined in the Declaration of Helsinki, and Good Clinical Practice guidelines. The study was ethically approved by the Ethics Committee of Chengdu Seventh People’s Hospital (reference number AF-SOP-09-2.1). Written informed consents were signed with all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Chen, P., Ding, M., Li, C. et al. Distinct experiences and care needs of advanced cancer patients with good ECOG performance status: a qualitative phenomenological study. BMC Palliat Care 23 , 102 (2024). https://doi.org/10.1186/s12904-024-01425-3

Download citation

Received : 08 May 2023

Accepted : 28 March 2024

Published : 17 April 2024

DOI : https://doi.org/10.1186/s12904-024-01425-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Advanced cancer patient
• Eastern cooperative oncology group performance status
• Qualitative phenomenology

BMC Palliative Care

ISSN: 1472-684X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Writing Survey Questions

Perhaps the most important part of the survey process is the creation of questions that accurately measure the opinions, experiences and behaviors of the public. Accurate random sampling will be wasted if the information gathered is built on a shaky foundation of ambiguous or biased questions. Creating good measures involves both writing good questions and organizing them to form the questionnaire.

Questionnaire design is a multistage process that requires attention to many details at once. Designing the questionnaire is complicated because surveys can ask about topics in varying degrees of detail, questions can be asked in different ways, and questions asked earlier in a survey may influence how people respond to later questions. Researchers are also often interested in measuring change over time and therefore must be attentive to how opinions or behaviors have been measured in prior surveys.

Surveyors may conduct pilot tests or focus groups in the early stages of questionnaire development in order to better understand how people think about an issue or comprehend a question. Pretesting a survey is an essential step in the questionnaire design process to evaluate how people respond to the overall questionnaire and specific questions, especially when questions are being introduced for the first time.

For many years, surveyors approached questionnaire design as an art, but substantial research over the past forty years has demonstrated that there is a lot of science involved in crafting a good survey questionnaire. Here, we discuss the pitfalls and best practices of designing questionnaires.

Question development

There are several steps involved in developing a survey questionnaire. The first is identifying what topics will be covered in the survey. For Pew Research Center surveys, this involves thinking about what is happening in our nation and the world and what will be relevant to the public, policymakers and the media. We also track opinion on a variety of issues over time so we often ensure that we update these trends on a regular basis to better understand whether people’s opinions are changing.

At Pew Research Center, questionnaire development is a collaborative and iterative process where staff meet to discuss drafts of the questionnaire several times over the course of its development. We frequently test new survey questions ahead of time through qualitative research methods such as  focus groups , cognitive interviews, pretesting (often using an  online, opt-in sample ), or a combination of these approaches. Researchers use insights from this testing to refine questions before they are asked in a production survey, such as on the ATP.

Measuring change over time

Many surveyors want to track changes over time in people’s attitudes, opinions and behaviors. To measure change, questions are asked at two or more points in time. A cross-sectional design surveys different people in the same population at multiple points in time. A panel, such as the ATP, surveys the same people over time. However, it is common for the set of people in survey panels to change over time as new panelists are added and some prior panelists drop out. Many of the questions in Pew Research Center surveys have been asked in prior polls. Asking the same questions at different points in time allows us to report on changes in the overall views of the general public (or a subset of the public, such as registered voters, men or Black Americans), or what we call “trending the data”.

When measuring change over time, it is important to use the same question wording and to be sensitive to where the question is asked in the questionnaire to maintain a similar context as when the question was asked previously (see  question wording  and  question order  for further information). All of our survey reports include a topline questionnaire that provides the exact question wording and sequencing, along with results from the current survey and previous surveys in which we asked the question.

The Center’s transition from conducting U.S. surveys by live telephone interviewing to an online panel (around 2014 to 2020) complicated some opinion trends, but not others. Opinion trends that ask about sensitive topics (e.g., personal finances or attending religious services ) or that elicited volunteered answers (e.g., “neither” or “don’t know”) over the phone tended to show larger differences than other trends when shifting from phone polls to the online ATP. The Center adopted several strategies for coping with changes to data trends that may be related to this change in methodology. If there is evidence suggesting that a change in a trend stems from switching from phone to online measurement, Center reports flag that possibility for readers to try to head off confusion or erroneous conclusions.

Open- and closed-ended questions

One of the most significant decisions that can affect how people answer questions is whether the question is posed as an open-ended question, where respondents provide a response in their own words, or a closed-ended question, where they are asked to choose from a list of answer choices.

For example, in a poll conducted after the 2008 presidential election, people responded very differently to two versions of the question: “What one issue mattered most to you in deciding how you voted for president?” One was closed-ended and the other open-ended. In the closed-ended version, respondents were provided five options and could volunteer an option not on the list.

When explicitly offered the economy as a response, more than half of respondents (58%) chose this answer; only 35% of those who responded to the open-ended version volunteered the economy. Moreover, among those asked the closed-ended version, fewer than one-in-ten (8%) provided a response other than the five they were read. By contrast, fully 43% of those asked the open-ended version provided a response not listed in the closed-ended version of the question. All of the other issues were chosen at least slightly more often when explicitly offered in the closed-ended version than in the open-ended version. (Also see  “High Marks for the Campaign, a High Bar for Obama”  for more information.)

Researchers will sometimes conduct a pilot study using open-ended questions to discover which answers are most common. They will then develop closed-ended questions based off that pilot study that include the most common responses as answer choices. In this way, the questions may better reflect what the public is thinking, how they view a particular issue, or bring certain issues to light that the researchers may not have been aware of.

When asking closed-ended questions, the choice of options provided, how each option is described, the number of response options offered, and the order in which options are read can all influence how people respond. One example of the impact of how categories are defined can be found in a Pew Research Center poll conducted in January 2002. When half of the sample was asked whether it was “more important for President Bush to focus on domestic policy or foreign policy,” 52% chose domestic policy while only 34% said foreign policy. When the category “foreign policy” was narrowed to a specific aspect – “the war on terrorism” – far more people chose it; only 33% chose domestic policy while 52% chose the war on terrorism.

In most circumstances, the number of answer choices should be kept to a relatively small number – just four or perhaps five at most – especially in telephone surveys. Psychological research indicates that people have a hard time keeping more than this number of choices in mind at one time. When the question is asking about an objective fact and/or demographics, such as the religious affiliation of the respondent, more categories can be used. In fact, they are encouraged to ensure inclusivity. For example, Pew Research Center’s standard religion questions include more than 12 different categories, beginning with the most common affiliations (Protestant and Catholic). Most respondents have no trouble with this question because they can expect to see their religious group within that list in a self-administered survey.

In addition to the number and choice of response options offered, the order of answer categories can influence how people respond to closed-ended questions. Research suggests that in telephone surveys respondents more frequently choose items heard later in a list (a “recency effect”), and in self-administered surveys, they tend to choose items at the top of the list (a “primacy” effect).

Because of concerns about the effects of category order on responses to closed-ended questions, many sets of response options in Pew Research Center’s surveys are programmed to be randomized to ensure that the options are not asked in the same order for each respondent. Rotating or randomizing means that questions or items in a list are not asked in the same order to each respondent. Answers to questions are sometimes affected by questions that precede them. By presenting questions in a different order to each respondent, we ensure that each question gets asked in the same context as every other question the same number of times (e.g., first, last or any position in between). This does not eliminate the potential impact of previous questions on the current question, but it does ensure that this bias is spread randomly across all of the questions or items in the list. For instance, in the example discussed above about what issue mattered most in people’s vote, the order of the five issues in the closed-ended version of the question was randomized so that no one issue appeared early or late in the list for all respondents. Randomization of response items does not eliminate order effects, but it does ensure that this type of bias is spread randomly.

Questions with ordinal response categories – those with an underlying order (e.g., excellent, good, only fair, poor OR very favorable, mostly favorable, mostly unfavorable, very unfavorable) – are generally not randomized because the order of the categories conveys important information to help respondents answer the question. Generally, these types of scales should be presented in order so respondents can easily place their responses along the continuum, but the order can be reversed for some respondents. For example, in one of Pew Research Center’s questions about abortion, half of the sample is asked whether abortion should be “legal in all cases, legal in most cases, illegal in most cases, illegal in all cases,” while the other half of the sample is asked the same question with the response categories read in reverse order, starting with “illegal in all cases.” Again, reversing the order does not eliminate the recency effect but distributes it randomly across the population.

Question wording

The choice of words and phrases in a question is critical in expressing the meaning and intent of the question to the respondent and ensuring that all respondents interpret the question the same way. Even small wording differences can substantially affect the answers people provide.

[View more Methods 101 Videos ]

An example of a wording difference that had a significant impact on responses comes from a January 2003 Pew Research Center survey. When people were asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule,” 68% said they favored military action while 25% said they opposed military action. However, when asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule  even if it meant that U.S. forces might suffer thousands of casualties, ” responses were dramatically different; only 43% said they favored military action, while 48% said they opposed it. The introduction of U.S. casualties altered the context of the question and influenced whether people favored or opposed military action in Iraq.

There has been a substantial amount of research to gauge the impact of different ways of asking questions and how to minimize differences in the way respondents interpret what is being asked. The issues related to question wording are more numerous than can be treated adequately in this short space, but below are a few of the important things to consider:

First, it is important to ask questions that are clear and specific and that each respondent will be able to answer. If a question is open-ended, it should be evident to respondents that they can answer in their own words and what type of response they should provide (an issue or problem, a month, number of days, etc.). Closed-ended questions should include all reasonable responses (i.e., the list of options is exhaustive) and the response categories should not overlap (i.e., response options should be mutually exclusive). Further, it is important to discern when it is best to use forced-choice close-ended questions (often denoted with a radio button in online surveys) versus “select-all-that-apply” lists (or check-all boxes). A 2019 Center study found that forced-choice questions tend to yield more accurate responses, especially for sensitive questions.  Based on that research, the Center generally avoids using select-all-that-apply questions.

It is also important to ask only one question at a time. Questions that ask respondents to evaluate more than one concept (known as double-barreled questions) – such as “How much confidence do you have in President Obama to handle domestic and foreign policy?” – are difficult for respondents to answer and often lead to responses that are difficult to interpret. In this example, it would be more effective to ask two separate questions, one about domestic policy and another about foreign policy.

In general, questions that use simple and concrete language are more easily understood by respondents. It is especially important to consider the education level of the survey population when thinking about how easy it will be for respondents to interpret and answer a question. Double negatives (e.g., do you favor or oppose  not  allowing gays and lesbians to legally marry) or unfamiliar abbreviations or jargon (e.g., ANWR instead of Arctic National Wildlife Refuge) can result in respondent confusion and should be avoided.

Similarly, it is important to consider whether certain words may be viewed as biased or potentially offensive to some respondents, as well as the emotional reaction that some words may provoke. For example, in a 2005 Pew Research Center survey, 51% of respondents said they favored “making it legal for doctors to give terminally ill patients the means to end their lives,” but only 44% said they favored “making it legal for doctors to assist terminally ill patients in committing suicide.” Although both versions of the question are asking about the same thing, the reaction of respondents was different. In another example, respondents have reacted differently to questions using the word “welfare” as opposed to the more generic “assistance to the poor.” Several experiments have shown that there is much greater public support for expanding “assistance to the poor” than for expanding “welfare.”

We often write two versions of a question and ask half of the survey sample one version of the question and the other half the second version. Thus, we say we have two  forms  of the questionnaire. Respondents are assigned randomly to receive either form, so we can assume that the two groups of respondents are essentially identical. On questions where two versions are used, significant differences in the answers between the two forms tell us that the difference is a result of the way we worded the two versions.

One of the most common formats used in survey questions is the “agree-disagree” format. In this type of question, respondents are asked whether they agree or disagree with a particular statement. Research has shown that, compared with the better educated and better informed, less educated and less informed respondents have a greater tendency to agree with such statements. This is sometimes called an “acquiescence bias” (since some kinds of respondents are more likely to acquiesce to the assertion than are others). This behavior is even more pronounced when there’s an interviewer present, rather than when the survey is self-administered. A better practice is to offer respondents a choice between alternative statements. A Pew Research Center experiment with one of its routinely asked values questions illustrates the difference that question format can make. Not only does the forced choice format yield a very different result overall from the agree-disagree format, but the pattern of answers between respondents with more or less formal education also tends to be very different.

One other challenge in developing questionnaires is what is called “social desirability bias.” People have a natural tendency to want to be accepted and liked, and this may lead people to provide inaccurate answers to questions that deal with sensitive subjects. Research has shown that respondents understate alcohol and drug use, tax evasion and racial bias. They also may overstate church attendance, charitable contributions and the likelihood that they will vote in an election. Researchers attempt to account for this potential bias in crafting questions about these topics. For instance, when Pew Research Center surveys ask about past voting behavior, it is important to note that circumstances may have prevented the respondent from voting: “In the 2012 presidential election between Barack Obama and Mitt Romney, did things come up that kept you from voting, or did you happen to vote?” The choice of response options can also make it easier for people to be honest. For example, a question about church attendance might include three of six response options that indicate infrequent attendance. Research has also shown that social desirability bias can be greater when an interviewer is present (e.g., telephone and face-to-face surveys) than when respondents complete the survey themselves (e.g., paper and web surveys).

Lastly, because slight modifications in question wording can affect responses, identical question wording should be used when the intention is to compare results to those from earlier surveys. Similarly, because question wording and responses can vary based on the mode used to survey respondents, researchers should carefully evaluate the likely effects on trend measurements if a different survey mode will be used to assess change in opinion over time.

Question order

Once the survey questions are developed, particular attention should be paid to how they are ordered in the questionnaire. Surveyors must be attentive to how questions early in a questionnaire may have unintended effects on how respondents answer subsequent questions. Researchers have demonstrated that the order in which questions are asked can influence how people respond; earlier questions can unintentionally provide context for the questions that follow (these effects are called “order effects”).

One kind of order effect can be seen in responses to open-ended questions. Pew Research Center surveys generally ask open-ended questions about national problems, opinions about leaders and similar topics near the beginning of the questionnaire. If closed-ended questions that relate to the topic are placed before the open-ended question, respondents are much more likely to mention concepts or considerations raised in those earlier questions when responding to the open-ended question.

For closed-ended opinion questions, there are two main types of order effects: contrast effects ( where the order results in greater differences in responses), and assimilation effects (where responses are more similar as a result of their order).

An example of a contrast effect can be seen in a Pew Research Center poll conducted in October 2003, a dozen years before same-sex marriage was legalized in the U.S. That poll found that people were more likely to favor allowing gays and lesbians to enter into legal agreements that give them the same rights as married couples when this question was asked after one about whether they favored or opposed allowing gays and lesbians to marry (45% favored legal agreements when asked after the marriage question, but 37% favored legal agreements without the immediate preceding context of a question about same-sex marriage). Responses to the question about same-sex marriage, meanwhile, were not significantly affected by its placement before or after the legal agreements question.

Another experiment embedded in a December 2008 Pew Research Center poll also resulted in a contrast effect. When people were asked “All in all, are you satisfied or dissatisfied with the way things are going in this country today?” immediately after having been asked “Do you approve or disapprove of the way George W. Bush is handling his job as president?”; 88% said they were dissatisfied, compared with only 78% without the context of the prior question.

Responses to presidential approval remained relatively unchanged whether national satisfaction was asked before or after it. A similar finding occurred in December 2004 when both satisfaction and presidential approval were much higher (57% were dissatisfied when Bush approval was asked first vs. 51% when general satisfaction was asked first).

Several studies also have shown that asking a more specific question before a more general question (e.g., asking about happiness with one’s marriage before asking about one’s overall happiness) can result in a contrast effect. Although some exceptions have been found, people tend to avoid redundancy by excluding the more specific question from the general rating.

Assimilation effects occur when responses to two questions are more consistent or closer together because of their placement in the questionnaire. We found an example of an assimilation effect in a Pew Research Center poll conducted in November 2008 when we asked whether Republican leaders should work with Obama or stand up to him on important issues and whether Democratic leaders should work with Republican leaders or stand up to them on important issues. People were more likely to say that Republican leaders should work with Obama when the question was preceded by the one asking what Democratic leaders should do in working with Republican leaders (81% vs. 66%). However, when people were first asked about Republican leaders working with Obama, fewer said that Democratic leaders should work with Republican leaders (71% vs. 82%).

The order questions are asked is of particular importance when tracking trends over time. As a result, care should be taken to ensure that the context is similar each time a question is asked. Modifying the context of the question could call into question any observed changes over time (see  measuring change over time  for more information).

A questionnaire, like a conversation, should be grouped by topic and unfold in a logical order. It is often helpful to begin the survey with simple questions that respondents will find interesting and engaging. Throughout the survey, an effort should be made to keep the survey interesting and not overburden respondents with several difficult questions right after one another. Demographic questions such as income, education or age should not be asked near the beginning of a survey unless they are needed to determine eligibility for the survey or for routing respondents through particular sections of the questionnaire. Even then, it is best to precede such items with more interesting and engaging questions. One virtue of survey panels like the ATP is that demographic questions usually only need to be asked once a year, not in each survey.

U.S. Surveys

Other research methods, sign up for our weekly newsletter.

Fresh data delivered Saturday mornings

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Locum doctor working and quality and safety: a qualitative study in English primary and secondary care
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-8129-8376 Jane Ferguson 1 ,
• http://orcid.org/0000-0001-9325-3362 Gemma Stringer 2 ,
• http://orcid.org/0000-0002-0696-480X Kieran Walshe 2 ,
• http://orcid.org/0000-0002-2972-7911 Thomas Allen 3 , 4 ,
• http://orcid.org/0000-0003-1621-8648 Christos Grigoroglou 3 ,
• http://orcid.org/0000-0002-2958-915X Darren M Ashcroft 5 ,
• http://orcid.org/0000-0001-6450-5815 Evangelos Kontopantelis 6 , 7
• 1 Health Services Management Centre, School of Social Policy , University of Birmingham , Birmingham , UK
• 2 Alliance Manchester Business School , University of Manchester , Manchester , UK
• 3 Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care , University of Manchester , Manchester , UK
• 4 Danish Centre for Health Economics , University of Southern Denmark , Odense , Denmark
• 5 NIHR Greater Manchester Patient Safety Research Collaboration (PSRC), Division of Pharmacy and Optometry, Faculty of Biology Medicine and Health , University of Manchester , Manchester , UK
• 6 Division of Informatics, Imaging and Data Sciences , University of Manchester , Manchester , UK
• 7 NIHR School for Primary Care Research, Centre for Primary Care, Division of Population Health, Health Services Research and Primary Care , University of Manchester , Manchester , UK
• Correspondence to Dr Jane Ferguson, Health Services Management Centre, University of Birmingham, Birmingham, UK; j.ferguson.1{at}bham.ac.uk

Background The use of temporary doctors, known as locums, has been common practice for managing staffing shortages and maintaining service delivery internationally. However, there has been little empirical research on the implications of locum working for quality and safety. This study aimed to investigate the implications of locum working for quality and safety.

Methods Qualitative semi-structured interviews and focus groups were conducted with 130 participants, including locums, patients, permanently employed doctors, nurses and other healthcare professionals with governance and recruitment responsibilities for locums across primary and secondary healthcare organisations in the English NHS. Data were collected between March 2021 and April 2022. Data were analysed using reflexive thematic analysis and abductive analysis.

Results Participants described the implications of locum working for quality and safety across five themes: (1) ‘familiarity’ with an organisation and its patients and staff was essential to delivering safe care; (2) ‘balance and stability’ of services reliant on locums were seen as at risk of destabilisation and lacking leadership for quality improvement; (3) ‘discrimination and exclusion’ experienced by locums had negative implications for morale, retention and patient outcomes; (4) ‘defensive practice’ by locums as a result of perceptions of increased vulnerability and decreased support; (5) clinical governance arrangements, which often did not adequately cover locum doctors.

Conclusion Locum working and how locums were integrated into organisations posed some significant challenges and opportunities for patient safety and quality of care. Organisations should take stock of how they work with the locum workforce to improve not only quality and safety but also locum experience and retention.

• Health services research
• Patient safety
• Qualitative research
• Quality improvement

Data availability statement

No data are available.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjqs-2023-016699

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Despite longstanding policy concerns about the implications of locum working for quality and safety, there has been little empirical research. Understanding how organisations engage, support and work with locums and how locum doctors integrate and interact with the complex and changing systems in which they work is essential if quality and safety are to be improved.

WHAT THIS STUDY ADDS

This qualitative study examines the perspectives of locums, patients and people who work with locums to identify the implications of temporary medical working for quality and safety.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Organisations should examine how they engage, support and work with locums. Organisations and locums need to reflect on whether their practices support a collective approach to patient safety and quality of care.

Introduction

Temporary doctors, often known as locums, are a vital resource that enable healthcare organisations to deliver care by flexing capacity and covering staffing gaps. In the United Kingdom, all doctors, other than those in their first year of training after qualifying, can work as a locum. Locum work can vary from very short-term (a single shift) to longer-term assignments (weeks, months or even sometimes years). Locums find work through various platforms, including locum agencies, online job platforms, professional networks or word of mouth. Locum agencies typically have some governance responsibilities (such as compliance with regulations and licensing requirements), but the extent of these responsibilities varies and the NHS in England has no oversight over how recruitment agencies operate. Despite concerns among policymakers, healthcare providers, professional associations and professional regulators about the implications of locum working for quality and safety and cost, 1–3 there is limited robust empirical research to evidence or support those concerns.

The workforce retention crisis is a significant challenge in healthcare internationally 4–6 and persistent understaffing poses a serious risk to patient safety. 7 8 In the UK, high doctor turnover has been linked to poorer service and health outcomes 9 and has led NHS trusts and general practices (GPs) to be ‘overly reliant’ 3 on temporary staff to fill rota gaps. 10 11 Expenditure on temporary staff in the NHS in England increased from £3.45 billion to £5.2 billion between 2021 and 2022. 3 12 The NHS Long Term Workforce Plan aims to reduce reliance on temporary staff and make substantive employment the most cost-effective and attractive option. 3 However, with the vacancy rate in the NHS projected to increase, 13 locums are likely to continue to be essential to maintaining service provision, especially in shortage specialities such as psychiatry. 14

An obvious implication of locum working is a reduced likelihood of organisational and team integration, 15 familiarity and a shared understanding of ‘the way things are done around here’. 16 Locums are likely to be less familiar with teams and other contextual factors relevant to providing safe and effective care 17 and more likely to be situated on the periphery of organisational structures, teams and governance systems 1 18 Teamwork represents a powerful process to improve patient care, 19 20 and trust, shared understanding, communication and collaboration have been associated with better patient outcomes. 21 22 The ability of healthcare teams to develop and maintain team situational awareness, or a shared perception, comprehension and subsequent projection of what is going on in complex and changing clinical environments, has been described as crucial for patient safety. 23 24 Through participation and working together, 25 teams gain an understanding of the roles, skills and competencies of others to demonstrate ‘collective competence’, 26 27 which is critical for healthcare delivery, 28 29 and existing research on locums suggests a need for better integration into teams to improve quality and safety. 30 31

Context matters for patient safety and quality improvement, 32 33 yet the limited evidence 17 relating to locums practice is largely ‘acontextual’ and tends to ignore the role of the organisation in the integration of temporary staff, focusing instead on the potential risks locums present as individual clinicians, 17 30 which is perhaps unsurprising given the liminal space locums occupy. In the UK, responsibility for the quality and safety of healthcare services is shared primarily between organisations and the individual professionals working within them. 34 Organisations are responsible for creating systems and environments that promote and protect clinical governance and enable all doctors to meet their professional obligations, while doctors are expected to participate in the systems and processes put in place by regulators and organisations to protect and improve patient care. 35 However, NHS trusts and primary care organisations procure the services of locum doctors without assuming the responsibilities normally associated with an employer–employee relationship 30 and locums often struggle to participate in teams and governance systems that were designed for doctors working in conventional employment relationships. 18 36

There is longstanding debate about the role of individual accountability in patient safety and how responsibility is distributed between organisations and individuals. 37 A systems approach reasons that adverse events are likely to occur as a result of system failures rather than individual failures, 38 and patients are protected from mistakes by well designed systems and environments that promote safety cultures. 39 But locums are often positioned at the periphery of these systems, 30 and doctors who are new to and also peripheral to organisations, and organisations who are inexperienced with and unsupportive of locums are unlikely to be able to perform optimally. 40

The aim of this research was to provide evidence on how locum working arrangements impact quality and safety and the implications of locum working for patients, locums and health service organisations in primary and secondary care in the English NHS. Locum doctors are an essential and growing part of the healthcare workforce 1 who have been largely ignored in healthcare workforce research. This research addresses a gap in the empirical evidence base on how locum doctor working arrangements affect quality and safety, and provides, for the first time, an in-depth exploration that includes perspectives from patients, locums and the people they work with.

Study design and setting

A qualitative semi-structured interview and focus group study was conducted with locums, people working with locums, and patients with experience of being treated by locums. Participants were purposively sampled through 11 organisations, including NHS trusts, primary care practices, statutory NHS bodies and locum agencies. Locum doctor participants were recruited through these organisations, locum recruitment agencies and networks. We used purposive, snowball and convenience sampling, drawing on intelligence from stakeholders, including our project advisory group, to identify and recruit organisations and participants. Patient participants were recruited through patient and contributor forums. The forum involved active partnership between patients and researchers in the research process to develop research which is relevant and useful to patient and public needs. Participant demographics were monitored to ensure representation across a broad range of roles in primary and secondary care and to increase diversity in terms of gender and ethnicity (see table 1 ).

• View inline

Characteristics of study participants

Data collection

Three semi-structured interview and focus group guides were developed for use with locums, people working with locums and patients with experience of being treated by locums (as shown in online supplemental files 1-3 ). Our previous review of the literature relating to quality and safety and locum work 17 informed the schedules as well as the initial coding and thematic development. Schedules were also refined and informed by our patient and public involvement (PPI) forum and our project advisory group. Each schedule was intended to explore locum doctor working arrangements with a particular focus on understanding how locum doctor working may affect the safety and quality of care and what strategies or systems organisations and individuals used to assure or improve quality and safety. The topic guides for locums and people working with locums also covered governance and support, the impact of the COVID pandemic and policies and initiatives used to support locums.

Supplemental material

Interviews and focus groups were transcribed verbatim by a professional transcription company and organised into codes and themes using the software package NVivo. 41 Reflexive thematic analysis (RTA) 42 was used and involved familiarisation with the data by reading and re-reading the transcripts and field notes; coding the dataset and collating all relevant data extracts; generating initial themes by examining the codes and collated data to identify significant broader patterns of meaning across the dataset; reviewing themes by questioning whether themes answered the research question and told a convincing story of the data and combining, splitting and discarding themes as necessary; defining and naming themes by developing a detailed analysis of each theme; and finally the analytical write up which positioned the analysis in relation to existing literature. 43 RTA acknowledges the active role of researchers in knowledge production and the researcher’s subjectivity as the analytic resource. 42 RTA recognises interpretive variability between researchers based on differences in their knowledge and skills, theoretical assumptions and differences in how they responded to the dataset is acknowledged and expected. 42 The research team worked reflexively discussing their personal biases and their potential impact on the research at regular meetings throughout the data collection and analysis period. Our PPI forum were also involved in data collection and analysis, and offered a form of triangulation to enhance rigour, challenge and alternative interpretations of the findings. 44 Analysis adopted a constructionist epistemology, in that while we acknowledged the importance of recurrence in generating themes, meaning and meaningfulness were the central criteria in the coding process. 42

After themes were developed, an abductive approach was taken to position findings against a background of existing theory and knowledge. 17 30 This provided a way of constructing empirically based theorisations without confining theory to predefined concepts. 45 This approach integrated inductive data-driven coding with deductive theory-driven interpretation; aiming to find a middle ground between inductive and deductive methods and the most logical solution and useful explanation for phenomena. 45

We conducted 130 interviews with 88 participants who worked in healthcare and 42 patients took part in focus groups and one-to-one interviews. Participants included locums, permanently employed doctors; nurses and other health professionals; medical directors/clinical leaders; responsible officers (ROs are accountable for local clinical governance processes and focus on the performance of doctors) and appraisers; leads for medical staffing and clinical governance and practice managers (see table 2 ). Three experienced qualitative researchers (JF, GS and KW) and two members of the PPI forum (MM and MS) carried out five focus groups with 30 patients, and JF and GS carried out 12 one-to-one interviews. Data were collected between March 2021 and April 2022 during the COVID pandemic using video conferencing software (n=126) or over the phone (n=4) at a time convenient to participants. Interviews and focus groups ranged in length from 23 to 171 min, with the average interview being 59 min.

Healthcare organisations and participant roles

Thematic framework

Our findings are presented under five broad and interrelated themes that examine how locum work relates to and impacts quality and safety: ‘familiarity’ with an organisation and its patients and staff; ‘balance and stability’ in services with lots of locums; ‘discrimination and exclusion’ towards locums and their effects; ‘defensive practice’ by locums; and the positioning of locums outside clinical governance arrangements.

Familiarity: knowing who, where and how

Locums described often working in unfamiliar environments, sometimes with minimal induction and varying levels of support. Unfamiliarity, lack of access to or other restrictions on computer systems, policies, procedures and buildings meant that locums were not always able to do their job safely, productively or effectively.

That’s probably the biggest sort of safety aspect that sticks in my mind, is that it is unbelievably frustrating to have to learn a whole new set of patients from day to day … when I was signed up to four different hospitals, plus the locum agencies, I very quickly realised that not only is it the fact that you don’t know the patients from day to day, if you’re chopping and changing site the whole time, then store cupboards are laid out differently, ways of contacting relevant staff members are different, you’ve got to recognise what code to put in to bleep someone that’s different at every single site. (Interview 23, locum, secondary care)

Locum working sometimes created extra work for permanent staff who were responsible for inducting, training and supervising locums. The amount of additional workload was dependent on contextual factors, such as the experience of the locum, organisational support and length of placement, access to systems and what terms and conditions locums or organisations had negotiated. Locum reliance on permanent staff meant that care could be delayed, partially completed or not completed at all, which sometimes caused resentment.

Some of the things that we don’t … like, for example, procedures of limited clinical value that we don’t refer in for, they won’t know about those in our areas … So they’ll do referrals that we then will get pulled on. They’ll maybe prescribe medications that are not first line medications within our own formulary. So we see quite a bit of that, you know, there’s quite a lot of tidying up to be done afterwards or work. They generate that. So whilst we meet the patient numbers, they create a lot of work for the rest of the team. (Interview 3, practice manager, primary care)

Locums mitigated risks related to working in unfamiliar environments by avoiding organisations considered chaotic or unsafe, working below their grade to avoid having responsibility in unfamiliar organisations where they may not be supported or included in the team or working in a limited number of organisations to increase familiarity.

Most locums take jobs, locum work below their grade. So a person who’s at a registrar level would take a locum work as an SHO (senior house officer), because they don't know the trust that well. (Interview 55, locum, secondary care)

However, lack of familiarity and discontinuity could at times be beneficial for patients and organisations as fresh perspectives offered by locums led to different routes of treatment or management, and could alter organisational cultures or practices.

So that [locum] doctor, through that line of questioning and not having any sort of prior history … ordered the right tests and didn’t feel constrained in that practice about what tests that they could order. And someone subsequently … because when you get referred to hospital, the consultant said that that doctor was very much on the ball. And, of course, that’s a change to lifelong medication. And literally within a month of the medication kicking in, it transformed my life. (Focus group A, patient 1)

Balance and stability

The balance between locum and permanent staff had implications for quality and safety, organisational leadership, long-term planning and governance. Locums were often employed to deliver immediate services and consequently were less likely to be involved in team and organisational development. Locums recognised that having ‘an NHS run by locums’ was detrimental to quality and safety, and some avoided organisations that were locum dependent for this reason. Well functioning established teams were regarded as better able to incorporate a small number of locums without being significantly impacted.

Locum work, my view on it is they’re there to fill a gap. They shouldn’t be relied upon to deliver a service Monday to Friday, day in, day out, week in, week out. And unfortunately my trust see it as that, though, that’s my worry that they feel they’re not just plugging a gap, they’re almost as a workforce … (Interview 84, lead GP, primary and secondary care)

Departments that were disproportionately locum dependent were often perceived to lack clinical leadership and direction. An absence of consistent medical leadership meant that quality improvement was slower or less likely to happen, and trusting relationships between staff were harder to establish.

If you get a department that is disproportionately locum dependent, then it stagnates, it doesn't progress. Things like implementation of new NICE guidance, for example, that sort of thing tends not to happen or happen less well, less quickly. (Interview 30, responsible officer, secondary care)

Discrimination and exclusion

Most locums described negative behaviours and attitudes from staff and some patients, which impacted their involvement, inclusion and experiences in organisations. Negative attitudes and behaviours towards locums could affect turnover, locum well-being, team dynamics and potentially patient safety. Perceived disparities between pay, workload, competence and organisational and team commitment between locums and permanent staff could be sources of resentment and influenced how locums were treated and viewed. This compromised staff communication and reduced the sharing of important patient information.

I guess like any temporary post really, you struggle to invest in them, don't necessarily see them as being part of the team. Not very positive about them, particularly junior staff, particularly in the acute trusts. We'd have locums refusing to come back because of the treatment of the midwives. (Interview 86, clinical lead, secondary care)

Negative perceptions of competency and safety meant that locums were often stigmatised, marginalised and excluded. The identity of locum intersected and overlapped with other identities and was described as ‘layering up’ with ethnicity and gender to further exacerbate discrimination.

Oh, doctors coming over from Germany. There was one locum … that administered a dose of something and the patient died, and then there’s this whole layer of extra negativity attached to locum doctors in general because of what one doctor did, and that doctor happened to be someone from a different ethnicity … As a UK born and qualified doctor I can see that those overseas get it but I can also see that I have experienced that as well. So yeah, it can layer up with the whole locum thing. (Interview 59, locum GP, primary care)

A sense of othering and being seen as less was particularly evident during the COVID pandemic when resources were limited. Some locums described how they were not afforded the same protections as permanent members of staff and were sometimes expected to take on riskier work.

I’ve worked in another practice where, because they live on locums and they live on ad hoc locums, you’re a piece of dirt under the shoe. You don’t get gloves, you didn’t have aprons, you didn’t have a face visor, you didn’t have safety specs, you have to ask for a mask. Not only are you not treated as a service provider, you’re not treated as a colleague, someone with knowledge. (Interview 44, locum GP)

Defensive practice by locums

Locums recognised that they were likely to be scapegoated if things went wrong, and some locums described being more likely to practice defensively. Defensive practice has been defined as deviation from standard practice to avoid litigation, complaints or criticism. 46 Participants reported instances of defensive practice which involved providing services (eg, tests, referrals) or avoiding high-risk decisions, usually to reduce the risk of adverse outcomes such as patient complaints or potential termination of contract at short notice. Locums described practicing defensively because they were attempting to practice as safely as possible in complex unfamiliar environments where they were professionally isolated and perceived negatively. Permanent members of staff could perceive that locums practiced defensively because they lacked confidence in their abilities. The diversion of resources away from more clinically relevant activities placed additional burden on teams, who were already facing significant workload challenges.

Being risk averse and practising defensive medicine usually means more tests, more referrals, whereas holding risk tends to be disadvantageous for you as a locum because what’s the benefit to you of not doing that. You’re benefiting the system by rationing resource, the patient won’t thank you. (Interview 35, locum GP)

Locums described avoiding making decisions when risks to employment or medical licenses were perceived as high. Locums felt they were more vulnerable to criticisms of their clinical competence and disempowered to make decisions. Others felt that some locums were simply avoiding work and evaded responsibility for patients by pushing work onto others or into the future.

You don’t interfere, very simple. Over time locums have learned that if you interfere, if you participate in the team, you participate in patient care, [and this] is when you get into trouble … Well most of the locums that I know will just say, okay, there’s already somebody else who’s made a decision, it’s not my job to make a decision, I just follow through. If things go wrong, call the senior person and be done with it, that’s the end of my role. Actually doing something to protect a patient is not important for a locum because the risk is too high. (Interview 55, locum, secondary care)

Locums fall outside clinical governance arrangements

Governance practices in relation to locums varied widely and were not generally regarded as being as robust in comparison to permanently employed doctors. Responsibility for involving locum doctors in performance feedback, supervision, educational opportunities, appraisal and quality improvement was unclear. While some organisations included locums in their governance activities, others regarded locum work as transactional; where the locum was there to provide a finite service and the organisation assumed no responsibilities for their performance, development or oversight. There were concerns that governance structures were modelled on and designed for permanently employed doctors and did not work for locums. When deficits in performance were undetected or unaddressed, doctor performance and patient safety could be jeopardised.

I think it’s a remote world. It’s like a cloud, you know, it’s like the cloud. We talk about the cloud when it comes to storing information. And I think locum world is a bit like that … And I don’t know the doctors anywhere like as much as I did when I was an RO in the NHS, I knew them all personally. If I used to have a problem, I used to get them in my office there and then, chat it all through, sort it. Can’t do that in locum world, it might take me four days to get hold of the doctor, some of them won’t respond immediately … They don’t know me and I don’t know them. (Interview 51, responsible officer, locum agency)

The absence of typical recruitment processes (involving meeting a doctor, carrying out an interview and following up on references) meant that healthcare organisations were reliant on partial information from locum agencies, which made it difficult to determine competency, scope of practice and suitability for a role. However, staff shortages and a requirement to meet safe staffing ratios meant that organisational leaders had little recourse of action if they were unsure about a doctor’s capability, which caused anxiety and frustration. This suggests that the provision of healthcare superseded ensuring safety standards and necessitated accepting one of two objectionable alternatives; accepting gaps in staffing that may jeopardise patient safety or accepting unknown doctors; each of which may compromise patient safety.

If a locum turns up and I have serious doubts about their ability to do the job to the required standard, I don’t have any recourse … And therefore I’m in a position where either I accept this locum or I don’t. There’s not much in the way of middle ground. Not accepting them is a really unpalatable choice because if I say look, I’m sorry, I don’t think you’re up to this, I think you should go home, that leaves me with a gap. (Interview 30, consultant and responsible officer, secondary care)

Similar governance and information sharing problems were described by locum agencies and NHS organisations; both described difficulties in gathering and sharing feedback. When concerns were raised, participants were often uncertain as to what happened to the information they provided and whether it was shared or acted on. Locums often did not get to hear about concerns raised about them, meaning learning opportunities were missed.

It would give you more confidence if you heard back. And sometimes I'll pick up the phone and you try to do the best you can to make sure this information gets passed on. But I just have this nagging doubt that I'm not always convinced it does. (Interview 30, responsible officer, secondary care)

There was also a perception from some locum agency responsible officers that while most locum doctors were excellent, there were some locums who were isolated and in need of organisational and professional support.

You have to accept that whilst within the agency world, 80 per cent of the doctors we place are excellent, and have no problems, and do a great job, perhaps 20 per cent are those that have shaken down to that 20 per cent in the agency world, because they’ve not succeeded in the NHS, they’ve not got a substantive place, they are lost souls. And they are less able to cope with the vicissitudes of busy clinical life and professional life within a large organisation such as the NHS. (Interview 47, responsible officer, locum agency)

Our findings provide some profound and concerning insights for patient safety and quality of care. The ways in which locums were recruited, inducted, deployed and integrated, and supported by organisations undoubtedly affected quality and safety. Our findings indicate that regardless of their level of experience, it was unlikely that locum doctors would be able to function optimally in unfamiliar environments; and organisations who had poor supportive infrastructure and governance mechanisms for locums were less likely to deliver high-quality safe services.

Locums were often regarded as organisational outsiders—positioned at the periphery of the team and the organisation. The implications of transience and peripheral participation were weaker relationships with organisations, teams, peers and patients, leading some to suggest locum working is better suited to experienced doctors. 47 Consistent with previous research, 48 frequent variation in process, systems and equipment, combined with disruption in relationships and a lack of mutual awareness of team skills and competencies, decreased collective competence, placed additional burden on the wider healthcare team and reduced patient safety. As others have found in research on safe staffing and nursing, 49 temporary staff are not effective substitutes for staff who regularly work in the organisation. Safe medical staffing is not just achieved by filling rota gaps, but also team composition and doctors’ familiarity with the team and organisation must be taken into account. Regulatory agencies should consider locum usage in their inspections and perhaps be particularly concerned when organisations have ‘services run on locums’.

Our research found, as others have, 18 that organisations and doctors sometimes struggled to meet their governance obligations and that governance activities differed based on contractual status and organisational policies and norms, with systems being less robust for locums. This research has highlighted that much still needs to be done to develop governance systems that promote and protect the interests of patients and create an environment which supports locum doctors in meeting their professional obligations.

More positively, locum doctors are a potentially valuable source of information about safety concerns, faulty systems or poor conduct. 50 Locums move between organisations, have broad systems knowledge and are perhaps better placed to identify some quality and safety issues than permanent doctors. However, findings indicate opportunities for shared learning were often missed. Locums recognised their precarity and vulnerability when offering second opinions, sharing improvement ideas or voicing safety concerns; meaning opinions were not always offered and concerns were not always raised. Failure to voice concerns is a persistent problem in healthcare, 51 and locums may be even less inclined to offer potentially valuable information about safety concerns because of their perceptions of unsupportive organisational climates.

Our findings shed light on how temporary doctors fit into the enduring debate 37 around how responsibility between organisational systems and individual professionals is distributed. Locums appear to represent a subsection of the medical profession for whom the wider paradigm shift from a focus on individual blame to a systems approach 52 appeared not to have been made. Locums were often not regarded as a part of the organisation, and therefore the system, and not afforded the same protections as permanent staff when things went wrong. Blaming locums when things go wrong and punishing or sanctioning individuals who make errors in contexts that were not designed to incorporate temporary workers may divert attention from understanding inadequately designed, poorly functioning systems, or indeed the individual practice of other doctors. While we should take into account systemic factors that impede locums from performing safely, we should expect high standards of healthcare professionals, be cognisant of individual agency and recognise the distinction between blaming someone and holding them responsible. 53

Strengths and limitations

This large qualitative study explores locum working and quality and safety in an under-researched, yet growing area of the medical workforce. However, sites were all based in England, which means caution should be taken when extrapolating findings. Similar research in other countries and contexts to understand more about locum doctor working and quality and safety is therefore important. It is possible that our sample may have been skewed towards locums, healthcare professionals and patients who had more negative perceptions and experiences, although accounts resonate with previous research 30 and patient perspectives were generally positive. Our data were collected during the COVID pandemic, which may have affected findings as there was a reduction in locum working during that time 10 11 ; it also meant we were unable to carry out observations, which would have strengthened our findings and mitigated some of the inherent limitations of interviews, such as recall bias. We used both one-to-one interviews and focus groups in data collection. Although flexibility in data collection meant that participants had the option to take part in an interview or a focus group, these methods are used for different reasons and produce different data. There may have been differences in what participants disclosed depending on the method

Our findings show that the way in which doctors who worked on a temporary basis were integrated into organisations posed some significant challenges and opportunities for patient safety and quality of care, and that both organisations and locums had a part to play in improvement. Doctors working as locums are a heterogeneous group with differing backgrounds, experiences, skills and capabilities that likely reflect the variability seen in the wider population of doctors. Locums are working in the same pressured and imperfect systems as other health workers; it is vital that systemic problems are not mistaken for problems about individuals and important to recognise that a locum is not a type of doctor but a way of working. Our findings are a call to action for organisations to take stock of how they engage, support and work with locums, and asks both locums and organisations to reflect on whether their practices support a collective approach to patient safety and quality of care.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This study involves human participants and was approved by the Health Research Authority North West—Haydock Research Ethics Committee 20/NW/0386. Participants gave informed consent to participate in the study before taking part.

• General Medical Council
• NHS England
• World Health Organization
• Health Social Care Committee
• Harvey PR ,
• Trudgill NJ
• Bower P , et al
• Grigoroglou C ,
• Kontopantelis E , et al
• Shembavnekar N ,
• Bazeer N , et al
• Glatter R ,
• Papadakos P ,
• Ferguson J ,
• Schmutz JB ,
• DiazGranados D ,
• Dietz AS , et al
• Sutradhar R ,
• Jerath A , et al
• Oeppen RS , et al
• Pollack AH ,
• Mishra SR ,
• Apodaca C , et al
• McLaney E ,
• Morassaei S ,
• Hughes L , et al
• Firth-Cozens J
• Tazzyman A ,
• Walshe K , et al
• Pouwels KB ,
• van Hecke O , et al
• Aveling E-L ,
• Dixon-Woods M
• Blumenthal DM ,
• Olenski AR ,
• Tsugawa Y , et al
• QSR International Pty Ltd
• Stewart D ,
• Tierney T , et al
• Timmermans S
• Baungaard N ,
• Skovvang PL ,
• Assing Hvidt E , et al
• Lewis L , et al
• Liberati EG ,
• Tarrant C ,
• Willars J , et al
• Zaranko B ,
• Sanford NJ ,
• Kelly E , et al
• Martin GP ,
• Dixon-Woods M ,
• Aveling EL ,
• Campbell A , et al
• Karsh B-T ,
• Holden RJ ,
• Alper SJ , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2
• Data supplement 3
• Data supplement 4

Twitter @janefergo, @@kieran_walshe

Contributors JF, KW, DA, TA and EK conceived the study. Recruitment was led by JF and supported by GS. JF, GS and KW conducted the interviews, reviewed and analysed the transcripts, and JF wrote the first version of the manuscript. Two members of the patient and public involvement (PPI) forum also assisted with focus groups. JF conducted data analysis with input from KW and GS, the PPI forum, and review by all authors. JF and KW were involved in initial critical review and revision of the manuscript, followed by all authors. All authors read and approved the final version of the manuscript. JF is the guarantor.

Funding This study was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR128349), and the NIHR Greater Manchester Patient Safety Research Collaboration (PSRC). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles

• Editorial Locums: threat or opportunity Richard Lilford BMJ Quality & Safety 2024; - Published Online First: 16 Apr 2024. doi: 10.1136/bmjqs-2023-016951

Read the full text or download the PDF:

SYSTEMATIC REVIEW article

This article is part of the research topic.

Ethnic Inequalities in Diabetes Care and Outcomes

Acceptability of Community Health Worker and Peer Supported Interventions for Minoritized Populations with Type 2 Diabetes: A Qualitative Systematic Review Provisionally Accepted

• 1 University of Birmingham, United Kingdom

The final, formatted version of the article will be published soon.

Ethnic minority groups in high income countries in North America, Europe, and elsewhere are disproportionately affected by T2DM with a higher risk of mortality and morbidity. The use of community health workers and peer supporters offer a way of ensuring the benefits of self-management support observed in the general population are shared by those in minoritized communities.The major databases were searched for existing qualitative evidence of participants' experiences and perspectives of self-management support for type 2 diabetes delivered by community health workers and peer supporters (CHWPs) in ethnically minoritized populations. The data were analysed using Sekhon's Theoretical Framework of Acceptability.The results are described within five domains of the framework of acceptability collapsed from seven for reasons of clarity and concision: Affective attitude described participants' satisfaction with CHWPs delivering the intervention including the open, trusting relationships that developed in contrast to those with clinical providers. In considering Burden and Opportunity Costs, participants reflected on the impact of health, transport, and the responsibilities of work and childcare on their attendance, alongside a lack of resources necessary to maintain healthy diets and active lifestyles. In relation to Cultural Sensitivity participants appreciated the greater understanding of the specific cultural needs and challenges exhibited by CHWPs. The evidence related to Intervention Coherence indicated that participants responded positively to the practical and applied content, the range of teaching materials, and interactive practical sessions. Finally, in examining the impact of Effectiveness and Self-efficacy participants described how they changed a range of healthrelated behaviours, had more confidence in dealing with their condition and interacting with senior clinicians and benefitted from the social support of fellow participants and CHWPs.Many of the same barriers around attendance and engagement adherence towith usual self-management support interventions delivered to general populations were observed, including lack of time and resource. However, the insight of CHWPs, their culturallysensitive and specific strategies for self-management and their development of trusting relationships presented considerable advantages.

Keywords: type 2 diabetes, self-management, Community Health, health inequalities, Ethnic minorities Library, Places of Worship, Sports centres, Community Halls, a Mixed methods studies

Received: 03 Oct 2023; Accepted: 26 Feb 2024.

Copyright: © 2024 Grant and Litchfield. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mx. Ian Litchfield, University of Birmingham, Birmingham, United Kingdom

People also looked at

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Grad Med Educ
• v.7(4); 2015 Dec

Choosing a Qualitative Research Approach

Associated data.

Editor's Note: The online version of this article contains a list of further reading resources and the authors' professional information .

The Challenge

Educators often pose questions about qualitative research. For example, a program director might say: “I collect data from my residents about their learning experiences in a new longitudinal clinical rotation. If I want to know about their learning experiences, should I use qualitative methods? I have been told that there are many approaches from which to choose. Someone suggested that I use grounded theory, but how do I know this is the best approach? Are there others?”

What Is Known

Qualitative research is the systematic inquiry into social phenomena in natural settings. These phenomena can include, but are not limited to, how people experience aspects of their lives, how individuals and/or groups behave, how organizations function, and how interactions shape relationships. In qualitative research, the researcher is the main data collection instrument. The researcher examines why events occur, what happens, and what those events mean to the participants studied. 1 , 2

Qualitative research starts from a fundamentally different set of beliefs—or paradigms—than those that underpin quantitative research. Quantitative research is based on positivist beliefs that there is a singular reality that can be discovered with the appropriate experimental methods. Post-positivist researchers agree with the positivist paradigm, but believe that environmental and individual differences, such as the learning culture or the learners' capacity to learn, influence this reality, and that these differences are important. Constructivist researchers believe that there is no single reality, but that the researcher elicits participants' views of reality. 3 Qualitative research generally draws on post-positivist or constructivist beliefs.

Qualitative scholars develop their work from these beliefs—usually post-positivist or constructivist—using different approaches to conduct their research. In this Rip Out, we describe 3 different qualitative research approaches commonly used in medical education: grounded theory, ethnography, and phenomenology. Each acts as a pivotal frame that shapes the research question(s), the method(s) of data collection, and how data are analyzed. 4 , 5

Choosing a Qualitative Approach

Before engaging in any qualitative study, consider how your views about what is possible to study will affect your approach. Then select an appropriate approach within which to work. Alignment between the belief system underpinning the research approach, the research question, and the research approach itself is a prerequisite for rigorous qualitative research. To enhance the understanding of how different approaches frame qualitative research, we use this introductory challenge as an illustrative example.

The clinic rotation in a program director's training program was recently redesigned as a longitudinal clinical experience. Resident satisfaction with this rotation improved significantly following implementation of the new longitudinal experience. The program director wants to understand how the changes made in the clinic rotation translated into changes in learning experiences for the residents.

Qualitative research can support this program director's efforts. Qualitative research focuses on the events that transpire and on outcomes of those events from the perspectives of those involved. In this case, the program director can use qualitative research to understand the impact of the new clinic rotation on the learning experiences of residents. The next step is to decide which approach to use as a frame for the study.

The table lists the purpose of 3 commonly used approaches to frame qualitative research. For each frame, we provide an example of a research question that could direct the study and delineate what outcomes might be gained by using that particular approach.

Methodology Overview

How You Can Start TODAY

• 1 Examine the foundations of the existing literature: As part of the literature review, make note of what is known about the topic and which approaches have been used in prior studies. A decision should be made to determine the extent to which the new study is exploratory and the extent to which findings will advance what is already known about the topic.
• 2 Find a qualitatively skilled collaborator: If you are interested in doing qualitative research, you should consult with a qualitative expert. Be prepared to talk to the qualitative scholar about what you would like to study and why . Furthermore, be ready to describe the literature to date on the topic (remember, you are asking for this person's expertise regarding qualitative approaches—he or she won't necessarily have content expertise). Qualitative research must be designed and conducted with rigor (rigor will be discussed in Rip Out No. 8 of this series). Input from a qualitative expert will ensure that rigor is employed from the study's inception.
• 3 Consider the approach: With a literature review completed and a qualitatively skilled collaborator secured, it is time to decide which approach would be best suited to answering the research question. Questions to consider when weighing approaches might include the following:
• • Will my findings contribute to the creation of a theoretical model to better understand the area of study? ( grounded theory )
• • Will I need to spend an extended amount of time trying to understand the culture and process of a particular group of learners in their natural context? ( ethnography )
• • Is there a particular phenomenon I want to better understand/describe? ( phenomenology )

What You Can Do LONG TERM

• 1 Develop your qualitative research knowledge and skills : A basic qualitative research textbook is a valuable investment to learn about qualitative research (further reading is provided as online supplemental material). A novice qualitative researcher will also benefit from participating in a massive online open course or a mini-course (often offered by professional organizations or conferences) that provides an introduction to qualitative research. Most of all, collaborating with a qualitative researcher can provide the support necessary to design, execute, and report on the study.
• 2 Undertake a pilot study: After learning about qualitative methodology, the next best way to gain expertise in qualitative research is to try it in a small scale pilot study with the support of a qualitative expert. Such application provides an appreciation for the thought processes that go into designing a study, analyzing the data, and reporting on the findings. Alternatively, if you have the opportunity to work on a study led by a qualitative expert, take it! The experience will provide invaluable opportunities for learning how to engage in qualitative research.

Supplementary Material

The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Uniformed Services University of the Health Sciences, the Department of the Navy, the Department of Defense, or the US government.

References and Resources for Further Reading

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 April 2024

The economic commitment of climate change

• Maximilian Kotz   ORCID: orcid.org/0000-0003-2564-5043 1 , 2 ,
• Anders Levermann   ORCID: orcid.org/0000-0003-4432-4704 1 , 2 &
• Leonie Wenz   ORCID: orcid.org/0000-0002-8500-1568 1 , 3  

Nature volume  628 ,  pages 551–557 ( 2024 ) Cite this article

39k Accesses

2856 Altmetric

Metrics details

• Environmental economics
• Environmental health
• Interdisciplinary studies
• Projection and prediction

Global projections of macroeconomic climate-change damages typically consider impacts from average annual and national temperatures over long time horizons 1 , 2 , 3 , 4 , 5 , 6 . Here we use recent empirical findings from more than 1,600 regions worldwide over the past 40 years to project sub-national damages from temperature and precipitation, including daily variability and extremes 7 , 8 . Using an empirical approach that provides a robust lower bound on the persistence of impacts on economic growth, we find that the world economy is committed to an income reduction of 19% within the next 26 years independent of future emission choices (relative to a baseline without climate impacts, likely range of 11–29% accounting for physical climate and empirical uncertainty). These damages already outweigh the mitigation costs required to limit global warming to 2 °C by sixfold over this near-term time frame and thereafter diverge strongly dependent on emission choices. Committed damages arise predominantly through changes in average temperature, but accounting for further climatic components raises estimates by approximately 50% and leads to stronger regional heterogeneity. Committed losses are projected for all regions except those at very high latitudes, at which reductions in temperature variability bring benefits. The largest losses are committed at lower latitudes in regions with lower cumulative historical emissions and lower present-day income.

Similar content being viewed by others

Climate damage projections beyond annual temperature

Paul Waidelich, Fulden Batibeniz, … Sonia I. Seneviratne

Investment incentive reduced by climate damages can be restored by optimal policy

Sven N. Willner, Nicole Glanemann & Anders Levermann

Climate economics support for the UN climate targets

Martin C. Hänsel, Moritz A. Drupp, … Thomas Sterner

Projections of the macroeconomic damage caused by future climate change are crucial to informing public and policy debates about adaptation, mitigation and climate justice. On the one hand, adaptation against climate impacts must be justified and planned on the basis of an understanding of their future magnitude and spatial distribution 9 . This is also of importance in the context of climate justice 10 , as well as to key societal actors, including governments, central banks and private businesses, which increasingly require the inclusion of climate risks in their macroeconomic forecasts to aid adaptive decision-making 11 , 12 . On the other hand, climate mitigation policy such as the Paris Climate Agreement is often evaluated by balancing the costs of its implementation against the benefits of avoiding projected physical damages. This evaluation occurs both formally through cost–benefit analyses 1 , 4 , 5 , 6 , as well as informally through public perception of mitigation and damage costs 13 .

Projections of future damages meet challenges when informing these debates, in particular the human biases relating to uncertainty and remoteness that are raised by long-term perspectives 14 . Here we aim to overcome such challenges by assessing the extent of economic damages from climate change to which the world is already committed by historical emissions and socio-economic inertia (the range of future emission scenarios that are considered socio-economically plausible 15 ). Such a focus on the near term limits the large uncertainties about diverging future emission trajectories, the resulting long-term climate response and the validity of applying historically observed climate–economic relations over long timescales during which socio-technical conditions may change considerably. As such, this focus aims to simplify the communication and maximize the credibility of projected economic damages from future climate change.

In projecting the future economic damages from climate change, we make use of recent advances in climate econometrics that provide evidence for impacts on sub-national economic growth from numerous components of the distribution of daily temperature and precipitation 3 , 7 , 8 . Using fixed-effects panel regression models to control for potential confounders, these studies exploit within-region variation in local temperature and precipitation in a panel of more than 1,600 regions worldwide, comprising climate and income data over the past 40 years, to identify the plausibly causal effects of changes in several climate variables on economic productivity 16 , 17 . Specifically, macroeconomic impacts have been identified from changing daily temperature variability, total annual precipitation, the annual number of wet days and extreme daily rainfall that occur in addition to those already identified from changing average temperature 2 , 3 , 18 . Moreover, regional heterogeneity in these effects based on the prevailing local climatic conditions has been found using interactions terms. The selection of these climate variables follows micro-level evidence for mechanisms related to the impacts of average temperatures on labour and agricultural productivity 2 , of temperature variability on agricultural productivity and health 7 , as well as of precipitation on agricultural productivity, labour outcomes and flood damages 8 (see Extended Data Table 1 for an overview, including more detailed references). References  7 , 8 contain a more detailed motivation for the use of these particular climate variables and provide extensive empirical tests about the robustness and nature of their effects on economic output, which are summarized in Methods . By accounting for these extra climatic variables at the sub-national level, we aim for a more comprehensive description of climate impacts with greater detail across both time and space.

Constraining the persistence of impacts

A key determinant and source of discrepancy in estimates of the magnitude of future climate damages is the extent to which the impact of a climate variable on economic growth rates persists. The two extreme cases in which these impacts persist indefinitely or only instantaneously are commonly referred to as growth or level effects 19 , 20 (see Methods section ‘Empirical model specification: fixed-effects distributed lag models’ for mathematical definitions). Recent work shows that future damages from climate change depend strongly on whether growth or level effects are assumed 20 . Following refs.  2 , 18 , we provide constraints on this persistence by using distributed lag models to test the significance of delayed effects separately for each climate variable. Notably, and in contrast to refs.  2 , 18 , we use climate variables in their first-differenced form following ref.  3 , implying a dependence of the growth rate on a change in climate variables. This choice means that a baseline specification without any lags constitutes a model prior of purely level effects, in which a permanent change in the climate has only an instantaneous effect on the growth rate 3 , 19 , 21 . By including lags, one can then test whether any effects may persist further. This is in contrast to the specification used by refs.  2 , 18 , in which climate variables are used without taking the first difference, implying a dependence of the growth rate on the level of climate variables. In this alternative case, the baseline specification without any lags constitutes a model prior of pure growth effects, in which a change in climate has an infinitely persistent effect on the growth rate. Consequently, including further lags in this alternative case tests whether the initial growth impact is recovered 18 , 19 , 21 . Both of these specifications suffer from the limiting possibility that, if too few lags are included, one might falsely accept the model prior. The limitations of including a very large number of lags, including loss of data and increasing statistical uncertainty with an increasing number of parameters, mean that such a possibility is likely. By choosing a specification in which the model prior is one of level effects, our approach is therefore conservative by design, avoiding assumptions of infinite persistence of climate impacts on growth and instead providing a lower bound on this persistence based on what is observable empirically (see Methods section ‘Empirical model specification: fixed-effects distributed lag models’ for further exposition of this framework). The conservative nature of such a choice is probably the reason that ref.  19 finds much greater consistency between the impacts projected by models that use the first difference of climate variables, as opposed to their levels.

We begin our empirical analysis of the persistence of climate impacts on growth using ten lags of the first-differenced climate variables in fixed-effects distributed lag models. We detect substantial effects on economic growth at time lags of up to approximately 8–10 years for the temperature terms and up to approximately 4 years for the precipitation terms (Extended Data Fig. 1 and Extended Data Table 2 ). Furthermore, evaluation by means of information criteria indicates that the inclusion of all five climate variables and the use of these numbers of lags provide a preferable trade-off between best-fitting the data and including further terms that could cause overfitting, in comparison with model specifications excluding climate variables or including more or fewer lags (Extended Data Fig. 3 , Supplementary Methods Section  1 and Supplementary Table 1 ). We therefore remove statistically insignificant terms at later lags (Supplementary Figs. 1 – 3 and Supplementary Tables 2 – 4 ). Further tests using Monte Carlo simulations demonstrate that the empirical models are robust to autocorrelation in the lagged climate variables (Supplementary Methods Section  2 and Supplementary Figs. 4 and 5 ), that information criteria provide an effective indicator for lag selection (Supplementary Methods Section  2 and Supplementary Fig. 6 ), that the results are robust to concerns of imperfect multicollinearity between climate variables and that including several climate variables is actually necessary to isolate their separate effects (Supplementary Methods Section  3 and Supplementary Fig. 7 ). We provide a further robustness check using a restricted distributed lag model to limit oscillations in the lagged parameter estimates that may result from autocorrelation, finding that it provides similar estimates of cumulative marginal effects to the unrestricted model (Supplementary Methods Section 4 and Supplementary Figs. 8 and 9 ). Finally, to explicitly account for any outstanding uncertainty arising from the precise choice of the number of lags, we include empirical models with marginally different numbers of lags in the error-sampling procedure of our projection of future damages. On the basis of the lag-selection procedure (the significance of lagged terms in Extended Data Fig. 1 and Extended Data Table 2 , as well as information criteria in Extended Data Fig. 3 ), we sample from models with eight to ten lags for temperature and four for precipitation (models shown in Supplementary Figs. 1 – 3 and Supplementary Tables 2 – 4 ). In summary, this empirical approach to constrain the persistence of climate impacts on economic growth rates is conservative by design in avoiding assumptions of infinite persistence, but nevertheless provides a lower bound on the extent of impact persistence that is robust to the numerous tests outlined above.

Committed damages until mid-century

We combine these empirical economic response functions (Supplementary Figs. 1 – 3 and Supplementary Tables 2 – 4 ) with an ensemble of 21 climate models (see Supplementary Table 5 ) from the Coupled Model Intercomparison Project Phase 6 (CMIP-6) 22 to project the macroeconomic damages from these components of physical climate change (see Methods for further details). Bias-adjusted climate models that provide a highly accurate reproduction of observed climatological patterns with limited uncertainty (Supplementary Table 6 ) are used to avoid introducing biases in the projections. Following a well-developed literature 2 , 3 , 19 , these projections do not aim to provide a prediction of future economic growth. Instead, they are a projection of the exogenous impact of future climate conditions on the economy relative to the baselines specified by socio-economic projections, based on the plausibly causal relationships inferred by the empirical models and assuming ceteris paribus. Other exogenous factors relevant for the prediction of economic output are purposefully assumed constant.

A Monte Carlo procedure that samples from climate model projections, empirical models with different numbers of lags and model parameter estimates (obtained by 1,000 block-bootstrap resamples of each of the regressions in Supplementary Figs. 1 – 3 and Supplementary Tables 2 – 4 ) is used to estimate the combined uncertainty from these sources. Given these uncertainty distributions, we find that projected global damages are statistically indistinguishable across the two most extreme emission scenarios until 2049 (at the 5% significance level; Fig. 1 ). As such, the climate damages occurring before this time constitute those to which the world is already committed owing to the combination of past emissions and the range of future emission scenarios that are considered socio-economically plausible 15 . These committed damages comprise a permanent income reduction of 19% on average globally (population-weighted average) in comparison with a baseline without climate-change impacts (with a likely range of 11–29%, following the likelihood classification adopted by the Intergovernmental Panel on Climate Change (IPCC); see caption of Fig. 1 ). Even though levels of income per capita generally still increase relative to those of today, this constitutes a permanent income reduction for most regions, including North America and Europe (each with median income reductions of approximately 11%) and with South Asia and Africa being the most strongly affected (each with median income reductions of approximately 22%; Fig. 1 ). Under a middle-of-the road scenario of future income development (SSP2, in which SSP stands for Shared Socio-economic Pathway), this corresponds to global annual damages in 2049 of 38 trillion in 2005 international dollars (likely range of 19–59 trillion 2005 international dollars). Compared with empirical specifications that assume pure growth or pure level effects, our preferred specification that provides a robust lower bound on the extent of climate impact persistence produces damages between these two extreme assumptions (Extended Data Fig. 3 ).

Estimates of the projected reduction in income per capita from changes in all climate variables based on empirical models of climate impacts on economic output with a robust lower bound on their persistence (Extended Data Fig. 1 ) under a low-emission scenario compatible with the 2 °C warming target and a high-emission scenario (SSP2-RCP2.6 and SSP5-RCP8.5, respectively) are shown in purple and orange, respectively. Shading represents the 34% and 10% confidence intervals reflecting the likely and very likely ranges, respectively (following the likelihood classification adopted by the IPCC), having estimated uncertainty from a Monte Carlo procedure, which samples the uncertainty from the choice of physical climate models, empirical models with different numbers of lags and bootstrapped estimates of the regression parameters shown in Supplementary Figs. 1 – 3 . Vertical dashed lines show the time at which the climate damages of the two emission scenarios diverge at the 5% and 1% significance levels based on the distribution of differences between emission scenarios arising from the uncertainty sampling discussed above. Note that uncertainty in the difference of the two scenarios is smaller than the combined uncertainty of the two respective scenarios because samples of the uncertainty (climate model and empirical model choice, as well as model parameter bootstrap) are consistent across the two emission scenarios, hence the divergence of damages occurs while the uncertainty bounds of the two separate damage scenarios still overlap. Estimates of global mitigation costs from the three IAMs that provide results for the SSP2 baseline and SSP2-RCP2.6 scenario are shown in light green in the top panel, with the median of these estimates shown in bold.

Damages already outweigh mitigation costs

We compare the damages to which the world is committed over the next 25 years to estimates of the mitigation costs required to achieve the Paris Climate Agreement. Taking estimates of mitigation costs from the three integrated assessment models (IAMs) in the IPCC AR6 database 23 that provide results under comparable scenarios (SSP2 baseline and SSP2-RCP2.6, in which RCP stands for Representative Concentration Pathway), we find that the median committed climate damages are larger than the median mitigation costs in 2050 (six trillion in 2005 international dollars) by a factor of approximately six (note that estimates of mitigation costs are only provided every 10 years by the IAMs and so a comparison in 2049 is not possible). This comparison simply aims to compare the magnitude of future damages against mitigation costs, rather than to conduct a formal cost–benefit analysis of transitioning from one emission path to another. Formal cost–benefit analyses typically find that the net benefits of mitigation only emerge after 2050 (ref.  5 ), which may lead some to conclude that physical damages from climate change are simply not large enough to outweigh mitigation costs until the second half of the century. Our simple comparison of their magnitudes makes clear that damages are actually already considerably larger than mitigation costs and the delayed emergence of net mitigation benefits results primarily from the fact that damages across different emission paths are indistinguishable until mid-century (Fig. 1 ).

Although these near-term damages constitute those to which the world is already committed, we note that damage estimates diverge strongly across emission scenarios after 2049, conveying the clear benefits of mitigation from a purely economic point of view that have been emphasized in previous studies 4 , 24 . As well as the uncertainties assessed in Fig. 1 , these conclusions are robust to structural choices, such as the timescale with which changes in the moderating variables of the empirical models are estimated (Supplementary Figs. 10 and 11 ), as well as the order in which one accounts for the intertemporal and international components of currency comparison (Supplementary Fig. 12 ; see Methods for further details).

Damages from variability and extremes

Committed damages primarily arise through changes in average temperature (Fig. 2 ). This reflects the fact that projected changes in average temperature are larger than those in other climate variables when expressed as a function of their historical interannual variability (Extended Data Fig. 4 ). Because the historical variability is that on which the empirical models are estimated, larger projected changes in comparison with this variability probably lead to larger future impacts in a purely statistical sense. From a mechanistic perspective, one may plausibly interpret this result as implying that future changes in average temperature are the most unprecedented from the perspective of the historical fluctuations to which the economy is accustomed and therefore will cause the most damage. This insight may prove useful in terms of guiding adaptation measures to the sources of greatest damage.

Estimates of the median projected reduction in sub-national income per capita across emission scenarios (SSP2-RCP2.6 and SSP2-RCP8.5) as well as climate model, empirical model and model parameter uncertainty in the year in which climate damages diverge at the 5% level (2049, as identified in Fig. 1 ). a , Impacts arising from all climate variables. b – f , Impacts arising separately from changes in annual mean temperature ( b ), daily temperature variability ( c ), total annual precipitation ( d ), the annual number of wet days (>1 mm) ( e ) and extreme daily rainfall ( f ) (see Methods for further definitions). Data on national administrative boundaries are obtained from the GADM database version 3.6 and are freely available for academic use ( https://gadm.org/ ).

Nevertheless, future damages based on empirical models that consider changes in annual average temperature only and exclude the other climate variables constitute income reductions of only 13% in 2049 (Extended Data Fig. 5a , likely range 5–21%). This suggests that accounting for the other components of the distribution of temperature and precipitation raises net damages by nearly 50%. This increase arises through the further damages that these climatic components cause, but also because their inclusion reveals a stronger negative economic response to average temperatures (Extended Data Fig. 5b ). The latter finding is consistent with our Monte Carlo simulations, which suggest that the magnitude of the effect of average temperature on economic growth is underestimated unless accounting for the impacts of other correlated climate variables (Supplementary Fig. 7 ).

In terms of the relative contributions of the different climatic components to overall damages, we find that accounting for daily temperature variability causes the largest increase in overall damages relative to empirical frameworks that only consider changes in annual average temperature (4.9 percentage points, likely range 2.4–8.7 percentage points, equivalent to approximately 10 trillion international dollars). Accounting for precipitation causes smaller increases in overall damages, which are—nevertheless—equivalent to approximately 1.2 trillion international dollars: 0.01 percentage points (−0.37–0.33 percentage points), 0.34 percentage points (0.07–0.90 percentage points) and 0.36 percentage points (0.13–0.65 percentage points) from total annual precipitation, the number of wet days and extreme daily precipitation, respectively. Moreover, climate models seem to underestimate future changes in temperature variability 25 and extreme precipitation 26 , 27 in response to anthropogenic forcing as compared with that observed historically, suggesting that the true impacts from these variables may be larger.

The distribution of committed damages

The spatial distribution of committed damages (Fig. 2a ) reflects a complex interplay between the patterns of future change in several climatic components and those of historical economic vulnerability to changes in those variables. Damages resulting from increasing annual mean temperature (Fig. 2b ) are negative almost everywhere globally, and larger at lower latitudes in regions in which temperatures are already higher and economic vulnerability to temperature increases is greatest (see the response heterogeneity to mean temperature embodied in Extended Data Fig. 1a ). This occurs despite the amplified warming projected at higher latitudes 28 , suggesting that regional heterogeneity in economic vulnerability to temperature changes outweighs heterogeneity in the magnitude of future warming (Supplementary Fig. 13a ). Economic damages owing to daily temperature variability (Fig. 2c ) exhibit a strong latitudinal polarisation, primarily reflecting the physical response of daily variability to greenhouse forcing in which increases in variability across lower latitudes (and Europe) contrast decreases at high latitudes 25 (Supplementary Fig. 13b ). These two temperature terms are the dominant determinants of the pattern of overall damages (Fig. 2a ), which exhibits a strong polarity with damages across most of the globe except at the highest northern latitudes. Future changes in total annual precipitation mainly bring economic benefits except in regions of drying, such as the Mediterranean and central South America (Fig. 2d and Supplementary Fig. 13c ), but these benefits are opposed by changes in the number of wet days, which produce damages with a similar pattern of opposite sign (Fig. 2e and Supplementary Fig. 13d ). By contrast, changes in extreme daily rainfall produce damages in all regions, reflecting the intensification of daily rainfall extremes over global land areas 29 , 30 (Fig. 2f and Supplementary Fig. 13e ).

The spatial distribution of committed damages implies considerable injustice along two dimensions: culpability for the historical emissions that have caused climate change and pre-existing levels of socio-economic welfare. Spearman’s rank correlations indicate that committed damages are significantly larger in countries with smaller historical cumulative emissions, as well as in regions with lower current income per capita (Fig. 3 ). This implies that those countries that will suffer the most from the damages already committed are those that are least responsible for climate change and which also have the least resources to adapt to it.

Estimates of the median projected change in national income per capita across emission scenarios (RCP2.6 and RCP8.5) as well as climate model, empirical model and model parameter uncertainty in the year in which climate damages diverge at the 5% level (2049, as identified in Fig. 1 ) are plotted against cumulative national emissions per capita in 2020 (from the Global Carbon Project) and coloured by national income per capita in 2020 (from the World Bank) in a and vice versa in b . In each panel, the size of each scatter point is weighted by the national population in 2020 (from the World Bank). Inset numbers indicate the Spearman’s rank correlation ρ and P -values for a hypothesis test whose null hypothesis is of no correlation, as well as the Spearman’s rank correlation weighted by national population.

To further quantify this heterogeneity, we assess the difference in committed damages between the upper and lower quartiles of regions when ranked by present income levels and historical cumulative emissions (using a population weighting to both define the quartiles and estimate the group averages). On average, the quartile of countries with lower income are committed to an income loss that is 8.9 percentage points (or 61%) greater than the upper quartile (Extended Data Fig. 6 ), with a likely range of 3.8–14.7 percentage points across the uncertainty sampling of our damage projections (following the likelihood classification adopted by the IPCC). Similarly, the quartile of countries with lower historical cumulative emissions are committed to an income loss that is 6.9 percentage points (or 40%) greater than the upper quartile, with a likely range of 0.27–12 percentage points. These patterns reemphasize the prevalence of injustice in climate impacts 31 , 32 , 33 in the context of the damages to which the world is already committed by historical emissions and socio-economic inertia.

Contextualizing the magnitude of damages

The magnitude of projected economic damages exceeds previous literature estimates 2 , 3 , arising from several developments made on previous approaches. Our estimates are larger than those of ref.  2 (see first row of Extended Data Table 3 ), primarily because of the facts that sub-national estimates typically show a steeper temperature response (see also refs.  3 , 34 ) and that accounting for other climatic components raises damage estimates (Extended Data Fig. 5 ). However, we note that our empirical approach using first-differenced climate variables is conservative compared with that of ref.  2 in regard to the persistence of climate impacts on growth (see introduction and Methods section ‘Empirical model specification: fixed-effects distributed lag models’), an important determinant of the magnitude of long-term damages 19 , 21 . Using a similar empirical specification to ref.  2 , which assumes infinite persistence while maintaining the rest of our approach (sub-national data and further climate variables), produces considerably larger damages (purple curve of Extended Data Fig. 3 ). Compared with studies that do take the first difference of climate variables 3 , 35 , our estimates are also larger (see second and third rows of Extended Data Table 3 ). The inclusion of further climate variables (Extended Data Fig. 5 ) and a sufficient number of lags to more adequately capture the extent of impact persistence (Extended Data Figs. 1 and 2 ) are the main sources of this difference, as is the use of specifications that capture nonlinearities in the temperature response when compared with ref.  35 . In summary, our estimates develop on previous studies by incorporating the latest data and empirical insights 7 , 8 , as well as in providing a robust empirical lower bound on the persistence of impacts on economic growth, which constitutes a middle ground between the extremes of the growth-versus-levels debate 19 , 21 (Extended Data Fig. 3 ).

Compared with the fraction of variance explained by the empirical models historically (<5%), the projection of reductions in income of 19% may seem large. This arises owing to the fact that projected changes in climatic conditions are much larger than those that were experienced historically, particularly for changes in average temperature (Extended Data Fig. 4 ). As such, any assessment of future climate-change impacts necessarily requires an extrapolation outside the range of the historical data on which the empirical impact models were evaluated. Nevertheless, these models constitute the most state-of-the-art methods for inference of plausibly causal climate impacts based on observed data. Moreover, we take explicit steps to limit out-of-sample extrapolation by capping the moderating variables of the interaction terms at the 95th percentile of the historical distribution (see Methods ). This avoids extrapolating the marginal effects outside what was observed historically. Given the nonlinear response of economic output to annual mean temperature (Extended Data Fig. 1 and Extended Data Table 2 ), this is a conservative choice that limits the magnitude of damages that we project. Furthermore, back-of-the-envelope calculations indicate that the projected damages are consistent with the magnitude and patterns of historical economic development (see Supplementary Discussion Section  5 ).

Missing impacts and spatial spillovers

Despite assessing several climatic components from which economic impacts have recently been identified 3 , 7 , 8 , this assessment of aggregate climate damages should not be considered comprehensive. Important channels such as impacts from heatwaves 31 , sea-level rise 36 , tropical cyclones 37 and tipping points 38 , 39 , as well as non-market damages such as those to ecosystems 40 and human health 41 , are not considered in these estimates. Sea-level rise is unlikely to be feasibly incorporated into empirical assessments such as this because historical sea-level variability is mostly small. Non-market damages are inherently intractable within our estimates of impacts on aggregate monetary output and estimates of these impacts could arguably be considered as extra to those identified here. Recent empirical work suggests that accounting for these channels would probably raise estimates of these committed damages, with larger damages continuing to arise in the global south 31 , 36 , 37 , 38 , 39 , 40 , 41 , 42 .

Moreover, our main empirical analysis does not explicitly evaluate the potential for impacts in local regions to produce effects that ‘spill over’ into other regions. Such effects may further mitigate or amplify the impacts we estimate, for example, if companies relocate production from one affected region to another or if impacts propagate along supply chains. The current literature indicates that trade plays a substantial role in propagating spillover effects 43 , 44 , making their assessment at the sub-national level challenging without available data on sub-national trade dependencies. Studies accounting for only spatially adjacent neighbours indicate that negative impacts in one region induce further negative impacts in neighbouring regions 45 , 46 , 47 , 48 , suggesting that our projected damages are probably conservative by excluding these effects. In Supplementary Fig. 14 , we assess spillovers from neighbouring regions using a spatial-lag model. For simplicity, this analysis excludes temporal lags, focusing only on contemporaneous effects. The results show that accounting for spatial spillovers can amplify the overall magnitude, and also the heterogeneity, of impacts. Consistent with previous literature, this indicates that the overall magnitude (Fig. 1 ) and heterogeneity (Fig. 3 ) of damages that we project in our main specification may be conservative without explicitly accounting for spillovers. We note that further analysis that addresses both spatially and trade-connected spillovers, while also accounting for delayed impacts using temporal lags, would be necessary to adequately address this question fully. These approaches offer fruitful avenues for further research but are beyond the scope of this manuscript, which primarily aims to explore the impacts of different climate conditions and their persistence.

Policy implications

We find that the economic damages resulting from climate change until 2049 are those to which the world economy is already committed and that these greatly outweigh the costs required to mitigate emissions in line with the 2 °C target of the Paris Climate Agreement (Fig. 1 ). This assessment is complementary to formal analyses of the net costs and benefits associated with moving from one emission path to another, which typically find that net benefits of mitigation only emerge in the second half of the century 5 . Our simple comparison of the magnitude of damages and mitigation costs makes clear that this is primarily because damages are indistinguishable across emissions scenarios—that is, committed—until mid-century (Fig. 1 ) and that they are actually already much larger than mitigation costs. For simplicity, and owing to the availability of data, we compare damages to mitigation costs at the global level. Regional estimates of mitigation costs may shed further light on the national incentives for mitigation to which our results already hint, of relevance for international climate policy. Although these damages are committed from a mitigation perspective, adaptation may provide an opportunity to reduce them. Moreover, the strong divergence of damages after mid-century reemphasizes the clear benefits of mitigation from a purely economic perspective, as highlighted in previous studies 1 , 4 , 6 , 24 .

Historical climate data

Historical daily 2-m temperature and precipitation totals (in mm) are obtained for the period 1979–2019 from the W5E5 database. The W5E5 dataset comes from ERA-5, a state-of-the-art reanalysis of historical observations, but has been bias-adjusted by applying version 2.0 of the WATCH Forcing Data to ERA-5 reanalysis data and precipitation data from version 2.3 of the Global Precipitation Climatology Project to better reflect ground-based measurements 49 , 50 , 51 . We obtain these data on a 0.5° × 0.5° grid from the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) database. Notably, these historical data have been used to bias-adjust future climate projections from CMIP-6 (see the following section), ensuring consistency between the distribution of historical daily weather on which our empirical models were estimated and the climate projections used to estimate future damages. These data are publicly available from the ISIMIP database. See refs.  7 , 8 for robustness tests of the empirical models to the choice of climate data reanalysis products.

Future climate data

Daily 2-m temperature and precipitation totals (in mm) are taken from 21 climate models participating in CMIP-6 under a high (RCP8.5) and a low (RCP2.6) greenhouse gas emission scenario from 2015 to 2100. The data have been bias-adjusted and statistically downscaled to a common half-degree grid to reflect the historical distribution of daily temperature and precipitation of the W5E5 dataset using the trend-preserving method developed by the ISIMIP 50 , 52 . As such, the climate model data reproduce observed climatological patterns exceptionally well (Supplementary Table 5 ). Gridded data are publicly available from the ISIMIP database.

Historical economic data

Historical economic data come from the DOSE database of sub-national economic output 53 . We use a recent revision to the DOSE dataset that provides data across 83 countries, 1,660 sub-national regions with varying temporal coverage from 1960 to 2019. Sub-national units constitute the first administrative division below national, for example, states for the USA and provinces for China. Data come from measures of gross regional product per capita (GRPpc) or income per capita in local currencies, reflecting the values reported in national statistical agencies, yearbooks and, in some cases, academic literature. We follow previous literature 3 , 7 , 8 , 54 and assess real sub-national output per capita by first converting values from local currencies to US dollars to account for diverging national inflationary tendencies and then account for US inflation using a US deflator. Alternatively, one might first account for national inflation and then convert between currencies. Supplementary Fig. 12 demonstrates that our conclusions are consistent when accounting for price changes in the reversed order, although the magnitude of estimated damages varies. See the documentation of the DOSE dataset for further discussion of these choices. Conversions between currencies are conducted using exchange rates from the FRED database of the Federal Reserve Bank of St. Louis 55 and the national deflators from the World Bank 56 .

Future socio-economic data

Baseline gridded gross domestic product (GDP) and population data for the period 2015–2100 are taken from the middle-of-the-road scenario SSP2 (ref.  15 ). Population data have been downscaled to a half-degree grid by the ISIMIP following the methodologies of refs.  57 , 58 , which we then aggregate to the sub-national level of our economic data using the spatial aggregation procedure described below. Because current methodologies for downscaling the GDP of the SSPs use downscaled population to do so, per-capita estimates of GDP with a realistic distribution at the sub-national level are not readily available for the SSPs. We therefore use national-level GDP per capita (GDPpc) projections for all sub-national regions of a given country, assuming homogeneity within countries in terms of baseline GDPpc. Here we use projections that have been updated to account for the impact of the COVID-19 pandemic on the trajectory of future income, while remaining consistent with the long-term development of the SSPs 59 . The choice of baseline SSP alters the magnitude of projected climate damages in monetary terms, but when assessed in terms of percentage change from the baseline, the choice of socio-economic scenario is inconsequential. Gridded SSP population data and national-level GDPpc data are publicly available from the ISIMIP database. Sub-national estimates as used in this study are available in the code and data replication files.

Climate variables

Following recent literature 3 , 7 , 8 , we calculate an array of climate variables for which substantial impacts on macroeconomic output have been identified empirically, supported by further evidence at the micro level for plausible underlying mechanisms. See refs.  7 , 8 for an extensive motivation for the use of these particular climate variables and for detailed empirical tests on the nature and robustness of their effects on economic output. To summarize, these studies have found evidence for independent impacts on economic growth rates from annual average temperature, daily temperature variability, total annual precipitation, the annual number of wet days and extreme daily rainfall. Assessments of daily temperature variability were motivated by evidence of impacts on agricultural output and human health, as well as macroeconomic literature on the impacts of volatility on growth when manifest in different dimensions, such as government spending, exchange rates and even output itself 7 . Assessments of precipitation impacts were motivated by evidence of impacts on agricultural productivity, metropolitan labour outcomes and conflict, as well as damages caused by flash flooding 8 . See Extended Data Table 1 for detailed references to empirical studies of these physical mechanisms. Marked impacts of daily temperature variability, total annual precipitation, the number of wet days and extreme daily rainfall on macroeconomic output were identified robustly across different climate datasets, spatial aggregation schemes, specifications of regional time trends and error-clustering approaches. They were also found to be robust to the consideration of temperature extremes 7 , 8 . Furthermore, these climate variables were identified as having independent effects on economic output 7 , 8 , which we further explain here using Monte Carlo simulations to demonstrate the robustness of the results to concerns of imperfect multicollinearity between climate variables (Supplementary Methods Section  2 ), as well as by using information criteria (Supplementary Table 1 ) to demonstrate that including several lagged climate variables provides a preferable trade-off between optimally describing the data and limiting the possibility of overfitting.

We calculate these variables from the distribution of daily, d , temperature, T x , d , and precipitation, P x , d , at the grid-cell, x , level for both the historical and future climate data. As well as annual mean temperature, \({\bar{T}}_{x,y}\) , and annual total precipitation, P x , y , we calculate annual, y , measures of daily temperature variability, \({\widetilde{T}}_{x,y}\) :

the number of wet days, Pwd x , y :

and extreme daily rainfall:

in which T x , d , m , y is the grid-cell-specific daily temperature in month m and year y , \({\bar{T}}_{x,m,{y}}\) is the year and grid-cell-specific monthly, m , mean temperature, D m and D y the number of days in a given month m or year y , respectively, H the Heaviside step function, 1 mm the threshold used to define wet days and P 99.9 x is the 99.9th percentile of historical (1979–2019) daily precipitation at the grid-cell level. Units of the climate measures are degrees Celsius for annual mean temperature and daily temperature variability, millimetres for total annual precipitation and extreme daily precipitation, and simply the number of days for the annual number of wet days.

We also calculated weighted standard deviations of monthly rainfall totals as also used in ref.  8 but do not include them in our projections as we find that, when accounting for delayed effects, their effect becomes statistically indistinct and is better captured by changes in total annual rainfall.

Spatial aggregation

We aggregate grid-cell-level historical and future climate measures, as well as grid-cell-level future GDPpc and population, to the level of the first administrative unit below national level of the GADM database, using an area-weighting algorithm that estimates the portion of each grid cell falling within an administrative boundary. We use this as our baseline specification following previous findings that the effect of area or population weighting at the sub-national level is negligible 7 , 8 .

Empirical model specification: fixed-effects distributed lag models

Following a wide range of climate econometric literature 16 , 60 , we use panel regression models with a selection of fixed effects and time trends to isolate plausibly exogenous variation with which to maximize confidence in a causal interpretation of the effects of climate on economic growth rates. The use of region fixed effects, μ r , accounts for unobserved time-invariant differences between regions, such as prevailing climatic norms and growth rates owing to historical and geopolitical factors. The use of yearly fixed effects, η y , accounts for regionally invariant annual shocks to the global climate or economy such as the El Niño–Southern Oscillation or global recessions. In our baseline specification, we also include region-specific linear time trends, k r y , to exclude the possibility of spurious correlations resulting from common slow-moving trends in climate and growth.

The persistence of climate impacts on economic growth rates is a key determinant of the long-term magnitude of damages. Methods for inferring the extent of persistence in impacts on growth rates have typically used lagged climate variables to evaluate the presence of delayed effects or catch-up dynamics 2 , 18 . For example, consider starting from a model in which a climate condition, C r , y , (for example, annual mean temperature) affects the growth rate, Δlgrp r , y (the first difference of the logarithm of gross regional product) of region r in year y :

which we refer to as a ‘pure growth effects’ model in the main text. Typically, further lags are included,

and the cumulative effect of all lagged terms is evaluated to assess the extent to which climate impacts on growth rates persist. Following ref.  18 , in the case that,

the implication is that impacts on the growth rate persist up to NL years after the initial shock (possibly to a weaker or a stronger extent), whereas if

then the initial impact on the growth rate is recovered after NL years and the effect is only one on the level of output. However, we note that such approaches are limited by the fact that, when including an insufficient number of lags to detect a recovery of the growth rates, one may find equation ( 6 ) to be satisfied and incorrectly assume that a change in climatic conditions affects the growth rate indefinitely. In practice, given a limited record of historical data, including too few lags to confidently conclude in an infinitely persistent impact on the growth rate is likely, particularly over the long timescales over which future climate damages are often projected 2 , 24 . To avoid this issue, we instead begin our analysis with a model for which the level of output, lgrp r , y , depends on the level of a climate variable, C r , y :

Given the non-stationarity of the level of output, we follow the literature 19 and estimate such an equation in first-differenced form as,

which we refer to as a model of ‘pure level effects’ in the main text. This model constitutes a baseline specification in which a permanent change in the climate variable produces an instantaneous impact on the growth rate and a permanent effect only on the level of output. By including lagged variables in this specification,

we are able to test whether the impacts on the growth rate persist any further than instantaneously by evaluating whether α L  > 0 are statistically significantly different from zero. Even though this framework is also limited by the possibility of including too few lags, the choice of a baseline model specification in which impacts on the growth rate do not persist means that, in the case of including too few lags, the framework reverts to the baseline specification of level effects. As such, this framework is conservative with respect to the persistence of impacts and the magnitude of future damages. It naturally avoids assumptions of infinite persistence and we are able to interpret any persistence that we identify with equation ( 9 ) as a lower bound on the extent of climate impact persistence on growth rates. See the main text for further discussion of this specification choice, in particular about its conservative nature compared with previous literature estimates, such as refs.  2 , 18 .

We allow the response to climatic changes to vary across regions, using interactions of the climate variables with historical average (1979–2019) climatic conditions reflecting heterogenous effects identified in previous work 7 , 8 . Following this previous work, the moderating variables of these interaction terms constitute the historical average of either the variable itself or of the seasonal temperature difference, \({\hat{T}}_{r}\) , or annual mean temperature, \({\bar{T}}_{r}\) , in the case of daily temperature variability 7 and extreme daily rainfall, respectively 8 .

The resulting regression equation with N and M lagged variables, respectively, reads:

in which Δlgrp r , y is the annual, regional GRPpc growth rate, measured as the first difference of the logarithm of real GRPpc, following previous work 2 , 3 , 7 , 8 , 18 , 19 . Fixed-effects regressions were run using the fixest package in R (ref.  61 ).

Estimates of the coefficients of interest α i , L are shown in Extended Data Fig. 1 for N  =  M  = 10 lags and for our preferred choice of the number of lags in Supplementary Figs. 1 – 3 . In Extended Data Fig. 1 , errors are shown clustered at the regional level, but for the construction of damage projections, we block-bootstrap the regressions by region 1,000 times to provide a range of parameter estimates with which to sample the projection uncertainty (following refs.  2 , 31 ).

Spatial-lag model

In Supplementary Fig. 14 , we present the results from a spatial-lag model that explores the potential for climate impacts to ‘spill over’ into spatially neighbouring regions. We measure the distance between centroids of each pair of sub-national regions and construct spatial lags that take the average of the first-differenced climate variables and their interaction terms over neighbouring regions that are at distances of 0–500, 500–1,000, 1,000–1,500 and 1,500–2000 km (spatial lags, ‘SL’, 1 to 4). For simplicity, we then assess a spatial-lag model without temporal lags to assess spatial spillovers of contemporaneous climate impacts. This model takes the form:

in which SL indicates the spatial lag of each climate variable and interaction term. In Supplementary Fig. 14 , we plot the cumulative marginal effect of each climate variable at different baseline climate conditions by summing the coefficients for each climate variable and interaction term, for example, for average temperature impacts as:

These cumulative marginal effects can be regarded as the overall spatially dependent impact to an individual region given a one-unit shock to a climate variable in that region and all neighbouring regions at a given value of the moderating variable of the interaction term.

Constructing projections of economic damage from future climate change

We construct projections of future climate damages by applying the coefficients estimated in equation ( 10 ) and shown in Supplementary Tables 2 – 4 (when including only lags with statistically significant effects in specifications that limit overfitting; see Supplementary Methods Section  1 ) to projections of future climate change from the CMIP-6 models. Year-on-year changes in each primary climate variable of interest are calculated to reflect the year-to-year variations used in the empirical models. 30-year moving averages of the moderating variables of the interaction terms are calculated to reflect the long-term average of climatic conditions that were used for the moderating variables in the empirical models. By using moving averages in the projections, we account for the changing vulnerability to climate shocks based on the evolving long-term conditions (Supplementary Figs. 10 and 11 show that the results are robust to the precise choice of the window of this moving average). Although these climate variables are not differenced, the fact that the bias-adjusted climate models reproduce observed climatological patterns across regions for these moderating variables very accurately (Supplementary Table 6 ) with limited spread across models (<3%) precludes the possibility that any considerable bias or uncertainty is introduced by this methodological choice. However, we impose caps on these moderating variables at the 95th percentile at which they were observed in the historical data to prevent extrapolation of the marginal effects outside the range in which the regressions were estimated. This is a conservative choice that limits the magnitude of our damage projections.

Time series of primary climate variables and moderating climate variables are then combined with estimates of the empirical model parameters to evaluate the regression coefficients in equation ( 10 ), producing a time series of annual GRPpc growth-rate reductions for a given emission scenario, climate model and set of empirical model parameters. The resulting time series of growth-rate impacts reflects those occurring owing to future climate change. By contrast, a future scenario with no climate change would be one in which climate variables do not change (other than with random year-to-year fluctuations) and hence the time-averaged evaluation of equation ( 10 ) would be zero. Our approach therefore implicitly compares the future climate-change scenario to this no-climate-change baseline scenario.

The time series of growth-rate impacts owing to future climate change in region r and year y , δ r , y , are then added to the future baseline growth rates, π r , y (in log-diff form), obtained from the SSP2 scenario to yield trajectories of damaged GRPpc growth rates, ρ r , y . These trajectories are aggregated over time to estimate the future trajectory of GRPpc with future climate impacts:

in which GRPpc r , y =2020 is the initial log level of GRPpc. We begin damage estimates in 2020 to reflect the damages occurring since the end of the period for which we estimate the empirical models (1979–2019) and to match the timing of mitigation-cost estimates from most IAMs (see below).

For each emission scenario, this procedure is repeated 1,000 times while randomly sampling from the selection of climate models, the selection of empirical models with different numbers of lags (shown in Supplementary Figs. 1 – 3 and Supplementary Tables 2 – 4 ) and bootstrapped estimates of the regression parameters. The result is an ensemble of future GRPpc trajectories that reflect uncertainty from both physical climate change and the structural and sampling uncertainty of the empirical models.

Estimates of mitigation costs

We obtain IPCC estimates of the aggregate costs of emission mitigation from the AR6 Scenario Explorer and Database hosted by IIASA 23 . Specifically, we search the AR6 Scenarios Database World v1.1 for IAMs that provided estimates of global GDP and population under both a SSP2 baseline and a SSP2-RCP2.6 scenario to maintain consistency with the socio-economic and emission scenarios of the climate damage projections. We find five IAMs that provide data for these scenarios, namely, MESSAGE-GLOBIOM 1.0, REMIND-MAgPIE 1.5, AIM/GCE 2.0, GCAM 4.2 and WITCH-GLOBIOM 3.1. Of these five IAMs, we use the results only from the first three that passed the IPCC vetting procedure for reproducing historical emission and climate trajectories. We then estimate global mitigation costs as the percentage difference in global per capita GDP between the SSP2 baseline and the SSP2-RCP2.6 emission scenario. In the case of one of these IAMs, estimates of mitigation costs begin in 2020, whereas in the case of two others, mitigation costs begin in 2010. The mitigation cost estimates before 2020 in these two IAMs are mostly negligible, and our choice to begin comparison with damage estimates in 2020 is conservative with respect to the relative weight of climate damages compared with mitigation costs for these two IAMs.

Data availability

Data on economic production and ERA-5 climate data are publicly available at https://doi.org/10.5281/zenodo.4681306 (ref. 62 ) and https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5 , respectively. Data on mitigation costs are publicly available at https://data.ene.iiasa.ac.at/ar6/#/downloads . Processed climate and economic data, as well as all other necessary data for reproduction of the results, are available at the public repository https://doi.org/10.5281/zenodo.10562951  (ref. 63 ).

Code availability

All code necessary for reproduction of the results is available at the public repository https://doi.org/10.5281/zenodo.10562951  (ref. 63 ).

Glanemann, N., Willner, S. N. & Levermann, A. Paris Climate Agreement passes the cost-benefit test. Nat. Commun. 11 , 110 (2020).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Burke, M., Hsiang, S. M. & Miguel, E. Global non-linear effect of temperature on economic production. Nature 527 , 235–239 (2015).

Article   ADS   CAS   PubMed   Google Scholar  

Kalkuhl, M. & Wenz, L. The impact of climate conditions on economic production. Evidence from a global panel of regions. J. Environ. Econ. Manag. 103 , 102360 (2020).

Article   Google Scholar  

Moore, F. C. & Diaz, D. B. Temperature impacts on economic growth warrant stringent mitigation policy. Nat. Clim. Change 5 , 127–131 (2015).

Article   ADS   Google Scholar  

Drouet, L., Bosetti, V. & Tavoni, M. Net economic benefits of well-below 2°C scenarios and associated uncertainties. Oxf. Open Clim. Change 2 , kgac003 (2022).

Ueckerdt, F. et al. The economically optimal warming limit of the planet. Earth Syst. Dyn. 10 , 741–763 (2019).

Kotz, M., Wenz, L., Stechemesser, A., Kalkuhl, M. & Levermann, A. Day-to-day temperature variability reduces economic growth. Nat. Clim. Change 11 , 319–325 (2021).

Kotz, M., Levermann, A. & Wenz, L. The effect of rainfall changes on economic production. Nature 601 , 223–227 (2022).

Kousky, C. Informing climate adaptation: a review of the economic costs of natural disasters. Energy Econ. 46 , 576–592 (2014).

Harlan, S. L. et al. in Climate Change and Society: Sociological Perspectives (eds Dunlap, R. E. & Brulle, R. J.) 127–163 (Oxford Univ. Press, 2015).

Bolton, P. et al. The Green Swan (BIS Books, 2020).

Alogoskoufis, S. et al. ECB Economy-wide Climate Stress Test: Methodology and Results European Central Bank, 2021).

Weber, E. U. What shapes perceptions of climate change? Wiley Interdiscip. Rev. Clim. Change 1 , 332–342 (2010).

Markowitz, E. M. & Shariff, A. F. Climate change and moral judgement. Nat. Clim. Change 2 , 243–247 (2012).

Riahi, K. et al. The shared socioeconomic pathways and their energy, land use, and greenhouse gas emissions implications: an overview. Glob. Environ. Change 42 , 153–168 (2017).

Auffhammer, M., Hsiang, S. M., Schlenker, W. & Sobel, A. Using weather data and climate model output in economic analyses of climate change. Rev. Environ. Econ. Policy 7 , 181–198 (2013).

Kolstad, C. D. & Moore, F. C. Estimating the economic impacts of climate change using weather observations. Rev. Environ. Econ. Policy 14 , 1–24 (2020).

Dell, M., Jones, B. F. & Olken, B. A. Temperature shocks and economic growth: evidence from the last half century. Am. Econ. J. Macroecon. 4 , 66–95 (2012).

Newell, R. G., Prest, B. C. & Sexton, S. E. The GDP-temperature relationship: implications for climate change damages. J. Environ. Econ. Manag. 108 , 102445 (2021).

Kikstra, J. S. et al. The social cost of carbon dioxide under climate-economy feedbacks and temperature variability. Environ. Res. Lett. 16 , 094037 (2021).

Article   ADS   CAS   Google Scholar  

Bastien-Olvera, B. & Moore, F. Persistent effect of temperature on GDP identified from lower frequency temperature variability. Environ. Res. Lett. 17 , 084038 (2022).

Eyring, V. et al. Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci. Model Dev. 9 , 1937–1958 (2016).

Byers, E. et al. AR6 scenarios database. Zenodo https://zenodo.org/records/7197970 (2022).

Burke, M., Davis, W. M. & Diffenbaugh, N. S. Large potential reduction in economic damages under UN mitigation targets. Nature 557 , 549–553 (2018).

Kotz, M., Wenz, L. & Levermann, A. Footprint of greenhouse forcing in daily temperature variability. Proc. Natl Acad. Sci. 118 , e2103294118 (2021).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Myhre, G. et al. Frequency of extreme precipitation increases extensively with event rareness under global warming. Sci. Rep. 9 , 16063 (2019).

Min, S.-K., Zhang, X., Zwiers, F. W. & Hegerl, G. C. Human contribution to more-intense precipitation extremes. Nature 470 , 378–381 (2011).

England, M. R., Eisenman, I., Lutsko, N. J. & Wagner, T. J. The recent emergence of Arctic Amplification. Geophys. Res. Lett. 48 , e2021GL094086 (2021).

Fischer, E. M. & Knutti, R. Anthropogenic contribution to global occurrence of heavy-precipitation and high-temperature extremes. Nat. Clim. Change 5 , 560–564 (2015).

Pfahl, S., O’Gorman, P. A. & Fischer, E. M. Understanding the regional pattern of projected future changes in extreme precipitation. Nat. Clim. Change 7 , 423–427 (2017).

Callahan, C. W. & Mankin, J. S. Globally unequal effect of extreme heat on economic growth. Sci. Adv. 8 , eadd3726 (2022).

Diffenbaugh, N. S. & Burke, M. Global warming has increased global economic inequality. Proc. Natl Acad. Sci. 116 , 9808–9813 (2019).

Callahan, C. W. & Mankin, J. S. National attribution of historical climate damages. Clim. Change 172 , 40 (2022).

Burke, M. & Tanutama, V. Climatic constraints on aggregate economic output. National Bureau of Economic Research, Working Paper 25779. https://doi.org/10.3386/w25779 (2019).

Kahn, M. E. et al. Long-term macroeconomic effects of climate change: a cross-country analysis. Energy Econ. 104 , 105624 (2021).

Desmet, K. et al. Evaluating the economic cost of coastal flooding. National Bureau of Economic Research, Working Paper 24918. https://doi.org/10.3386/w24918 (2018).

Hsiang, S. M. & Jina, A. S. The causal effect of environmental catastrophe on long-run economic growth: evidence from 6,700 cyclones. National Bureau of Economic Research, Working Paper 20352. https://doi.org/10.3386/w2035 (2014).

Ritchie, P. D. et al. Shifts in national land use and food production in Great Britain after a climate tipping point. Nat. Food 1 , 76–83 (2020).

Dietz, S., Rising, J., Stoerk, T. & Wagner, G. Economic impacts of tipping points in the climate system. Proc. Natl Acad. Sci. 118 , e2103081118 (2021).

Bastien-Olvera, B. A. & Moore, F. C. Use and non-use value of nature and the social cost of carbon. Nat. Sustain. 4 , 101–108 (2021).

Carleton, T. et al. Valuing the global mortality consequences of climate change accounting for adaptation costs and benefits. Q. J. Econ. 137 , 2037–2105 (2022).

Bastien-Olvera, B. A. et al. Unequal climate impacts on global values of natural capital. Nature 625 , 722–727 (2024).

Malik, A. et al. Impacts of climate change and extreme weather on food supply chains cascade across sectors and regions in Australia. Nat. Food 3 , 631–643 (2022).

Article   ADS   PubMed   Google Scholar  

Kuhla, K., Willner, S. N., Otto, C., Geiger, T. & Levermann, A. Ripple resonance amplifies economic welfare loss from weather extremes. Environ. Res. Lett. 16 , 114010 (2021).

Schleypen, J. R., Mistry, M. N., Saeed, F. & Dasgupta, S. Sharing the burden: quantifying climate change spillovers in the European Union under the Paris Agreement. Spat. Econ. Anal. 17 , 67–82 (2022).

Dasgupta, S., Bosello, F., De Cian, E. & Mistry, M. Global temperature effects on economic activity and equity: a spatial analysis. European Institute on Economics and the Environment, Working Paper 22-1 (2022).

Neal, T. The importance of external weather effects in projecting the macroeconomic impacts of climate change. UNSW Economics Working Paper 2023-09 (2023).

Deryugina, T. & Hsiang, S. M. Does the environment still matter? Daily temperature and income in the United States. National Bureau of Economic Research, Working Paper 20750. https://doi.org/10.3386/w20750 (2014).

Hersbach, H. et al. The ERA5 global reanalysis. Q. J. R. Meteorol. Soc. 146 , 1999–2049 (2020).

Cucchi, M. et al. WFDE5: bias-adjusted ERA5 reanalysis data for impact studies. Earth Syst. Sci. Data 12 , 2097–2120 (2020).

Adler, R. et al. The New Version 2.3 of the Global Precipitation Climatology Project (GPCP) Monthly Analysis Product 1072–1084 (University of Maryland, 2016).

Lange, S. Trend-preserving bias adjustment and statistical downscaling with ISIMIP3BASD (v1.0). Geosci. Model Dev. 12 , 3055–3070 (2019).

Wenz, L., Carr, R. D., Kögel, N., Kotz, M. & Kalkuhl, M. DOSE – global data set of reported sub-national economic output. Sci. Data 10 , 425 (2023).

Article   PubMed   PubMed Central   Google Scholar  

Gennaioli, N., La Porta, R., Lopez De Silanes, F. & Shleifer, A. Growth in regions. J. Econ. Growth 19 , 259–309 (2014).

Board of Governors of the Federal Reserve System (US). U.S. dollars to euro spot exchange rate. https://fred.stlouisfed.org/series/AEXUSEU (2022).

World Bank. GDP deflator. https://data.worldbank.org/indicator/NY.GDP.DEFL.ZS (2022).

Jones, B. & O’Neill, B. C. Spatially explicit global population scenarios consistent with the Shared Socioeconomic Pathways. Environ. Res. Lett. 11 , 084003 (2016).

Murakami, D. & Yamagata, Y. Estimation of gridded population and GDP scenarios with spatially explicit statistical downscaling. Sustainability 11 , 2106 (2019).

Koch, J. & Leimbach, M. Update of SSP GDP projections: capturing recent changes in national accounting, PPP conversion and Covid 19 impacts. Ecol. Econ. 206 (2023).

Carleton, T. A. & Hsiang, S. M. Social and economic impacts of climate. Science 353 , aad9837 (2016).

Article   PubMed   Google Scholar  

Bergé, L. Efficient estimation of maximum likelihood models with multiple fixed-effects: the R package FENmlm. DEM Discussion Paper Series 18-13 (2018).

Kalkuhl, M., Kotz, M. & Wenz, L. DOSE - The MCC-PIK Database Of Subnational Economic output. Zenodo https://zenodo.org/doi/10.5281/zenodo.4681305 (2021).

Kotz, M., Wenz, L. & Levermann, A. Data and code for “The economic commitment of climate change”. Zenodo https://zenodo.org/doi/10.5281/zenodo.10562951 (2024).

Dasgupta, S. et al. Effects of climate change on combined labour productivity and supply: an empirical, multi-model study. Lancet Planet. Health 5 , e455–e465 (2021).

Lobell, D. B. et al. The critical role of extreme heat for maize production in the United States. Nat. Clim. Change 3 , 497–501 (2013).

Zhao, C. et al. Temperature increase reduces global yields of major crops in four independent estimates. Proc. Natl Acad. Sci. 114 , 9326–9331 (2017).

Wheeler, T. R., Craufurd, P. Q., Ellis, R. H., Porter, J. R. & Prasad, P. V. Temperature variability and the yield of annual crops. Agric. Ecosyst. Environ. 82 , 159–167 (2000).

Rowhani, P., Lobell, D. B., Linderman, M. & Ramankutty, N. Climate variability and crop production in Tanzania. Agric. For. Meteorol. 151 , 449–460 (2011).

Ceglar, A., Toreti, A., Lecerf, R., Van der Velde, M. & Dentener, F. Impact of meteorological drivers on regional inter-annual crop yield variability in France. Agric. For. Meteorol. 216 , 58–67 (2016).

Shi, L., Kloog, I., Zanobetti, A., Liu, P. & Schwartz, J. D. Impacts of temperature and its variability on mortality in New England. Nat. Clim. Change 5 , 988–991 (2015).

Xue, T., Zhu, T., Zheng, Y. & Zhang, Q. Declines in mental health associated with air pollution and temperature variability in China. Nat. Commun. 10 , 2165 (2019).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Liang, X.-Z. et al. Determining climate effects on US total agricultural productivity. Proc. Natl Acad. Sci. 114 , E2285–E2292 (2017).

Desbureaux, S. & Rodella, A.-S. Drought in the city: the economic impact of water scarcity in Latin American metropolitan areas. World Dev. 114 , 13–27 (2019).

Damania, R. The economics of water scarcity and variability. Oxf. Rev. Econ. Policy 36 , 24–44 (2020).

Davenport, F. V., Burke, M. & Diffenbaugh, N. S. Contribution of historical precipitation change to US flood damages. Proc. Natl Acad. Sci. 118 , e2017524118 (2021).

Dave, R., Subramanian, S. S. & Bhatia, U. Extreme precipitation induced concurrent events trigger prolonged disruptions in regional road networks. Environ. Res. Lett. 16 , 104050 (2021).

Download references

Acknowledgements

We gratefully acknowledge financing from the Volkswagen Foundation and the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH on behalf of the Government of the Federal Republic of Germany and Federal Ministry for Economic Cooperation and Development (BMZ).

Open access funding provided by Potsdam-Institut für Klimafolgenforschung (PIK) e.V.

Author information

Authors and affiliations.

Research Domain IV, Research Domain IV, Potsdam Institute for Climate Impact Research, Potsdam, Germany

Maximilian Kotz, Anders Levermann & Leonie Wenz

Institute of Physics, Potsdam University, Potsdam, Germany

Maximilian Kotz & Anders Levermann

Mercator Research Institute on Global Commons and Climate Change, Berlin, Germany

Leonie Wenz

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the design of the analysis. M.K. conducted the analysis and produced the figures. All authors contributed to the interpretation and presentation of the results. M.K. and L.W. wrote the manuscript.

Corresponding author

Correspondence to Leonie Wenz .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature thanks Xin-Zhong Liang, Chad Thackeray and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended data fig. 1 constraining the persistence of historical climate impacts on economic growth rates..

The results of a panel-based fixed-effects distributed lag model for the effects of annual mean temperature ( a ), daily temperature variability ( b ), total annual precipitation ( c ), the number of wet days ( d ) and extreme daily precipitation ( e ) on sub-national economic growth rates. Point estimates show the effects of a 1 °C or one standard deviation increase (for temperature and precipitation variables, respectively) at the lower quartile, median and upper quartile of the relevant moderating variable (green, orange and purple, respectively) at different lagged periods after the initial shock (note that these are not cumulative effects). Climate variables are used in their first-differenced form (see main text for discussion) and the moderating climate variables are the annual mean temperature, seasonal temperature difference, total annual precipitation, number of wet days and annual mean temperature, respectively, in panels a – e (see Methods for further discussion). Error bars show the 95% confidence intervals having clustered standard errors by region. The within-region R 2 , Bayesian and Akaike information criteria for the model are shown at the top of the figure. This figure shows results with ten lags for each variable to demonstrate the observed levels of persistence, but our preferred specifications remove later lags based on the statistical significance of terms shown above and the information criteria shown in Extended Data Fig. 2 . The resulting models without later lags are shown in Supplementary Figs. 1 – 3 .

Extended Data Fig. 2 Incremental lag-selection procedure using information criteria and within-region R 2 .

Starting from a panel-based fixed-effects distributed lag model estimating the effects of climate on economic growth using the real historical data (as in equation ( 4 )) with ten lags for all climate variables (as shown in Extended Data Fig. 1 ), lags are incrementally removed for one climate variable at a time. The resulting Bayesian and Akaike information criteria are shown in a – e and f – j , respectively, and the within-region R 2 and number of observations in k – o and p – t , respectively. Different rows show the results when removing lags from different climate variables, ordered from top to bottom as annual mean temperature, daily temperature variability, total annual precipitation, the number of wet days and extreme annual precipitation. Information criteria show minima at approximately four lags for precipitation variables and ten to eight for temperature variables, indicating that including these numbers of lags does not lead to overfitting. See Supplementary Table 1 for an assessment using information criteria to determine whether including further climate variables causes overfitting.

Extended Data Fig. 3 Damages in our preferred specification that provides a robust lower bound on the persistence of climate impacts on economic growth versus damages in specifications of pure growth or pure level effects.

Estimates of future damages as shown in Fig. 1 but under the emission scenario RCP8.5 for three separate empirical specifications: in orange our preferred specification, which provides an empirical lower bound on the persistence of climate impacts on economic growth rates while avoiding assumptions of infinite persistence (see main text for further discussion); in purple a specification of ‘pure growth effects’ in which the first difference of climate variables is not taken and no lagged climate variables are included (the baseline specification of ref.  2 ); and in pink a specification of ‘pure level effects’ in which the first difference of climate variables is taken but no lagged terms are included.

Extended Data Fig. 4 Climate changes in different variables as a function of historical interannual variability.

Changes in each climate variable of interest from 1979–2019 to 2035–2065 under the high-emission scenario SSP5-RCP8.5, expressed as a percentage of the historical variability of each measure. Historical variability is estimated as the standard deviation of each detrended climate variable over the period 1979–2019 during which the empirical models were identified (detrending is appropriate because of the inclusion of region-specific linear time trends in the empirical models). See Supplementary Fig. 13 for changes expressed in standard units. Data on national administrative boundaries are obtained from the GADM database version 3.6 and are freely available for academic use ( https://gadm.org/ ).

Extended Data Fig. 5 Contribution of different climate variables to overall committed damages.

a , Climate damages in 2049 when using empirical models that account for all climate variables, changes in annual mean temperature only or changes in both annual mean temperature and one other climate variable (daily temperature variability, total annual precipitation, the number of wet days and extreme daily precipitation, respectively). b , The cumulative marginal effects of an increase in annual mean temperature of 1 °C, at different baseline temperatures, estimated from empirical models including all climate variables or annual mean temperature only. Estimates and uncertainty bars represent the median and 95% confidence intervals obtained from 1,000 block-bootstrap resamples from each of three different empirical models using eight, nine or ten lags of temperature terms.

Extended Data Fig. 6 The difference in committed damages between the upper and lower quartiles of countries when ranked by GDP and cumulative historical emissions.

Quartiles are defined using a population weighting, as are the average committed damages across each quartile group. The violin plots indicate the distribution of differences between quartiles across the two extreme emission scenarios (RCP2.6 and RCP8.5) and the uncertainty sampling procedure outlined in Methods , which accounts for uncertainty arising from the choice of lags in the empirical models, uncertainty in the empirical model parameter estimates, as well as the climate model projections. Bars indicate the median, as well as the 10th and 90th percentiles and upper and lower sixths of the distribution reflecting the very likely and likely ranges following the likelihood classification adopted by the IPCC.

Supplementary information

Supplementary information, peer review file, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kotz, M., Levermann, A. & Wenz, L. The economic commitment of climate change. Nature 628 , 551–557 (2024). https://doi.org/10.1038/s41586-024-07219-0

Download citation

Received : 25 January 2023

Accepted : 21 February 2024

Published : 17 April 2024

Issue Date : 18 April 2024

DOI : https://doi.org/10.1038/s41586-024-07219-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.