case study case evaluation

• Open access
• Published: 27 November 2020

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials

• Aileen Grant 1 ,
• Carol Bugge 2 &
• Mary Wells 3  

Trials volume  21 , Article number:  982 ( 2020 ) Cite this article

10k Accesses

9 Citations

5 Altmetric

Metrics details

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail, and whether they can be transferred to other settings and populations. However, historically, context has not been sufficiently explored and reported resulting in the poor uptake of trial results. Therefore, suitable methodologies are needed to guide the investigation of context. Case study is one appropriate methodology, but there is little guidance about what case study design can offer the study of context in trials. We address this gap in the literature by presenting a number of important considerations for process evaluation using a case study design.

In this paper, we define context, the relationship between complex interventions and context, and describe case study design methodology. A well-designed process evaluation using case study should consider the following core components: the purpose; definition of the intervention; the trial design, the case, the theories or logic models underpinning the intervention, the sampling approach and the conceptual or theoretical framework. We describe each of these in detail and highlight with examples from recently published process evaluations.

Conclusions

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation. We provide a comprehensive overview of the issues for process evaluation design to consider when using a case study design.

Trial registration

DQIP - ClinicalTrials.gov number, NCT01425502 - OPAL - ISRCTN57746448

Peer Review reports

Contribution to the literature

We illustrate how case study methodology can explore the complex, dynamic and uncertain relationship between context and interventions within trials.

We depict different case study designs and illustrate there is not one formula and that design needs to be tailored to the context and trial design.

Case study can support comparisons between intervention and control arms and between cases within arms to uncover and explain differences in detail.

We argue that case study can illustrate how components have evolved and been redefined through implementation.

Key issues for consideration in case study design within process evaluations are presented and illustrated with examples.

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail and whether they can be transferred to other settings and populations. However, historically, not all trials have had a process evaluation component, nor have they sufficiently reported aspects of context, resulting in poor uptake of trial findings [ 1 ]. Considerations of context are often absent from published process evaluations, with few studies acknowledging, taking account of or describing context during implementation, or assessing the impact of context on implementation [ 2 , 3 ]. At present, evidence from trials is not being used in a timely manner [ 4 , 5 ], and this can negatively impact on patient benefit and experience [ 6 ]. It takes on average 17 years for knowledge from research to be implemented into practice [ 7 ]. Suitable methodologies are therefore needed that allow for context to be exposed; one appropriate methodological approach is case study [ 8 , 9 ].

In 2015, the Medical Research Council (MRC) published guidance for process evaluations [ 10 ]. This was a key milestone in legitimising as well as providing tools, methods and a framework for conducting process evaluations. Nevertheless, as with all guidance, there is a need for reflection, challenge and refinement. There have been a number of critiques of the MRC guidance, including that interventions should be considered as events in systems [ 11 , 12 , 13 , 14 ]; a need for better use, critique and development of theories [ 15 , 16 , 17 ]; and a need for more guidance on integrating qualitative and quantitative data [ 18 , 19 ]. Although the MRC process evaluation guidance does consider appropriate qualitative and quantitative methods, it does not mention case study design and what it can offer the study of context in trials.

The case study methodology is ideally suited to real-world, sustainable intervention development and evaluation because it can explore and examine contemporary complex phenomena, in depth, in numerous contexts and using multiple sources of data [ 8 ]. Case study design can capture the complexity of the case, the relationship between the intervention and the context and how the intervention worked (or not) [ 8 ]. There are a number of textbooks on a case study within the social science fields [ 8 , 9 , 20 ], but there are no case study textbooks and a paucity of useful texts on how to design, conduct and report case study within the health arena. Few examples exist within the trial design and evaluation literature [ 3 , 21 ]. Therefore, guidance to enable well-designed process evaluations using case study methodology is required.

We aim to address the gap in the literature by presenting a number of important considerations for process evaluation using a case study design. First, we define the context and describe the relationship between complex health interventions and context.

What is context?

While there is growing recognition that context interacts with the intervention to impact on the intervention’s effectiveness [ 22 ], context is still poorly defined and conceptualised. There are a number of different definitions in the literature, but as Bate et al. explained ‘almost universally, we find context to be an overworked word in everyday dialogue but a massively understudied and misunderstood concept’ [ 23 ]. Ovretveit defines context as ‘everything the intervention is not’ [ 24 ]. This last definition is used by the MRC framework for process evaluations [ 25 ]; however; the problem with this definition is that it is highly dependent on how the intervention is defined. We have found Pfadenhauer et al.’s definition useful:

Context is conceptualised as a set of characteristics and circumstances that consist of active and unique factors that surround the implementation. As such it is not a backdrop for implementation but interacts, influences, modifies and facilitates or constrains the intervention and its implementation. Context is usually considered in relation to an intervention or object, with which it actively interacts. A boundary between the concepts of context and setting is discernible: setting refers to the physical, specific location in which the intervention is put into practice. Context is much more versatile, embracing not only the setting but also roles, interactions and relationships [ 22 ].

Traditionally, context has been conceptualised in terms of barriers and facilitators, but what is a barrier in one context may be a facilitator in another, so it is the relationship and dynamics between the intervention and context which are the most important [ 26 ]. There is a need for empirical research to really understand how different contextual factors relate to each other and to the intervention. At present, research studies often list common contextual factors, but without a depth of meaning and understanding, such as government or health board policies, organisational structures, professional and patient attitudes, behaviours and beliefs [ 27 ]. The case study methodology is well placed to understand the relationship between context and intervention where these boundaries may not be clearly evident. It offers a means of unpicking the contextual conditions which are pertinent to effective implementation.

The relationship between complex health interventions and context

Health interventions are generally made up of a number of different components and are considered complex due to the influence of context on their implementation and outcomes [ 3 , 28 ]. Complex interventions are often reliant on the engagement of practitioners and patients, so their attitudes, behaviours, beliefs and cultures influence whether and how an intervention is effective or not. Interventions are context-sensitive; they interact with the environment in which they are implemented. In fact, many argue that interventions are a product of their context, and indeed, outcomes are likely to be a product of the intervention and its context [ 3 , 29 ]. Within a trial, there is also the influence of the research context too—so the observed outcome could be due to the intervention alone, elements of the context within which the intervention is being delivered, elements of the research process or a combination of all three. Therefore, it can be difficult and unhelpful to separate the intervention from the context within which it was evaluated because the intervention and context are likely to have evolved together over time. As a result, the same intervention can look and behave differently in different contexts, so it is important this is known, understood and reported [ 3 ]. Finally, the intervention context is dynamic; the people, organisations and systems change over time, [ 3 ] which requires practitioners and patients to respond, and they may do this by adapting the intervention or contextual factors. So, to enable researchers to replicate successful interventions, or to explain why the intervention was not successful, it is not enough to describe the components of the intervention, they need to be described by their relationship to their context and resources [ 3 , 28 ].

What is a case study?

Case study methodology aims to provide an in-depth, holistic, balanced, detailed and complete picture of complex contemporary phenomena in its natural context [ 8 , 9 , 20 ]. In this case, the phenomena are the implementation of complex interventions in a trial. Case study methodology takes the view that the phenomena can be more than the sum of their parts and have to be understood as a whole [ 30 ]. It is differentiated from a clinical case study by its analytical focus [ 20 ].

The methodology is particularly useful when linked to trials because some of the features of the design naturally fill the gaps in knowledge generated by trials. Given the methodological focus on understanding phenomena in the round, case study methodology is typified by the use of multiple sources of data, which are more commonly qualitatively guided [ 31 ]. The case study methodology is not epistemologically specific, like realist evaluation, and can be used with different epistemologies [ 32 ], and with different theories, such as Normalisation Process Theory (which explores how staff work together to implement a new intervention) or the Consolidated Framework for Implementation Research (which provides a menu of constructs associated with effective implementation) [ 33 , 34 , 35 ]. Realist evaluation can be used to explore the relationship between context, mechanism and outcome, but case study differs from realist evaluation by its focus on a holistic and in-depth understanding of the relationship between an intervention and the contemporary context in which it was implemented [ 36 ]. Case study enables researchers to choose epistemologies and theories which suit the nature of the enquiry and their theoretical preferences.

Designing a process evaluation using case study

An important part of any study is the research design. Due to their varied philosophical positions, the seminal authors in the field of case study have different epistemic views as to how a case study should be conducted [ 8 , 9 ]. Stake takes an interpretative approach (interested in how people make sense of their world), and Yin has more positivistic leanings, arguing for objectivity, validity and generalisability [ 8 , 9 ].

Regardless of the philosophical background, a well-designed process evaluation using case study should consider the following core components: the purpose; the definition of the intervention, the trial design, the case, and the theories or logic models underpinning the intervention; the sampling approach; and the conceptual or theoretical framework [ 8 , 9 , 20 , 31 , 33 ]. We now discuss these critical components in turn, with reference to two process evaluations that used case study design, the DQIP and OPAL studies [ 21 , 37 , 38 , 39 , 40 , 41 ].

The purpose of a process evaluation is to evaluate and explain the relationship between the intervention and its components, to context and outcome. It can help inform judgements about validity (by exploring the intervention components and their relationship with one another (construct validity), the connections between intervention and outcomes (internal validity) and the relationship between intervention and context (external validity)). It can also distinguish between implementation failure (where the intervention is poorly delivered) and intervention failure (intervention design is flawed) [ 42 , 43 ]. By using a case study to explicitly understand the relationship between context and the intervention during implementation, the process evaluation can explain the intervention effects and the potential generalisability and optimisation into routine practice [ 44 ].

The DQIP process evaluation aimed to qualitatively explore how patients and GP practices responded to an intervention designed to reduce high-risk prescribing of nonsteroidal anti-inflammatory drugs (NSAIDs) and/or antiplatelet agents (see Table  1 ) and quantitatively examine how change in high-risk prescribing was associated with practice characteristics and implementation processes. The OPAL process evaluation (see Table  2 ) aimed to quantitatively understand the factors which influenced the effectiveness of a pelvic floor muscle training intervention for women with urinary incontinence and qualitatively explore the participants’ experiences of treatment and adherence.

Defining the intervention and exploring the theories or assumptions underpinning the intervention design

Process evaluations should also explore the utility of the theories or assumptions underpinning intervention design [ 49 ]. Not all theories underpinning interventions are based on a formal theory, but they based on assumptions as to how the intervention is expected to work. These can be depicted as a logic model or theory of change [ 25 ]. To capture how the intervention and context evolve requires the intervention and its expected mechanisms to be clearly defined at the outset [ 50 ]. Hawe and colleagues recommend defining interventions by function (what processes make the intervention work) rather than form (what is delivered) [ 51 ]. However, in some cases, it may be useful to know if some of the components are redundant in certain contexts or if there is a synergistic effect between all the intervention components.

The DQIP trial delivered two interventions, one intervention was delivered to professionals with high fidelity and then professionals delivered the other intervention to patients by form rather than function allowing adaptations to the local context as appropriate. The assumptions underpinning intervention delivery were prespecified in a logic model published in the process evaluation protocol [ 52 ].

Case study is well placed to challenge or reinforce the theoretical assumptions or redefine these based on the relationship between the intervention and context. Yin advocates the use of theoretical propositions; these direct attention to specific aspects of the study for investigation [ 8 ] can be based on the underlying assumptions and tested during the course of the process evaluation. In case studies, using an epistemic position more aligned with Yin can enable research questions to be designed, which seek to expose patterns of unanticipated as well as expected relationships [ 9 ]. The OPAL trial was more closely aligned with Yin, where the research team predefined some of their theoretical assumptions, based on how the intervention was expected to work. The relevant parts of the data analysis then drew on data to support or refute the theoretical propositions. This was particularly useful for the trial as the prespecified theoretical propositions linked to the mechanisms of action on which the intervention was anticipated to have an effect (or not).

Tailoring to the trial design

Process evaluations need to be tailored to the trial, the intervention and the outcomes being measured [ 45 ]. For example, in a stepped wedge design (where the intervention is delivered in a phased manner), researchers should try to ensure process data are captured at relevant time points or in a two-arm or multiple arm trial, ensure data is collected from the control group(s) as well as the intervention group(s). In the DQIP trial, a stepped wedge trial, at least one process evaluation case, was sampled per cohort. Trials often continue to measure outcomes after delivery of the intervention has ceased, so researchers should also consider capturing ‘follow-up’ data on contextual factors, which may continue to influence the outcome measure. The OPAL trial had two active treatment arms so collected process data from both arms. In addition, as the trial was interested in long-term adherence, the trial and the process evaluation collected data from participants for 2 years after the intervention was initially delivered, providing 24 months follow-up data, in line with the primary outcome for the trial.

Defining the case

Case studies can include single or multiple cases in their design. Single case studies usually sample typical or unique cases, their advantage being the depth and richness that can be achieved over a long period of time. The advantages of multiple case study design are that cases can be compared to generate a greater depth of analysis. Multiple case study sampling may be carried out in order to test for replication or contradiction [ 8 ]. Given that trials are often conducted over a number of sites, a multiple case study design is more sensible for process evaluations, as there is likely to be variation in implementation between sites. Case definition may occur at a variety of levels but is most appropriate if it reflects the trial design. For example, a case in an individual patient level trial is likely to be defined as a person/patient (e.g. a woman with urinary incontinence—OPAL trial) whereas in a cluster trial, a case is like to be a cluster, such as an organisation (e.g. a general practice—DQIP trial). Of course, the process evaluation could explore cases with less distinct boundaries, such as communities or relationships; however, the clarity with which these cases are defined is important, in order to scope the nature of the data that will be generated.

Carefully sampled cases are critical to a good case study as sampling helps inform the quality of the inferences that can be made from the data [ 53 ]. In both qualitative and quantitative research, how and how many participants to sample must be decided when planning the study. Quantitative sampling techniques generally aim to achieve a random sample. Qualitative research generally uses purposive samples to achieve data saturation, occurring when the incoming data produces little or no new information to address the research questions. The term data saturation has evolved from theoretical saturation in conventional grounded theory studies; however, its relevance to other types of studies is contentious as the term saturation seems to be widely used but poorly justified [ 54 ]. Empirical evidence suggests that for in-depth interview studies, saturation occurs at 12 interviews for thematic saturation, but typically more would be needed for a heterogenous sample higher degrees of saturation [ 55 , 56 ]. Both DQIP and OPAL case studies were huge with OPAL designed to interview each of the 40 individual cases four times and DQIP designed to interview the lead DQIP general practitioner (GP) twice (to capture change over time), another GP and the practice manager from each of the 10 organisational cases. Despite the plethora of mixed methods research textbooks, there is very little about sampling as discussions typically link to method (e.g. interviews) rather than paradigm (e.g. case study).

Purposive sampling can improve the generalisability of the process evaluation by sampling for greater contextual diversity. The typical or average case is often not the richest source of information. Outliers can often reveal more important insights, because they may reflect the implementation of the intervention using different processes. Cases can be selected from a number of criteria, which are not mutually exclusive, to enable a rich and detailed picture to be built across sites [ 53 ]. To avoid the Hawthorne effect, it is recommended that process evaluations sample from both intervention and control sites, which enables comparison and explanation. There is always a trade-off between breadth and depth in sampling, so it is important to note that often quantity does not mean quality and that carefully sampled cases can provide powerful illustrative examples of how the intervention worked in practice, the relationship between the intervention and context and how and why they evolved together. The qualitative components of both DQIP and OPAL process evaluations aimed for maximum variation sampling. Please see Table  1 for further information on how DQIP’s sampling frame was important for providing contextual information on processes influencing effective implementation of the intervention.

Conceptual and theoretical framework

A conceptual or theoretical framework helps to frame data collection and analysis [ 57 ]. Theories can also underpin propositions, which can be tested in the process evaluation. Process evaluations produce intervention-dependent knowledge, and theories help make the research findings more generalizable by providing a common language [ 16 ]. There are a number of mid-range theories which have been designed to be used with process evaluation [ 34 , 35 , 58 ]. The choice of the appropriate conceptual or theoretical framework is, however, dependent on the philosophical and professional background of the research. The two examples within this paper used our own framework for the design of process evaluations, which proposes a number of candidate processes which can be explored, for example, recruitment, delivery, response, maintenance and context [ 45 ]. This framework was published before the MRC guidance on process evaluations, and both the DQIP and OPAL process evaluations were designed before the MRC guidance was published. The DQIP process evaluation explored all candidates in the framework whereas the OPAL process evaluation selected four candidates, illustrating that process evaluations can be selective in what they explore based on the purpose, research questions and resources. Furthermore, as Kislov and colleagues argue, we also have a responsibility to critique the theoretical framework underpinning the evaluation and refine theories to advance knowledge [ 59 ].

Data collection

An important consideration is what data to collect or measure and when. Case study methodology supports a range of data collection methods, both qualitative and quantitative, to best answer the research questions. As the aim of the case study is to gain an in-depth understanding of phenomena in context, methods are more commonly qualitative or mixed method in nature. Qualitative methods such as interviews, focus groups and observation offer rich descriptions of the setting, delivery of the intervention in each site and arm, how the intervention was perceived by the professionals delivering the intervention and the patients receiving the intervention. Quantitative methods can measure recruitment, fidelity and dose and establish which characteristics are associated with adoption, delivery and effectiveness. To ensure an understanding of the complexity of the relationship between the intervention and context, the case study should rely on multiple sources of data and triangulate these to confirm and corroborate the findings [ 8 ]. Process evaluations might consider using routine data collected in the trial across all sites and additional qualitative data across carefully sampled sites for a more nuanced picture within reasonable resource constraints. Mixed methods allow researchers to ask more complex questions and collect richer data than can be collected by one method alone [ 60 ]. The use of multiple sources of data allows data triangulation, which increases a study’s internal validity but also provides a more in-depth and holistic depiction of the case [ 20 ]. For example, in the DQIP process evaluation, the quantitative component used routinely collected data from all sites participating in the trial and purposively sampled cases for a more in-depth qualitative exploration [ 21 , 38 , 39 ].

The timing of data collection is crucial to study design, especially within a process evaluation where data collection can potentially influence the trial outcome. Process evaluations are generally in parallel or retrospective to the trial. The advantage of a retrospective design is that the evaluation itself is less likely to influence the trial outcome. However, the disadvantages include recall bias, lack of sensitivity to nuances and an inability to iteratively explore the relationship between intervention and outcome as it develops. To capture the dynamic relationship between intervention and context, the process evaluation needs to be parallel and longitudinal to the trial. Longitudinal methodological design is rare, but it is needed to capture the dynamic nature of implementation [ 40 ]. How the intervention is delivered is likely to change over time as it interacts with context. For example, as professionals deliver the intervention, they become more familiar with it, and it becomes more embedded into systems. The OPAL process evaluation was a longitudinal, mixed methods process evaluation where the quantitative component had been predefined and built into trial data collection systems. Data collection in both the qualitative and quantitative components mirrored the trial data collection points, which were longitudinal to capture adherence and contextual changes over time.

There is a lot of attention in the recent literature towards a systems approach to understanding interventions in context, which suggests interventions are ‘events within systems’ [ 61 , 62 ]. This framing highlights the dynamic nature of context, suggesting that interventions are an attempt to change systems dynamics. This conceptualisation would suggest that the study design should collect contextual data before and after implementation to assess the effect of the intervention on the context and vice versa.

Data analysis

Designing a rigorous analysis plan is particularly important for multiple case studies, where researchers must decide whether their approach to analysis is case or variable based. Case-based analysis is the most common, and analytic strategies must be clearly articulated for within and across case analysis. A multiple case study design can consist of multiple cases, where each case is analysed at the case level, or of multiple embedded cases, where data from all the cases are pulled together for analysis at some level. For example, OPAL analysis was at the case level, but all the cases for the intervention and control arms were pulled together at the arm level for more in-depth analysis and comparison. For Yin, analytical strategies rely on theoretical propositions, but for Stake, analysis works from the data to develop theory. In OPAL and DQIP, case summaries were written to summarise the cases and detail within-case analysis. Each of the studies structured these differently based on the phenomena of interest and the analytic technique. DQIP applied an approach more akin to Stake [ 9 ], with the cases summarised around inductive themes whereas OPAL applied a Yin [ 8 ] type approach using theoretical propositions around which the case summaries were structured. As the data for each case had been collected through longitudinal interviews, the case summaries were able to capture changes over time. It is beyond the scope of this paper to discuss different analytic techniques; however, to ensure the holistic examination of the intervention(s) in context, it is important to clearly articulate and demonstrate how data is integrated and synthesised [ 31 ].

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation [ 38 ]. Case study can enable comparisons within and across intervention and control arms and enable the evolving relationship between intervention and context to be captured holistically rather than considering processes in isolation. Utilising a longitudinal design can enable the dynamic relationship between context and intervention to be captured in real time. This information is fundamental to holistically explaining what intervention was implemented, understanding how and why the intervention worked or not and informing the transferability of the intervention into routine clinical practice.

Case study designs are not prescriptive, but process evaluations using case study should consider the purpose, trial design, the theories or assumptions underpinning the intervention, and the conceptual and theoretical frameworks informing the evaluation. We have discussed each of these considerations in turn, providing a comprehensive overview of issues for process evaluations using a case study design. There is no single or best way to conduct a process evaluation or a case study, but researchers need to make informed choices about the process evaluation design. Although this paper focuses on process evaluations, we recognise that case study design could also be useful during intervention development and feasibility trials. Elements of this paper are also applicable to other study designs involving trials.

Availability of data and materials

No data and materials were used.

Abbreviations

Data-driven Quality Improvement in Primary Care

Medical Research Council

Nonsteroidal anti-inflammatory drugs

Optimizing Pelvic Floor Muscle Exercises to Achieve Long-term benefits

Blencowe NB. Systematic review of intervention design and delivery in pragmatic and explanatory surgical randomized clinical trials. Br J Surg. 2015;102:1037–47.

Article   CAS   PubMed   Google Scholar  

Dixon-Woods M. The problem of context in quality improvement. In: Foundation TH, editor. Perspectives on context: The Health Foundation; 2014.

Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13(1):95.

Article   PubMed   PubMed Central   Google Scholar  

Grant A, Sullivan F, Dowell J. An ethnographic exploration of influences on prescribing in general practice: why is there variation in prescribing practices? Implement Sci. 2013;8(1):72.

Lang ES, Wyer PC, Haynes RB. Knowledge translation: closing the evidence-to-practice gap. Ann Emerg Med. 2007;49(3):355–63.

Article   PubMed   Google Scholar  

Ward V, House AF, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy. 2009;14(3):156–64.

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

Yin R. Case study research and applications: design and methods. Los Angeles: Sage Publications Inc; 2018.

Google Scholar  

Stake R. The art of case study research. Thousand Oaks, California: Sage Publications Ltd; 1995.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350.

Hawe P. Minimal, negligible and negligent interventions. Soc Sci Med. 2015;138:265–8.

Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, Murphy S. From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation. 2018;25(1):23–45.

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95.

Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, Greaves F, Harper L, Hawe P, Moore L, et al. The need for a complex systems model of evidence for public health. Lancet. 2017;390(10112):2602–4.

Moore G, Cambon L, Michie S, Arwidson P, Ninot G, Ferron C, Potvin L, Kellou N, Charlesworth J, Alla F, et al. Population health intervention research: the place of theories. Trials. 2019;20(1):285.

Kislov R. Engaging with theory: from theoretically informed to theoretically informative improvement research. BMJ Qual Saf. 2019;28(3):177–9.

Boulton R, Sandall J, Sevdalis N. The cultural politics of ‘Implementation Science’. J Med Human. 2020;41(3):379-94. h https://doi.org/10.1007/s10912-020-09607-9 .

Cheng KKF, Metcalfe A. Qualitative methods and process evaluation in clinical trials context: where to head to? Int J Qual Methods. 2018;17(1):1609406918774212.

Article   Google Scholar  

Richards DA, Bazeley P, Borglin G, Craig P, Emsley R, Frost J, Hill J, Horwood J, Hutchings HA, Jinks C, et al. Integrating quantitative and qualitative data and findings when undertaking randomised controlled trials. BMJ Open. 2019;9(11):e032081.

Thomas G. How to do your case study, 2nd edition edn. London: Sage Publications Ltd; 2016.

Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing. BMJ Open. 2017;7(3).

Pfadenhauer L, Rohwer A, Burns J, Booth A, Lysdahl KB, Hofmann B, Gerhardus A, Mozygemba K, Tummers M, Wahlster P, et al. Guidance for the assessment of context and implementation in health technology assessments (HTA) and systematic reviews of complex interventions: the Context and Implementation of Complex Interventions (CICI) framework: Integrate-HTA; 2016.

Bate P, Robert G, Fulop N, Ovretveit J, Dixon-Woods M. Perspectives on context. London: The Health Foundation; 2014.

Ovretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011;20.

Medical Research Council: Process evaluation of complex interventions: UK Medical Research Council (MRC) guidance. 2015.

May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141.

Bate P. Context is everything. In: Perpesctives on Context. The Health Foundation 2014.

Horton TJ, Illingworth JH, Warburton WHP. Overcoming challenges in codifying and replicating complex health care interventions. Health Aff. 2018;37(2):191–7.

O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns. 1998;33:267–79.

Creswell J, Poth C. Qualiative inquiry and research design, fourth edition edn. Thousan Oaks, California: Sage Publications; 2018.

Carolan CM, Forbat L, Smith A. Developing the DESCARTE model: the design of case study research in health care. Qual Health Res. 2016;26(5):626–39.

Takahashi ARW, Araujo L. Case study research: opening up research opportunities. RAUSP Manage J. 2020;55(1):100–11.

Tight M. Understanding case study research, small-scale research with meaning. London: Sage Publications; 2017.

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalisation process theory. Sociology. 2009;43:535.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice. A consolidated framework for advancing implementation science. Implement Sci. 2009;4.

Pawson R, Tilley N. Realist evaluation. London: Sage; 1997.

Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer prescribing - a trial of education, informatics & financial incentives. N Engl J Med. 2016;374:1053–64.

Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement Sci. 2017;12(1):4.

Dreischulte T, Grant A, Hapca A, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: quantitative examination of variation between practices in recruitment, implementation and effectiveness. BMJ Open. 2018;8(1):e017133.

Grant A, Dean S, Hay-Smith J, Hagen S, McClurg D, Taylor A, Kovandzic M, Bugge C. Effectiveness and cost-effectiveness randomised controlled trial of basic versus biofeedback-mediated intensive pelvic floor muscle training for female stress or mixed urinary incontinence: protocol for the OPAL (Optimising Pelvic Floor Exercises to Achieve Long-term benefits) trial mixed methods longitudinal qualitative case study and process evaluation. BMJ Open. 2019;9(2):e024152.

Hagen S, McClurg D, Bugge C, Hay-Smith J, Dean SG, Elders A, Glazener C, Abdel-fattah M, Agur WI, Booth J, et al. Effectiveness and cost-effectiveness of basic versus biofeedback-mediated intensive pelvic floor muscle training for female stress or mixed urinary incontinence: protocol for the OPAL randomised trial. BMJ Open. 2019;9(2):e024153.

Steckler A, Linnan L. Process evaluation for public health interventions and research; 2002.

Durlak JA. Why programme implementation is so important. J Prev Intervent Commun. 1998;17(2):5–18.

Bonell C, Oakley A, Hargreaves J, VS, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333(7563):346–9.

Article   CAS   Google Scholar  

Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15.

Yin R. Case study research: design and methods. London: Sage Publications; 2003.

Bugge C, Hay-Smith J, Grant A, Taylor A, Hagen S, McClurg D, Dean S: A 24 month longitudinal qualitative study of women’s experience of electromyography biofeedback pelvic floor muscle training (PFMT) and PFMT alone for urinary incontinence: adherence, outcome and context. ICS Gothenburg 2019 2019. https://www.ics.org/2019/abstract/473 . Access 10.9.2020.

Suzanne Hagen, Andrew Elders, Susan Stratton, Nicole Sergenson, Carol Bugge, Sarah Dean, Jean Hay-Smith, Mary Kilonzo, Maria Dimitrova, Mohamed Abdel-Fattah, Wael Agur, Jo Booth, Cathryn Glazener, Karen Guerrero, Alison McDonald, John Norrie, Louise R Williams, Doreen McClurg. Effectiveness of pelvic floor muscle training with and without electromyographic biofeedback for urinary incontinence in women: multicentre randomised controlled trial BMJ 2020;371. https://doi.org/10.1136/bmj.m3719 .

Cook TD. Emergent principles for the design, implementation, and analysis of cluster-based experiments in social science. Ann Am Acad Pol Soc Sci. 2005;599(1):176–98.

Hoffmann T, Glasziou P, Boutron I, Milne R, Perera R, Moher D. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348.

Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? Br Med J. 2004;328(7455):1561–3.

Grant A, Dreischulte T, Treweek S, Guthrie B. Study protocol of a mixed-methods evaluation of a cluster randomised trial to improve the safety of NSAID and antiplatelet prescribing: Data-driven Quality Improvement in Primary Care. Trials. 2012;13:154.

Flyvbjerg B. Five misunderstandings about case-study research. Qual Inq. 2006;12(2):219–45.

Thorne S. The great saturation debate: what the “S word” means and doesn’t mean in qualitative research reporting. Can J Nurs Res. 2020;52(1):3–5.

Guest G, Bunce A, Johnson L. How many interviews are enough?: an experiment with data saturation and variability. Field Methods. 2006;18(1):59–82.

Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One. 2020;15(5):e0232076.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.

Rycroft-Malone J. The PARIHS framework: a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;4:297-304.

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):103.

Cresswell JW, Plano Clark VL. Designing and conducting mixed methods research. Thousand Oaks: Sage Publications Ltd; 2007.

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

Craig P, Ruggiero E, Frohlich KL, Mykhalovskiy E, White M. Taking account of context in population health intervention research: guidance for producers, users and funders of research: National Institute for Health Research; 2018. https://www.ncbi.nlm.nih.gov/books/NBK498645/pdf/Bookshelf_NBK498645.pdf .

Download references

Acknowledgements

We would like to thank Professor Shaun Treweek for the discussions about context in trials.

No funding was received for this work.

Author information

Authors and affiliations.

School of Nursing, Midwifery and Paramedic Practice, Robert Gordon University, Garthdee Road, Aberdeen, AB10 7QB, UK

Aileen Grant

Faculty of Health Sciences and Sport, University of Stirling, Pathfoot Building, Stirling, FK9 4LA, UK

Carol Bugge

Department of Surgery and Cancer, Imperial College London, Charing Cross Campus, London, W6 8RP, UK

You can also search for this author in PubMed   Google Scholar

Contributions

AG, CB and MW conceptualised the study. AG wrote the paper. CB and MW commented on the drafts. All authors have approved the final manuscript.

Corresponding author

Correspondence to Aileen Grant .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval and consent to participate is not appropriate as no participants were included.

Consent for publication

Consent for publication is not required as no participants were included.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Grant, A., Bugge, C. & Wells, M. Designing process evaluations using case study to explore the context of complex interventions evaluated in trials. Trials 21 , 982 (2020). https://doi.org/10.1186/s13063-020-04880-4

Download citation

Received : 09 April 2020

Accepted : 06 November 2020

Published : 27 November 2020

DOI : https://doi.org/10.1186/s13063-020-04880-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Process evaluation
• Case study design

ISSN: 1745-6215

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

Prevent plagiarism. Run a free check.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved March 20, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Open access
• Published: 10 November 2020

Case study research for better evaluations of complex interventions: rationale and challenges

• Sara Paparini   ORCID: orcid.org/0000-0002-1909-2481 1 ,
• Judith Green 2 ,
• Chrysanthi Papoutsi 1 ,
• Jamie Murdoch 3 ,
• Mark Petticrew 4 ,
• Trish Greenhalgh 1 ,
• Benjamin Hanckel 5 &
• Sara Shaw 1  

BMC Medicine volume  18 , Article number:  301 ( 2020 ) Cite this article

16k Accesses

39 Citations

35 Altmetric

Metrics details

The need for better methods for evaluation in health research has been widely recognised. The ‘complexity turn’ has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research—currently denigrated as poor evidence—is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear.

Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports.

Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.

Peer Review reports

The need for methodological development to address the most urgent challenges in health research has been well-documented. Many of the most pressing questions for public health research, where the focus is on system-level determinants [ 1 , 2 ], and for health services research, where provisions typically vary across sites and are provided through interlocking networks of services [ 3 ], require methodological approaches that can attend to complexity. The need for methodological advance has arisen, in part, as a result of the diminishing returns from randomised controlled trials (RCTs) where they have been used to answer questions about the effects of interventions in complex systems [ 4 , 5 , 6 ]. In conditions of complexity, there is limited value in maintaining the current orientation to experimental trial designs in the health sciences as providing ‘gold standard’ evidence of effect.

There are increasing calls for methodological pluralism [ 7 , 8 ], with the recognition that complex intervention and context are not easily or usefully separated (as is often the situation when using trial design), and that system interruptions may have effects that are not reducible to linear causal pathways between intervention and outcome. These calls are reflected in a shifting and contested discourse of trial design, seen with the emergence of realist [ 9 ], adaptive and hybrid (types 1, 2 and 3) [ 10 , 11 ] trials that blend studies of effectiveness with a close consideration of the contexts of implementation. Similarly, process evaluation has now become a core component of complex healthcare intervention trials, reflected in MRC guidance on how to explore implementation, causal mechanisms and context [ 12 ].

Evidence about the context of an intervention is crucial for questions of external validity. As Woolcock [ 4 ] notes, even if RCT designs are accepted as robust for maximising internal validity, questions of transferability (how well the intervention works in different contexts) and generalisability (how well the intervention can be scaled up) remain unanswered [ 5 , 13 ]. For research evidence to have impact on policy and systems organisation, and thus to improve population and patient health, there is an urgent need for better methods for strengthening external validity, including a better understanding of the relationship between intervention and context [ 14 ].

Policymakers, healthcare commissioners and other research users require credible evidence of relevance to their settings and populations [ 15 ], to perform what Rosengarten and Savransky [ 16 ] call ‘careful abstraction’ to the locales that matter for them. They also require robust evidence for understanding complex causal pathways. Case study research, currently under-utilised in public health and health services evaluation, can offer considerable potential for strengthening faith in both external and internal validity. For example, in an empirical case study of how the policy of free bus travel had specific health effects in London, UK, a quasi-experimental evaluation (led by JG) identified how important aspects of context (a good public transport system) and intervention (that it was universal) were necessary conditions for the observed effects, thus providing useful, actionable evidence for decision-makers in other contexts [ 17 ].

The overall approach of case study research is based on the in-depth exploration of complex phenomena in their natural, or ‘real-life’, settings. Empirical case studies typically enable dynamic understanding of complex challenges rather than restricting the focus on narrow problem delineations and simple fixes. Case study research is a diverse and somewhat contested field, with multiple definitions and perspectives grounded in different ways of viewing the world, and involving different combinations of methods. In this paper, we raise awareness of such plurality and highlight the contribution that case study research can make to the evaluation of complex system-level interventions. We review some of the challenges in exploiting the current evidence base from empirical case studies and conclude by recommending that further guidance and minimum reporting criteria for evaluation using case studies, appropriate for audiences in the health sciences, can enhance the take-up of evidence from case study research.

Case study research offers evidence about context, causal inference in complex systems and implementation

Well-conducted and described empirical case studies provide evidence on context, complexity and mechanisms for understanding how, where and why interventions have their observed effects. Recognition of the importance of context for understanding the relationships between interventions and outcomes is hardly new. In 1943, Canguilhem berated an over-reliance on experimental designs for determining universal physiological laws: ‘As if one could determine a phenomenon’s essence apart from its conditions! As if conditions were a mask or frame which changed neither the face nor the picture!’ ([ 18 ] p126). More recently, a concern with context has been expressed in health systems and public health research as part of what has been called the ‘complexity turn’ [ 1 ]: a recognition that many of the most enduring challenges for developing an evidence base require a consideration of system-level effects [ 1 ] and the conceptualisation of interventions as interruptions in systems [ 19 ].

The case study approach is widely recognised as offering an invaluable resource for understanding the dynamic and evolving influence of context on complex, system-level interventions [ 20 , 21 , 22 , 23 ]. Empirically, case studies can directly inform assessments of where, when, how and for whom interventions might be successfully implemented, by helping to specify the necessary and sufficient conditions under which interventions might have effects and to consolidate learning on how interdependencies, emergence and unpredictability can be managed to achieve and sustain desired effects. Case study research has the potential to address four objectives for improving research and reporting of context recently set out by guidance on taking account of context in population health research [ 24 ], that is to (1) improve the appropriateness of intervention development for specific contexts, (2) improve understanding of ‘how’ interventions work, (3) better understand how and why impacts vary across contexts and (4) ensure reports of intervention studies are most useful for decision-makers and researchers.

However, evaluations of complex healthcare interventions have arguably not exploited the full potential of case study research and can learn much from other disciplines. For evaluative research, exploratory case studies have had a traditional role of providing data on ‘process’, or initial ‘hypothesis-generating’ scoping, but might also have an increasing salience for explanatory aims. Across the social and political sciences, different kinds of case studies are undertaken to meet diverse aims (description, exploration or explanation) and across different scales (from small N qualitative studies that aim to elucidate processes, or provide thick description, to more systematic techniques designed for medium-to-large N cases).

Case studies with explanatory aims vary in terms of their positioning within mixed-methods projects, with designs including (but not restricted to) (1) single N of 1 studies of interventions in specific contexts, where the overall design is a case study that may incorporate one or more (randomised or not) comparisons over time and between variables within the case; (2) a series of cases conducted or synthesised to provide explanation from variations between cases; and (3) case studies of particular settings within RCT or quasi-experimental designs to explore variation in effects or implementation.

Detailed qualitative research (typically done as ‘case studies’ within process evaluations) provides evidence for the plausibility of mechanisms [ 25 ], offering theoretical generalisations for how interventions may function under different conditions. Although RCT designs reduce many threats to internal validity, the mechanisms of effect remain opaque, particularly when the causal pathways between ‘intervention’ and ‘effect’ are long and potentially non-linear: case study research has a more fundamental role here, in providing detailed observational evidence for causal claims [ 26 ] as well as producing a rich, nuanced picture of tensions and multiple perspectives [ 8 ].

Longitudinal or cross-case analysis may be best suited for evidence generation in system-level evaluative research. Turner [ 27 ], for instance, reflecting on the complex processes in major system change, has argued for the need for methods that integrate learning across cases, to develop theoretical knowledge that would enable inferences beyond the single case, and to develop generalisable theory about organisational and structural change in health systems. Qualitative Comparative Analysis (QCA) [ 28 ] is one such formal method for deriving causal claims, using set theory mathematics to integrate data from empirical case studies to answer questions about the configurations of causal pathways linking conditions to outcomes [ 29 , 30 ].

Nonetheless, the single N case study, too, provides opportunities for theoretical development [ 31 ], and theoretical generalisation or analytical refinement [ 32 ]. How ‘the case’ and ‘context’ are conceptualised is crucial here. Findings from the single case may seem to be confined to its intrinsic particularities in a specific and distinct context [ 33 ]. However, if such context is viewed as exemplifying wider social and political forces, the single case can be ‘telling’, rather than ‘typical’, and offer insight into a wider issue [ 34 ]. Internal comparisons within the case can offer rich possibilities for logical inferences about causation [ 17 ]. Further, case studies of any size can be used for theory testing through refutation [ 22 ]. The potential lies, then, in utilising the strengths and plurality of case study to support theory-driven research within different methodological paradigms.

Evaluation research in health has much to learn from a range of social sciences where case study methodology has been used to develop various kinds of causal inference. For instance, Gerring [ 35 ] expands on the within-case variations utilised to make causal claims. For Gerring [ 35 ], case studies come into their own with regard to invariant or strong causal claims (such as X is a necessary and/or sufficient condition for Y) rather than for probabilistic causal claims. For the latter (where experimental methods might have an advantage in estimating effect sizes), case studies offer evidence on mechanisms: from observations of X affecting Y, from process tracing or from pattern matching. Case studies also support the study of emergent causation, that is, the multiple interacting properties that account for particular and unexpected outcomes in complex systems, such as in healthcare [ 8 ].

Finally, efficacy (or beliefs about efficacy) is not the only contributor to intervention uptake, with a range of organisational and policy contingencies affecting whether an intervention is likely to be rolled out in practice. Case study research is, therefore, invaluable for learning about contextual contingencies and identifying the conditions necessary for interventions to become normalised (i.e. implemented routinely) in practice [ 36 ].

The challenges in exploiting evidence from case study research

At present, there are significant challenges in exploiting the benefits of case study research in evaluative health research, which relate to status, definition and reporting. Case study research has been marginalised at the bottom of an evidence hierarchy, seen to offer little by way of explanatory power, if nonetheless useful for adding descriptive data on process or providing useful illustrations for policymakers [ 37 ]. This is an opportune moment to revisit this low status. As health researchers are increasingly charged with evaluating ‘natural experiments’—the use of face masks in the response to the COVID-19 pandemic being a recent example [ 38 ]—rather than interventions that take place in settings that can be controlled, research approaches using methods to strengthen causal inference that does not require randomisation become more relevant.

A second challenge for improving the use of case study evidence in evaluative health research is that, as we have seen, what is meant by ‘case study’ varies widely, not only across but also within disciplines. There is indeed little consensus amongst methodologists as to how to define ‘a case study’. Definitions focus, variously, on small sample size or lack of control over the intervention (e.g. [ 39 ] p194), on in-depth study and context [ 40 , 41 ], on the logic of inference used [ 35 ] or on distinct research strategies which incorporate a number of methods to address questions of ‘how’ and ‘why’ [ 42 ]. Moreover, definitions developed for specific disciplines do not capture the range of ways in which case study research is carried out across disciplines. Multiple definitions of case study reflect the richness and diversity of the approach. However, evidence suggests that a lack of consensus across methodologists results in some of the limitations of published reports of empirical case studies [ 43 , 44 ]. Hyett and colleagues [ 43 ], for instance, reviewing reports in qualitative journals, found little match between methodological definitions of case study research and how authors used the term.

This raises the third challenge we identify that case study reports are typically not written in ways that are accessible or useful for the evaluation research community and policymakers. Case studies may not appear in journals widely read by those in the health sciences, either because space constraints preclude the reporting of rich, thick descriptions, or because of the reported lack of willingness of some biomedical journals to publish research that uses qualitative methods [ 45 ], signalling the persistence of the aforementioned evidence hierarchy. Where they do, however, the term ‘case study’ is used to indicate, interchangeably, a qualitative study, an N of 1 sample, or a multi-method, in-depth analysis of one example from a population of phenomena. Definitions of what constitutes the ‘case’ are frequently lacking and appear to be used as a synonym for the settings in which the research is conducted. Despite offering insights for evaluation, the primary aims may not have been evaluative, so the implications may not be explicitly drawn out. Indeed, some case study reports might properly be aiming for thick description without necessarily seeking to inform about context or causality.

Acknowledging plurality and developing guidance

We recognise that definitional and methodological plurality is not only inevitable, but also a necessary and creative reflection of the very different epistemological and disciplinary origins of health researchers, and the aims they have in doing and reporting case study research. Indeed, to provide some clarity, Thomas [ 46 ] has suggested a typology of subject/purpose/approach/process for classifying aims (e.g. evaluative or exploratory), sample rationale and selection and methods for data generation of case studies. We also recognise that the diversity of methods used in case study research, and the necessary focus on narrative reporting, does not lend itself to straightforward development of formal quality or reporting criteria.

Existing checklists for reporting case study research from the social sciences—for example Lincoln and Guba’s [ 47 ] and Stake’s [ 33 ]—are primarily orientated to the quality of narrative produced, and the extent to which they encapsulate thick description, rather than the more pragmatic issues of implications for intervention effects. Those designed for clinical settings, such as the CARE (CAse REports) guidelines, provide specific reporting guidelines for medical case reports about single, or small groups of patients [ 48 ], not for case study research.

The Design of Case Study Research in Health Care (DESCARTE) model [ 44 ] suggests a series of questions to be asked of a case study researcher (including clarity about the philosophy underpinning their research), study design (with a focus on case definition) and analysis (to improve process). The model resembles toolkits for enhancing the quality and robustness of qualitative and mixed-methods research reporting, and it is usefully open-ended and non-prescriptive. However, even if it does include some reflections on context, the model does not fully address aspects of context, logic and causal inference that are perhaps most relevant for evaluative research in health.

Hence, for evaluative research where the aim is to report empirical findings in ways that are intended to be pragmatically useful for health policy and practice, this may be an opportune time to consider how to best navigate plurality around what is (minimally) important to report when publishing empirical case studies, especially with regards to the complex relationships between context and interventions, information that case study research is well placed to provide.

The conventional scientific quest for certainty, predictability and linear causality (maximised in RCT designs) has to be augmented by the study of uncertainty, unpredictability and emergent causality [ 8 ] in complex systems. This will require methodological pluralism, and openness to broadening the evidence base to better understand both causality in and the transferability of system change intervention [ 14 , 20 , 23 , 25 ]. Case study research evidence is essential, yet is currently under exploited in the health sciences. If evaluative health research is to move beyond the current impasse on methods for understanding interventions as interruptions in complex systems, we need to consider in more detail how researchers can conduct and report empirical case studies which do aim to elucidate the contextual factors which interact with interventions to produce particular effects. To this end, supported by the UK’s Medical Research Council, we are embracing the challenge to develop guidance for case study researchers studying complex interventions. Following a meta-narrative review of the literature, we are planning a Delphi study to inform guidance that will, at minimum, cover the value of case study research for evaluating the interrelationship between context and complex system-level interventions; for situating and defining ‘the case’, and generalising from case studies; as well as provide specific guidance on conducting, analysing and reporting case study research. Our hope is that such guidance can support researchers evaluating interventions in complex systems to better exploit the diversity and richness of case study research.

Availability of data and materials

Not applicable (article based on existing available academic publications)

Abbreviations

Qualitative comparative analysis

Quasi-experimental design

Randomised controlled trial

Diez Roux AV. Complex systems thinking and current impasses in health disparities research. Am J Public Health. 2011;101(9):1627–34.

Article   Google Scholar  

Ogilvie D, Mitchell R, Mutrie N, M P, Platt S. Evaluating health effects of transport interventions: methodologic case study. Am J Prev Med 2006;31:118–126.

Walshe C. The evaluation of complex interventions in palliative care: an exploration of the potential of case study research strategies. Palliat Med. 2011;25(8):774–81.

Woolcock M. Using case studies to explore the external validity of ‘complex’ development interventions. Evaluation. 2013;19:229–48.

Cartwright N. Are RCTs the gold standard? BioSocieties. 2007;2(1):11–20.

Deaton A, Cartwright N. Understanding and misunderstanding randomized controlled trials. Soc Sci Med. 2018;210:2–21.

Salway S, Green J. Towards a critical complex systems approach to public health. Crit Public Health. 2017;27(5):523–4.

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95.

Bonell C, Warren E, Fletcher A. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials. 2016;17:478.

Pallmann P, Bedding AW, Choodari-Oskooei B. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018;16:29.

Curran G, Bauer M, Mittman B, Pyne J, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. https://doi.org/10.1097/MLR.0b013e3182408812 .

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015 [cited 2020 Jun 27];350. Available from: https://www.bmj.com/content/350/bmj.h1258 .

Evans RE, Craig P, Hoddinott P, Littlecott H, Moore L, Murphy S, et al. When and how do ‘effective’ interventions need to be adapted and/or re-evaluated in new contexts? The need for guidance. J Epidemiol Community Health. 2019;73(6):481–2.

Shoveller J. A critical examination of representations of context within research on population health interventions. Crit Public Health. 2016;26(5):487–500.

Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10(1):37.

Rosengarten M, Savransky M. A careful biomedicine? Generalization and abstraction in RCTs. Crit Public Health. 2019;29(2):181–91.

Green J, Roberts H, Petticrew M, Steinbach R, Goodman A, Jones A, et al. Integrating quasi-experimental and inductive designs in evaluation: a case study of the impact of free bus travel on public health. Evaluation. 2015;21(4):391–406.

Canguilhem G. The normal and the pathological. New York: Zone Books; 1991. (1949).

Google Scholar  

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

King G, Keohane RO, Verba S. Designing social inquiry: scientific inference in qualitative research: Princeton University Press; 1994.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

Yin R. Enhancing the quality of case studies in health services research. Health Serv Res. 1999;34(5 Pt 2):1209.

CAS   PubMed   PubMed Central   Google Scholar  

Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016 [cited 2020 Jun 30];4(16). Available from: https://www.journalslibrary.nihr.ac.uk/hsdr/hsdr04160#/abstract .

Craig P, Di Ruggiero E, Frohlich KL, E M, White M, Group CCGA. Taking account of context in population health intervention research: guidance for producers, users and funders of research. NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.

Grant RL, Hood R. Complex systems, explanation and policy: implications of the crisis of replication for public health research. Crit Public Health. 2017;27(5):525–32.

Mahoney J. Strategies of causal inference in small-N analysis. Sociol Methods Res. 2000;4:387–424.

Turner S. Major system change: a management and organisational research perspective. In: Rosalind Raine, Ray Fitzpatrick, Helen Barratt, Gywn Bevan, Nick Black, Ruth Boaden, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016;4(16) 2016. https://doi.org/10.3310/hsdr04160.

Ragin CC. Using qualitative comparative analysis to study causal complexity. Health Serv Res. 1999;34(5 Pt 2):1225.

Hanckel B, Petticrew M, Thomas J, Green J. Protocol for a systematic review of the use of qualitative comparative analysis for evaluative questions in public health research. Syst Rev. 2019;8(1):252.

Schneider CQ, Wagemann C. Set-theoretic methods for the social sciences: a guide to qualitative comparative analysis: Cambridge University Press; 2012. 369 p.

Flyvbjerg B. Five misunderstandings about case-study research. Qual Inq. 2006;12:219–45.

Tsoukas H. Craving for generality and small-N studies: a Wittgensteinian approach towards the epistemology of the particular in organization and management studies. Sage Handb Organ Res Methods. 2009:285–301.

Stake RE. The art of case study research. London: Sage Publications Ltd; 1995.

Mitchell JC. Typicality and the case study. Ethnographic research: A guide to general conduct. Vol. 238241. 1984.

Gerring J. What is a case study and what is it good for? Am Polit Sci Rev. 2004;98(2):341–54.

May C, Mort M, Williams T, F M, Gask L. Health technology assessment in its local contexts: studies of telehealthcare. Soc Sci Med 2003;57:697–710.

McGill E. Trading quality for relevance: non-health decision-makers’ use of evidence on the social determinants of health. BMJ Open. 2015;5(4):007053.

Greenhalgh T. We can’t be 100% sure face masks work – but that shouldn’t stop us wearing them | Trish Greenhalgh. The Guardian. 2020 [cited 2020 Jun 27]; Available from: https://www.theguardian.com/commentisfree/2020/jun/05/face-masks-coronavirus .

Hammersley M. So, what are case studies? In: What’s wrong with ethnography? New York: Routledge; 1992.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(1):100.

Luck L, Jackson D, Usher K. Case study: a bridge across the paradigms. Nurs Inq. 2006;13(2):103–9.

Yin RK. Case study research and applications: design and methods: Sage; 2017.

Hyett N, A K, Dickson-Swift V. Methodology or method? A critical review of qualitative case study reports. Int J Qual Stud Health Well-Being. 2014;9:23606.

Carolan CM, Forbat L, Smith A. Developing the DESCARTE model: the design of case study research in health care. Qual Health Res. 2016;26(5):626–39.

Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N, Bleakley A, et al. An open letter to the BMJ editors on qualitative research. Bmj. 2016;352.

Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qual Inq. 2011;17(6):511–21.

Lincoln YS, Guba EG. Judging the quality of case study reports. Int J Qual Stud Educ. 1990;3(1):53–9.

Riley DS, Barber MS, Kienle GS, Aronson JK, Schoen-Angerer T, Tugwell P, et al. CARE guidelines for case reports: explanation and elaboration document. J Clin Epidemiol. 2017;89:218–35.

Download references

Acknowledgements

Not applicable

This work was funded by the Medical Research Council - MRC Award MR/S014632/1 HCS: Case study, Context and Complex interventions (TRIPLE C). SP was additionally funded by the University of Oxford's Higher Education Innovation Fund (HEIF).

Author information

Authors and affiliations.

Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Sara Paparini, Chrysanthi Papoutsi, Trish Greenhalgh & Sara Shaw

Wellcome Centre for Cultures & Environments of Health, University of Exeter, Exeter, UK

Judith Green

School of Health Sciences, University of East Anglia, Norwich, UK

Jamie Murdoch

Public Health, Environments and Society, London School of Hygiene & Tropical Medicin, London, UK

Mark Petticrew

Institute for Culture and Society, Western Sydney University, Penrith, Australia

Benjamin Hanckel

You can also search for this author in PubMed   Google Scholar

Contributions

JG, MP, SP, JM, TG, CP and SS drafted the initial paper; all authors contributed to the drafting of the final version, and read and approved the final manuscript.

Corresponding author

Correspondence to Sara Paparini .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Paparini, S., Green, J., Papoutsi, C. et al. Case study research for better evaluations of complex interventions: rationale and challenges. BMC Med 18 , 301 (2020). https://doi.org/10.1186/s12916-020-01777-6

Download citation

Received : 03 July 2020

Accepted : 07 September 2020

Published : 10 November 2020

DOI : https://doi.org/10.1186/s12916-020-01777-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative
• Case studies
• Mixed-method
• Public health
• Health services research
• Interventions

BMC Medicine

ISSN: 1741-7015

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

A case study research paper examines a person, place, event, condition, phenomenon, or other type of subject of analysis in order to extrapolate  key themes and results that help predict future trends, illuminate previously hidden issues that can be applied to practice, and/or provide a means for understanding an important research problem with greater clarity. A case study research paper usually examines a single subject of analysis, but case study papers can also be designed as a comparative investigation that shows relationships between two or more subjects. The methods used to study a case can rest within a quantitative, qualitative, or mixed-method investigative paradigm.

Case Studies. Writing@CSU. Colorado State University; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010 ; “What is a Case Study?” In Swanborn, Peter G. Case Study Research: What, Why and How? London: SAGE, 2010.

How to Approach Writing a Case Study Research Paper

General information about how to choose a topic to investigate can be found under the " Choosing a Research Problem " tab in the Organizing Your Social Sciences Research Paper writing guide. Review this page because it may help you identify a subject of analysis that can be investigated using a case study design.

However, identifying a case to investigate involves more than choosing the research problem . A case study encompasses a problem contextualized around the application of in-depth analysis, interpretation, and discussion, often resulting in specific recommendations for action or for improving existing conditions. As Seawright and Gerring note, practical considerations such as time and access to information can influence case selection, but these issues should not be the sole factors used in describing the methodological justification for identifying a particular case to study. Given this, selecting a case includes considering the following:

• The case represents an unusual or atypical example of a research problem that requires more in-depth analysis? Cases often represent a topic that rests on the fringes of prior investigations because the case may provide new ways of understanding the research problem. For example, if the research problem is to identify strategies to improve policies that support girl's access to secondary education in predominantly Muslim nations, you could consider using Azerbaijan as a case study rather than selecting a more obvious nation in the Middle East. Doing so may reveal important new insights into recommending how governments in other predominantly Muslim nations can formulate policies that support improved access to education for girls.
• The case provides important insight or illuminate a previously hidden problem? In-depth analysis of a case can be based on the hypothesis that the case study will reveal trends or issues that have not been exposed in prior research or will reveal new and important implications for practice. For example, anecdotal evidence may suggest drug use among homeless veterans is related to their patterns of travel throughout the day. Assuming prior studies have not looked at individual travel choices as a way to study access to illicit drug use, a case study that observes a homeless veteran could reveal how issues of personal mobility choices facilitate regular access to illicit drugs. Note that it is important to conduct a thorough literature review to ensure that your assumption about the need to reveal new insights or previously hidden problems is valid and evidence-based.
• The case challenges and offers a counter-point to prevailing assumptions? Over time, research on any given topic can fall into a trap of developing assumptions based on outdated studies that are still applied to new or changing conditions or the idea that something should simply be accepted as "common sense," even though the issue has not been thoroughly tested in current practice. A case study analysis may offer an opportunity to gather evidence that challenges prevailing assumptions about a research problem and provide a new set of recommendations applied to practice that have not been tested previously. For example, perhaps there has been a long practice among scholars to apply a particular theory in explaining the relationship between two subjects of analysis. Your case could challenge this assumption by applying an innovative theoretical framework [perhaps borrowed from another discipline] to explore whether this approach offers new ways of understanding the research problem. Taking a contrarian stance is one of the most important ways that new knowledge and understanding develops from existing literature.
• The case provides an opportunity to pursue action leading to the resolution of a problem? Another way to think about choosing a case to study is to consider how the results from investigating a particular case may result in findings that reveal ways in which to resolve an existing or emerging problem. For example, studying the case of an unforeseen incident, such as a fatal accident at a railroad crossing, can reveal hidden issues that could be applied to preventative measures that contribute to reducing the chance of accidents in the future. In this example, a case study investigating the accident could lead to a better understanding of where to strategically locate additional signals at other railroad crossings so as to better warn drivers of an approaching train, particularly when visibility is hindered by heavy rain, fog, or at night.
• The case offers a new direction in future research? A case study can be used as a tool for an exploratory investigation that highlights the need for further research about the problem. A case can be used when there are few studies that help predict an outcome or that establish a clear understanding about how best to proceed in addressing a problem. For example, after conducting a thorough literature review [very important!], you discover that little research exists showing the ways in which women contribute to promoting water conservation in rural communities of east central Africa. A case study of how women contribute to saving water in a rural village of Uganda can lay the foundation for understanding the need for more thorough research that documents how women in their roles as cooks and family caregivers think about water as a valuable resource within their community. This example of a case study could also point to the need for scholars to build new theoretical frameworks around the topic [e.g., applying feminist theories of work and family to the issue of water conservation].

Eisenhardt, Kathleen M. “Building Theories from Case Study Research.” Academy of Management Review 14 (October 1989): 532-550; Emmel, Nick. Sampling and Choosing Cases in Qualitative Research: A Realist Approach . Thousand Oaks, CA: SAGE Publications, 2013; Gerring, John. “What Is a Case Study and What Is It Good for?” American Political Science Review 98 (May 2004): 341-354; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Seawright, Jason and John Gerring. "Case Selection Techniques in Case Study Research." Political Research Quarterly 61 (June 2008): 294-308.

Structure and Writing Style

The purpose of a paper in the social sciences designed around a case study is to thoroughly investigate a subject of analysis in order to reveal a new understanding about the research problem and, in so doing, contributing new knowledge to what is already known from previous studies. In applied social sciences disciplines [e.g., education, social work, public administration, etc.], case studies may also be used to reveal best practices, highlight key programs, or investigate interesting aspects of professional work.

In general, the structure of a case study research paper is not all that different from a standard college-level research paper. However, there are subtle differences you should be aware of. Here are the key elements to organizing and writing a case study research paper.

I.  Introduction

As with any research paper, your introduction should serve as a roadmap for your readers to ascertain the scope and purpose of your study . The introduction to a case study research paper, however, should not only describe the research problem and its significance, but you should also succinctly describe why the case is being used and how it relates to addressing the problem. The two elements should be linked. With this in mind, a good introduction answers these four questions:

• What is being studied? Describe the research problem and describe the subject of analysis [the case] you have chosen to address the problem. Explain how they are linked and what elements of the case will help to expand knowledge and understanding about the problem.
• Why is this topic important to investigate? Describe the significance of the research problem and state why a case study design and the subject of analysis that the paper is designed around is appropriate in addressing the problem.
• What did we know about this topic before I did this study? Provide background that helps lead the reader into the more in-depth literature review to follow. If applicable, summarize prior case study research applied to the research problem and why it fails to adequately address the problem. Describe why your case will be useful. If no prior case studies have been used to address the research problem, explain why you have selected this subject of analysis.
• How will this study advance new knowledge or new ways of understanding? Explain why your case study will be suitable in helping to expand knowledge and understanding about the research problem.

Each of these questions should be addressed in no more than a few paragraphs. Exceptions to this can be when you are addressing a complex research problem or subject of analysis that requires more in-depth background information.

II.  Literature Review

The literature review for a case study research paper is generally structured the same as it is for any college-level research paper. The difference, however, is that the literature review is focused on providing background information and  enabling historical interpretation of the subject of analysis in relation to the research problem the case is intended to address . This includes synthesizing studies that help to:

• Place relevant works in the context of their contribution to understanding the case study being investigated . This would involve summarizing studies that have used a similar subject of analysis to investigate the research problem. If there is literature using the same or a very similar case to study, you need to explain why duplicating past research is important [e.g., conditions have changed; prior studies were conducted long ago, etc.].
• Describe the relationship each work has to the others under consideration that informs the reader why this case is applicable . Your literature review should include a description of any works that support using the case to investigate the research problem and the underlying research questions.
• Identify new ways to interpret prior research using the case study . If applicable, review any research that has examined the research problem using a different research design. Explain how your use of a case study design may reveal new knowledge or a new perspective or that can redirect research in an important new direction.
• Resolve conflicts amongst seemingly contradictory previous studies . This refers to synthesizing any literature that points to unresolved issues of concern about the research problem and describing how the subject of analysis that forms the case study can help resolve these existing contradictions.
• Point the way in fulfilling a need for additional research . Your review should examine any literature that lays a foundation for understanding why your case study design and the subject of analysis around which you have designed your study may reveal a new way of approaching the research problem or offer a perspective that points to the need for additional research.
• Expose any gaps that exist in the literature that the case study could help to fill . Summarize any literature that not only shows how your subject of analysis contributes to understanding the research problem, but how your case contributes to a new way of understanding the problem that prior research has failed to do.
• Locate your own research within the context of existing literature [very important!] . Collectively, your literature review should always place your case study within the larger domain of prior research about the problem. The overarching purpose of reviewing pertinent literature in a case study paper is to demonstrate that you have thoroughly identified and synthesized prior studies in relation to explaining the relevance of the case in addressing the research problem.

III.  Method

In this section, you explain why you selected a particular case [i.e., subject of analysis] and the strategy you used to identify and ultimately decide that your case was appropriate in addressing the research problem. The way you describe the methods used varies depending on the type of subject of analysis that constitutes your case study.

If your subject of analysis is an incident or event . In the social and behavioral sciences, the event or incident that represents the case to be studied is usually bounded by time and place, with a clear beginning and end and with an identifiable location or position relative to its surroundings. The subject of analysis can be a rare or critical event or it can focus on a typical or regular event. The purpose of studying a rare event is to illuminate new ways of thinking about the broader research problem or to test a hypothesis. Critical incident case studies must describe the method by which you identified the event and explain the process by which you determined the validity of this case to inform broader perspectives about the research problem or to reveal new findings. However, the event does not have to be a rare or uniquely significant to support new thinking about the research problem or to challenge an existing hypothesis. For example, Walo, Bull, and Breen conducted a case study to identify and evaluate the direct and indirect economic benefits and costs of a local sports event in the City of Lismore, New South Wales, Australia. The purpose of their study was to provide new insights from measuring the impact of a typical local sports event that prior studies could not measure well because they focused on large "mega-events." Whether the event is rare or not, the methods section should include an explanation of the following characteristics of the event: a) when did it take place; b) what were the underlying circumstances leading to the event; and, c) what were the consequences of the event in relation to the research problem.

If your subject of analysis is a person. Explain why you selected this particular individual to be studied and describe what experiences they have had that provide an opportunity to advance new understandings about the research problem. Mention any background about this person which might help the reader understand the significance of their experiences that make them worthy of study. This includes describing the relationships this person has had with other people, institutions, and/or events that support using them as the subject for a case study research paper. It is particularly important to differentiate the person as the subject of analysis from others and to succinctly explain how the person relates to examining the research problem [e.g., why is one politician in a particular local election used to show an increase in voter turnout from any other candidate running in the election]. Note that these issues apply to a specific group of people used as a case study unit of analysis [e.g., a classroom of students].

If your subject of analysis is a place. In general, a case study that investigates a place suggests a subject of analysis that is unique or special in some way and that this uniqueness can be used to build new understanding or knowledge about the research problem. A case study of a place must not only describe its various attributes relevant to the research problem [e.g., physical, social, historical, cultural, economic, political], but you must state the method by which you determined that this place will illuminate new understandings about the research problem. It is also important to articulate why a particular place as the case for study is being used if similar places also exist [i.e., if you are studying patterns of homeless encampments of veterans in open spaces, explain why you are studying Echo Park in Los Angeles rather than Griffith Park?]. If applicable, describe what type of human activity involving this place makes it a good choice to study [e.g., prior research suggests Echo Park has more homeless veterans].

If your subject of analysis is a phenomenon. A phenomenon refers to a fact, occurrence, or circumstance that can be studied or observed but with the cause or explanation to be in question. In this sense, a phenomenon that forms your subject of analysis can encompass anything that can be observed or presumed to exist but is not fully understood. In the social and behavioral sciences, the case usually focuses on human interaction within a complex physical, social, economic, cultural, or political system. For example, the phenomenon could be the observation that many vehicles used by ISIS fighters are small trucks with English language advertisements on them. The research problem could be that ISIS fighters are difficult to combat because they are highly mobile. The research questions could be how and by what means are these vehicles used by ISIS being supplied to the militants and how might supply lines to these vehicles be cut off? How might knowing the suppliers of these trucks reveal larger networks of collaborators and financial support? A case study of a phenomenon most often encompasses an in-depth analysis of a cause and effect that is grounded in an interactive relationship between people and their environment in some way.

NOTE:   The choice of the case or set of cases to study cannot appear random. Evidence that supports the method by which you identified and chose your subject of analysis should clearly support investigation of the research problem and linked to key findings from your literature review. Be sure to cite any studies that helped you determine that the case you chose was appropriate for examining the problem.

IV.  Discussion

The main elements of your discussion section are generally the same as any research paper, but centered around interpreting and drawing conclusions about the key findings from your analysis of the case study. Note that a general social sciences research paper may contain a separate section to report findings. However, in a paper designed around a case study, it is common to combine a description of the results with the discussion about their implications. The objectives of your discussion section should include the following:

Reiterate the Research Problem/State the Major Findings Briefly reiterate the research problem you are investigating and explain why the subject of analysis around which you designed the case study were used. You should then describe the findings revealed from your study of the case using direct, declarative, and succinct proclamation of the study results. Highlight any findings that were unexpected or especially profound.

Explain the Meaning of the Findings and Why They are Important Systematically explain the meaning of your case study findings and why you believe they are important. Begin this part of the section by repeating what you consider to be your most important or surprising finding first, then systematically review each finding. Be sure to thoroughly extrapolate what your analysis of the case can tell the reader about situations or conditions beyond the actual case that was studied while, at the same time, being careful not to misconstrue or conflate a finding that undermines the external validity of your conclusions.

Relate the Findings to Similar Studies No study in the social sciences is so novel or possesses such a restricted focus that it has absolutely no relation to previously published research. The discussion section should relate your case study results to those found in other studies, particularly if questions raised from prior studies served as the motivation for choosing your subject of analysis. This is important because comparing and contrasting the findings of other studies helps support the overall importance of your results and it highlights how and in what ways your case study design and the subject of analysis differs from prior research about the topic.

Consider Alternative Explanations of the Findings Remember that the purpose of social science research is to discover and not to prove. When writing the discussion section, you should carefully consider all possible explanations revealed by the case study results, rather than just those that fit your hypothesis or prior assumptions and biases. Be alert to what the in-depth analysis of the case may reveal about the research problem, including offering a contrarian perspective to what scholars have stated in prior research if that is how the findings can be interpreted from your case.

Acknowledge the Study's Limitations You can state the study's limitations in the conclusion section of your paper but describing the limitations of your subject of analysis in the discussion section provides an opportunity to identify the limitations and explain why they are not significant. This part of the discussion section should also note any unanswered questions or issues your case study could not address. More detailed information about how to document any limitations to your research can be found here .

Suggest Areas for Further Research Although your case study may offer important insights about the research problem, there are likely additional questions related to the problem that remain unanswered or findings that unexpectedly revealed themselves as a result of your in-depth analysis of the case. Be sure that the recommendations for further research are linked to the research problem and that you explain why your recommendations are valid in other contexts and based on the original assumptions of your study.

V.  Conclusion

As with any research paper, you should summarize your conclusion in clear, simple language; emphasize how the findings from your case study differs from or supports prior research and why. Do not simply reiterate the discussion section. Provide a synthesis of key findings presented in the paper to show how these converge to address the research problem. If you haven't already done so in the discussion section, be sure to document the limitations of your case study and any need for further research.

The function of your paper's conclusion is to: 1) reiterate the main argument supported by the findings from your case study; 2) state clearly the context, background, and necessity of pursuing the research problem using a case study design in relation to an issue, controversy, or a gap found from reviewing the literature; and, 3) provide a place to persuasively and succinctly restate the significance of your research problem, given that the reader has now been presented with in-depth information about the topic.

Consider the following points to help ensure your conclusion is appropriate:

• If the argument or purpose of your paper is complex, you may need to summarize these points for your reader.
• If prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the conclusion of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration of the case study's findings that returns the topic to the context provided by the introduction or within a new context that emerges from your case study findings.

Note that, depending on the discipline you are writing in or the preferences of your professor, the concluding paragraph may contain your final reflections on the evidence presented as it applies to practice or on the essay's central research problem. However, the nature of being introspective about the subject of analysis you have investigated will depend on whether you are explicitly asked to express your observations in this way.

Problems to Avoid

Overgeneralization One of the goals of a case study is to lay a foundation for understanding broader trends and issues applied to similar circumstances. However, be careful when drawing conclusions from your case study. They must be evidence-based and grounded in the results of the study; otherwise, it is merely speculation. Looking at a prior example, it would be incorrect to state that a factor in improving girls access to education in Azerbaijan and the policy implications this may have for improving access in other Muslim nations is due to girls access to social media if there is no documentary evidence from your case study to indicate this. There may be anecdotal evidence that retention rates were better for girls who were engaged with social media, but this observation would only point to the need for further research and would not be a definitive finding if this was not a part of your original research agenda.

Failure to Document Limitations No case is going to reveal all that needs to be understood about a research problem. Therefore, just as you have to clearly state the limitations of a general research study , you must describe the specific limitations inherent in the subject of analysis. For example, the case of studying how women conceptualize the need for water conservation in a village in Uganda could have limited application in other cultural contexts or in areas where fresh water from rivers or lakes is plentiful and, therefore, conservation is understood more in terms of managing access rather than preserving access to a scarce resource.

Failure to Extrapolate All Possible Implications Just as you don't want to over-generalize from your case study findings, you also have to be thorough in the consideration of all possible outcomes or recommendations derived from your findings. If you do not, your reader may question the validity of your analysis, particularly if you failed to document an obvious outcome from your case study research. For example, in the case of studying the accident at the railroad crossing to evaluate where and what types of warning signals should be located, you failed to take into consideration speed limit signage as well as warning signals. When designing your case study, be sure you have thoroughly addressed all aspects of the problem and do not leave gaps in your analysis that leave the reader questioning the results.

Case Studies. Writing@CSU. Colorado State University; Gerring, John. Case Study Research: Principles and Practices . New York: Cambridge University Press, 2007; Merriam, Sharan B. Qualitative Research and Case Study Applications in Education . Rev. ed. San Francisco, CA: Jossey-Bass, 1998; Miller, Lisa L. “The Use of Case Studies in Law and Social Science Research.” Annual Review of Law and Social Science 14 (2018): TBD; Mills, Albert J., Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Putney, LeAnn Grogan. "Case Study." In Encyclopedia of Research Design , Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE Publications, 2010), pp. 116-120; Simons, Helen. Case Study Research in Practice . London: SAGE Publications, 2009;  Kratochwill,  Thomas R. and Joel R. Levin, editors. Single-Case Research Design and Analysis: New Development for Psychology and Education .  Hilldsale, NJ: Lawrence Erlbaum Associates, 1992; Swanborn, Peter G. Case Study Research: What, Why and How? London : SAGE, 2010; Yin, Robert K. Case Study Research: Design and Methods . 6th edition. Los Angeles, CA, SAGE Publications, 2014; Walo, Maree, Adrian Bull, and Helen Breen. “Achieving Economic Benefits at Local Events: A Case Study of a Local Sports Event.” Festival Management and Event Tourism 4 (1996): 95-106.

Writing Tip

At Least Five Misconceptions about Case Study Research

Social science case studies are often perceived as limited in their ability to create new knowledge because they are not randomly selected and findings cannot be generalized to larger populations. Flyvbjerg examines five misunderstandings about case study research and systematically "corrects" each one. To quote, these are:

Misunderstanding 1 :  General, theoretical [context-independent] knowledge is more valuable than concrete, practical [context-dependent] knowledge. Misunderstanding 2 :  One cannot generalize on the basis of an individual case; therefore, the case study cannot contribute to scientific development. Misunderstanding 3 :  The case study is most useful for generating hypotheses; that is, in the first stage of a total research process, whereas other methods are more suitable for hypotheses testing and theory building. Misunderstanding 4 :  The case study contains a bias toward verification, that is, a tendency to confirm the researcher’s preconceived notions. Misunderstanding 5 :  It is often difficult to summarize and develop general propositions and theories on the basis of specific case studies [p. 221].

While writing your paper, think introspectively about how you addressed these misconceptions because to do so can help you strengthen the validity and reliability of your research by clarifying issues of case selection, the testing and challenging of existing assumptions, the interpretation of key findings, and the summation of case outcomes. Think of a case study research paper as a complete, in-depth narrative about the specific properties and key characteristics of your subject of analysis applied to the research problem.

Flyvbjerg, Bent. “Five Misunderstandings About Case-Study Research.” Qualitative Inquiry 12 (April 2006): 219-245.

• << Previous: Writing a Case Analysis Paper
• Next: Writing a Field Report >>
• Last Updated: Mar 6, 2024 1:00 PM
• URL: https://libguides.usc.edu/writingguide/assignments

15.7 Evaluation: Presentation and Analysis of Case Study

Learning outcomes.

By the end of this section, you will be able to:

• Revise writing to follow the genre conventions of case studies.
• Evaluate the effectiveness and quality of a case study report.

Case studies follow a structure of background and context , methods , findings , and analysis . Body paragraphs should have main points and concrete details. In addition, case studies are written in formal language with precise wording and with a specific purpose and audience (generally other professionals in the field) in mind. Case studies also adhere to the conventions of the discipline’s formatting guide ( APA Documentation and Format in this study). Compare your case study with the following rubric as a final check.

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/writing-guide/pages/1-unit-introduction

• Authors: Michelle Bachelor Robinson, Maria Jerskey, featuring Toby Fulwiler
• Publisher/website: OpenStax
• Book title: Writing Guide with Handbook
• Publication date: Dec 21, 2021
• Location: Houston, Texas
• Book URL: https://openstax.org/books/writing-guide/pages/1-unit-introduction
• Section URL: https://openstax.org/books/writing-guide/pages/15-7-evaluation-presentation-and-analysis-of-case-study

© Dec 19, 2023 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

Case Study Evaluation: Past, Present and Future Challenges: Volume 15

Table of contents, case study evaluation: past, present and future challenges, advances in program evaluation, copyright page, list of contributors, introduction, case study, methodology and educational evaluation: a personal view.

This chapter gives one version of the recent history of evaluation case study. It looks back over the emergence of case study as a sociological method, developed in the early years of the 20th Century and celebrated and elaborated by the Chicago School of urban sociology at Chicago University, starting throughout the 1920s and 1930s. Some of the basic methods, including constant comparison, were generated at that time. Only partly influenced by this methodological movement, an alliance between an Illinois-based team in the United States and a team at the University of East Anglia in the United Kingdom recast the case method as a key tool for the evaluation of social and educational programmes.

Letters from a Headmaster ☆ Originally published in Simons, H. (Ed.) (1980). Towards a Science of the Singular: Essays about Case Study in Educational Research and Evaluation. Occasional Papers No. 10. Norwich, UK: Centre for Applied Research, University of East Anglia.

Story telling and educational understanding ☆ previously published in occasional papers #12, evaluation centre, university of western michigan, 1978..

The full ‘storytelling’ paper was written in 1978 and was influential in its time. It is reprinted here, introduced by an Author's reflection on it in 2014. The chapter describes the author’s early disenchantment with traditional approaches to educational research.

He regards educational research as, at best, a misnomer, since little of it is preceded by a search . Entitled educational researchers often fancy themselves as scientists at work. But those whom they attempt to describe are often artists at work. Statistical methodologies enable educational researchers to measure something, but their measurements can neither capture nor explain splendid teaching.

Since such a tiny fraction of what is published in educational research journals influences school practitioners, professional researchers should risk trying alternative approaches to uncovering what is going on in schools.

Story telling is posited as a possible key to producing insights that inform and ultimately improve educational practice. It advocates openness to broad inquiry into the culture of the educational setting.

Case Study as Antidote to the Literal

Much programme and policy evaluation yields to the pressure to report on the productivity of programmes and is perforce compliant with the conditions of contract. Too often the view of these evaluations is limited to a literal reading of the analytical challenge. If we are evaluating X we look critically at X1, X2 and X3. There might be cause for embracing adjoining data sources such as W1 and Y1. This ignores frequent realities that an evaluation specification is only an approximate starting point for an unpredictable journey into comprehensive understanding; that the specification represents only that which is wanted by the sponsor, and not all that may be needed ; and that the contractual specification too often insists on privileging the questions and concerns of a few. Case study evaluation proves an alternative that allows for the less-than-literal in the form of analysis of contingencies – how people, phenomena and events may be related in dynamic ways, how context and action have only a blurred dividing line and how what defines the case as a case may only emerge late in the study.

Thinking about Case Studies in 3-D: Researching the NHS Clinical Commissioning Landscape in England

What is our unit of analysis and by implication what are the boundaries of our cases? This is a question we grapple with at the start of every new project. We observe that case studies are often referred to in an unreflective manner and are often conflated with geographical location. Neat units of analysis and clearly bounded cases usually do not reflect the messiness encountered during qualitative fieldwork. Others have puzzled over these questions. We briefly discuss work to problematise the use of households as units of analysis in the context of apartheid South Africa and then consider work of other anthropologists engaged in multi-site ethnography. We have found the notion of ‘following’ chains, paths and threads across sites to be particularly insightful.

We present two examples from our work studying commissioning in the English National Health Service (NHS) to illustrate our struggles with case studies. The first is a study of Practice-based Commissioning groups and the second is a study of the early workings of Clinical Commissioning Groups. In both instances we show how ideas of what constituted our unit of analysis and the boundaries of our cases became less clear as our research progressed. We also discuss pressures we experienced to add more case studies to our projects. These examples illustrate the primacy for us of understanding interactions between place, local history and rapidly developing policy initiatives. Understanding cases in this way can be challenging in a context where research funders hold different views of what constitutes a case.

The Case for Evaluating Process and Worth: Evaluation of a Programme for Carers and People with Dementia

A case study methodology was applied as a major component of a mixed-methods approach to the evaluation of a mobile dementia education and support service in the Bega Valley Shire, New South Wales, Australia. In-depth interviews with people with dementia (PWD), their carers, programme staff, family members and service providers and document analysis including analysis of client case notes and client database were used.

The strengths of the case study approach included: (i) simultaneous evaluation of programme process and worth, (ii) eliciting the theory of change and addressing the problem of attribution, (iii) demonstrating the impact of the programme on earlier steps identified along the causal pathway (iv) understanding the complexity of confounding factors, (v) eliciting the critical role of the social, cultural and political context, (vi) understanding the importance of influences contributing to differences in programme impact for different participants and (vii) providing insight into how programme participants experience the value of the programme including unintended benefits.

The broader case of the collective experience of dementia and as part of this experience, the impact of a mobile programme of support and education, in a predominately rural area grew from the investigation of the programme experience of ‘individual cases’ of carers and PWD. Investigation of living conditions, relationships, service interactions through observation and increased depth of interviews with service providers and family members would have provided valuable perspectives and thicker description of the case for increased understanding of the case and strength of the evaluation.

The Collapse of “Primary Care” in Medical Education: A Case Study of Michigan’s Community/University Health Partnerships Project

This chapter describes a case study of a social change project in medical education (primary care), in which the critical interpretive evaluation methodology I sought to use came up against the “positivist” approach preferred by senior figures in the medical school who commissioned the evaluation.

I describe the background to the study and justify the evaluation approach and methods employed in the case study – drawing on interviews, document analysis, survey research, participant observation, literature reviews, and critical incidents – one of which was the decision by the medical school hierarchy to restrict my contact with the lay community in my official evaluation duties. The use of critical ethnography also embraced wider questions about circuits of power and the social and political contexts within which the “social change” effort occurred.

Central to my analysis is John Gaventa’s theory of power as “the internalization of values that inhibit consciousness and participation while encouraging powerlessness and dependency.” Gaventa argued, essentially, that the evocation of power has as much to do with preventing decisions as with bringing them about. My chosen case illustrated all three dimensions of power that Gaventa originally uncovered in his portrait of self-interested Appalachian coal mine owners: (1) communities were largely excluded from decision making power; (2) issues were avoided or suppressed; and (3) the interests of the oppressed went largely unrecognized.

The account is auto-ethnographic, hence the study is limited by my abilities, biases, and subject positions. I reflect on these in the chapter.

The study not only illustrates the unique contribution of case study as a research methodology but also its low status in the positivist paradigm adhered to by many doctors. Indeed, the tension between the potential of case study to illuminate the complexities of community engagement through thick description and the rejection of this very method as inherently “flawed” suggests that medical education may be doomed to its neoliberal fate for some time to come.

‘Lead’ Standard Evaluation

This is a personal narrative, but I trust not a self-regarding one. For more years than I care to remember I have been working in the field of curriculum (or ‘program’) evaluation. The field by any standards is dispersed and fragmented, with variously ascribed purposes, roles, implicit values, political contexts, and social research methods. Attempts to organize this territory into an ‘evaluation theory tree’ (e.g. Alkin, M., & Christie, C. (2003). An evaluation theory tree. In M. Alkin (Ed.), Evaluation roots: Tracing theorists’ views and influences (pp. 12–65). Thousand Oaks, CA: Sage) have identified broad types or ‘branches’, but the migration of specific characteristics (like ‘case study’) or individual practitioners across the boundaries has tended to undermine the analysis at the level of detail, and there is no suggestion that it represents a cladistic taxonomy. There is, however, general agreement that the roots of evaluation practice tap into a variety of cultural sources, being grounded bureaucratically in (potentially conflicting) doctrines of accountability and methodologically in discipline-based or pragmatically eclectic formats for systematic social enquiry.

In general, this diversity is not treated as problematic. The professional evaluation community has increasingly taken the view (‘let all the flowers grow’) that evaluation models can be deemed appropriate across a wide spectrum, with their appropriateness determined by the nature of the task and its context, including in relation to hybrid studies using mixed models or displaying what Geertz (Geertz, C. (1980/1993). Blurred genres: The refiguration of social thought. The American Scholar , 49(2), 165–179) called ‘blurred genres’. However, from time to time historic tribal rivalries re-emerge as particular practitioners feel the need to defend their modus operandi (and thereby their livelihood) against paradigm shifts or governments and other sponsors of program evaluation seeking for ideological reasons to prioritize certain types of study at the expense of others. The latter possibility poses a potential threat that needs to be taken seriously by evaluators within the broad tradition showcased in this volume, interpretive qualitative case studies of educational programs that combine naturalistic description (often ‘thick’; Geertz, C. (1973). Thick description: Towards an interpretive theory of culture. In The interpretation of culture (pp. 3–30). New York, NY: Basic Books.) description with a values-orientated analysis of their implications. Such studies are more likely to seek inspiration from anthropology or critical discourse analysis than from the randomly controlled trials familiar in medical research or laboratory practice in the physical sciences, despite the impressive rigour of the latter in appropriate contexts. It is the risk of ideological allegiance that I address in this chapter.

Freedom from the Rubric

Twice-told tales how public inquiry could inform n of 1 case study research.

This chapter considers the usefulness and validity of public inquiries as a source of data and preliminary interpretation for case study research. Using two contrasting examples – the Bristol Inquiry into excess deaths in a children’s cardiac surgery unit and the Woolf Inquiry into a breakdown of governance at the London School of Economics (LSE) – I show how academics can draw fruitfully on, and develop further analysis from, the raw datasets, published summaries and formal judgements of public inquiries.

Academic analysis of public inquiries can take two broad forms, corresponding to the two main approaches to individual case study defined by Stake: instrumental (selecting the public inquiry on the basis of pre-defined theoretical features and using the material to develop and test theoretical propositions) and intrinsic (selecting the public inquiry on the basis of the particular topic addressed and using the material to explore questions about what was going on and why).

The advantages of a public inquiry as a data source for case study research typically include a clear and uncontested focus of inquiry; the breadth and richness of the dataset collected; the exceptional level of support available for the tasks of transcribing, indexing, collating, summarising and so on; and the expert interpretations and insights of the inquiry’s chair (with which the researcher may or may not agree). A significant disadvantage is that whilst the dataset collected for a public inquiry is typically ‘rich’, it has usually been collected under far from ideal research conditions. Hence, while public inquiries provide a potentially rich resource for researchers, those who seek to use public inquiry data for research must justify their choice on both ethical and scientific grounds.

Evaluation as the Co-Construction of Knowledge: Case Studies of Place-Based Leadership and Public Service Innovation

This chapter introduces the notion of the ‘Innovation Story’ as a methodological approach to public policy evaluation, which builds in greater opportunity for learning and reflexivity.

The Innovation Story is an adaptation of the case study approach and draws on participatory action research traditions. It is a structured narrative that describes a particular public policy innovation in the personalised contexts in which it is experienced by innovators. Its construction involves a discursive process through which involved actors tell their story, explain it to others, listen to their questions and co-construct knowledge of change together.

The approach was employed to elaborate five case studies of place-based leadership and public service innovation in the United Kingdom, The Netherlands and Mexico. The key findings are that spaces in which civic leaders come together from different ‘realms’ of leadership in a locality (community, business, professional managers and political leaders) can become innovation zones that foster inventive behaviour. Much depends on the quality of civic leadership, and its capacity to foster genuine dialogue and co-responsibility. This involves the evaluation seeking out influential ideas from below the level of strategic management, and documenting leadership activities of those who are skilled at ‘boundary crossing’ – for example, communicating between sectors.

The evaluator can be a key player in this process, as a convenor of safe spaces for actors to come together to discuss and deliberate before returning to practice. Our approach therefore argues for a particular awareness of the political nature of policy evaluation in terms of negotiating these spaces, and the need for politically engaged evaluators who are skilled in facilitating collective learning processes.

Evaluation Noir: The Other Side of the Experience

What are the boundaries of a case study, and what should new evaluators do when these boundaries are breached? How does a new evaluator interpret the breakdown of communication, how do new evaluators protect themselves when the evaluation fails? This chapter discusses the journey of an evaluator new to the field of qualitative evaluative inquiry. Integrating the perspective of a senior evaluator, the authors reflect on three key experiences that informed the new evaluator. The authors hope to provide a rare insight into case study practice as emotional issues turn out to be just as complex as the methodology used.

About the Editors

About the authors.

• Jill Russell
• Trisha Greenhalgh
• Saville Kushner

We’re listening — tell us what you think

Something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

• Case Study Evaluation Approach
• Learning Center

A case study evaluation approach can be an incredibly powerful tool for monitoring and evaluating complex programs and policies. By identifying common themes and patterns, this approach allows us to better understand the successes and challenges faced by the program. In this article, we’ll explore the benefits of using a case study evaluation approach in the monitoring and evaluation of projects, programs, and public policies.

Table of Contents

Introduction to Case Study Evaluation Approach

The advantages of a case study evaluation approach, types of case studies, potential challenges with a case study evaluation approach, guiding principles for successful implementation of a case study evaluation approach.

• Benefits of Incorporating the Case Study Evaluation Approach in the Monitoring and Evaluation of Projects and Programs

A case study evaluation approach is a great way to gain an in-depth understanding of a particular issue or situation. This type of approach allows the researcher to observe, analyze, and assess the effects of a particular situation on individuals or groups.

An individual, a location, or a project may serve as the focal point of a case study’s attention. Quantitative and qualitative data are frequently used in conjunction with one another.

It also allows the researcher to gain insights into how people react to external influences. By using a case study evaluation approach, researchers can gain insights into how certain factors such as policy change or a new technology have impacted individuals and communities. The data gathered through this approach can be used to formulate effective strategies for responding to changes and challenges. Ultimately, this monitoring and evaluation approach helps organizations make better decision about the implementation of their plans.

This approach can be used to assess the effectiveness of a policy, program, or initiative by considering specific elements such as implementation processes, outcomes, and impact. A case study evaluation approach can provide an in-depth understanding of the effectiveness of a program by closely examining the processes involved in its implementation. This includes understanding the context, stakeholders, and resources to gain insight into how well a program is functioning or has been executed. By evaluating these elements, it can help to identify areas for improvement and suggest potential solutions. The findings from this approach can then be used to inform decisions about policies, programs, and initiatives for improved outcomes.

It is also useful for determining if other policies, programs, or initiatives could be applied to similar situations in order to achieve similar results or improved outcomes. All in all, the case study monitoring evaluation approach is an effective method for determining the effectiveness of specific policies, programs, or initiatives. By researching and analyzing the successes of previous cases, this approach can be used to identify similar approaches that could be applied to similar situations in order to achieve similar results or improved outcomes.

A case study evaluation approach offers the advantage of providing in-depth insight into a particular program or policy. This can be accomplished by analyzing data and observations collected from a range of stakeholders such as program participants, service providers, and community members. The monitoring and evaluation approach is used to assess the impact of programs and inform the decision-making process to ensure successful implementation. The case study monitoring and evaluation approach can help identify any underlying issues that need to be addressed in order to improve program effectiveness. It also provides a reality check on how successful programs are actually working, allowing organizations to make adjustments as needed. Overall, a case study monitoring and evaluation approach helps to ensure that policies and programs are achieving their objectives while providing valuable insight into how they are performing overall.

By taking a qualitative approach to data collection and analysis, case study evaluations are able to capture nuances in the context of a particular program or policy that can be overlooked when relying solely on quantitative methods. Using this approach, insights can be gleaned from looking at the individual experiences and perspectives of actors involved, providing a more detailed understanding of the impact of the program or policy than is possible with other evaluation methodologies. As such, case study monitoring evaluation is an invaluable tool in assessing the effectiveness of a particular initiative, enabling more informed decision-making as well as more effective implementation of programs and policies.

Furthermore, this approach is an effective way to uncover experiential information that can help to inform the ongoing improvement of policy and programming over time All in all, the case study monitoring evaluation approach offers an effective way to uncover experiential information necessary to inform the ongoing improvement of policy and programming. By analyzing the data gathered from this systematic approach, stakeholders can gain deeper insight into how best to make meaningful and long-term changes in their respective organizations.

Case studies come in a variety of forms, each of which can be put to a unique set of evaluation tasks. Evaluators have come to a consensus on describing six distinct sorts of case studies, which are as follows: illustrative, exploratory, critical instance, program implementation, program effects, and cumulative.

Illustrative Case Study

An illustrative case study is a type of case study that is used to provide a detailed and descriptive account of a particular event, situation, or phenomenon. It is often used in research to provide a clear understanding of a complex issue, and to illustrate the practical application of theories or concepts.

An illustrative case study typically uses qualitative data, such as interviews, surveys, or observations, to provide a detailed account of the unit being studied. The case study may also include quantitative data, such as statistics or numerical measurements, to provide additional context or to support the qualitative data.

The goal of an illustrative case study is to provide a rich and detailed description of the unit being studied, and to use this information to illustrate broader themes or concepts. For example, an illustrative case study of a successful community development project may be used to illustrate the importance of community engagement and collaboration in achieving development goals.

One of the strengths of an illustrative case study is its ability to provide a detailed and nuanced understanding of a particular issue or phenomenon. By focusing on a single case, the researcher is able to provide a detailed and in-depth analysis that may not be possible through other research methods.

However, one limitation of an illustrative case study is that the findings may not be generalizable to other contexts or populations. Because the case study focuses on a single unit, it may not be representative of other similar units or situations.

A well-executed case study can shed light on wider research topics or concepts through its thorough and descriptive analysis of a specific event or phenomenon.

Exploratory Case Study

An exploratory case study is a type of case study that is used to investigate a new or previously unexplored phenomenon or issue. It is often used in research when the topic is relatively unknown or when there is little existing literature on the topic.

Exploratory case studies are typically qualitative in nature and use a variety of methods to collect data, such as interviews, observations, and document analysis. The focus of the study is to gather as much information as possible about the phenomenon being studied and to identify new and emerging themes or patterns.

The goal of an exploratory case study is to provide a foundation for further research and to generate hypotheses about the phenomenon being studied. By exploring the topic in-depth, the researcher can identify new areas of research and generate new questions to guide future research.

One of the strengths of an exploratory case study is its ability to provide a rich and detailed understanding of a new or emerging phenomenon. By using a variety of data collection methods, the researcher can gather a broad range of data and perspectives to gain a more comprehensive understanding of the phenomenon being studied.

However, one limitation of an exploratory case study is that the findings may not be generalizable to other contexts or populations. Because the study is focused on a new or previously unexplored phenomenon, the findings may not be applicable to other situations or populations.

Exploratory case studies are an effective research strategy for learning about novel occurrences, developing research hypotheses, and gaining a deep familiarity with a topic of study.

Critical Instance Case Study

A critical instance case study is a type of case study that focuses on a specific event or situation that is critical to understanding a broader issue or phenomenon. The goal of a critical instance case study is to analyze the event in depth and to draw conclusions about the broader issue or phenomenon based on the analysis.

A critical instance case study typically uses qualitative data, such as interviews, observations, or document analysis, to provide a detailed and nuanced understanding of the event being studied. The data are analyzed using various methods, such as content analysis or thematic analysis, to identify patterns and themes that emerge from the data.

The critical instance case study is often used in research when a particular event or situation is critical to understanding a broader issue or phenomenon. For example, a critical instance case study of a successful disaster response effort may be used to identify key factors that contributed to the success of the response, and to draw conclusions about effective disaster response strategies more broadly.

One of the strengths of a critical instance case study is its ability to provide a detailed and in-depth analysis of a particular event or situation. By focusing on a critical instance, the researcher is able to provide a rich and nuanced understanding of the event, and to draw conclusions about broader issues or phenomena based on the analysis.

However, one limitation of a critical instance case study is that the findings may not be generalizable to other contexts or populations. Because the case study focuses on a specific event or situation, the findings may not be applicable to other similar events or situations.

A critical instance case study is a valuable research method that can provide a detailed and nuanced understanding of a particular event or situation and can be used to draw conclusions about broader issues or phenomena based on the analysis.

Program Implementation Program Implementation

A program implementation case study is a type of case study that focuses on the implementation of a particular program or intervention. The goal of the case study is to provide a detailed and comprehensive account of the program implementation process, and to identify factors that contributed to the success or failure of the program.

Program implementation case studies typically use qualitative data, such as interviews, observations, and document analysis, to provide a detailed and nuanced understanding of the program implementation process. The data are analyzed using various methods, such as content analysis or thematic analysis, to identify patterns and themes that emerge from the data.

The program implementation case study is often used in research to evaluate the effectiveness of a particular program or intervention, and to identify strategies for improving program implementation in the future. For example, a program implementation case study of a school-based health program may be used to identify key factors that contributed to the success or failure of the program, and to make recommendations for improving program implementation in similar settings.

One of the strengths of a program implementation case study is its ability to provide a detailed and comprehensive account of the program implementation process. By using qualitative data, the researcher is able to capture the complexity and nuance of the implementation process, and to identify factors that may not be captured by quantitative data alone.

However, one limitation of a program implementation case study is that the findings may not be generalizable to other contexts or populations. Because the case study focuses on a specific program or intervention, the findings may not be applicable to other programs or interventions in different settings.

An effective research tool, a case study of program implementation may illuminate the intricacies of the implementation process and point the way towards future enhancements.

Program Effects Case Study

A program effects case study is a research method that evaluates the effectiveness of a particular program or intervention by examining its outcomes or effects. The purpose of this type of case study is to provide a detailed and comprehensive account of the program’s impact on its intended participants or target population.

A program effects case study typically employs both quantitative and qualitative data collection methods, such as surveys, interviews, and observations, to evaluate the program’s impact on the target population. The data is then analyzed using statistical and thematic analysis to identify patterns and themes that emerge from the data.

The program effects case study is often used to evaluate the success of a program and identify areas for improvement. For example, a program effects case study of a community-based HIV prevention program may evaluate the program’s effectiveness in reducing HIV transmission rates among high-risk populations and identify factors that contributed to the program’s success.

One of the strengths of a program effects case study is its ability to provide a detailed and nuanced understanding of a program’s impact on its intended participants or target population. By using both quantitative and qualitative data, the researcher can capture both the objective and subjective outcomes of the program and identify factors that may have contributed to the outcomes.

However, a limitation of the program effects case study is that it may not be generalizable to other populations or contexts. Since the case study focuses on a particular program and population, the findings may not be applicable to other programs or populations in different settings.

A program effects case study is a good way to do research because it can give a detailed look at how a program affects the people it is meant for. This kind of case study can be used to figure out what needs to be changed and how to make programs that work better.

Cumulative Case Study

A cumulative case study is a type of case study that involves the collection and analysis of multiple cases to draw broader conclusions. Unlike a single-case study, which focuses on one specific case, a cumulative case study combines multiple cases to provide a more comprehensive understanding of a phenomenon.

The purpose of a cumulative case study is to build up a body of evidence through the examination of multiple cases. The cases are typically selected to represent a range of variations or perspectives on the phenomenon of interest. Data is collected from each case using a range of methods, such as interviews, surveys, and observations.

The data is then analyzed across cases to identify common themes, patterns, and trends. The analysis may involve both qualitative and quantitative methods, such as thematic analysis and statistical analysis.

The cumulative case study is often used in research to develop and test theories about a phenomenon. For example, a cumulative case study of successful community-based health programs may be used to identify common factors that contribute to program success, and to develop a theory about effective community-based health program design.

One of the strengths of the cumulative case study is its ability to draw on a range of cases to build a more comprehensive understanding of a phenomenon. By examining multiple cases, the researcher can identify patterns and trends that may not be evident in a single case study. This allows for a more nuanced understanding of the phenomenon and helps to develop more robust theories.

However, one limitation of the cumulative case study is that it can be time-consuming and resource-intensive to collect and analyze data from multiple cases. Additionally, the selection of cases may introduce bias if the cases are not representative of the population of interest.

In summary, a cumulative case study is a valuable research method that can provide a more comprehensive understanding of a phenomenon by examining multiple cases. This type of case study is particularly useful for developing and testing theories and identifying common themes and patterns across cases.

When conducting a case study evaluation approach, one of the main challenges is the need to establish a contextually relevant research design that accounts for the unique factors of the case being studied. This requires close monitoring of the case, its environment, and relevant stakeholders. In addition, the researcher must build a framework for the collection and analysis of data that is able to draw meaningful conclusions and provide valid insights into the dynamics of the case. Ultimately, an effective case study monitoring evaluation approach will allow researchers to form an accurate understanding of their research subject.

Additionally, depending on the size and scope of the case, there may be concerns regarding the availability of resources and personnel that could be allocated to data collection and analysis. To address these issues, a case study monitoring evaluation approach can be adopted, which would involve a mix of different methods such as interviews, surveys, focus groups and document reviews. Such an approach could provide valuable insights into the effectiveness and implementation of the case in question. Additionally, this type of evaluation can be tailored to the specific needs of the case study to ensure that all relevant data is collected and respected.

When dealing with a highly sensitive or confidential subject matter within a case study, researchers must take extra measures to prevent bias during data collection as well as protect participant anonymity while also collecting valid data in order to ensure reliable results

Moreover, when conducting a case study evaluation it is important to consider the potential implications of the data gathered. By taking extra measures to prevent bias and protect participant anonymity, researchers can ensure reliable results while also collecting valid data. Maintaining confidentiality and deploying ethical research practices are essential when conducting a case study to ensure an unbiased and accurate monitoring evaluation.

When planning and implementing a case study evaluation approach, it is important to ensure the guiding principles of research quality, data collection, and analysis are met. To ensure these principles are upheld, it is essential to develop a comprehensive monitoring and evaluation plan. This plan should clearly outline the steps to be taken during the data collection and analysis process. Furthermore, the plan should provide detailed descriptions of the project objectives, target population, key indicators, and timeline. It is also important to include metrics or benchmarks to monitor progress and identify any potential areas for improvement. By implementing such an approach, it will be possible to ensure that the case study evaluation approach yields valid and reliable results.

To ensure successful implementation, it is essential to establish a reliable data collection process that includes detailed information such as the scope of the study, the participants involved, and the methods used to collect data. Additionally, it is important to have a clear understanding of what will be examined through the evaluation process and how the results will be used. All in all, it is essential to establish a sound monitoring evaluation approach for a successful case study implementation. This includes creating a reliable data collection process that encompasses the scope of the study, the participants involved, and the methods used to collect data. It is also imperative to have an understanding of what will be examined and how the results will be utilized. Ultimately, effective planning is key to ensure that the evaluation process yields meaningful insights.

Benefits of Incorporating the Case Study Evaluation Approach in the Monitoring and Evaluation of Projects and Programmes

Using a case study approach in monitoring and evaluation allows for a more detailed and in-depth exploration of the project’s success, helping to identify key areas of improvement and successes that may have been overlooked through traditional evaluation. Through this case study method, specific data can be collected and analyzed to identify trends and different perspectives that can support the evaluation process. This data can allow stakeholders to gain a better understanding of the project’s successes and failures, helping them make informed decisions on how to strengthen current activities or shape future initiatives. From a monitoring and evaluation standpoint, this approach can provide an increased level of accuracy in terms of accurately assessing the effectiveness of the project.

This can provide valuable insights into what works—and what doesn’t—when it comes to implementing projects and programs, aiding decision-makers in making future plans that better meet their objectives However, monitoring and evaluation is just one approach to assessing the success of a case study. It does provide a useful insight into what initiatives may be successful, but it is important to note that there are other effective research methods, such as surveys and interviews, that can also help to further evaluate the success of a project or program.

In conclusion, a case study evaluation approach can be incredibly useful in monitoring and evaluating complex programs and policies. By exploring key themes, patterns and relationships, organizations can gain a detailed understanding of the successes, challenges and limitations of their program or policy. This understanding can then be used to inform decision-making and improve outcomes for those involved. With its ability to provide an in-depth understanding of a program or policy, the case study evaluation approach has become an invaluable tool for monitoring and evaluation professionals.

Leave a Comment Cancel Reply

Your email address will not be published.

Featured Jobs

Data & report coordinator, knowledge management specialist, director organizational development/hicd director, director of monitoring, evaluation, and research (mer), chief of party – bosnia and herzegovina.

• Bosnia and Herzegovina

Director of Collaborating, Learning, and Adapting (CLA)

Director of collaborating, learning, and adapting (cla) – bosnia and herzegovina, senior director of monitoring and evaluation – bosnia and herzegovina, learning and impact advisor, global ftf fork.

• Washington, DC, USA

Monitoring, Evaluation & Learning (MEL) Coordinator

• Center for Victims of Torture

Land a better career with members' services

How strong is my Resume?

Only 2% of resumes land interviews.

READY TO LAND M&E JOB YOU LOVE?

Get our FREE walkthrough guide to landing a job in International Development

We will never spam or sell your data! You can unsubscribe at any time.

Services you might be interested in

Useful Guides ...

Masters, PhD and Certificate in M&E

What is Evaluation?

What is the difference between Monitoring and Evaluation?

Types of Evaluation

Monitoring, Evaluation, Accountability, and Learning (MEAL)

LAND A JOB REFERRAL IN 2 WEEKS (NO ONLINE APPS!)

Sign Up & To Get My Free Referral Toolkit Now:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials

Aileen grant.

1 School of Nursing, Midwifery and Paramedic Practice, Robert Gordon University, Garthdee Road, Aberdeen, AB10 7QB UK

Carol Bugge

2 Faculty of Health Sciences and Sport, University of Stirling, Pathfoot Building, Stirling, FK9 4LA UK

3 Department of Surgery and Cancer, Imperial College London, Charing Cross Campus, London, W6 8RP UK

Associated Data

No data and materials were used.

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail, and whether they can be transferred to other settings and populations. However, historically, context has not been sufficiently explored and reported resulting in the poor uptake of trial results. Therefore, suitable methodologies are needed to guide the investigation of context. Case study is one appropriate methodology, but there is little guidance about what case study design can offer the study of context in trials. We address this gap in the literature by presenting a number of important considerations for process evaluation using a case study design.

In this paper, we define context, the relationship between complex interventions and context, and describe case study design methodology. A well-designed process evaluation using case study should consider the following core components: the purpose; definition of the intervention; the trial design, the case, the theories or logic models underpinning the intervention, the sampling approach and the conceptual or theoretical framework. We describe each of these in detail and highlight with examples from recently published process evaluations.

Conclusions

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation. We provide a comprehensive overview of the issues for process evaluation design to consider when using a case study design.

Trial registration

DQIP - ClinicalTrials.gov number, {"type":"clinical-trial","attrs":{"text":"NCT01425502","term_id":"NCT01425502"}} NCT01425502 - OPAL - ISRCTN57746448

Contribution to the literature

• We illustrate how case study methodology can explore the complex, dynamic and uncertain relationship between context and interventions within trials.
• We depict different case study designs and illustrate there is not one formula and that design needs to be tailored to the context and trial design.
• Case study can support comparisons between intervention and control arms and between cases within arms to uncover and explain differences in detail.
• We argue that case study can illustrate how components have evolved and been redefined through implementation.
• Key issues for consideration in case study design within process evaluations are presented and illustrated with examples.

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail and whether they can be transferred to other settings and populations. However, historically, not all trials have had a process evaluation component, nor have they sufficiently reported aspects of context, resulting in poor uptake of trial findings [ 1 ]. Considerations of context are often absent from published process evaluations, with few studies acknowledging, taking account of or describing context during implementation, or assessing the impact of context on implementation [ 2 , 3 ]. At present, evidence from trials is not being used in a timely manner [ 4 , 5 ], and this can negatively impact on patient benefit and experience [ 6 ]. It takes on average 17 years for knowledge from research to be implemented into practice [ 7 ]. Suitable methodologies are therefore needed that allow for context to be exposed; one appropriate methodological approach is case study [ 8 , 9 ].

In 2015, the Medical Research Council (MRC) published guidance for process evaluations [ 10 ]. This was a key milestone in legitimising as well as providing tools, methods and a framework for conducting process evaluations. Nevertheless, as with all guidance, there is a need for reflection, challenge and refinement. There have been a number of critiques of the MRC guidance, including that interventions should be considered as events in systems [ 11 – 14 ]; a need for better use, critique and development of theories [ 15 – 17 ]; and a need for more guidance on integrating qualitative and quantitative data [ 18 , 19 ]. Although the MRC process evaluation guidance does consider appropriate qualitative and quantitative methods, it does not mention case study design and what it can offer the study of context in trials.

The case study methodology is ideally suited to real-world, sustainable intervention development and evaluation because it can explore and examine contemporary complex phenomena, in depth, in numerous contexts and using multiple sources of data [ 8 ]. Case study design can capture the complexity of the case, the relationship between the intervention and the context and how the intervention worked (or not) [ 8 ]. There are a number of textbooks on a case study within the social science fields [ 8 , 9 , 20 ], but there are no case study textbooks and a paucity of useful texts on how to design, conduct and report case study within the health arena. Few examples exist within the trial design and evaluation literature [ 3 , 21 ]. Therefore, guidance to enable well-designed process evaluations using case study methodology is required.

We aim to address the gap in the literature by presenting a number of important considerations for process evaluation using a case study design. First, we define the context and describe the relationship between complex health interventions and context.

What is context?

While there is growing recognition that context interacts with the intervention to impact on the intervention’s effectiveness [ 22 ], context is still poorly defined and conceptualised. There are a number of different definitions in the literature, but as Bate et al. explained ‘almost universally, we find context to be an overworked word in everyday dialogue but a massively understudied and misunderstood concept’ [ 23 ]. Ovretveit defines context as ‘everything the intervention is not’ [ 24 ]. This last definition is used by the MRC framework for process evaluations [ 25 ]; however; the problem with this definition is that it is highly dependent on how the intervention is defined. We have found Pfadenhauer et al.’s definition useful:

Context is conceptualised as a set of characteristics and circumstances that consist of active and unique factors that surround the implementation. As such it is not a backdrop for implementation but interacts, influences, modifies and facilitates or constrains the intervention and its implementation. Context is usually considered in relation to an intervention or object, with which it actively interacts. A boundary between the concepts of context and setting is discernible: setting refers to the physical, specific location in which the intervention is put into practice. Context is much more versatile, embracing not only the setting but also roles, interactions and relationships [ 22 ].

Traditionally, context has been conceptualised in terms of barriers and facilitators, but what is a barrier in one context may be a facilitator in another, so it is the relationship and dynamics between the intervention and context which are the most important [ 26 ]. There is a need for empirical research to really understand how different contextual factors relate to each other and to the intervention. At present, research studies often list common contextual factors, but without a depth of meaning and understanding, such as government or health board policies, organisational structures, professional and patient attitudes, behaviours and beliefs [ 27 ]. The case study methodology is well placed to understand the relationship between context and intervention where these boundaries may not be clearly evident. It offers a means of unpicking the contextual conditions which are pertinent to effective implementation.

The relationship between complex health interventions and context

Health interventions are generally made up of a number of different components and are considered complex due to the influence of context on their implementation and outcomes [ 3 , 28 ]. Complex interventions are often reliant on the engagement of practitioners and patients, so their attitudes, behaviours, beliefs and cultures influence whether and how an intervention is effective or not. Interventions are context-sensitive; they interact with the environment in which they are implemented. In fact, many argue that interventions are a product of their context, and indeed, outcomes are likely to be a product of the intervention and its context [ 3 , 29 ]. Within a trial, there is also the influence of the research context too—so the observed outcome could be due to the intervention alone, elements of the context within which the intervention is being delivered, elements of the research process or a combination of all three. Therefore, it can be difficult and unhelpful to separate the intervention from the context within which it was evaluated because the intervention and context are likely to have evolved together over time. As a result, the same intervention can look and behave differently in different contexts, so it is important this is known, understood and reported [ 3 ]. Finally, the intervention context is dynamic; the people, organisations and systems change over time, [ 3 ] which requires practitioners and patients to respond, and they may do this by adapting the intervention or contextual factors. So, to enable researchers to replicate successful interventions, or to explain why the intervention was not successful, it is not enough to describe the components of the intervention, they need to be described by their relationship to their context and resources [ 3 , 28 ].

What is a case study?

Case study methodology aims to provide an in-depth, holistic, balanced, detailed and complete picture of complex contemporary phenomena in its natural context [ 8 , 9 , 20 ]. In this case, the phenomena are the implementation of complex interventions in a trial. Case study methodology takes the view that the phenomena can be more than the sum of their parts and have to be understood as a whole [ 30 ]. It is differentiated from a clinical case study by its analytical focus [ 20 ].

The methodology is particularly useful when linked to trials because some of the features of the design naturally fill the gaps in knowledge generated by trials. Given the methodological focus on understanding phenomena in the round, case study methodology is typified by the use of multiple sources of data, which are more commonly qualitatively guided [ 31 ]. The case study methodology is not epistemologically specific, like realist evaluation, and can be used with different epistemologies [ 32 ], and with different theories, such as Normalisation Process Theory (which explores how staff work together to implement a new intervention) or the Consolidated Framework for Implementation Research (which provides a menu of constructs associated with effective implementation) [ 33 – 35 ]. Realist evaluation can be used to explore the relationship between context, mechanism and outcome, but case study differs from realist evaluation by its focus on a holistic and in-depth understanding of the relationship between an intervention and the contemporary context in which it was implemented [ 36 ]. Case study enables researchers to choose epistemologies and theories which suit the nature of the enquiry and their theoretical preferences.

Designing a process evaluation using case study

An important part of any study is the research design. Due to their varied philosophical positions, the seminal authors in the field of case study have different epistemic views as to how a case study should be conducted [ 8 , 9 ]. Stake takes an interpretative approach (interested in how people make sense of their world), and Yin has more positivistic leanings, arguing for objectivity, validity and generalisability [ 8 , 9 ].

Regardless of the philosophical background, a well-designed process evaluation using case study should consider the following core components: the purpose; the definition of the intervention, the trial design, the case, and the theories or logic models underpinning the intervention; the sampling approach; and the conceptual or theoretical framework [ 8 , 9 , 20 , 31 , 33 ]. We now discuss these critical components in turn, with reference to two process evaluations that used case study design, the DQIP and OPAL studies [ 21 , 37 – 41 ].

The purpose of a process evaluation is to evaluate and explain the relationship between the intervention and its components, to context and outcome. It can help inform judgements about validity (by exploring the intervention components and their relationship with one another (construct validity), the connections between intervention and outcomes (internal validity) and the relationship between intervention and context (external validity)). It can also distinguish between implementation failure (where the intervention is poorly delivered) and intervention failure (intervention design is flawed) [ 42 , 43 ]. By using a case study to explicitly understand the relationship between context and the intervention during implementation, the process evaluation can explain the intervention effects and the potential generalisability and optimisation into routine practice [ 44 ].

The DQIP process evaluation aimed to qualitatively explore how patients and GP practices responded to an intervention designed to reduce high-risk prescribing of nonsteroidal anti-inflammatory drugs (NSAIDs) and/or antiplatelet agents (see Table  1 ) and quantitatively examine how change in high-risk prescribing was associated with practice characteristics and implementation processes. The OPAL process evaluation (see Table  2 ) aimed to quantitatively understand the factors which influenced the effectiveness of a pelvic floor muscle training intervention for women with urinary incontinence and qualitatively explore the participants’ experiences of treatment and adherence.

Data-driven Quality Improvement in Primary Care (DQIP)

Optimising Pelvic Floor Exercises to Achieve Long-term benefits (OPAL)

Defining the intervention and exploring the theories or assumptions underpinning the intervention design

Process evaluations should also explore the utility of the theories or assumptions underpinning intervention design [ 49 ]. Not all theories underpinning interventions are based on a formal theory, but they based on assumptions as to how the intervention is expected to work. These can be depicted as a logic model or theory of change [ 25 ]. To capture how the intervention and context evolve requires the intervention and its expected mechanisms to be clearly defined at the outset [ 50 ]. Hawe and colleagues recommend defining interventions by function (what processes make the intervention work) rather than form (what is delivered) [ 51 ]. However, in some cases, it may be useful to know if some of the components are redundant in certain contexts or if there is a synergistic effect between all the intervention components.

The DQIP trial delivered two interventions, one intervention was delivered to professionals with high fidelity and then professionals delivered the other intervention to patients by form rather than function allowing adaptations to the local context as appropriate. The assumptions underpinning intervention delivery were prespecified in a logic model published in the process evaluation protocol [ 52 ].

Case study is well placed to challenge or reinforce the theoretical assumptions or redefine these based on the relationship between the intervention and context. Yin advocates the use of theoretical propositions; these direct attention to specific aspects of the study for investigation [ 8 ] can be based on the underlying assumptions and tested during the course of the process evaluation. In case studies, using an epistemic position more aligned with Yin can enable research questions to be designed, which seek to expose patterns of unanticipated as well as expected relationships [ 9 ]. The OPAL trial was more closely aligned with Yin, where the research team predefined some of their theoretical assumptions, based on how the intervention was expected to work. The relevant parts of the data analysis then drew on data to support or refute the theoretical propositions. This was particularly useful for the trial as the prespecified theoretical propositions linked to the mechanisms of action on which the intervention was anticipated to have an effect (or not).

Tailoring to the trial design

Process evaluations need to be tailored to the trial, the intervention and the outcomes being measured [ 45 ]. For example, in a stepped wedge design (where the intervention is delivered in a phased manner), researchers should try to ensure process data are captured at relevant time points or in a two-arm or multiple arm trial, ensure data is collected from the control group(s) as well as the intervention group(s). In the DQIP trial, a stepped wedge trial, at least one process evaluation case, was sampled per cohort. Trials often continue to measure outcomes after delivery of the intervention has ceased, so researchers should also consider capturing ‘follow-up’ data on contextual factors, which may continue to influence the outcome measure. The OPAL trial had two active treatment arms so collected process data from both arms. In addition, as the trial was interested in long-term adherence, the trial and the process evaluation collected data from participants for 2 years after the intervention was initially delivered, providing 24 months follow-up data, in line with the primary outcome for the trial.

Defining the case

Case studies can include single or multiple cases in their design. Single case studies usually sample typical or unique cases, their advantage being the depth and richness that can be achieved over a long period of time. The advantages of multiple case study design are that cases can be compared to generate a greater depth of analysis. Multiple case study sampling may be carried out in order to test for replication or contradiction [ 8 ]. Given that trials are often conducted over a number of sites, a multiple case study design is more sensible for process evaluations, as there is likely to be variation in implementation between sites. Case definition may occur at a variety of levels but is most appropriate if it reflects the trial design. For example, a case in an individual patient level trial is likely to be defined as a person/patient (e.g. a woman with urinary incontinence—OPAL trial) whereas in a cluster trial, a case is like to be a cluster, such as an organisation (e.g. a general practice—DQIP trial). Of course, the process evaluation could explore cases with less distinct boundaries, such as communities or relationships; however, the clarity with which these cases are defined is important, in order to scope the nature of the data that will be generated.

Carefully sampled cases are critical to a good case study as sampling helps inform the quality of the inferences that can be made from the data [ 53 ]. In both qualitative and quantitative research, how and how many participants to sample must be decided when planning the study. Quantitative sampling techniques generally aim to achieve a random sample. Qualitative research generally uses purposive samples to achieve data saturation, occurring when the incoming data produces little or no new information to address the research questions. The term data saturation has evolved from theoretical saturation in conventional grounded theory studies; however, its relevance to other types of studies is contentious as the term saturation seems to be widely used but poorly justified [ 54 ]. Empirical evidence suggests that for in-depth interview studies, saturation occurs at 12 interviews for thematic saturation, but typically more would be needed for a heterogenous sample higher degrees of saturation [ 55 , 56 ]. Both DQIP and OPAL case studies were huge with OPAL designed to interview each of the 40 individual cases four times and DQIP designed to interview the lead DQIP general practitioner (GP) twice (to capture change over time), another GP and the practice manager from each of the 10 organisational cases. Despite the plethora of mixed methods research textbooks, there is very little about sampling as discussions typically link to method (e.g. interviews) rather than paradigm (e.g. case study).

Purposive sampling can improve the generalisability of the process evaluation by sampling for greater contextual diversity. The typical or average case is often not the richest source of information. Outliers can often reveal more important insights, because they may reflect the implementation of the intervention using different processes. Cases can be selected from a number of criteria, which are not mutually exclusive, to enable a rich and detailed picture to be built across sites [ 53 ]. To avoid the Hawthorne effect, it is recommended that process evaluations sample from both intervention and control sites, which enables comparison and explanation. There is always a trade-off between breadth and depth in sampling, so it is important to note that often quantity does not mean quality and that carefully sampled cases can provide powerful illustrative examples of how the intervention worked in practice, the relationship between the intervention and context and how and why they evolved together. The qualitative components of both DQIP and OPAL process evaluations aimed for maximum variation sampling. Please see Table  1 for further information on how DQIP’s sampling frame was important for providing contextual information on processes influencing effective implementation of the intervention.

Conceptual and theoretical framework

A conceptual or theoretical framework helps to frame data collection and analysis [ 57 ]. Theories can also underpin propositions, which can be tested in the process evaluation. Process evaluations produce intervention-dependent knowledge, and theories help make the research findings more generalizable by providing a common language [ 16 ]. There are a number of mid-range theories which have been designed to be used with process evaluation [ 34 , 35 , 58 ]. The choice of the appropriate conceptual or theoretical framework is, however, dependent on the philosophical and professional background of the research. The two examples within this paper used our own framework for the design of process evaluations, which proposes a number of candidate processes which can be explored, for example, recruitment, delivery, response, maintenance and context [ 45 ]. This framework was published before the MRC guidance on process evaluations, and both the DQIP and OPAL process evaluations were designed before the MRC guidance was published. The DQIP process evaluation explored all candidates in the framework whereas the OPAL process evaluation selected four candidates, illustrating that process evaluations can be selective in what they explore based on the purpose, research questions and resources. Furthermore, as Kislov and colleagues argue, we also have a responsibility to critique the theoretical framework underpinning the evaluation and refine theories to advance knowledge [ 59 ].

Data collection

An important consideration is what data to collect or measure and when. Case study methodology supports a range of data collection methods, both qualitative and quantitative, to best answer the research questions. As the aim of the case study is to gain an in-depth understanding of phenomena in context, methods are more commonly qualitative or mixed method in nature. Qualitative methods such as interviews, focus groups and observation offer rich descriptions of the setting, delivery of the intervention in each site and arm, how the intervention was perceived by the professionals delivering the intervention and the patients receiving the intervention. Quantitative methods can measure recruitment, fidelity and dose and establish which characteristics are associated with adoption, delivery and effectiveness. To ensure an understanding of the complexity of the relationship between the intervention and context, the case study should rely on multiple sources of data and triangulate these to confirm and corroborate the findings [ 8 ]. Process evaluations might consider using routine data collected in the trial across all sites and additional qualitative data across carefully sampled sites for a more nuanced picture within reasonable resource constraints. Mixed methods allow researchers to ask more complex questions and collect richer data than can be collected by one method alone [ 60 ]. The use of multiple sources of data allows data triangulation, which increases a study’s internal validity but also provides a more in-depth and holistic depiction of the case [ 20 ]. For example, in the DQIP process evaluation, the quantitative component used routinely collected data from all sites participating in the trial and purposively sampled cases for a more in-depth qualitative exploration [ 21 , 38 , 39 ].

The timing of data collection is crucial to study design, especially within a process evaluation where data collection can potentially influence the trial outcome. Process evaluations are generally in parallel or retrospective to the trial. The advantage of a retrospective design is that the evaluation itself is less likely to influence the trial outcome. However, the disadvantages include recall bias, lack of sensitivity to nuances and an inability to iteratively explore the relationship between intervention and outcome as it develops. To capture the dynamic relationship between intervention and context, the process evaluation needs to be parallel and longitudinal to the trial. Longitudinal methodological design is rare, but it is needed to capture the dynamic nature of implementation [ 40 ]. How the intervention is delivered is likely to change over time as it interacts with context. For example, as professionals deliver the intervention, they become more familiar with it, and it becomes more embedded into systems. The OPAL process evaluation was a longitudinal, mixed methods process evaluation where the quantitative component had been predefined and built into trial data collection systems. Data collection in both the qualitative and quantitative components mirrored the trial data collection points, which were longitudinal to capture adherence and contextual changes over time.

There is a lot of attention in the recent literature towards a systems approach to understanding interventions in context, which suggests interventions are ‘events within systems’ [ 61 , 62 ]. This framing highlights the dynamic nature of context, suggesting that interventions are an attempt to change systems dynamics. This conceptualisation would suggest that the study design should collect contextual data before and after implementation to assess the effect of the intervention on the context and vice versa.

Data analysis

Designing a rigorous analysis plan is particularly important for multiple case studies, where researchers must decide whether their approach to analysis is case or variable based. Case-based analysis is the most common, and analytic strategies must be clearly articulated for within and across case analysis. A multiple case study design can consist of multiple cases, where each case is analysed at the case level, or of multiple embedded cases, where data from all the cases are pulled together for analysis at some level. For example, OPAL analysis was at the case level, but all the cases for the intervention and control arms were pulled together at the arm level for more in-depth analysis and comparison. For Yin, analytical strategies rely on theoretical propositions, but for Stake, analysis works from the data to develop theory. In OPAL and DQIP, case summaries were written to summarise the cases and detail within-case analysis. Each of the studies structured these differently based on the phenomena of interest and the analytic technique. DQIP applied an approach more akin to Stake [ 9 ], with the cases summarised around inductive themes whereas OPAL applied a Yin [ 8 ] type approach using theoretical propositions around which the case summaries were structured. As the data for each case had been collected through longitudinal interviews, the case summaries were able to capture changes over time. It is beyond the scope of this paper to discuss different analytic techniques; however, to ensure the holistic examination of the intervention(s) in context, it is important to clearly articulate and demonstrate how data is integrated and synthesised [ 31 ].

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation [ 38 ]. Case study can enable comparisons within and across intervention and control arms and enable the evolving relationship between intervention and context to be captured holistically rather than considering processes in isolation. Utilising a longitudinal design can enable the dynamic relationship between context and intervention to be captured in real time. This information is fundamental to holistically explaining what intervention was implemented, understanding how and why the intervention worked or not and informing the transferability of the intervention into routine clinical practice.

Case study designs are not prescriptive, but process evaluations using case study should consider the purpose, trial design, the theories or assumptions underpinning the intervention, and the conceptual and theoretical frameworks informing the evaluation. We have discussed each of these considerations in turn, providing a comprehensive overview of issues for process evaluations using a case study design. There is no single or best way to conduct a process evaluation or a case study, but researchers need to make informed choices about the process evaluation design. Although this paper focuses on process evaluations, we recognise that case study design could also be useful during intervention development and feasibility trials. Elements of this paper are also applicable to other study designs involving trials.

Acknowledgements

We would like to thank Professor Shaun Treweek for the discussions about context in trials.

Abbreviations

Authors’ contributions.

AG, CB and MW conceptualised the study. AG wrote the paper. CB and MW commented on the drafts. All authors have approved the final manuscript.

No funding was received for this work.

Availability of data and materials

Ethics approval and consent to participate.

Ethics approval and consent to participate is not appropriate as no participants were included.

Consent for publication

Consent for publication is not required as no participants were included.

Competing interests

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Aileen Grant, Email: [email protected] .

Carol Bugge, Email: [email protected] .

Mary Wells, Email: [email protected] .

No internet connection.

All search filters on the page have been cleared., your search has been saved..

• All content
• Dictionaries
• Encyclopedias
• Expert Insights
• Foundations
• How-to Guides
• Journal Articles
• Little Blue Books
• Little Green Books
• Project Planner
• Tools Directory
• Sign in to my profile My Profile

• Sign in Signed in
• My profile My Profile

Retrospective Impact Evaluation Using Administrative Data: Lessons from a Study on the Effect of Capitation Policy Withdrawal on Maternal Health Service Provision in Ghana

• By: Naasegnibe Kuunibe , John K. Yambah & Samuel Sekyi
• Product: Sage Research Methods Cases Part 1
• Publisher: SAGE Publications Ltd
• Publication year: 2024
• Online pub date: January 20, 2024
• Discipline: Economics
• Methods: Case study research , Administrative data , Evaluation
• DOI: https:// doi. org/10.4135/9781529683264
• Keywords: administrative data , capitation , Ghana , health service provision , maternal health services , withdrawal Show all Show less
• Online ISBN: 9781529683264 Copyright: © 2024 SAGE Publications Ltd More information Less information

This case study is guided by our earlier research on the effect of the capitation policy withdrawal on maternal health service provision in Ashanti, Ghana, which was a time-series analysis. Healthcare provider payment methods affect the provision of care in general and maternal care in particular. In 2017, Ghana suspended its capitation policy in the Ashanti Region after piloting it for 5 years. Without evidence of the effect of the policy on health service provision, capitation was suspended, and Ghana missed out on learning any lessons for the future. Given that the policy was suspended and the need to learn meaningful policy lessons and bridge the gap in the literature was missed, only a retrospective evaluation using administrative data was possible. This case study walks students through the process of planning a retrospective evaluation. Critical considerations at the planning, data-acquisition, and management stages are explained. We describe the data used for our retrospective impact evaluation, the challenges that arose, and how we addressed those challenges. We also demonstrate how the retrospective design informed the analytical approach—the interrupted time-series analysis (ITSA). We explain single versus multiple ITSA and show why single ITSA was appropriate for our case study.

Learning Outcomes

By the end of this case study, readers should be able to

• describe a retrospective impact evaluation.
• distinguish single and multiple interrupted time-series designs.
• explain level change and trend change.

Project Overview and Context

Our research evaluated a piloted health provider payment system (capitation) in the context of Ghana’s health insurance policy (Yambah et al., 2022). Since implementing a national health insurance policy in 2004, Ghana has used various provider payment systems to achieve different objectives. Concerns about cost containment, for example, prompted the piloting of the capitation payment policy. A capitation payment system involves a fixed amount of money paid to a provider (health facility) to address an individual’s healthcare needs over a specified period.

The policy was introduced for primary care in 2012 in the Ashanti Region but discontinued in 2017. In that pilot program, clients of the national health insurance scheme in Ghana were allowed to choose their preferred primary provider (PPP) for outpatient health services for a period of 6 months. In principle, the clients were given the option to change their PPP after 6 months for reasons such as location change or dissatisfaction with service provision by the current PPP. The national health insurance scheme would then pay a provider for the number of clients who sign up to seek care at the specific facility. For care providers, capitation represented a predictable funding source for service provision such that its withdrawal could affect service provision in general and maternal healthcare in particular. Yet, since its suspension and until our study, it was difficult to find any study that assessed the effect of the policy withdrawal on the provision of maternal care. In our thinking, it was necessary to evaluate the policy withdrawal and offer evidence for the debate on the relevance or otherwise of the capitation payment policy in Ghana.

In contemplating our study, one potential problem was setting up an appropriate research design and obtaining the needed data. Note that in impact evaluation, the “gold standard” is the randomized, controlled trial, where an experiment is conducted for a group and the effects are measured against a control group. For validity, the assignment of study participants to the experimental and control groups should be random—each participant is given an equal chance of being selected into either group. Baseline data are then collected from both the experimental group (also called the intervention or treatment group ) and the control group, and then the intervention is introduced for the intended period, after which endpoint data are collected from both groups. A comparison is then made between the two groups before and after the treatment (the difference-in-difference analysis). The random assignment of study participants is expected to cancel out any confounding factors and allow for isolation of the impact of the treatment. For many reasons, including it being considered unethical under some conditions and the fact that policy may target particular subjects or indeed the entire population, quasi-experimental procedures are used as alternatives (for further reading on impact evaluation, see Gertler et al., 2016; Khandker, Koolwal, & Samad, 2010). Until recently, most impact evaluations require the collection of primary data before and after an intervention in treatment and control groups, even where the assignment of study participants is nonrandom (which is to say that the opportunity for equal chances of being selected was not given). Where it is not possible to collect primary data (e.g., an entire population was treated in time past), impact evaluation is only possible retrospectively.

In our study, the treatment was given in the past. We did not have baseline data on service provision from the piloted facilities. Such an opportunity could never have been possible because pilot study was already withdrawn. We had to think retrospectively about where to get appropriate good-quality data to serve our purpose We also thought about how to handle and analyze such data. This process set us exploring various existing data relevant to our policy. Because this was a health policy intervention, we focused our attention on existing data in the health sector that contained information relevant to our study variables. Fortunately, we found data on key variables of interest—the District Health Information Management System (DHIMS2) database. The data are routine administrative data collected across health facilities over time (longitudinal in nature). This means that they contained information on key variables before the introduction and during and after the withdrawal of capitation.

We used an interrupted time-series design because the data were longitudinal (collected across time) and employed the segmented regression model for the analysis to identify the change in the level and trend in service provision following withdrawal of the policy. Our outcome of variables were three key maternal healthcare services (the number of women who received at least their fourth scheduled antenatal care in a particular month [ANC4+], the number of laboratory tests done for women in their 36th week of pregnancy [HB36], and the number of births at a facility in each month ), all measured as counts.

Section Summary

• Capitation payment was piloted in one of Ghana’s regions by the national health insurance scheme for cost containment but got suspended after 5 years.
• We felt the need to evaluate the effect of the policy withdrawal on three key maternal service indicators.
• Due to the lack of primary data, we relied, retrospectively, on routine administrative data collected by the Ghana Health Service on service provision, used an interrupted time-series design, and employed a segmented regression model to analyze the change in levels and trends in the provision of care for three maternal care indicators.

Research Design

We used a single-group interrupted time-series design (for more information on interrupted time-series designs, see Linden, 2015; Shin, 2017). To avoid sounding technical, the interrupted time-series analysis or design refers to time series with an intervention point (point of interruption). The idea is that the impact of a program or policy can be estimated by comparing the difference in the series of observations after the intervention point (so, in this case, after withdrawal of capitation) and the so-called counterfactual (which is the modeled before the intervention series into the postintervention period). This design allows us to identify whether an intervention has caused a change in either the level and trend of a time series or both. The level and trend changes are identified by assuming that had the intervention not taken place, there would not be a change in the underlying trend of the outcome. This hypothetically unchanged trend continuing, in the absence of the intervention and given the preexisting trend, provides the counterfactual for evaluation of the impact by examining the change occurring in the postintervention period (Bernal et al., 2017; Shin, 2017). In the context of our study, the design allowed us to compare the levels and trends of service provision before and after the capitation policy was withdrawn. A positive level change, for example, means that service provision increased immediately as opposed to the period before policy withdrawal. Similarly, a positive change in trend means that the month-to-month provision after the withdrawal is higher than the month-to-moth provision in the period before the policy was withdrawn. Therefore, the interrupted time-series design is used when a policy or program has a clear starting point and at least 10 data points before and after the policy. If the data points are fewer than 10, it might be difficult to detect a meaningful trend change. In our case, the policy had a specific starting point (July 2017). We also had data that we could aggregate monthly from January 2015 to December 2019, which means that we could get data points for at least 10 months before and after withdrawal of capitation. The reader may now wonder, why do we call this design a single group? We do so because we did not include data from other regions where capitation was not implemented. If we had included data from a region where capitation was never introduced, this would have acted as a control group, meaning that we would have had two groups (or multiple groups) instead of a single group. However, we focused only on the pilot region. There are advantages when a control group can be included (see Linden, 2015), but this could be challenging. In multiple groups, the main identification assumption is that the changes in the level and trend in the outcome variables are presumed to be the same for both the intervention and control groups and that confounding omitted variables affect both groups similarly (Linden, 2015). Therefore, outcomes in the control group are not expected to diverge from underlying secular trends (McLintock et al., 2014). This allows for a more robust estimation of the impact. In our study, we had challenges accessing data from control regions but believed that we could still make a strong case using the single-group design, as has been done by many studies elsewhere (Serumaga et al., 2011).

Data Sources

A key consideration when using the interrupted time-series design is the source of data. The fact that secondary data are used mostly when this design is employed means that it is important to be transparent about the credibility and quality of the data. Generally, the data should come from a credible source. This could be a national or international database. Our data were from a credible data source. DHIMS2 was rolled out as a national cluster of the District Health Information Management System in Ghana to routinely collect and compile health data for decision making. The platform was implemented by the Ghana Health Service (GHS) in collaboration with University of Oslo (Odei-Lartey et al., 2020). The quality of the DHIMS2 also has been accessed previously (Amoakoh-Coleman et al., 2015) and found to be good. This notwithstanding, most administrative data normally will not be 100% complete. Therefore, it is important to still check for missing data points and, if any, take remedial measures (see (Kuunibe, 2023) for firsthand experience.

Study Variables

As much as possible, variables should be clearly defined. In our study, we used three maternal care indicators (the number of women who received at least their fourth scheduled antenatal care in a particular month [ANC4+], the number of laboratory tests done for women in their 36 th week of pregnancy in each month [HB36], and the number of births at a facility in each month). These indicators signal quality maternal care from the provider’s point of view and in line with Sustainable Development Goals Targets 3.1 and 3.2, which address reducing global maternal mortality to <70 per 100,000 live births by 2030 and ending preventable deaths of newborns and children <5 years of age, with all countries aiming to reduce neonatal mortality to at least as low as 12 per 1000 live births and under age 5 mortality to at least as low as 25 per 1000 live births by 2030, respectively. The progress of these goals targets also depends on how policies such as capitation affect the capacity of health facilities to provide maternal and childcare services.

Analytical Technique

We chose our analytical method in line with the data structure and the design. The segmented regression model allowed us to account for the preintervention trends and compare the slopes over time and the intercepts from the end of the preintervention period to the beginning of the postintervention period. In line with theory, we estimated the slope in the preintervention period and determined difference in slope between the preintervention and postintervention periods (Shin, 2017). In addition, we estimated the change in the mean of each outcome from the end of the preintervention period to the start of the intervention period. Because our data were collected over time from the same healthcare facilities, we considered the possibility that observations made in one month would be correlated with those made in the next month (a phenomenon known as autocorrelation ). We used the Dubbin–Watson test to test for the presence of autocorrelation and estimated the generalized linear model framework to correct for autocorrelation.

• The single-group interrupted time-series design was used to identify whether the withdrawal of capitation caused a change in either or both the level and trend of provision of maternal care services.
• We used data from the DHIMS2, a national cluster of the District Health Information Management System in Ghana, to routinely collect and compile health data for decision making.
• We used the segmented regression and estimated the generalized linear model to correct for autocorrelation.

Research Practicalities

In designing and implementing this study, we dealt with many practical issues. Our initial design was a multigroup interrupted time-series analysis aimed at including regions where capitation was never piloted. This was to have a valid control group and improve the robustness of our results. We also planned to include more than three maternal indicators. However, because we relied on secondary data, getting access and overcoming bureaucratic procedures affected implementation. The DHIMS2 database is not open source in Ghana. This means that access requests had to be made. National-level access to the data would have been best, but this was enormously challenging because of the many bureaucratic procedures involved and especially because this was initially part of the lead author’s graduate thesis. Not having access to the data at the national level meant that we could not extract as much data as we needed. We contemplated visiting selected health facilities from regions that could qualify as a control to explore the possibility of extracting data directly at the facility level. However, the COVID-19 pandemic made it impossible for us to do so. Even if we were to observe all the protocols, health facilities were particularly strict about allowing people into their facilities for nonmedical reasons.

Given these difficulties, it was more practical to discard the idea of a multigroup analysis (and to acknowledge this in our study) and to opt for a single-group analysis (using data from only the capitation-piloted region). This option was necessitated by the fact that the lead author (a graduate student at the time of the study) was expected to complete the study in not more than 3 years. Having to attend to many bureaucratic procedures to obtain access to the database at the national level was likely to delay completion of the thesis. Fortunately, the lead author was then working (as a medical doctor) in the capitation-piloted region and had networks at the subnational (regional) level that we could exploit for a less cumbersome process in obtaining access to the database for information customized for the region only. Given that we used routine data, which is anonymized, there were not going to be serious ethical concerns. However, it is standard procedure when doing research to think about ethical considerations. Thus, before seeking access to extract and use the data, the lead author submitted the study protocol and obtained ethical clearance for the study from his host university.

Data extraction, organization, and administration were the subsequent practical problems we had to solve. After access to the database was granted, we realized that the data set was not organized in the format we envisioned (as is typical in most situations). It was therefore more practical to extract the data in the existing format and then reorganize them into a format suitable for our analysis. One of the authors, who had previous expertise in organizing large routine data into usable formats (Kuunibe et al., 2020), served as mentor to the lead author. The lead author is an incredibly fast learner, which made the process of coaching and implementing the necessary procedures much easier and faster. Online discussions were used in most cases because the study was conducted during the COVID-19 pandemic.

The third issue was the need to examine the data for completeness and deal with incomplete cases. Even though most analyses using routine data are carried out using complete case analyses (without recognizing or dealing with missing data) we also know that such analyses could lead to biased results and that, where possible, imputation is always better (Honaker & King, 2010; Pratama et al., 2016). Of course, if the level (percentage) of missing data is so high (>30%), imputation might not make sense because such data might not be representative of the sample. Fortunately for us, because maternal care is a public health priority, reporting on these indicators is generally better. Despite this, however, we found a few missing cases in our selected variables (indicators) and went ahead to do single imputation using the mean value. Though multiple imputation would have been the best option, it would have required that at least one of the indicators should not have missing data at all and be used as the independent variable in the imputation model. One of the authors had implemented multiple imputation in a previous study (see Kuunibe et al., 2020) and had firsthand experience with the practical considerations in doing so (see Kuunibe et al, 2023). Thus it was relatively easier to navigate the question of whether to impute for missing data and which approach to use. The expertise of the lead author as a medical doctor who had practiced under the different provider payment systems used by the national health insurance authority in Ghana was very handy in relating the study findings to the practical issues on the ground and in discussing those findings. It was easier to see why the results made or did not make sense and tailor the search for supporting literature appropriately. Understanding the context of your study is a critical issue to consider when carrying out any research.

• Designing and implementing a retrospective impact evaluation are often more tedious than most people imagine.
• Issues regarding access to data and information can be particularly frustrating because of institutional restrictions and bureaucracies. One needs to be open minded about what is practical without compromising the overall value of the research.
• Because retrospective impact evaluation often involves using secondary data, which may have missing values, the data should be examined for completeness. It may be necessary to take appropriate remedial measures.

Method in Action

Doing a retrospective impact evaluation may not seem too involving from the initial stages until the process is started. Using routine data meant that we did not have to design a survey and collect data from individuals. This was good in terms of budget and the costs of data collection. Yet, in reality, the process was demanding in terms of finding the right data when considering quality and credibility. Our approach in getting access to the data began by reading the literature on research that used routine health data, more specifically looking for studies that employed relevant health service provision indicators (DHIMS2), like our study. Because at the point of contemplating the study we did not already have access to the database, our proxy for quality was to find out if others had used information from the same database for different or similar studies. Fortunately, we found some studies that directly addressed the completeness and quality of routine maternal and neonatal services data in Ghana and specifically related to the DHIMS2 (Amoakoh-Coleman et al., 2015; Kayode et al., 2014).

Once we got this information, our next step was to explore whether we could gain access to the database and at what level (national or subnational). Initially, this process involved making informal contacts with officials at the regional and national directorates of the GHS, the institution mandated to collect and keep the DHIMS2, to ascertain the availability of data for our indicators of interest and for what period. Remember that once we intended to use the interrupted time-series analysis, we needed to be sure that even if the data were available, they would also cover the period under consideration. We got the indication that the data were available in the database but that we needed to go through several bureaucratic steps if we needed access at the national level.

We preferred having access at the national level because we could then implement the multigroup interrupted time-series design. However, after weighing the possible time loss versus the time to completion of a student thesis, we had to be realistic about our design. And being realistic about what is possible is important under such conditions because one does not have control over how such data are accessed and used. At this point, we considered the option of using data from only the piloted region and changing the design to a single interrupted time-series. As already alluded to earlier, our lead author was at the time working in the piloted region and had some networks that could facilitate access to a restricted database for that region. Given that our analytical approach can account for the effect of time and other confounders and has been used in similar circumstances elsewhere (Tung, Chang, & Cheng, 2015; Zombre, De Allegri, & Ridde, 2017), we adjusted our design slightly to accommodate the reality with which we were confronted.

With this hurdle cleared, we designed a data-extraction sheet that guided us in extracting the data on our indicators of choice per the facility type of interest and for the needed period. The extracted data were in Microsoft Excel format, yet we needed to perform the analysis in STATA 14. We transferred the data to STATA, where we examined them for completeness and so on. STATA is a versatile software that allowed us to check by indicators the level (percentage) of data missing per indicator and the pattern of missingness (whether this was random or related to specific indicators, period, or facility type). This is a quick and easy way of checking routine data for completeness and quality. We did find some percentage missingness (average 20% per indicator and not particularly related to any period or indicator). However, there were more missing cases for lower-level facilities than for secondary facilities. We did not want to do complete case analysis without imputing for missing data. This was possible because in the end the data were aggregated on a monthly basis. Even though the debate regarding the significance of imputation is ongoing (Arel-Bundock & Pelc, 2018; Dettori, Norvell, & Chapman, 2018), our experience (Kuunibe et al., 2020) and some literature (Horton & Kleinman, 2007) show that imputed data lead to less biased estimates than complete case analysis. So our decision was to impute and do so using multiple imputations (employing a regression) rather than single imputation (using the mean value, for example). Here again there was a practical challenge. Multiple imputation requires that at least one indicator should not contain missing values so that it can be used as an independent variable based on which imputed values are calculated (StataCorp LP, 2013). Our data did not meet this condition, so we were left with the practical option of single imputation—and we imputed using the mean value of each indicator to replace the missing values.

With data management completed, our next step was to check the data for consistency with time-series analysis. We had to check for autocorrelation. The problem of autocorrelation is common in time-series analysis and, if not checked and remedied, can lead to biased estimates. We checked and noticed the presence of autocorrelation at lag period one. Using the ordinary least squares (OLS) regression would not have corrected this problem. We employed the generalized linear model, which adjusted for the presence of autocorrelation (Shin, 2017) in the data (see Table 2 of our study; Yambah et al., 2022).

Our analysis was carried out for two categories of facility types (hospitals or secondary facilities and community-based health planning and services [CHPS] compound or primary facilities). We did so because we wanted to see the differential impact of the withdrawal of the policy on these facility categories, which have different capacities and mandates. One strength of time-series analysis is the graphical display of otherwise technical statistical estimates. The graphs allow for easy interpretation of results, especially in the case of segmented regression, where the interpretation is not straightforward. As presented in our study (Yambah et al., 2022), the graphs clearly show what happened to each indicator in each facility category immediately after withdrawal of the policy as well as the trend. In the case of HB36 in CHPS and delivery in hospitals, we see a clear downward shift immediately after the withdrawal. While the slope (trend) of HB36 in CHPS continued downward, that of delivery in hospitals increased after the initial downward shift. Vividly, even without looking at the statistics, we see a clear picture of what happened to the provision of maternal care after withdrawal of the capitation payment policy. Thus, by using the interrupted time-series design, we were able to evaluate retrospectively the effect of the policy withdrawal on service provision.

• Data management, including checking data for completeness and taking remedial actions when missing values are detected, is key.
• In carrying out the main analysis, it is important to check for and account for autocorrelation.
• Even though the model statistics convey the direct message of the amount of change in the level and trend, interrupted time-series graphs may be more appealing, especially to the nonquantitative person.

Practical Lessons Learned

Did we learn any practical lessons? The answer is yes, the greatest being that we underestimated the effort needed in using routine data for retrospective impact evaluation. We also learned that networks remain an important component of many things in life, including conducting research. Finally, if you do not have a personal drive and the motivation to maintain the tempo and seek alternatives without losing your focus, retrospective impact evaluation can easily turn out to be a nightmare. We relied on our networks in the early stages of the study to determine the availability and accessibility of data. Given earlier hiccups at obtaining the data at the national level, we relied on formal and informal networks to obtain the data. The ability to obtain the raw data from a single access point also was a great plus, although there were issues with internet and network connectivity issues.

The fact that we needed to be open minded and adaptive, including toward the possibility of completely abandoning the research project, is yet another lesson learned. For example, we had to scale down the initial number of indicators from 17 to 3 due to issues with access.

Finally, we learned that we needed to find a way to engage nontechnical readers in a study that might be too technical for some to follow. The use of graphs to complement the regression output, in our view, makes our findings more appealing to noneconometric students.

• Do not underestimate the time and effort needed for a retrospective impact evaluation just because you will not collect primary data.
• Be open minded and ready to adjust.
• Always think about the consumers (readers) of your final product, and complement otherwise technical results with alternative presentations (e.g., graphs) that can be easily understood.

This case study highlights the procedures and steps necessary for a retrospective impact evaluation when using routine secondary data. Retrospective impact evaluation may be necessitated by the need to learn from an important program or policy that was implemented in the past or implemented halfway, as in the case of our study. Even where primary data collect events over time (e.g., parallel programs and interventions), it may be necessary to use retrospective thinking and analysis to isolate the effect of the program of interest.

Success depends largely on having access to a credible data source that is of good quality. When employing the interrupted time-series design, ensure that your data contain enough time points (at least 10 periods before the interruption and 10 periods after the intervention). In contemplating the use of DHIMS2 data for our study, we had to make an enquiry not just about the availability of the data but also about the period the available data covered. Data preparation, including checking for missing values and imputing where necessary, is important because the completeness and quality of the data will be reflected in the validity of the results at the end of the day. For example, in our study, we checked for missingness and accordingly imputed for missing values. These important procedures also depend on the assumption around the missingness and the requirement for imputation. For example, multiple imputation requires that the data are missing at random and that there is no systematic reason (e.g., missing only for some participants or indicators or some specific time), and that at least one of the available indicators should have zero missing data so that it can be used as the independent variable in the imputation model.

In time-series regressions such as the segmented regression, one must always check for autocorrelation and, if found it is to be present, use the appropriate models to correct for it. The generalized linear model, for example, will easily correct for autocorrelation and report the corrected test values. Interpreting results from interrupted time-series analysis is somewhat technical. Remember here that the aim is to determine whether the program or policy has led to a change in the level and trend of the outcome of interest. This is done by comparing the pre- and postintervention levels and trends. Intuitively, this might be difficult for nonquantitative readers to follow. Fortunately, it is possible to display vivid graphics of at least the level change and the slope of the trend. The graphs do not display quantities, but when used together with the regression coefficients, they become easier to understand.

Important lessons: Never underestimate the amount of work involved in retrospective impact evaluation. Always be open and willing to adjust because much of what can be done depends on data access, which can be incredibly frustrating especially in low -and middle-income settings, and the quality of the data. Finally, always remember that your output is useful only when readers can understand your message.

• Retrospective impact evaluation may be necessitated by the need to learn from an important program or policy that was implemented in the past or implemented halfway, as in our study.
• Even where primary data collect events over time (e.g., parallel programs and interventions), it may be necessary to use retrospective thinking and analysis to isolate the effect of the program of interest.
• In time-series regressions such as segmented regression, one must always check for autocorrelation and, if it is found to be present, use the appropriate models to correct for it.

Discussion Questions

• 1. What are the key attributes of retrospective impact evaluation?
• 2. If you plan to undertake a retrospective impact evaluation, what steps will you follow for data access, and why?
• 3. Discuss the steps involved at the preanalysis stage when using segmented regression for program impact evaluation.
• 4. Under what circumstances would you employ an interrupted time-series design for impact evaluation?
• 5. Complete case analysis is always better than imputing data for missing values. Discuss.

Multiple-Choice Quiz

1. One of the following is not necessary when employing the interrupted time-series design.

Incorrect Answer

Feedback: This is not the correct answer. The correct answer is C.

Correct Answer

Feedback: Well done, correct answer

2. Which of these is not a data imputation method?

Feedback: This is not the correct answer. The correct answer is B.

3. Which of the following is a test for autocorrelation?

4. Autocorrelation can be remedied in which of the following models?

Further Reading

Web resources, sign in to access this content, get a 30 day free trial, more like this, sage recommends.

We found other relevant content for you on other Sage platforms.

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

• Sign in/register

Navigating away from this page will delete your results

Please save your results to "My Self-Assessments" in your profile before navigating away from this page.

Sign in to my profile

Sign up for a free trial and experience all Sage Learning Resources have to offer.

You must have a valid academic email address to sign up.

Get off-campus access

• View or download all content my institution has access to.

Sign up for a free trial and experience all Sage Research Methods has to offer.

• view my profile
• view my lists

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Study on the methodology of emergency decision-making for water transfer project contingencies: A case-based reasoning and regret theory approach

Roles Formal analysis, Supervision, Writing – original draft, Writing – review & editing

Affiliation School of Water Conservancy, North China University of Water Resources and Electric Power, Zhengzhou, China

Roles Conceptualization, Methodology, Writing – original draft, Writing – review & editing

Roles Data curation, Investigation, Resources, Writing – review & editing

* E-mail: [email protected]

Affiliation College of Agricultural Science and Engineering, Hohai University, Nanjing, China

Roles Data curation, Resources, Supervision

Affiliation Henan Water & Power Engineering Consulting Co., Ltd., Zhengzhou, China

• Feng Li, 
• Xuewan Du, 
• Xin Huang, 
• Xiaoxia Fei

• Published: March 21, 2024
• https://doi.org/10.1371/journal.pone.0300272
• Peer Review
• Reader Comments

To tackle the global water imbalance problem, a multitude of inter-basin water transfer projects have been built worldwide in recent decades. Nevertheless, given the complexity and safety challenges associated with project operation, effective emergency decision-making is crucial for addressing unforeseen incidents. Hence, this research has developed a two-stage emergency decision-making framework to tackle the uncertainty in the development trends of emergencies in inter-basin water transfer projects. (1) The first stage mainly utilizes case-based reasoning techniques to extract historical case information and disposal plans for inter-basin water transfer projects. Subsequently, a holistic similarity model is built by employing structural similarity and local attribute similarity algorithms to identify highly similar historical cases. (2) The second stage involves the optimization and adjustment of decision-making plans based on the dynamic evolution characteristics of emergencies. It utilizes the theory of decision-makers regret psychology and combines it with practical case studies to verify the scientific rationality of the method. This enables it to achieve effective multidimensional expression and rapid matching of scenarios, satisfying the decision-making requirements of "scenario response". Finally, this study compares the results obtained from this method with those computed using the traditional TOPSIS method and fuzzy comprehensive evaluation method, further validating its feasibility and effectiveness. In practice, this method can provide effective support for decision-makers work.

Citation: Li F, Du X, Huang X, Fei X (2024) Study on the methodology of emergency decision-making for water transfer project contingencies: A case-based reasoning and regret theory approach. PLoS ONE 19(3): e0300272. https://doi.org/10.1371/journal.pone.0300272

Editor: Ahmed Mancy Mosa, Al Mansour University College-Baghdad-Iraq, IRAQ

Received: November 16, 2023; Accepted: February 24, 2024; Published: March 21, 2024

Copyright: © 2024 Li et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript ( Table 1 ).

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Cross-basin water transfer projects refer to major water conservancy projects that cross two or more river basins or regions to carry out significant water diversion or transfer [ 1 ], and will significantly affect water supplies, hydrology, and the environment in both donor and receiving basins [ 2 – 4 ]. In recent years, a number of major inter-basin and inter-regional water diversion and transfer projects have been built around the world in order to promote water disaster prevention and control, water resource conservation, water ecology protection and restoration, and water environment management, as well as to solve the problems of global summer floods and winter droughts, the lack of the north and the south, and the imbalance in the distribution of water resources in time and space. It provides strong support for addressing insufficient water resource demand, and water scarcity [ 5 ], and promoting high-quality development of water conservancy in a new stage, thus providing strong support for sustained and healthy economic and social development [ 6 , 7 ].

According to statistics, there are currently over 160 water transfer projects worldwide [ 8 , 9 ], all of which have played an important role in flood control, alleviating water shortages in various regions. As a major water transfer project in China, the South-to-North Water Diversion Project has transferred 65.4 billion cubic meters of water since the completion of the main construction of the Middle Route Phase I and the Eastern Route, directly benefiting as many as 176 million people and replenished 10 billion cubic meters of water for ecological purposes [ 10 ]. It has played an important role in economic and social development as well as ecological environment protection, serving as a strategic infrastructure project fundamentally addressing water scarcity in North China and Northwest China [ 9 ]. However, the inter-basin water transfer project is a typical interconnected system engineering. The route of middle route of the South-to-North Water Diversion Project shown in Fig 1 has the characteristics of crossing different river basins and a long water diversion route. Once construction safety and quality issues or sudden events such as water resource pollution occur, it is highly likely to cause huge environmental and economic losses to the project itself or the surrounding environment [ 11 , 12 ], often accompanied by significant hidden dangers, and may even result in serious injuries or deaths. Therefore, strengthening research on emergency rescue and disposal decision-making for major water diversion projects, establishing effective emergency management measures for sudden events, ensuring the normal and continuous operation of the project, maintaining the safety of people’s lives and property, and promoting the stable development of the social economy are of great significance and practical importance.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0300272.g001

The serious consequences of environmental emergencies continually remind governments of the importance of emergency decision-making, rousing meaningful exploration and practice around the emergency organization structure, legal system, and other aspects [ 13 – 15 ]. Currently, there are a large number of Inter-basin water transfer projects that have been the subject of numerous studies [ 16 – 19 ]. However, emergency response and decision-making, which are essential components of emergency management, still lack sufficient support in the research on emergency response to sudden incidents in cross-basin water transfer projects. When a disaster occurs, decision-makers must develop emergency action plans within a limited time to maximize the objectives of emergency decision-making. However, due to the complex and intertwined causes of sudden events, decision-makers not only need to identify and anticipate risks before an event occurs but also need to make timely and effective solutions when a danger arises. Under such immense pressure, the dynamic disposal process based on emergency decision-making [ 20 ] is likely to lead to failed emergency measures that are difficult to handle using conventional methods. Therefore, it is crucial to improve the quality and efficiency of emergency decision-making by establishing emergency decision-making models that are suitable for the characteristics of sudden events. Consequently, case-based reasoning technology has emerged and matured, being applied in multiple fields [ 13 , 21 – 23 ]. Additionally, in existing research, scholars have mostly focused on selecting emergency plans based on the expected utility of one-time decision plans, without considering the expected regrets of the plans. In practical decision-making processes, decision-makers not only consider the expected utility of emergency plans but also take into account the expected regrets of overreacting. Therefore, in case-based decision-making, decision-makers often exhibit the following psychological behavior: when retrieving the most similar historical cases, if they find that choosing other historical cases is more similar to the target case, they will feel delighted with their decision. Conversely, they will feel regretful. Compared to current commonly used methods such as prospect theory [ 24 , 25 ], regret theory does not require decision-makers to provide reference points and involves fewer parameters in calculations, making it computationally simpler [ 26 ]. In recent years, this theory has been widely applied in fields such as group decision-making and hesitant fuzzy multi-criteria decision-making [ 27 ], effectively avoiding decision-makers selection of historical cases that would cause regret [ 28 ].

In conclusion, this study focuses on the uncertainty and multiple attribute issues of emergency events in inter-basin water transfer projects. Taking into consideration the decision-makers regret psychology, a case-based reasoning approach is adopted to analyze the decision-making of emergency events in a specific section of the South-to-North Water Diversion Project based on historical experience, data, and knowledge inference. This study makes three main contributions: (1) In response to the uncertainty of the development trends of emergency events in inter-basin water transfer projects, a two-stage emergency decision-making plan is constructed. (2) By fully considering decision-makers regret avoidance behavior, the theoretical and practical significance of regret theory in selecting emergency response plans for sudden events is studied and compared with other methods to verify its applicability. (3) Strict differentiation is applied to accident information indicators in emergency events of inter-basin water transfer projects, and hesitant fuzzy numbers are used to represent decision-making information, which enhances the effectiveness of emergency response.

The structure of the remaining parts of this study is as follows. Part 2 introduces the research methodology of this paper and proposes research questions and solutions based on it. Part 3 takes the decision-making research on the siphon backflow emergency event in a certain section of the South-to-North Water Diversion Project as an example to validate the applicability of the research methodology. Part 4 selects the TOPSIS method and fuzzy comprehensive evaluation method to optimize the selection of preliminary plans and compares them with regret theory. Finally, Part 5 draws conclusions based on the entire paper, while summarizing the main contributions and shortcomings as well as suggesting directions for further research.

Research methodology

Case-Based Reasoning (CBR) theory is a method that solves current case problems by using the solutions to similar historical cases. It helps decision-makers make choices based on similarity and implementation efficiency. If the historical cases used for reference are less relevant to the target case, the generated solutions based on the historical cases will also be less effective. Therefore, applying solutions from similar historical cases with poor implementation effects to the target case could result in poor decision-making [ 29 ]. Through case reasoning, decision-makers reasoning speed can be improved, the efficiency of emergency rescue and disposal decision-making can be increased, and the feasibility of the final plan can be ensured. To solve the problem of emergency decision-making for sudden events in the South-to-North Water Diversion Project, this study describes and analyzes the problem using this method. In the following sections, some related background knowledge is introduced to make the method more accessible.

Problem description

(1) Question 1: How to generate an initial emergency plan from the emergency plan repository based on the current operation status of the water diversion project?

In anticipation of future accident scenarios, let us assume that the emergency decision rescue team has developed m emergency response plans. Where A i represents the i -th emergency response plan ( i ∈ M ). In the course of emergency rescue operations, the assessment of emergency response plans must take into account the evaluation criteria encompassing personnel casualties, economic losses, and social impacts. Notably, personnel casualties and economic losses are categorized as cost-type attributes, whereas social impact is considered a benefit-type attribute, denoted as D = { D 1 , D 2 ,⋯, D n }, with D j denoting the j -th attribute ( j ∈ N ).

(2) Question 2: How can decision-makers dynamically adjust in a risk environment based on the dynamics of water diversion project accidents and the uncertainty of information? In the following chapters, this study specifically addresses the above issues.

Background knowledge

In order to address the above-mentioned issues, this article uses a research methodology that combines Case-Based Reasoning (CBR) and regret theory to seek solutions to decision problems in water diversion project emergencies. Therefore, in this section, the concepts and definitions of scoring function, deviation function, utility function, and regret theory are introduced in sequence, as well as their meanings in the research on emergency decision-making problems, in order to better understand the research methodology of this article.

Scoring function.

The scoring function quantifies and scores the features of emergency decision-making solutions based on specific evaluation indicators or criteria, aiming to provide a quantitative assessment tool for emergency decision-making, helping decision-makers compare and select different decision-making solutions. In recent years, the scoring function has been recognized as an important tool for studying interval intuitionistic fuzzy multi-criteria decision-making problems, attracting significant attention from numerous experts and scholars and yielding many research achievements. According to Wang Xuefang, a reasonable and effective scoring function needs to consider membership, non-membership, hesitation, and their tendencies simultaneously [ 30 ]. Based on this, the definition of the scoring function used in this paper is as follows:

Bias function.

The deviation function is a function used to describe the cognitive biases or decision biases that decision-makers may have in emergency decision-making. It measures the difference between the decision-maker’s decision and the rational decision, considering this difference as a manifestation of decision bias. The application of the deviation function in emergency decision-making research can help decision-makers identify and diagnose decision biases, and then propose corresponding decision support methods and intervention measures. In specific decision analysis models, the deviation function is usually defined together with utility functions or loss functions, such as prospect theory, mental accounting, and regret theory, helping decision-makers integrate considerations of benefits and risks to make optimal emergency decisions. Here is the definition of the deviation function:

Utility function.

The utility function is a function used to evaluate the relative value of different decision options. It maps the outcomes of emergency decisions onto a numerical range, reflecting the utility or benefits to decision-makers from different strategies. In the dynamic adjustment of emergency plans, different decision-makers have varying utilities for adjusting costs, adjustment losses, and disposal effects of emergency plans, meaning that the corresponding utility functions are not the same. Specifically, the utility function maps the decision outcomes to a real number domain, reflecting the value or utility of the results for decision-makers. Since the utility function embodies the preferences of decision-makers, utilizing it to perform utility analysis on various adjustment plans leads to decision outcomes that are more in line with reality [ 31 ]. The utility function used in this paper is defined as follows:

Regret theory.

Among them, V ( x ) and V ( y ) respectively represent the utilities that the decision-maker can obtain from the outcomes of options A and B , and R ( V ( x )− V ( y )) is the regret-rejoicing value. When R ( V ( x )− V ( y ))>0 is a rejoice value, it indicates that the decision-maker feels rejoice for choosing option A over option B . When R ( V ( x )− V ( y ))<0 is a regret value, it indicates that the decision-maker feels regret for choosing option A over option B . Here, the regret-rejoicing function is a monotonically concave function, and when V ( x )− V ( y ) = 0, then R ( V ( x )− V ( y )) = 0.

This modified utility function incorporates the factor of regret, which can better explain decision phenomena that deviate from the traditional expected utility function theory as found in many empirical studies. Later, Quiggin further extended this theory to the case of a general choice set, enabling it to include the selection of the optimal option from a set of multiple decision alternatives.

Where x * = max{ x i }, R ( V ( x i )− V ( x* ))≥0 indicates a rejoicing value and R ( V ( x i )− V ( x* ))≤0 represents a regretful value. The regret-rejoicing function still follows a monotonically increasing concave pattern.

Model construction

In this chapter, from a two-stage perspective, we study how decision makers should generate preliminary decision plans and how to make dynamic adjustments. The first stage is based on CBR theory to generate preliminary emergency decision plans. The second stage is based on regret theory, combined with hesitant fuzzy utility function, to calculate the regret value of each emergency plan. Then, the comprehensive utility value and regret value are integrated to obtain the perceived utility of each emergency plan. Finally, the plans are ranked using perceived utility and emergency plans are adjusted according to the actual situation. The specific process is shown in Fig 2 .

https://doi.org/10.1371/journal.pone.0300272.g002

Stage 1: Existing cross-basin water transfer project accident cases are integrated as a case library, and the indicator information of historical cases is extracted to describe the current information based on weather forecast information and the status of historical accident cases. In addition, due to the complexity and diversity of cases, their attribute values may exist in various forms. Therefore, this paper considers using a heterogeneous calculation method that combines qualitative and quantitative aspects as the attribute type for the problem. This method includes four types: symbolic, ordinal enumeration, precise numerical, and fuzzy linguistic types. For instance, in the case of accidents in cross-basin water transfer projects, the evaluation value of "whether there is flood discharge upstream" is calculated using the symbolic types "yes" and "no" to measure similarity. "Rainfall," "distance to town," and "population of town" are expressed using precise numerical values. "Flood control water level" is represented using fuzzy linguistic type, while "response level" is represented using ordinal enumeration type. The similarity calculation methods for each attribute are as follows:

In the equation: X j represents the j -th attribute of the target case X , Y ij represents the j -th attribute of the source case Y i . sim ( X j , Y ij ) represents the similarity in the j -th attribute between the target case X and the source case Y i . x j and y ij represent the symbol attribute values corresponding to attribute j of target case X and source case Y i , respectively.

In the formula: x j and y ij are the data corresponding to the j -th attribute of the target case X and the source case Y i , respectively. min( j ) and max( j ) represent the minimum and maximum values of attribute j in the case base.

In the formula: g is the number of levels for attribute j .

SIM ( X , Y ) represents the overall similarity between the target case and the source case. By considering the overall similarity, we can identify source cases that are relatively similar. The incident handling method of the selected source case can be used as the disposal plan for the current incident, providing a reliable reference for the current incident handling.

Stage 2: Building on the first stage’s disposal plan, utilize current valid information to predict and analyze the accident’s development trend, and provide corresponding disposal plans. This involves normalizing hesitant fuzzy numbers and using hesitant fuzzy utility functions to transform the hesitant fuzzy risk decision matrix into a comprehensive utility decision matrix. Furthermore, based on regret theory, calculate the regret values for implementing different emergency plans in each scenario following the occurrence of inter-basin water transfer project accidents. Then, calculate the sum of the comprehensive utility value and regret value for each emergency plan to determine the emergency plan’s comprehensive perceived utility. Finally, based on the magnitude of the comprehensive perceived utility of the emergency plan, establish the emergency plan ranking.

(1) Calculation of the combined utility value

(2) Regret value calculation

Based on regret theory, in the decision-making process, decision makers typically compare the selected option with other alternative options. If choosing another option results in greater utility, the decision maker experiences regret. If choosing another option would lead to unfavorable outcomes, the decision maker experiences delight.

In the equation, δ is the regret aversion coefficient, δ >0. The greater δ is, the higher the degree of regret aversion for the decision-maker. Δ V represents the utility difference of the solution.

https://doi.org/10.1371/journal.pone.0300272.g003

(3) Sequencing of emergency programmes

Finally, emergency plans are ranked based on the magnitude of the comprehensive perceived utility φ ( A i ), where a larger φ ( A i )value indicates a better outcome after implementing emergency plan A i .

In this chapter, we use the decision-making process for a sudden backflow incident in a cross-basin water transfer project in the South-to-North Water Diversion as the research background to validate the applicability of the research methodology proposed in this paper. The current accident situation involves flooding caused by a combination of high upstream discharge and rainfall in the downstream section of the river. The water level near the intake of the backflow phenomenon is close to the design water level, and the upstream head of the intake is eroded by the flood. The emergency management department has urgently initiated a level III emergency response. To effectively respond to potential sudden incidents in the cross-basin water transfer project, the water transfer project management department has corresponding disposal plans and accident cases for reference. However, as these cases are derived from the information since the operation of the South-to-North Water Diversion Project on the one hand, and on the other hand, it needs to be combined with practical experience and consultation with the relevant experts. Therefore, this paper sets target case and 10 inverted siphon project contingency scenarios as source cases to verify the effectiveness and applicability of this paper’s model based on practical experience and consultation with relevant experts. The specific data of the source and target cases are shown in Table 1 .

https://doi.org/10.1371/journal.pone.0300272.t001

Using Formula ( 10 ), the similarity between the current flood season siphonage engineering accidents and and the source cases set out in this paper, and the results are shown in Table 2 :

https://doi.org/10.1371/journal.pone.0300272.t002

Through analysis of the computational results, it is evident that the similarity score SIM ( Y 8 , X 0 ) = 0.911 is the highest, indicating that the source case Y 8 is most similar to the target case X 0 . Analyzing the specific information contained in the current accident situation, it is apparent that the historical accident details of source case Y 8 closely align with it. Hence, its disposal measures can be utilized to address the current accident. Choosing source case Y 8 as the disposal plan for the current accident is scientifically justified.

Step 1: following the completion of the preliminary disposal, experts in water diversion project accident management, on-site management leaders, and rescue command leaders analyze the development trends and disposal measures for the back-siphonage engineering accident during the current flood season based on a comprehensive analysis of the on-site accident scenario. Through extensive deliberation, the expert team predicts the potential occurrence of the following three scenarios:

• Scenario S 1 : As analyzed by the meteorological department, the rainfall is expected to weaken, the water level of the flood is starting to decrease, the riverbank slopes are experiencing minor erosion, and the flood is beginning to scour the right bank embankment. The probability of occurrence for this scenario is 0.4.
• Scenario S 2 : As analyzed by the meteorological department, the rainfall is projected to continue, the flood control water level will be maintained at a constant level, the riverbank slopes will undergo continuous erosion, and hindrance to flood flow will occur at the slope section of the connecting channel of the upstream and downstream of the inverted siphon on the right bank. The safety of the masonry head of the inverted siphon on the right bank is under threat. The probability of occurrence for this scenario is 0.4.
• Scenario S 3 : As analyzed by the meteorological department, the rainfall is expected to intensify continuously. Upstream reservoirs have initiated flood discharge, resulting in a rapid surge of river water levels. The severe erosion of riverbank slopes poses a threat to the base of the slope. Moreover, the pipeline section exhibits hidden risks, and there is an escalated risk of flood overflow at this time. It is highly likely to breach the flood protection embankment, leading to infrastructure destruction and detrimental impacts on the safety of residents’ lives and properties. The probability of this scenario occurring is 0.2.

In addition, after adopting the disposal plan of source case Y 8 as the initial disposal measure for the current accident, adjustments need to be made according to the development trend of the accident. There are three possible options to choose from:

• Plan A 1 : The plan involves setting up on-site alerts and using multimedia to remind the surrounding population to minimize outdoor activities and stay away from hazardous river areas while organizing mass evacuations. It also includes opening up rescue channels, ensuring unobstructed access for on-site rescue roads, maintaining normal electricity supply, guaranteeing the proper functioning of emergency communication equipment, organizing the orderly entry of various disaster relief equipment, rescue materials, and rescue personnel, and maintaining on-site order.
• Plan A 2 : Building on the implementation of the Plan A 1 , the rescue team sets up lighting at the scene, organizes the lighting, removes the mesh fence around the headgear, and the rescue personnel produce and throw willow stones to reinforce the protective slope. They also produce and throw wire mesh stone cages to the collapsed section of the headgear, and use mechanical equipment to reinforce the backflow phenomenon of the water transfer project.
• Plan A 3 : Building on the implementation of Plan A 2 , the specialized rescue team seals the breach in the flood embankment, provides protection for the eroded landslide channel slopes, raises the flood embankment, and constructs a new modular flood sub-embankment for protection.

When decision experts consider the selection of accident disposal plans, the primary attributes to be considered include three indicators: "casualties D 1 ", "economic losses D 2 ", and "social impact D 3 " within 24 hours after the accident. Here, indicators D 1 and D 2 are cost-based indicators, while D 3 is a benefit-based indicator. The attribute weight vector provided by the decision experts is denoted as w = (0.7,0.2,0.1) T .

https://doi.org/10.1371/journal.pone.0300272.t003

https://doi.org/10.1371/journal.pone.0300272.t004

https://doi.org/10.1371/journal.pone.0300272.t005

https://doi.org/10.1371/journal.pone.0300272.t006

Based on the principle that the greater the comprehensive perceptual utility of the emergency response plan, the better the effectiveness of its implementation, the emergency response plans are ranked as A 1 > A 2 > A 3 . Therefore, the implementation effect of emergency response plan A 1 should be the best. Consequently, after a back-siphonage engineering accident occurs during the flood season, first, measures should be taken to lead the public to a safe zone and conduct basic treatment of the accident to minimize the damage of floods to riverbank slopes and the back-siphonage engineering. Second, professional emergency rescue teams should be called upon to carry out targeted and specialized salvage operations on the key points of the accident using professional equipment to ensure safety and reduce losses. Finally, reinforcements should be made to flood control embankments to effectively reduce the risk of flood overflow and river channel collapse, thereby maximizing the protection of people’s lives and property.

To validate the differences between the regret theory and traditional objective methods, this study adopts two decision-making methods, TOPSIS and fuzzy comprehensive evaluation. Upon completing the initial case selection, the solutions are further optimized, and compared with the regret theory proposed in this study. Firstly, we need to preprocess the interval numbers of the hesitant fuzzy risk matrix in Table 2 and defuzzify it into a fixed value. Then, the matrix is standardized and substituted into the formulas of the two methods.

The fuzzy comprehensive evaluation method is a comprehensive evaluation and decision-making method based on fuzzy mathematics. It employs mathematical language to describe real-life problems with fuzzy boundaries and multiple levels, and provides a scientific solution using mathematical methods. In this study, M (∧,∨), M (•,+), M (•,∨), M (∧,⊕) four operators are selected to calculate the standardized emergency decision matrix, and the higher the comprehensive evaluation value, the better the solution. In comparison of the two algorithms, TOPSIS assumes that the relationship between evaluation indicators is linear and does not consider the mutual influence between indicators. This method is simple and easy to understand, has small computational complexity, and can intuitively reflect the degree of closeness between each solution and the ideal solution. On the other hand, fuzzy comprehensive evaluation method introduces fuzzy set theory to divide the values of evaluation indicators into fuzzy sets in order to obtain evaluation results. This method can handle the uncertainty and fuzziness between evaluation indicators, and is more suitable for decision-making problems where evaluation indicators cannot be accurately quantified in practical situations.

In Fig 4 , the evaluation results of the schemes are obtained based on five mathematical calculation methods of two models. According to the table, the results obtained by the TOPSIS method and M (•,+), M (•,∨) operator are A 3 > A 2 > A 1 . The results obtained by M (∧,⊕), M (∧,∨), using two operators, are A 3 = A 2 > A 1 . The result calculated using regret theory in this paper is A 1 > A 3 > A 2 , and the results obtained by multiple methods are different. Decision information is one of the important bases for emergency decision-making in unconventional emergencies. The important content of emergency decision-making is the process of information collection, processing, and feedback. Considering that the scenarios of engineering operational safety accidents are not exactly the same, different accidents have differences in weather, environment, human factors, and so on. Therefore, emergencies are characterized by sudden occurrence, high time pressure, and lack of precedent reference, which determines the extreme lack of decision information in the context of emergencies and the difficulty in obtaining it. Objective decision-making methods often fail to fully consider the actual situation and cannot achieve a correct and objective judgment based on the real circumstances. Comparing TOPSIS method and fuzzy comprehensive evaluation method, regret theory can take into account psychological behavioral characteristics of decision-makers such as reference dependence, loss aversion, diminishing sensitivity, and regret aversion. Compared to directly evaluating the quality of solutions, regret theory can more fully consider decision-makers emotional attitudes towards different outcomes and reduce the feeling of regret caused by decision-making. In addition, regret theory can better handle uncertainties in decision-making problems, not just limited to numerical evaluation indicators. Therefore, it is of significant importance and imperative to apply regret theory in the development and further exploration of emergency plan selection methods for sudden events, taking into account decision-makers’ psychological behaviors.

https://doi.org/10.1371/journal.pone.0300272.g004

In conclusion, the comparative analysis results illustrate the applicability and effectiveness of the proposed regret theory approach for decision-making in engineering operational safety accident handling schemes. This suggests that the new decision-making method combining CBR and regret theory, when applied to emergency decision-making research for unexpected events in the South-to-North Water Diversion Project, is both reasonable and practical.

The siphon-style inter-basin water transfer project, as an important cross-basin water transfer project, its safe operation is crucial to ensure the normal water transfer of the water diversion project. From the perspective of systems engineering, this paper combines historical data and practical experience, based on CBR theory and regret theory, to calculate the perceived utility of the schemes, and then conducts a detailed analysis of the generation and dynamic adjustment of emergency plans in the accident handling of the water diversion project. The main research conclusions of this paper are as follows:

• The selection problem of emergency plans for cross-basin water transfer project sudden events is an important issue worthy of attention. Its research can provide guidance and reference for the defense and rescue of sudden events in practical life, with practical application significance. This paper conducts decision-making research on the siphon-type sudden event in a section of the South-to-North Water Diversion cross-basin water transfer project. Based on the ranking results of the maximum comprehensive perceived utility of emergency response plans, Plan A 1 is finally determined as the optimal plan. In other words, when a sudden event occurs, decision-makers should first take measures to protect the public and carry out basic treatment of the accident, reducing the damage of floods to riverbank slopes and siphon-type projects. Secondly, professional emergency rescue teams should be called upon to conduct targeted and specialized rescue work on key points of the accident using professional equipment. Finally, the flood embankment should be reinforced to effectively reduce the possibility of flood overflow and river channel collapse, maximizing the protection of people’s lives and property.
• Considering the decision-maker’s behavior is necessary in the selection method of emergency response plans. This paper incorporates regret theory under the consideration of decision-makers’ psychological behavioral characteristics and uses TOPSIS method and fuzzy comprehensive evaluation method to optimize the plans, comparing the results with those obtained from regret theory. Finally, it is concluded that the calculation results of TOPSIS method and M (•,+), M (•,∨) operator are A 3 > A 2 > A 1 , the calculation results of M (∧,⊕), M (∧,∨) operator are A 3 = A 2 > A 1 , and the results calculated using regret theory are A 1 > A 3 > A 2 . Therefore, people often subconsciously consider regret factors before making decisions, and these psychological behaviors can have an impact on the decision analysis process and results. Considering decision-maker behavior in research is a more objective and scientific approach.
• Emergencies possess suddenness and complexity, and existing research on decision-making methods usually aims to solve specific problems, lacking systematic and accurate approaches. This study applies CBR theory and regret theory to investigate the emergency plan selection problem for inter-basin water transfer projects, taking the case of a siphonage emergency in a section of the South-to-North Water Diversion Project. It verifies the rationality and applicability of this new decision-making method.

The main contributions and innovations of this paper are as follows:

• The main contribution of this study is the construction of an emergency decision-making plan for cross-basin water diversion projects, which consists of two stages. In the first stage, case-based reasoning techniques are employed to extract historical case information and disposal plans of engineering projects, and to select highly similar historical cases. In the second stage, the decision-making plans are optimized and adjusted based on the dynamic evolution characteristics of emergencies and the theory of decision-makers’ regret psychology. Finally, the scientific rationality of this method is validated through practical case studies, enabling it to achieve effective multidimensional expression and rapid matching of scenarios, thereby satisfying the decision-making requirements of "scenario response".
• Fully considering decision-makers’ regret avoidance behavior, this study investigates the theoretical and practical significance of regret theory in the selection of emergency plans for emergency events. Finally, the applicability of the TOPSIS method and fuzzy comprehensive evaluation method is compared with regret theory to validate its effectiveness, which not only overcomes the impact of decision experts’ bounded rationality on decision results but also provides a new direction for the knowledge of emergency plan selection methods for emergency events.
• The accident information indicators in the inter-basin water transfer project emergency events are strictly differentiated. Considering the variety of attribute indicators in emergency events, this study describes the accident information of water transfer projects using four types: symbolic, ordinal enumeration, precise numerical, and fuzzy linguistic. Combined with the CBR method, historical case information that is more similar to the current information is selected to provide reference for the early disposal of accidents. Additionally, taking into account the excessive uncertainty factors and divergent information from decision experts after the occurrence of water transfer project accidents, hesitant fuzzy numbers are employed to represent decision information, thus enhancing the effectiveness of emergency response.

Despite this, there are still certain shortcomings in this study that need further improvement. For example: (1) How to rapidly and accurately obtain the state probabilities and attribute weights involved in emergency risk decision-making for sudden events. (2) How to fully grasp the changes in a situation in a short period of time and accurately estimate potential losses. (3) How to optimize the allocation of resources and achieve the best implementation effect in the event of conflicts arising from different sub-plans occupying the same resources in an emergency plan combination.

• View Article
• PubMed/NCBI
• Google Scholar

• Open access
• Published: 16 March 2024

The relationship between pregnancy and temporomandibular disorder (TMD) through diagnostic criteria for temporomandibular disorders (DC/TMD) axis II evaluation: a case-control cross-sectional study

• Giuseppe Minervini 1 , 2 ,
• Maria Maddalena Marrapodi 3 ,
• Marco La Verde 3 ,
• Aida Meto 4 ,
• Yuliia Siurkel 5 ,
• Marco Cicciù 6 &
• Diana Russo 2  

BMC Oral Health volume  24 , Article number:  342 ( 2024 ) Cite this article

155 Accesses

Metrics details

Introduction

This study focuses on temporomandibular disorders (TMDs), which affect the temporomandibular joint and related muscles and have multiple causes. Recent studies have examined the connection between menstrual cycles, estrogen levels, and TMDs, but results are inconsistent, highlighting the need for more research. The aim is to explore the prevalence of TMDs in pregnant women and consider how hormonal changes during pregnancy might influence these disorders.

In this cross-sectional case-control study, we compared 32 pregnant women with 35 non-pregnant women. We evaluated several TMD-related factors such as pain levels, chronic pain classification, scores on the Jaw Functional Limitation Scale-20 and Oral Behaviors Checklist, and psychological health. We used various statistical methods including descriptive statistics, chi-square tests, linear regression, and adjustments for multiple comparisons to analyze the data.

Pregnant women showed different pain perceptions, generally reporting less pain and lower severity. Nonetheless, these differences were not uniform across all TMD-related measures. Linear regression did not find a consistent link between pregnancy and TMD scores, except for chronic pain grade, which was not significant after adjusting for multiple comparisons. There was a significant relationship between depression and TMD severity, emphasizing the need to consider mental health in TMD evaluations.

The findings suggest that pregnancy is neither a risk nor a protective factor for TMD. Differences in pain perception, functional status, and psychological health were observed in pregnant women but were not consistent for all TMD-related aspects. The role of estrogen in TMJ health and TMD risk is complex and requires further study. The research highlights the necessity of including mental health, especially depression, in TMD assessments. More comprehensive research with larger sample sizes is essential to better understand the connections between pregnancy, TMD, and hormones, aiming to improve TMD management in pregnant women and others.

Peer Review reports

Temporomandibular disorders (TMDs) encompass a variety of conditions impacting the temporomandibular joint (TMJ) and its associated musculature [ 1 , 2 ]. These conditions affect approximately 15% of adults and are most prevalent between the ages of 20 and 40, often manifesting as symptoms such as jaw discomfort, dysfunction, otalgia, facial pain, and headaches [ 2 , 3 , 4 , 5 ]. Their etiology is multifactorial, influenced by biological, environmental, social, and psychological factors [ 6 , 7 ]. Given this complexity, the diagnosis and treatment of TMDs require a comprehensive approach that encompasses various diagnostic and therapeutic modalities. In the evaluation of TMDs, polysomnography serves as a crucial diagnostic tool, particularly in understanding the relationship between sleep disorders, such as sleep bruxism, and TMDs. Treatment approaches include physiotherapy, pharmacotherapy, psychotherapy, and the use of occlusal splints, addressing both the physical and psychological aspects of the disorder.

Recent investigations have delved into the potential correlation between varying menstrual states and estrogen levels with the incidence of TMDs [ 8 , 9 ]. Specifically, numerous studies have explored the connection between pregnancy and TMD prevalence [ 8 ]. Estrogens, known for their role in regulating various physiological functions, including reproductive organ development, have been evaluated in relation to TMD [ 10 , 11 , 12 , 13 , 14 ]. The findings from these studies have reported conflicting findings, with some suggesting a possible association between elevated estrogen levels and heightened TMD risk, while others have found no such linkage [ 15 , 16 , 17 , 18 , 19 , 20 , 21 ]. Notably, estrogen levels can exert an influence on the structure and function of the TMJ [ 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 ].

Key insights have emerged from studies showing gender-related differences in estradiol concentration among individuals aged 15 to 45, implying a potential connection between estrogen levels and TMD, as well as other oral conditions like gingivitis [ 8 ]. However, epidemiological investigations investigating the relationship between estrogen levels and TMD prevalence and severity have yielded contradictory findings. On one hand, evidence indicates that TMDs predominantly affect women of childbearing age, implying a role for higher estrogen levels in increasing TMD risk [ 17 , 21 ]. On the other hand, several epidemiological studies have identified an elevated risk of TMDs following menopause [ 18 , 19 , 37 , 38 ].

From a pathogenic perspective, estrogen’s impact on cartilage has been noted, with certain studies demonstrating its protective effects on cartilage and bone [ 8 ]. Notably, estrogen treatment has resulted in a significant reduction in cartilage thickness and an increase in proteoglycan content [ 8 ]. These outcomes suggest that estrogen may play a part in maintaining TMJ cartilage integrity, potentially offering avenues for estrogen-based treatments for TMJ disorders.

Pregnant women often encounter an array of stressors that can potentially affect both maternal and fetal health. Chronic or severe stress during pregnancy has been associated with various adverse outcomes, including preterm birth, low birth weight, and developmental delays in offspring. Maternal stress may also lead to elevated levels of cortisol and other stress-related hormones, which can potentially traverse the placenta and impact fetal development. While a certain degree of stress during pregnancy is normal, healthcare providers should remain vigilant about potential stressors and offer support to help women manage stress levels, thus mitigating potential adverse effects on maternal and fetal health [ 35 , 39 , 40 , 41 , 42 ].

Of note, the study by Radwan-Oczko et al. focuses on pregnant women's knowledge regarding oral health during pregnancy. It found that awareness about the importance of oral hygiene in this period is generally low. Only a minority of women had oral examinations before or during early pregnancy. The research also highlighted a positive correlation between frequent tooth brushing and higher birth weights, indicating the significance of oral care in pregnancy outcomes. This study underscores the need for better education and awareness among both pregnant women and healthcare providers about oral health during pregnancy.

TMDs can significantly affect an individual's quality of life and sleep. The pain severity associated with TMDs correlates with reduced life satisfaction and sleep quality. Understanding this relationship is crucial in developing comprehensive treatment plans that not only address the physical symptoms of TMDs but also consider their broader impact on an individual's well-being.

In light of this substantial body of evidence, our objective is to investigate the prevalence of TMDs among pregnant women.

Participants

In this case control cross-sectional study, we consecutively enrolled pregnant women and non-pregnant woman.

The study was conducted in accordance with the Declaration of Helsinki and this study was approved by the institute’s ethical committee of ALDENT UNIVERSITY [Protocol no. 846/2022; Date: 05/05/2022]The study was developed, and all subjects gave their written informed consent for inclusion before they participated in the study. Minors or illiterate are not involved in this study.

Pregnant Women Group: A total of 32 pregnant women formed this group. Comprehensive demographic data, including age, education, and pregnancy status, were gathered for each participant within this group.

Healthy Control Non-Pregnant Women Group: In parallel, a control group was assembled, encompassing 35 healthy non-pregnant women. Similar to the pregnant women group, detailed demographic information was collected for each participant within this control group, including age and educational background.

The study investigated various TMD-related variables using different scales and assessments:

The Chronic Pain Grade (CPG) scale, also known as the Graded Chronic Pain Scale, is a tool used to assess and classify chronic pain in individuals. It was developed by researchers at the University of Washington in Seattle and is designed to provide a more comprehensive understanding of a person's chronic pain experience beyond just intensity. The CPG scale takes into account several dimensions of chronic pain, including:

Pain Intensity: This dimension assesses the severity of pain on a scale from 0 to 10, with 0 being no pain and 10 being the worst pain imaginable.

Pain-Related Disability: This dimension evaluates how much chronic pain interferes with a person's daily activities, including work, social life, and self-care.

Days in Pain: This dimension assesses how many days in the past six months a person has experienced significant pain.

Pain Intensity Variability: It considers whether the pain is relatively constant or if it fluctuates over time.

Pain Medication Use: This dimension examines the use of pain medication and whether it provides relief. Based on the scores in these dimensions, individuals can be categorized into one of five grades, ranging from Grade 0 to Grade IV.

JPLS-20: The Jaw Functional Limitation Scale-20 subscales, including Mastication, Mobility, Communication, and Global, provide a comprehensive assessment of TMJ-related function and overall health status.

Oral Behaviors Checklist (OBC): OBC was administered to ascertain the presence of oral behaviors associated with TMD. Scores on this checklist were compared between the pregnant women group and the control group.

Psychological Well-being: The psychological well-being of participants was a key aspect of the study and was evaluated using the Patient Health Questionnaire-9 (PHQ-9) to assess depression levels. Furthermore, the Generalized Anxiety Disorder-7 (GAD-7) scale was employed to assess anxiety levels within the study cohort.

Statistical analysis

A comprehensive suite of statistical analyses was undertaken to unveil relationships and discrepancies between pregnancy status (pregnant women vs. control group) and the diverse TMD-related variables. These analyses were detailed as follows:

An application of descriptive statistics was undertaken to provide an in-depth summary and comparative analysis of pain perception, chronic pain grade, JFLS scores, and OBC scores, quantifying the differences between pregnant women and the control group. Variables were treated as categorical variable based in the following standardized categories [ 43 , 44 , 45 ]. To compare the percentage of patients for each group between pregnant and non-pregnant women a chi square test was used.

To explore the intricate relationship between TMD scores and pregnancy status, a linear regression analysis was executed. This statistical technique allowed for the adjustment of potential confounding variables, including age, education, and depression (PHQ-9), providing a robust assessment of the associations. Independent variables were treated as continuous variables.

The rigorous application of corrections for multiple comparisons was paramount in this study when assessing the significance of the results. The most restrictive approach was used, Bonferroni correction. This approach was adopted to maintain the integrity of the Type I error rate control.

Sample size calculation

To calculate the sample size for this study, we considered two groups: the pregnant group with an OBC score of 11 plus 9 standard deviations (SD) and the control group consisting of non-pregnant individuals with 24 participants. With a desired statistical power of 90% (1 - β), an alpha (α) level of 0.05, and an expected result of 27 participants per group, we employed a statistical formula or software to estimate the required sample size. It was determined that a sample size of 27 participants per group would be necessary to achieve a power of 90% at a significance level of 0.05, ensuring that the study is adequately powered to detect the anticipated effects or differences between the two groups.

Population characteristics

Demographics are displayed in Table 1 . The study sample consisted of 32 participants, primarily women, whose socio-demographic and clinical characteristics are the described in the following lines. The maternal age distribution was heterogeneous, with the majority falling between the ages of 30 to 35 years (40.6%) and 25 to 30 years (18.8%). The mean BMI prior to pregnancy was 25.68 ± 4.53 kg/m 2 . Comorbidities were prevalent, with 37.5% experiencing them in the first trimester, 40.6% in the second trimester, and 21.9% in the third trimester of pregnancy. Most participants had a high school education (50%), followed by secondary school (25%). A subset of the population reported tobacco use (18.8%). Comorbidities specific to pregnancy included gestational diabetes (6.3%) and gestational hypertension (3.1%), while no cases of preeclampsia were noted.

The majority of conceptions in the study were spontaneous, accounting for 84.4% of cases. Regarding the marital status of participants, 71.9% were married, while the remaining 28.1% were celibate. The racial composition was predominantly White (96.9%), with a small representation of Black individuals (3.1%). This diverse sample provides valuable insights into the characteristics of the population under study.

In the control group, the age distribution reveals a diverse composition. The majority of participants fall into two distinct age brackets, with 40.91% aged between 35 and 40 years and 31.82% in the 30 to 35-year range. Meanwhile, 18.18% of participants are in the 40 to 45-year category. Notably, there are no observations in the youngest age bracket of 15 to 20 years, and only a minor presence in the 20 to 25 and 25 to 30-year brackets, accounting for 4.55% each. Turning our attention to education levels, 46.88% of participants have completed university education, showcasing a substantial presence in this group. High school completion is also represented, with 13.64% of participants falling into this category. These results shed light on the age and educational profile of the control group, which will be vital for further analysis and interpretation of research findings. Therefore, patients in the control group were aged older and reported a higher educational level compared to the pregnant group.

Descriptive statistics reveal significant differences in number of body areas with pain between pregnant women and the control group (Table 2 and Fig. 1 ). In the study, a larger proportion of women in the pregnant group did not report pain (62.5%) compared to those in the control group (25.7%), as indicated in Table 2 .However, there were no significant differences in pain intensity (Table 3 ) or pain interference (Table 4 ) between the two groups.

Mean value of "Number of Body Areas" for Pregnant and Control Subjects

When comparing chronic pain grade (Table 5 ), no significant differences emerged, indicating that both groups were similarly affected by chronic pain. The JFLS-20 scale (Table 6 ) indicated that pregnant women exhibited significantly lower scores across mastication, mobility, communication, and global functioning, implying lower dysfunction in these areas (Figs 2 , 3 , 4 and 5 ).

Mean value of "JFLS Mastication" for Pregnant and Control Subjects

Mean value of "JFLS Mobility" for Pregnant and Control Subjects

Mean value of "JFLS Communication" for Pregnant and Control Subjects

Mean value of "JFLS Global" for Pregnant and Control Subjects

The study also examined psychological well-being. Pregnant women showed a lower prevalence of moderate depression ( p= 0.04) and mild depression ( p= 0.03) on the PHQ-9 scale (Table 7 ) and no difference in the prevalence of severe depression. However, no significant differences were observed in anxiety levels between the two groups using the GAD-7 scale (Table 8 ).

In Table 9 , the comparison between pregnant and control groups reveals significant differences in high OBC Parafunction levels ( p= 0.001), with 3.1% pregnant and 45.7% control participants. No significant distinctions were found in other categories.

In summary, descriptive statistic showed that less pregnant women experienced pain in TMD region. They also exhibited lower dysfunction in various functional areas.

Correlation analysis

Then, we investigated the correlation between the assessed TMD scores and pregnancy status, while adjusting for age, education, and depression (PHQ9). Linear regression analyses were performed for each TMD score measure separately and then the p value was adjusted for multiple comparisons.

Number of body areas

The regression analysis for the number of body areas revealed no significant association between pregnancy status and the number of body areas affected (Coefficient = -0.1254, p = 0.697). However, depression (PHQ9) had a significant positive association with the number of body areas affected (Coefficient = 0.1144, p = 0.001), indicating that higher depression scores were associated with a greater number of affected body areas. This statistical significative difference remained consistent after multiple comparison correction ( p= 0.009).

Pain intensity

For pain intensity, the regression analysis showed no statistically significant association between pregnancy status and pain intensity (Coefficient = -0.4409, p = 0.093). Additionally, none of the covariates (age, education, and depression) had significant associations with pain intensity.

Interference

The analysis of interference scores revealed no significant relationship between pregnancy status and interference caused by TMD (Coefficient = -0.2080, p = 0.279). None of the covariates showed significant associations with interference.

Chronic pain grade

The regression analysis for chronic pain grade demonstrated a significant negative association between pregnancy status and chronic pain grade (Coefficient = -0.6734, p = 0.032), indicating that pregnant individuals had lower chronic pain grades compared to non-pregnant individuals. However, after correction for multiple comparisons the significance was lost. None of the other covariates showed significant associations with chronic pain grade.

JFLS mastication

The analysis of JFLS Mastication scores showed a marginally significant negative association with pregnancy status (Coefficient = -1.4679, p = 0.055). Age, education, and depression did not have significant associations with JFLS Mastication scores.

JFLS mobility

For JFLS Mobility scores, the regression analysis revealed a significant negative association with pregnancy status (Coefficient = -1.7803, p = 0.024), indicating that pregnant individuals had lower JFLS Mobility scores. However, after correction for multiple comparisons the significance was lost. None of the other covariates showed significant associations with JFLS Mobility.

JFLS communication

The analysis of JFLS Communication scores did not yield any significant association with pregnancy status or any of the covariates (age, education, and depression).

JFLS global

The regression analysis for JFLS Global scores demonstrated a significant negative association with pregnancy status (Coefficient = -1.6081, p = 0.031), indicating that pregnant individuals had lower JFLS Global scores. However, even in this case, after correction for multiple comparisons the significance was lost. None of the other covariates showed significant associations with JFLS Global scores.

The analysis of the OBC scores revealed a significant negative association with pregnancy status (Coefficient = -7.2970, p = 0.047), indicating that pregnant individuals had lower OBC scores. However, after correction for multiple comparisons the significance was lost. Additionally, depression (PHQ9) was significantly associated with OBC scores (Coefficient = 1.4353, p < 0.001), indicating that higher depression scores were related to higher OBC scores. This latter association remained consistent after Bonferroni correction.

Overall, these results suggest that pregnancy is neither a risk factor nor a protective one. However, our results indicate a negative trend between pregnancy status and numerous scales exploring TMD health. Additionally, depression (PHQ9) appears to be a significant predictor of several TMD score measures, highlighting the importance of considering mental health factors in the assessment of TMD.

In this study investigating the relationship between pregnancy and TMD, a wealth of valuable insights has emerged. The study encompassed pregnant women and healthy control non-pregnant women, with an array of TMD-related variables meticulously assessed and analyzed [ 46 , 47 , 48 , 49 , 50 ].

Descriptive statistics demonstrated that pregnant women exhibited significant differences in pain perception compared to the control group, with a higher proportion of pregnant women not reporting pain and experiencing lower pain severity [ 51 , 52 , 53 ]. However, no significant disparities were observed in characteristic pain intensity or pain interference between the two groups. The study employed the JFLS to evaluate various dimensions of functional status, including mastication, mobility, communication, and global functioning. The analysis indicated that pregnant women displayed lower dysfunction in these areas, as evidenced by significantly lower scores on the JFLS scale. Nevertheless, it's essential to note that after correcting for multiple comparisons, the significance was lost in some cases. Psychological well-being, a critical component of TMD assessment, was explored using the PHQ-9 and the GAD-7 scale. Pregnant women demonstrated a lower prevalence of moderate and mild depression on the PHQ-9 scale, highlighting potential differences in mental health between the two groups. However, no significant differences in anxiety levels were observed using the GAD-7 scale. Linear regression analyses were performed to delve deeper into the relationship between TMD scores and pregnancy status while adjusting for covariates such as age, education, and depression (PHQ-9) [ 54 ]. The results indicated that pregnancy status was not consistently associated with TMD-related variables, except for chronic pain grade, where pregnant individuals exhibited lower grades compared to non-pregnant individuals. However, this significance was lost after correction for multiple comparisons. An important finding from the study is the significant association between depression (PHQ-9) and several TMD-related measures. Higher depression scores were related to increased TMD severity, particularly as indicated by the OBC scores.

Overall, this study results suggest that pregnancy itself is neither a definitive risk factor nor a protective factor for TMD. While pregnant women demonstrated differences in pain perception, functional status, and psychological well-being, these differences were not consistent across all TMD-related variables, and some significance was lost after correcting for multiple comparisons. The findings do, however, indicate a negative trend between pregnancy status and certain TMD health scales, hinting at potential associations worth exploring further. Furthermore, the study underscores the significance of considering mental health factors, particularly depression (PHQ-9), in the assessment of TMD. Elevated depression scores were associated with increased TMD severity, emphasizing the need for a holistic approach to TMD evaluation that incorporates mental health assessments.

Estrogen, a hormone with a pivotal role in numerous physiological processes, including bone and joint health, has garnered attention for its influence on the TMJ [ 8 ]. Recent investigations have illuminated the potential link between estrogen levels and TMDs [ 9 ]. This hormone's intricate involvement in the TMJ is multifaceted. The TMJ, a complex joint encompassing muscles, ligaments, nerves, and cartilage, faces the modulatory effects of estrogen. Studies have revealed that estrogen deficiency is associated with an increased risk of TMD. Estrogen, it seems, exerts its influence on the TMJ by regulating the expression of crucial proteins, such as collagen and elastin [ 24 , 25 ]. Additionally, estrogen impacts the production of prostaglandins, hormones vital for cartilage balance [ 55 ]. Furthermore, estrogen's role extends to bone health, with its deficiency linked to a heightened risk of osteoporosis, characterized by fragile bones [ 56 ]. Notably, estrogen contributes to the strength of TMJ-supporting muscles like the masseter and temporalis [ 57 ]. From a pathogenic perspective, estrogen appears to play a role in TMD development [ 20 ]. While some evidence suggests that higher estrogen levels in women of childbearing age may increase TMD risk, other studies point to an elevated risk post-menopause [ 20 ]. Reduced collagen production due to estrogen deficiency may lead to joint instability, potentially contributing to TMD. Moreover, estrogen deficiency has been linked to heightened pain sensitivity. Interestingly, estrogen also emerges as a player in TMD treatment [ 20 ]. Research indicates that estrogen replacement therapy can alleviate pain, enhance function, and expedite healing post-TMD treatment [ 8 ]. In sum, estrogen occupies a significant role in both the maintenance and treatment of TMJ disorders. While pregnancy introduces a range of physiological changes in women, including hormonal fluctuations, increased weight, and postural adjustments, it's vital to recognize that these changes can impact the TMJ. Physiologically, hormonal shifts during pregnancy may cause structural alterations, such as swelling, in the TMJ and its surrounding structures [ 8 ]. This swelling can restrict jaw movement due to reduced space [ 21 , 58 ]. Additionally, pregnancy can weaken the muscles and ligaments supporting the jaw, potentially leading to reduced jaw strength and mobility. Studies have offered contrasting insights into pregnancy's influence on TMJ health [ 59 , 60 ]. Some suggest an increase in TMJ laxity and decreased musculoskeletal orofacial pain during pregnancy [ 21 ]. Conversely, other research associated pregnancy with systemic hypermobility, unrelated to TMJ hypermobility [ 14 ]. Additionally, experimental pain studies indicate that high levels of estrogen and progesterone possess antinociceptive properties [ 61 ]. In a recent systematic review [ 62 ], the prevalence of TMD in pregnant women was explored, with findings indicating that it does not significantly differ from non-pregnant childbearing women. However, variations in diagnostic criteria may have contributed to the diversity in reported prevalence rates across studies. Notably, studies applying the RDC/TMD provide more robust evidence, strengthening the assertion that pregnancy is not a significant risk factor for TMD.

In the context of our discussion, it's important to note the relevance of Cone-Beam Computed Tomography (CBCT) in diagnosing degenerative changes in the temporomandibular joint (TMJ). A recent study by Görürgöz et al. (ref) conducted a multicenter CBCT investigation, focusing on degenerative changes in the mandibular condyle and their relation to TMJ space, gender, and age. This research emphasized that CBCT offers multiplanar views of TMJ bone components and pathologies without distortion, with condylar flattening being a frequently observed degenerative change. However, it's crucial to consider that the use of CBCT is limited during pregnancy due to radiation concerns.

In conclusion, in line with the most recent studies, this study highlights the complexity of this association and strength the assumption that pregnancy is neither a risk factor nor a protective one. Further research, with larger sample sizes and more extensive investigations, is warranted to elucidate the nuanced interplay between pregnancy, TMD, and mental health factors. Such endeavors can enhance our understanding and ultimately contribute to improved care and management of TMD in pregnant women and beyond.

Availability of data and materials

The data will be available on reasonable request from the corresponding author.

Murphy MK, MacBarb RF, Wong ME, Athanasiou KA. Temporomandibular Disorders: A Review of Etiology, Clinical Management, and Tissue Engineering Strategies. Int J Oral Maxillofac Implants. 2013;28:e393–414.

Article   PubMed   PubMed Central   Google Scholar  

Asquini G, Pitance L, Michelotti A, Falla D. Effectiveness of manual therapy applied to craniomandibular structures in temporomandibular disorders: A systematic review. J Oral Rehabil. 2022;49:442–55.

Article   PubMed   Google Scholar  

Michelotti A, Wijer A, Steenks M, FARELLA M,. Home-exercise regimes for the management of non-specific temporomandibular disorders. J Oral Rehabil. 2005;32:779–85.

Article   CAS   PubMed   Google Scholar  

Iodice G, Cimino R, Vollaro S, Lobbezoo F, Michelotti A. Prevalence of temporomandibular disorder pain, jaw noises and oral behaviours in an adult Italian population sample. J Oral Rehabil. 2019;46(8):691–8.

Solak Ö, Turhan-Haktanır N, Köken G, Toktas H, Güler Ö, Kavuncu V, Demir Y. Prevalence of temporomandibular disorders in pregnancy. Electronic Journal of General Medicine. 2009;6:223–8.

Article   Google Scholar  

Fichera G, Polizzi A, Scapellato S, Palazzo G, Indelicato F. Craniomandibular Disorders in Pregnant Women: An Epidemiological Survey. J Funct Morphol Kinesiol. 2020;5:36.

Donnarumma V, Michelotti A, Cimino R, Vollaro S, Cioffi I. Short-term Effects of a First-Line Treatment Including Counseling and Self-Management Strategies on Chronic TMD Muscle Pain and Awake Bruxism in Women. J Oral Facial Pain Headache. 2022;36:36–48.

Robinson JL, Johnson PM, Kister K, Yin MT, Chen J, Wadhwa S. Estrogen signaling impacts temporomandibular joint and periodontal disease pathology. Odontology. 2020;108:153–65.

Ivković N, Racic M, Lecic R, Bozovic D, Kulic M. Relationship Between Symptoms of Temporomandibular Disorders and Estrogen Levels in Women With Different Menstrual Status. J Oral Facial Pain Headache. 2018;32:151–8.

Cauley JA. Estrogen and bone health in men and women. Steroids. 2015;99:11–5.

Minervini G, Basili M, Franco R, et al. Periodontal Disease and Pregnancy: Correlation with Underweight Birth. Eur J Dent. 2022. https://doi.org/10.1055/s-0042-1757906 .

Mayoral Verónica A, Espinosa Irene A, Montiel Álvaro J. Association between signs and symptoms of temporomandibular disorders and pregnancy (case control study). Acta odontológica Latinoamericana. 2013;26:3–7.

PubMed   Google Scholar  

Kochhar AS, Sidhu MS, Prabhakar M, Bhasin R, Kochhar GK, Dadlani H, Spagnuolo G, Mehta VV. Intra- and Interobserver Reliability of Bone Volume Estimation Using OsiriX Software in Patients with Cleft Lip and Palate Using Cone Beam Computed Tomography. Dent J (Basel). 2021;9:14.

Silveira EB, Rocabado M, Russo AK, Cogo JC, Osorio RAL. Incidence of Systemic Joint Hypermobility and Temporomandibular Joint Hypermobility in Pregnancy. Cranio. 2005;23:138–43.

Slade GD, Bair E, Greenspan JD, Dubner R, Fillingim RB, Diatchenko L, Maixner W, Knott C, Ohrbach R. Signs and Symptoms of First-Onset TMD and Sociodemographic Predictors of Its Development: The OPPERA Prospective Cohort Study. J Pain. 2013;14:T20–T32.e1-3.

Slade GD, Ohrbach R, Greenspan JD, et al. Painful Temporomandibular Disorder. J Dent Res. 2016;95:1084–92.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Macfarlane TV, Blinkhorn AS, Davies RM, Kincey J, Worthington HV. Oro-facial pain in the community: prevalence and associated impact. Community Dent Oral Epidemiol. 2002;30:52–60.

Lövgren A, Häggman-Henrikson B, Visscher CM, Lobbezoo F, Marklund S, Wänman A. Temporomandibular pain and jaw dysfunction at different ages covering the lifespan - A population based study. European Journal of Pain. 2016;20:532–40.

Maixner W, Fillingim RB, Williams DA, Smith SB, Slade GD. Overlapping Chronic Pain Conditions: Implications for Diagnosis and Classification. J Pain. 2016;17:T93–107.

Berger M, Szalewski L, Bakalczuk M, Bakalczuk G, Bakalczuk S, Szkutnik J. Association between estrogen levels and temporomandibular disorders: a systematic literature review. Menopausal Review. 2015;4:260–70.

LeResche L. Epidemiology of Temporomandibular Disorders: Implications for the Investigation of Etiologic Factors. Critical Reviews in Oral Biology & Medicine. 1997;8:291–305.

Article   CAS   Google Scholar  

Juran CM, Dolwick MF, McFetridge PS. Engineered Microporosity: Enhancing the Early Regenerative Potential of Decellularized Temporomandibular Joint Discs. Tissue Eng Part A. 2015;21:829–39.

McKay GS, Yemm R, Cadden SW. The structure and function of the temporomandibular joint. Br Dent J. 1992;173:127–32.

Nakano T, Scott PG. Changes in the chemical composition of the bovine temporomandibular joint disc with age. Arch Oral Biol. 1996;41:845–53.

Almarza AJ, Bean AC, Baggett LS, Athanasiou KA. Biochemical analysis of the porcine temporomandibular joint disc. British Journal of Oral and Maxillofacial Surgery. 2006;44:124–8.

Shengyi T, Xu Y. Biomechanical properties and collagen fiber orientation of TMJ discs in dogs: Part 1. Gross anatomy and collagen fiber orientation of the discs. J Craniomandib Disord. 1991;5:28–34.

CAS   PubMed   Google Scholar  

Minarelli AM, Del Santo Júnior M, Liberti EA. The structure of the human temporomandibular joint disc: a scanning electron microscopy study. J Orofac Pain. 1997;11:95–100.

Bollero P. Oral health and implant therapy in Parkinson’s patients: review. Oral Implantol (Rome). 2017;10:105.

Franco R, Gianfreda F, Miranda M, Barlattani A, Bollero P. The hemostatic properties of chitosan in oral surgery. Biomed Biotechnol Res J (BBRJ). 2020;4:186. https://doi.org/10.4103/bbrj.bbrj_43_20 .

Seweryn P, Orzeszek SM, Waliszewska-Prosół M, Jenča A, Osiewicz M, Paradowska-Stolarz A, Winocur-Arias O, Ziętek M, Bombała W, Więckiewicz M. Relationship between pain severity, satisfaction with life and the quality of sleep in Polish adults with temporomandibular disorders. Dent Med Probl. 2023;60:609–17.

Görürgöz C, İçen M, Kurt M, Aksoy S, Bakırarar B, Rozylo-Kalinowska I, Orhan K. Degenerative changes of the mandibular condyle in relation to the temporomandibular joint space, gender and age: A multicenter CBCT study. Dent Med Probl. 2023;60:127–35.

Bhavsar N, Trivedi S, VACHHANI KS, BRAHMBHATT N, SHAH S, PATEL N, GUPTA D, PERIASAMY R,. Association between preterm birth and low birth weight and maternal chronic periodontitis: A hospital-based case–control study. Dent Med Probl. 2023;60:207–17.

Radwan-Oczko M, Hirnle L, Szczepaniak M, Duś-Ilnicka I. How much do pregnant women know about the importance of oral health in pregnancy? Questionnaire-based survey BMC Pregnancy Childbirth. 2023;23:348.

Smardz J, Martynowicz H, Wojakowska A, Winocur-Arias O, Michalek-Zrabkowska M, Mazur G, Wieckiewicz M. A polysomnographic study on the relationship between sleep bruxism intensity and sleep quality. Cranio. 2022;40:107–12.

Cigdem Karacay B, Sahbaz T. Investigation of the relationship between probable sleep bruxism, awake bruxism and temporomandibular disorders using the Diagnostic Criteria for Temporomandibular Disorders (DC/TMD). Dent Med Probl. 2023;60:601–8.

Boening K, Wieckiewicz M, Paradowska-Stolarz A, Wiland P, Shiau Y-Y. Temporomandibular Disorders and Oral Parafunctions: Mechanism, Diagnostics, and Therapy. Biomed Res Int. 2015;2015:1–2.

Di Cosola M, Cazzolla AP, Charitos IA, Ballini A, Inchingolo F, Santacroce L. Candida albicans and Oral Carcinogenesis. A Brief Review Journal of Fungi. 2021;7:476.

Cantore S, Mirgaldi R, Ballini A, Coscia MF, Scacco S, Papa F, Inchingolo F, Dipalma G, De Vito D. Cytokine Gene Polymorphisms Associate with Microbiogical Agents in Periodontal Disease: Our Experience. Int J Med Sci. 2014;11:674–9.

Traylor CS, Johnson JD, Kimmel MC, Manuck TA. Effects of psychological stress on adverse pregnancy outcomes and nonpharmacologic approaches for reduction: an expert review. Am J Obstet Gynecol MFM. 2020;2: 100229.

Alshiddi IF. Survival rate and clinico-radiographic parameters around narrow-diameter dental implants for fixed dental prostheses in the posterior regions: A systematic review. Dent Med Probl. 2023;60:345–53.

Erdilek D, Gümüştaş B, Efes BG. Digitalization era of dental education: A systematic review. Dent Med Probl. 2023;60:513–25.

Afkhami F, Ebrahimi H, Aghazadeh A, Sooratgar A, Chiniforush N. Evaluation of the effect of the photobiomodulation therapy on the pain related to dental injections: A preliminary clinical trial. Dent Med Probl. 2022;59:421–5.

Soegiantho P, Suryawinata PG, Tran W, Kujan O, Koyi B, Khzam N, et al. Survival of single immediate implants and reasons for loss: a systematic review. Prosthesis. 2023;5:378–424. https://doi.org/10.3390/prosthesis5020028 .

Franco R, Barlattani A, Perrone MA, Basili M, Miranda M, Costacurta M, et al. Obesity, bariatric surgery and periodontal disease: a literature update. Eur Rev Med Pharmacol Sci. 2020;24:5036–45. https://doi.org/10.26355/eurrev_202005_21196 .

Minervini G, Franco R, Marrapodi MM, Di Blasio M, Ronsivalle V, Cicciù M. Children oral health and parents education status: a cross sectional study. BMC Oral Health. 2023;23:787. https://doi.org/10.1186/s12903-023-03424-x .

Shaikh S, Jamdade B, Chanda A. Effects of customized 3D-printed insoles in patients with foot-related musculoskeletal ailments—A survey-based study. Prosthesis. 2023;5:550–61. https://doi.org/10.3390/prosthesis5020038 .

Yokoyama M, Shiga H, Ogura S, Sano M, Komino M, Takamori H, et al. Functional differences between chewing sides of implant-supported denture wearers. Prosthesis. 2023;5:346–57. https://doi.org/10.3390/prosthesis5020025 .

Vozzo LM, Azevedo L, Fernandes JCH, Fonseca P, Araújo F, Teixeira W, et al. The success and complications of complete-arch implant-supported fixed monolithic zirconia restorations: a systematic review. Prosthesis. 2023;5:425–36. https://doi.org/10.3390/prosthesis5020029 .

Morar L, Băciuț G, Băciuț M, Bran S, Colosi H, Manea A, Almășan O, Dinu C. Analysis of CBCT Bone Density Using the Hounsfield Scale. Prosthesis. 2022;4:414–23.

Murabayashi M, Mitani T, Inoue K. Development and Evaluation of a Passive Mechanism for a Transfemoral Prosthetic Knee That Prevents Falls during Running Stance. Prosthesis. 2022;4:172–83.

Di Stasio D, Romano A, Paparella RS, Gentile C, Serpico R, Minervini G, et al. How social media meet patients’ questions: YouTube™ review for mouth sores in children. J Biol Regul Homeost Agents. 2018;32:117–21.

Minervini G, Franco R, Marrapodi MM, Fiorillo L, Cervino G, Cicciù M. Economic inequalities and temporomandibular disorders: A systematic review with meta-analysis. J Oral Rehabil. 2023;50:715–23.

Minervini G, Nucci L, Lanza A, Femiano F, Contaldo M, Grassia V. Temporomandibular disc displacement with reduction treated with anterior repositioning splint: a 2-year clinical and Magnetic Resonance Imaging (MRI) follow-up. J Biol Regul Homeost Agents. 2020;34:151–60. Dental Supplement.

Lucchese A, Dolci A, Minervini G, Salerno C, DI Stasio D, Minervini G, et al. Vulvovaginal Gingival Lichen Planus: report of two cases and review of literature. Oral Implantol (Rome). 2016;9:54–60. https://doi.org/10.11138/orl/2016.9.2.054 .

Joldersma M, Klein-Nulend J, Oleksik AM, Heyligers IC, Burger EH. Estrogen enhances mechanical stress-induced prostaglandin production by bone cells from elderly women. American Journal of Physiology-Endocrinology and Metabolism. 2001;280:E436–42.

Kalervo Väänänen H, Härkönen PL. Estrogen and bone metabolism. Maturitas. 1996;23:S65–9.

Dong X-D, Mann MK, Kumar U, Svensson P, Arendt-Nielsen L, Hu JW, Sessle BJ, Cairns BE. Sex-related differences in NMDA-evoked rat masseter muscle afferent discharge result from estrogen-mediated modulation of peripheral NMDA receptor activity. Neuroscience. 2007;146:822–32.

Schauberger CW, Rooney BL, Goldsmith L, Shenton D, Silva PD, Schaper A. Peripheral joint laxity increases in pregnancy but does not correlate with serum relaxin levels. Am J Obstet Gynecol. 1996;174:667–71.

Calguneri M, Bird HA, Wright V. Changes in joint laxity occurring during pregnancy. Ann Rheum Dis. 1982;41:126–8.

Mushayandebvu T. Elevated Maternal Serum Relaxin Concentrations Throughout Pregnancy in Singleton Gestations After Superovulation. Obstetrics & Gynecology. 1998;92:17–20.

Dawson-Basoa ME, Gintzler AR. Estrogen and progesterone activate spinal kappa-opiate receptor analgesic mechanisms. Pain. 1996;64:169–77.

Minervini G, Franco R, Marrapodi MM, Fiorillo L, Cervino G, Cicciù M. Prevalence of temporomandibular disorders ( <scp>TMD</scp> ) in pregnancy: A systematic review with meta-analysis. J Oral Rehabil. 2023;50:627–34.

Download references

Acknowledgements

The authors want to acknowledge Dr. Attilio La Verde for their valuable support in the management and increase in the clinical data.

Not applicable.

Author information

Authors and affiliations.

Saveetha Dental College and Hospitals, Saveetha Institute of Medical and Technical Sciences (SIMATS), Saveetha University, Chennai, Tamil Nadu, India

Giuseppe Minervini

Multidisciplinary Department of Medical-Surgical and Odontostomatological Specialties, University of Campania “Luigi Vanvitelli”, 80121, Naples, Italy

Giuseppe Minervini & Diana Russo

Department of Woman, Child and General and Specialized Surgery, Obstetrics and Gynecology Unit, University of Campania “Luigi Vanvitelli”, Largo Madonna delle Grazie 1, 80138, Naples, Italy

Maria Maddalena Marrapodi & Marco La Verde

Department of Dental Therapy, Faculty of Dental Medicine, University of Medicine, Tirana, Albania

International European University School of Medicine, Akademika Hlushkova Ave, 42B, Kyiv, 03187, Ukraine

Yuliia Siurkel

Department of Biomedical and Surgical and Biomedical Sciences, Catania University, 95123, Catania, Italy

Marco Cicciù

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization – GM, MMM; Methodology – MMM, MLV; Formal analysis – MMM, AM; Investigation – YS, GM; Roles/Writing - original draft –GM, YS , MC; Writing - review & editing – M.C, DR, G.M., YS; Funding acquisition: YS; Resources – MLV, GM; Supervision – MC, GM;; project administration, MMM; All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Marco La Verde , Yuliia Siurkel or Diana Russo .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Minervini, G., Marrapodi, M.M., La Verde, M. et al. The relationship between pregnancy and temporomandibular disorder (TMD) through diagnostic criteria for temporomandibular disorders (DC/TMD) axis II evaluation: a case-control cross-sectional study. BMC Oral Health 24 , 342 (2024). https://doi.org/10.1186/s12903-024-04009-y

Download citation

Received : 20 December 2023

Accepted : 09 February 2024

Published : 16 March 2024

DOI : https://doi.org/10.1186/s12903-024-04009-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Temporomandibular disorders

BMC Oral Health

ISSN: 1472-6831

• Submission enquiries: [email protected]
• General enquiries: [email protected]

This website may not work correctly because your browser is out of date. Please update your browser .

This website from EuropeAid  provides a detailed overview of case studies including why, when and how they are used  and how they are carried out.

• What is a case study?
• What is meant by a "case study"?
• Where does this tool come from?
• Are there various case study types?
• Why and when is the case study used?
• In which situations is this tool appropriate?
• What are its advantages and its limitations?
• Can it be combined with other tools?
• What are the pre-conditions for its use in evaluation?
• Why and how is a case study used in country/region evaluation?
• Why are case studies used in country/region evaluation?
• What kind of use for each type of case study in country/region evaluation?
• How is a case study carried out?
• What are the stages to follow when carrying out a case study?
• What are the pitfalls to avoid?
• What are the key issues relating to the case study's quality control?
• Examples of the use of a case study in country/region evaluation
• Example 1: How to select case studies.
• Example 2: What instructions should be given to the teams in charge of case studies?

EuropeAid. (2005, April 27).  Case study . Retrieved from http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_cas_som_e...

Originally sourced from MyM&E

Related links

• https://web.archive.org/web/20101025231440/http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_cas_som_en.htm

Back to top

© 2022 BetterEvaluation. All right reserved.