qualitative research methods and validity

No internet connection.

All search filters on the page have been cleared., your search has been saved..

• All content
• Dictionaries
• Encyclopedias
• Expert Insights
• Foundations
• How-to Guides
• Journal Articles
• Little Blue Books
• Little Green Books
• Project Planner
• Tools Directory
• Sign in to my profile My Profile

• Sign in Signed in
• My profile My Profile

Reliability and Validity in Qualitative Research

• By: Jerome Kirk & Marc L. Miller
• Publisher: SAGE Publications, Inc.
• Series: Qualitative Research Methods
• Publication year: 1986
• Online pub date: January 01, 2011
• Discipline: Anthropology
• Methods: Validity , Reliability , Measurement
• DOI: https:// doi. org/10.4135/9781412985659
• Keywords: natural sciences , reliability and validity , social science , social scientists , sociology , tradition Show all Show less
• Print ISBN: 9780803924703
• Online ISBN: 9781412985659
• Buy the book icon link

Kirk and Miller define what is -- and what is not -- qualitative research. They suggest that the use of numbers in the process of recording and analyzing observations is less important than that the research should involve sustained interaction with the people being studied, in their own language and on their own turf. Following a chapter on objectivity, the authors discuss the role of reliability and validity and the problems that arise when these issues are neglected. They present a paradigm for the qualitative research process that makes it possible to pursue validity without neglecting reliability.

Front Matter

• Series Introduction
• Editors' Introduction
• Objectivity in Qualitative Research
• Reliability and Validity
• The Problem of Validity
• Toward Theoretical Validity
• The Problem of Reliability
• Ethnographic Decision Making: The Four Phases of Qualitative Research

Back Matter

• About the Authors
• Qualitative Research Methods

Sign in to access this content

Get a 30 day free trial, more like this, sage recommends.

We found other relevant content for you on other Sage platforms.

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

• Sign in/register

Navigating away from this page will delete your results

Please save your results to "My Self-Assessments" in your profile before navigating away from this page.

Sign in to my profile

Sign up for a free trial and experience all Sage Learning Resources have to offer.

You must have a valid academic email address to sign up.

Get off-campus access

• View or download all content my institution has access to.

Sign up for a free trial and experience all Sage Research Methods has to offer.

• view my profile
• view my lists

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 18, Issue 2
• Issues of validity and reliability in qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Helen Noble 1 ,
• Joanna Smith 2
• 1 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• 2 School of Human and Health Sciences, University of Huddersfield , Huddersfield , UK
• Correspondence to Dr Helen Noble School of Nursing and Midwifery, Queens's University Belfast, Medical Biology Centre, 97 Lisburn Rd, Belfast BT9 7BL, UK; helen.noble{at}qub.ac.uk

https://doi.org/10.1136/eb-2015-102054

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Evaluating the quality of research is essential if findings are to be utilised in practice and incorporated into care delivery. In a previous article we explored ‘bias’ across research designs and outlined strategies to minimise bias. 1 The aim of this article is to further outline rigour, or the integrity in which a study is conducted, and ensure the credibility of findings in relation to qualitative research. Concepts such as reliability, validity and generalisability typically associated with quantitative research and alternative terminology will be compared in relation to their application to qualitative research. In addition, some of the strategies adopted by qualitative researchers to enhance the credibility of their research are outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research. 2–4 In the broadest context these terms are applicable, with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures. 4 However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alterative frameworks for establishing rigour are appropriate. 3 Lincoln and Guba 5 offer alternative criteria for demonstrating rigour within qualitative research namely truth value, consistency and neutrality and applicability. Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

• View inline

Terminology and criteria used to evaluate the credibility of research findings

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Unlike quantitative researchers, who apply statistical methods for establishing validity and reliability of research findings, qualitative researchers aim to design and incorporate methodological strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

Accounting for personal biases which may have influenced findings; 6

Acknowledging biases in sampling and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis; 3

Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent; 3 , 4

Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented; 6 , 7

Including rich and thick verbatim descriptions of participants’ accounts to support findings; 7

Demonstrating clarity in terms of thought processes during data analysis and subsequent interpretations 3 ;

Engaging with other researchers to reduce research bias; 3

Respondent validation: includes inviting participants to comment on the interview transcript and whether the final themes and concepts created adequately reflect the phenomena being investigated; 4

Data triangulation, 3 , 4 whereby different methods and perspectives help produce a more comprehensive set of findings. 8 , 9

Table 2 provides some specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the impact of being a family carer to patients with stage 5 chronic kidney disease managed without dialysis. 10

Strategies for enhancing the credibility of qualitative research

In summary, it is imperative that all qualitative researchers incorporate strategies to enhance the credibility of a study during research design and implementation. Although there is no universally accepted terminology and criteria used to evaluate qualitative research, we have briefly outlined some of the strategies that can enhance the credibility of study findings.

• Sandelowski M
• Lincoln YS ,
• Barrett M ,
• Mayan M , et al
• Greenhalgh T
• Lingard L ,

Twitter Follow Joanna Smith at @josmith175 and Helen Noble at @helnoble

Competing interests None.

Read the full text or download the PDF:

Qualitative Research and Content Validity

• Reference work entry
• First Online: 01 January 2024
• pp 5658–5666
• Cite this reference work entry

• Meryl Brod 2 ,
• Betsy Pohlman 2 &
• Laura Tesler Waldman 3  

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Beatty, P., & Willis, G. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71 (2), 287–311.

Article   Google Scholar  

Brod, M., Tesler, L., & Christensen, T. (2009). Qualitative research and content validity: Developing best practices based on science and experience. Quality of Life Research, 18 (9), 1263–1278.

Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. Holstein & J. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 311–330). Thousand Oaks: SAGE.

Google Scholar  

Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons and evaluative criteria. Qualitative Sociology, 13 (1), 3–21.

Corbin, J., & Strauss, A. (2008). Basics of qualitative research (3rd ed.). Newbury Park: SAGE.

Cutcliffe, J. (2000). Methodological issues in grounded theory. Journal of Advanced Nursing, 31 (6), 1476–1484.

FDA. (2009). Patient-reported outcome measures: Use in medical product development to support labeling claims . Rockville: Food and Drug Administration. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf .

Frost, M., Reeve, B., Liepa, A., Stauffer, J., & Hays, R. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health, 10 (2), S94–S105.

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough?: An experiment with data saturation and variability. Field Methods, 18 (1), 59–82.

Lasch, K., Marquis, P., Vigneux, M., Abetz, L., Arnould, B., Bayliss, M., et al. (2010). PRO development: Rigorous qualitative research as the crucial foundation. Quality of Life Research, 19 (8), 1087–1096.

Leidy, N., & Vernon, M. (2008). Perspectives on patient-reported outcomes: Content validity and qualitative research in a changing clinical trial environment. PharmacoEconomics, 26 (5), 363–370.

Magasi, S., Ryan, G., Revicki, D., Lenderking, W., Hays, R., Brod, M., et al. (2012). Content validity of patient-reported outcome measures: Perspectives from a PROMIS meeting. Quality of Life Research, 21 (5), 739–746.

Morgan, D. (1996). Focus groups. Annual Review of Sociology, 22 , 129–152.

Patrick, D., Burke, L., Powers, J., Scott, J., Rock, E., Dawisha, S., O’Neill, R., et al. (2007). Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value in Health, 10 (2), S125–S137.

Patrick, D., Burke, L., Gwaltney, C., Leidy, N., Martin, M., Molsen, E., et al. (2011a). Content validity – Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 1 – Eliciting concepts for a new PRO instrument. Value in Health, 14 (8), 967–977.

Patrick, D., Burke, L., Gwaltney, C., Leidy, N., Martin, M., Molsen, E., et al. (2011b). Content validity – Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2 – Assessing respondent understanding. Value in Health, 14 (8), 978–988.

Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks: SAGE.

Poland, B. (2003). Transcription quality. In J. Holstein & J. Gubrium (Eds.), Inside interviewing: New lenses, new concerns . Thousand Oaks: SAGE.

Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 219–262). London: SAGE.

Rothman, M., Beltran, P., Cappelleri, J., Lipscomb, J., & Teschendorf, B. (2007). Patient-reported outcomes: Conceptual issues. Value in Health, 10 (2), S66–S75.

Stewart, D., Shamdasani, P., & Rook, D. (2007). Focus groups (2nd ed.). Thousand Oaks: SAGE.

Book   Google Scholar  

Turner, R., Quittner, A., Parasuraman, B., Kallich, J., & Cleeland, C. (2007). Patient-reported outcomes: Instrument development and selection issues. Value in Health, 10 (2), S86–S93.

Willis, G. (1999). Cognitive interviewing: A “how to” guide . Research Triangle Park: Research Triangle Institute. http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf .

Willis, G. (2004). Cognitive interviewing revisited: A useful technique, in theory? In S. Presser, J. Rothgeb, M. Couper, J. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 23–44). Hoboken: Wiley.

Chapter   Google Scholar  

Willis, G. (2005). Cognitive interviewing: A tool for improving questionnaire design . Thousand Oaks: SAGE.

Download references

Author information

Authors and affiliations.

The Brod Group, Inc, Mill Valley, CA, USA

Meryl Brod & Betsy Pohlman

Dana-Farber Cancer Institute, Boston, MA, USA

Laura Tesler Waldman

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Meryl Brod .

Editor information

Editors and affiliations.

Dipartimento di Scienze Statistiche, Sapienza Università di Roma, Roma, Roma, Italy

Filomena Maggino

Section Editor information

Department of Political Science, University of Naples Federico II, Naples, Italy

Mara Tognetti

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Brod, M., Pohlman, B., Waldman, L.T. (2023). Qualitative Research and Content Validity. In: Maggino, F. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Cham. https://doi.org/10.1007/978-3-031-17299-1_3848

Download citation

DOI : https://doi.org/10.1007/978-3-031-17299-1_3848

Published : 11 February 2024

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-17298-4

Online ISBN : 978-3-031-17299-1

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Popular searches

• How to Get Participants For Your Study
• How to Do Segmentation?
• Conjoint Preference Share Simulator
• MaxDiff Analysis
• Likert Scales
• Reliability & Validity

Request consultation

Do you need support in running a pricing or product study? We can help you with agile consumer research and conjoint analysis.

Looking for an online survey platform?

Conjointly offers a great survey tool with multiple question types, randomisation blocks, and multilingual support. The Basic tier is always free.

Research Methods Knowledge Base

• Navigating the Knowledge Base
• Foundations
• Construct Validity
• Reliability
• Levels of Measurement
• Survey Research
• Scaling in Measurement
• The Qualitative Debate
• Qualitative Data
• Qualitative Approaches
• Qualitative Methods

Qualitative Validity

• Unobtrusive Measures
• Research Design
• Table of Contents

Fully-functional online survey tool with various question types, logic, randomisation, and reporting for unlimited number of surveys.

Completely free for academics and students .

Depending on their philosophical perspectives , some qualitative researchers reject the framework of validity that is commonly accepted in more quantitative research in the social sciences. They reject the basic realist assumption that there is a reality external to our perception of it. Consequently, it doesn’t make sense to be concerned with the “truth” or “falsity” of an observation with respect to an external reality (which is a primary concern of validity). These qualitative researchers argue for different standards for judging the quality of research.

For instance, Guba and Lincoln proposed four criteria for judging the soundness of qualitative research and explicitly offered these as an alternative to more traditional quantitatively-oriented criteria. They felt that their four criteria better reflected the underlying assumptions involved in much qualitative research. Their proposed criteria and the “analogous” quantitative criteria are listed in the table.

Credibility

The credibility criteria involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in the research. Since from this perspective, the purpose of qualitative research is to describe or understand the phenomena of interest from the participant’s eyes, the participants are the only ones who can legitimately judge the credibility of the results.

Transferability

Transferability refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings. From a qualitative perspective transferability is primarily the responsibility of the one doing the generalizing. The qualitative researcher can enhance transferability by doing a thorough job of describing the research context and the assumptions that were central to the research. The person who wishes to “transfer” the results to a different context is then responsible for making the judgment of how sensible the transfer is.

Dependability

The traditional quantitative view of reliability is based on the assumption of replicability or repeatability. Essentially it is concerned with whether we would obtain the same results if we could observe the same thing twice. But we can’t actually measure the same thing twice – by definition if we are measuring twice, we are measuring two different things. In order to estimate reliability, quantitative researchers construct various hypothetical notions (e.g., true score theory ) to try to get around this fact.

The idea of dependability, on the other hand, emphasizes the need for the researcher to account for the ever-changing context within which research occurs. The research is responsible for describing the changes that occur in the setting and how these changes affected the way the research approached the study.

Confirmability

Qualitative research tends to assume that each researcher brings a unique perspective to the study. Confirmability refers to the degree to which the results could be confirmed or corroborated by others. There are a number of strategies for enhancing confirmability. The researcher can document the procedures for checking and rechecking the data throughout the study. Another researcher can take a “devil’s advocate” role with respect to the results, and this process can be documented. The researcher can actively search for and describe and negative instances that contradict prior observations. And, after he study, one can conduct a data audit that examines the data collection and analysis procedures and makes judgements about the potential for bias or distortion.

There has been considerable debate among methodologists about the value and legitimacy of this alternative set of standards for judging qualitative research. On the one hand, many quantitative researchers see the alternative criteria as just a relabeling of the very successful quantitative criteria in order to accrue greater legitimacy for qualitative research. They suggest that a correct reading of the quantitative criteria would show that they are not limited to quantitative research alone and can be applied equally well to qualitative data. They argue that the alternative criteria represent a different philosophical perspective that is subjectivist rather than realist in nature. They claim that research inherently assumes that there is some reality that is being observed and can be observed with greater or less accuracy or validity. if you don’t make this assumption, they would contend, you simply are not engaged in research (although that doesn’t mean that what you are doing is not valuable or useful).

Perhaps there is some legitimacy to this counter argument. Certainly a broad reading of the traditional quantitative criteria might make them appropriate to the qualitative realm as well. But historically the traditional quantitative criteria have been described almost exclusively in terms of quantitative research. No one has yet done a thorough job of translating how the same criteria might apply in qualitative research contexts. For instance, the discussions of external validity have been dominated by the idea of statistical sampling as the basis for generalizing. And, considerations of reliability have traditionally been inextricably linked to the notion of true score theory.

But qualitative researchers do have a point about the irrelevance of traditional quantitative criteria. How could we judge the external validity of a qualitative study that does not use formalized sampling methods? And, how can we judge the reliability of qualitative data when there is no mechanism for estimating the true score? No one has adequately explained how the operational procedures used to assess validity and reliability in quantitative research can be translated into legitimate corresponding operations for qualitative research.

While alternative criteria may not in the end be necessary (and I personally hope that more work is done on broadening the “traditional” criteria so that they legitimately apply across the entire spectrum of research approaches), and they certainly can be confusing for students and newcomers to this discussion, these alternatives do serve to remind us that qualitative research cannot easily be considered only an extension of the quantitative paradigm into the realm of nonnumeric data.

Cookie Consent

Conjointly uses essential cookies to make our site work. We also use additional cookies in order to understand the usage of the site, gather audience analytics, and for remarketing purposes.

For more information on Conjointly's use of cookies, please read our Cookie Policy .

Which one are you?

I am new to conjointly, i am already using conjointly.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.84(1); 2020 Jan

A Review of the Quality Indicators of Rigor in Qualitative Research

Jessica l. johnson.

a William Carey University School of Pharmacy, Biloxi, Mississippi

Donna Adkins

Sheila chauvin.

b Louisiana State University, School of Medicine, New Orleans, Louisiana

Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework, both of which contribute to the selection of appropriate research methods that enhance trustworthiness and minimize researcher bias inherent in qualitative methodologies. Qualitative data collection and analyses are often modified through an iterative approach to answering the research question. Researcher reflexivity, essentially a researcher’s insight into their own biases and rationale for decision-making as the study progresses, is critical to rigor. This article reviews common standards of rigor, quality scholarship criteria, and best practices for qualitative research from design through dissemination.

INTRODUCTION

Within the past 20 years, qualitative research in health professions education has increased significantly, both in practice and publication. Today, one can pick up most any issue of a wide variety of health professions education journals and find at least one article that includes some type of qualitative research, whether a full study or the inclusion of a qualitative component within a quantitative or mixed methods study. Simultaneously, there have been recurrent calls for enhancing rigor and quality in qualitative research.

As members of the academic community, we share responsibility for ensuring rigor in qualitative research, whether as researchers who design and implement, manuscript reviewers who critique, colleagues who discuss and learn from each other, or scholarly teachers who draw upon results to enhance and innovate education. Therefore, the purpose of this article is to summarize standards of rigor and suggested best practices for designing, conducting, and reporting high-quality qualitative research. To begin, Denzin and Lincoln’s definition of qualitative research, a long-standing cornerstone in the field, provides a useful foundation for summarizing quality standards and best practices:

Qualitative research involves the studied use and collection of a variety of empirical materials – case study; personal experience; introspection; life story; interview; artifacts; cultural texts and productions; observational, historical, interactional, and visual texts – that describe the routine and problematic moments and meanings in individual lives. Accordingly, qualitative researchers deploy a wide range of interconnected interpretative practices, hoping always to get a better understanding of the subject matter at hand. It is understood, however, that each practice makes the world visible in a different way. Hence there is frequently a commitment to using more than one interpretative practice in any study. 1

In recent years, multiple publications have synthesized quality criteria and recommendations for use by researchers and peer reviewers alike, often in the form of checklists. 2-6 Some authors have raised concerns about the use of such checklists and adherence to strict, universal criteria because they do not afford sufficient flexibility to accommodate the diverse approaches and multiple interpretive practices often represented in qualitative studies. 7-11 They argue that a strict focus on using checklists of specific technical criteria may stifle the diversity and multiplicity of practices that are so much a part of achieving quality and rigor within the qualitative paradigm. As an alternative, some of these authors have published best practice guidelines for use by researchers and peer reviewers to achieve and assess methodological rigor and research quality. 12,13

Some journals within the field of health professions education have also established best practice guidance, as opposed to strict criteria or a checklist, for qualitative research. These have been disseminated as guiding questions or evaluation categories. In 2015, Academic Medicine produced an expanded second edition of a researcher/author manual that includes specific criteria with extensive explanations and examples. 14 Still others have disseminated best practice guidelines through a series of methodological articles within journal publications. 2

In this article, attributes of rigor and quality and suggested best practices are presented as they relate to the steps of designing, conducting, and reporting qualitative research in a step-wise approach.

BEST PRACTICES: STEP-WISE APPROACH

Step 1: identifying a research topic.

Identifying and developing a research topic is comprised of two major tasks: formulating a research question, and developing a conceptual framework to support the study. Formulating a research question is often stimulated by real-life observations, experiences, or events in the researcher’s local setting that reflect a perplexing problem begging for systematic inquiry. The research question begins as a problem statement or set of propositions that describe the relationship among certain concepts, behaviors, or experiences. Agee 15 and others 16,17 note that initial questions are usually too broad in focus and too vague regarding the specific context of the study to be answerable and researchable. Creswell reminds us that initial qualitative research questions guide inquiry, but they often change as the author’s understanding of the issue develops throughout the study. 16 Developing and refining a primary research question focused on both the phenomena of interest and the context in which it is situated is essential to research rigor and quality.

Glassick, Huber, and Maeroff identified six criteria applicable to assessing the quality of scholarship. 18,19 Now commonly referred to as the Glassick Criteria ( Table 1 ), these critical attributes outline the essential elements of any scholarly approach and serve as a general research framework for developing research questions and designing studies. The first two criteria, clear purpose and adequate preparation, are directly related to formulating effective research questions and a strong conceptual framework.

Glassick’s Criteria for Assessing the Quality of Scholarship of a Research Study 18

Generating and refining a qualitative research question requires thorough, systematic, and iterative review of the literature, and the use of those results to establish a clear context and foundation for the question and study design. Using an iterative approach, relevant concepts, principles, theories or models, and prior evidence are identified to establish what is known, and more importantly, what is not known. The iterative process contributes to forming a better research question, the criteria for which can be abbreviated by the acronym FINER, ie, f easible, i nteresting, n ovel, e thical, and r elevant, that is answerable and researchable, in terms of research focus, context specificity, and the availability of time, logistics, and resources to carry out the study. Developing a FINER research question is critical to study rigor and quality and should not be rushed, as all other aspects of research design depend on the focus and clarity of the research question(s) guiding the study. 15 Agee provides clear and worthwhile additional guidance for developing qualitative research questions. 15

Reflexivity, the idea that a researcher’s preconceptions and biases can influence decisions and actions throughout qualitative research activities, is a critical aspect of rigor even at the earliest stages of the study. A researcher’s background, beliefs, and experiences may affect any aspect of the research from choosing which specific question to investigate through determining how to present the results. Therefore, even at this early stage, the potential effect of researcher bias and any ethical considerations should be acknowledged and addressed. That is, how will the question’s influence on study design affect participants’ lives, position the researcher in relationship with others, or require specific methods for addressing potential areas of research bias and ethical considerations?

A conceptual framework is then actively constructed to provide a logical and convincing argument for the research. The framework defines and justifies the research question, the methodology selected to answer that question, and the perspectives from which interpretation of results and conclusions will be made. 5,6,20 Developing a well-integrated conceptual framework is essential to establishing a research topic based upon a thorough and integrated review of relevant literature (addressing Glassick criteria #1 and #2: clear purpose and adequate preparation). Key concepts, principles, assumptions, best practices, and theories are identified, defined, and integrated in ways that clearly demonstrate the problem statement and corresponding research question are answerable, researchable, and important to advancing thinking and practice.

Ringsted, Hodges, and Sherpbier describe three essential parts to an effective conceptual framework: theories and/or concepts and principles relevant to the phenomenon of interest; what is known and unknown from prior work, observations, and examples; and the researcher’s observations, ideas, and suppositions regarding the research problem statement and question. 21 Lingard describes four types of unknowns to pursue during literature review: what no one knows; what is not yet well understood; what controversy or conflicting results, understandings, or perspectives exist; and what are unproven assumptions. 22 In qualitative research, these unknowns are critical to achieving a well-developed conceptual framework and a corresponding rigorous study design.

Recent contributions from Ravitch and colleagues present best practices in developing frameworks for conceptual and methodological coherence within a study design, regardless of the research approach. 23,24 Their recommendations and arguments are highly relevant to qualitative research. Figure 1 reflects the primary components of a conceptual framework adapted from Ravitch and Carl 23 and how all components contribute to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice. Notice that each element of the framework interacts with and influences other elements in a dynamic and interactive process from the beginning to the end of a research project. The intersecting bidirectional arrows represent direct relationships between elements as they relate to specific aspects of a qualitative research study.

Adaptation of Ravitch and Carl’s Components of a Conceptual Framework 23

Maxwell also provides useful guidance for developing an effective conceptual framework specific to the qualitative research paradigm. 17 The 2015 second edition of the Review Criteria for Research Manuscripts 14 and work by Ravitch and colleagues 23,24 provide specific guidance for applying the conceptual framework to each stage of the research process to enhance rigor and quality. Quality criteria for assessing a study’s problem statement, conceptual framework, and research question include the following: introduction builds a logical case and provides context for the problem statement; problem statement is clear and well-articulated; conceptual framework is explicit and justified; research purpose and/or question is clearly stated; and constructs being investigated are clearly identified and presented. 14,24,25 As best practice guidelines, these criteria facilitate quality and rigor while providing sufficient flexibility in how each is achieved and demonstrated.

While a conceptual framework is important to rigor in qualitative research, Huberman and Miles caution qualitative researchers about developing and using a framework to the extent that it influences qualitative design deductively because this would violate the very principles of induction that define the qualitative research paradigm. 25 Our profession’s recent emphasis on a holistic admissions process for pharmacy students provides a reasonable example of inductive and deductive reasoning and their respective applications in qualitative and quantitative research studies. Principles of inductive reasoning are applied when a qualitative research study examines a representative group of competent pharmacy professionals to generate a theory about essential cognitive and affective skills for patient-centered care. Deductive reasoning could then be applied to design a hypothesis-driven prospective study that compares the outcomes of two cohorts of students, one group admitted using traditional criteria and one admitted based on a holistic admissions process revised to value the affective skills of applicants. Essentially, the qualitative researcher must carefully generate a conceptual framework that guides the research question and study design without allowing the conceptual framework to become so rigid as to dictate a testable hypothesis, which is the founding principle of deductive reasoning. 26

Step 2: Qualitative Study Design

The development of a strong conceptual framework facilitates selection of appropriate study methods to minimize the bias inherent in qualitative studies and help readers to trust the research and the researcher (see Glassick criteria #3 in Table 1 ). Although researchers can employ great flexibility in the selection of study methods, inclusion of best practice methods for assuring the rigor and trustworthiness of results is critical to study design. Lincoln and Guba outline four criteria for establishing the overall trustworthiness of qualitative research results: credibility, the researcher ensures and imparts to the reader supporting evidence that the results accurately represent what was studied; transferability, the researcher provides detailed contextual information such that readers can determine whether the results are applicable to their or other situations; dependability, the researcher describes the study process in sufficient detail that the work could be repeated; confirmability, the researcher ensures and communicates to the reader that the results are based on and reflective of the information gathered from the participants and not the interpretations or bias of the researcher. 27

Specific best practice methods used in the sampling and data collection processes to increase the rigor and trustworthiness of qualitative research include: clear rationale for sampling design decisions, determination of data saturation, ethics in research design, member checking, prolonged engagement with and persistent observation of study participants, and triangulation of data sources. 28

Qualitative research is focused on making sense of lived, observed phenomenon in a specific context with specifically selected individuals, rather than attempting to generalize from sample to population. Therefore, sampling design in qualitative research is not random but defined purposively to include the most appropriate participants in the most appropriate context for answering the research question. Qualitative researchers recognize that certain participants are more likely to be “rich” with data or insight than others, and therefore, more relevant and useful in achieving the research purpose and answering the question at hand. The conceptual framework contributes directly to determining sample definitions, size, and recruitment of participants. A typical best practice is purposive sampling methods, and when appropriate, convenience sampling may be justified. 29

Purposive sampling reflects intentional selection of research participants to optimize data sources for answering the research question. For example, the research question may be best answered by persons who have particular experience (critical case sampling) or certain expertise (key informant sampling). Similarly, additional participants may be referred for participation by active participants (snowball sampling) or may be selected to represent either similar or opposing viewpoints (confirming or disconfirming samples). Again, the process of developing and using a strong conceptual framework to guide and justify methodological decisions, in this case defining and establishing the study sample, is critical to rigor and quality. 30 Convenience sampling, using the most accessible research participants, is the least rigorous approach to defining a study sample and may result in low accuracy, poor representativeness, low credibility, and lack of transferability of study results.

Qualitative studies typically reflect designs in which data collection and analysis are done concurrently, with results of ongoing analysis informing continuing data collection. Determination of a final sample size is largely based on having sufficient opportunity to collect relevant data until new information is no longer emerging from data collection, new coding is not feasible, and/or no new themes are emerging; that is, reaching data saturation , a common standard of rigor for data collection in qualitative studies . Thus, accurately predicting a sample size during the planning phases of qualitative research can be challenging. 30 Care should be taken that sufficient quantity (think thick description) and quality (think rich description) of data have been collected prior to concluding that data saturation has been achieved. A poor decision regarding sample size is a direct consequence of sampling strategy and quality of data generated, which leaves the researcher unable to fully answer the research question in sufficient depth. 30

Though data saturation is probably the most common terminology used to describe the achievement of sufficient sample size, it does not apply to all study designs. For example, one could argue that in some approaches to qualitative research, data collection could continue infinitely if the event continues infinitely. In education, we often anecdotally observe variations in the personality and structure of a class of students, and as generations of students continue to evolve with time, so too would the data generated from observing each successive class. In such situations, data saturation might never be achieved. Conversely, the number of participants available for inclusion in a sample may be small and some risk of not reaching data saturation may be unavoidable. Thus, the idea of fully achieving data saturation may be unrealistic when applied to some populations or research questions. In other instances, attrition and factors related to time and resources may contribute to not reaching data saturation within the limits of the study. By being transparent in the process and reporting of results when saturation may not have been possible, the resulting data may still contribute to the field and to further inquiry. Replication of the study using other samples and conducting additional types of follow-up studies are other options for better understanding the research phenomenon at hand. 31

In addition to defining the sample and selecting participants, other considerations related to sampling bias may impact the quantity and quality of data generated and therefore the quality of the study result. These include: methods of recruiting, procedures for informed consent, timing of the interviews in relation to experience or emotion, procedures for ensuring participant anonymity/confidentiality, interview setting, and methods of recording/transcribing the data. Any of these factors could potentially change the nature of the relationship between the researcher and the study participants and influence the trustworthiness of data collected or the study result. Thus, ongoing application of previously mentioned researcher reflexivity is critical to the rigor of the study and quality of sampling. 29,30

Common qualitative data collection methods used in health professions education include interview, direct observation methods, and textual/document analysis. Given the unique and often highly sensitive nature of data being collected by the researcher, trustworthiness is an essential component of the researcher-participant relationship. Ethical conduct refers to how moral principles and values are part of the research process. Participants’ perceptions of ethical conduct are fundamental to a relationship likely to generate high quality data. During each step of the research process, care must be taken to protect the confidentiality of participants and shield them from harm relating to issues of respect and dignity. Researchers must be respectful of the participants’ contributions and quotes, and results must be reported truthfully and honestly. 8

Interview methods range from highly structured to increase dependability or completely open-ended to allow for interviewers to clarify a participant’s response for increased credibility and confirmability. Regardless, interview protocols and structure are often modified or refined, based on concurrent data collection and analysis processes to support or refute preliminary interpretations and refine focus and continuing inquiry. Researcher reflexivity, or acknowledgement of researcher bias, is absolutely critical to the credibility and trustworthiness of data collection and analysis in such study designs. 32

Interviews should be recorded and transcribed verbatim prior to coding and analysis. 28 Member checking, a common standard of rigor, is a practice to increase study credibility and confirmability that involves asking a research subject to verify the transcription of an interview. 1,16,28 The research subject is asked to verify the completeness and accuracy of an interview transcript to ensure the transcript truthfully reflects the meaning and intent of the subject’s contribution.

Prolonged engagement involves the researcher gaining familiarity and understanding of the culture and context surrounding the persons or situations being studied. This strategy supports reflexivity, allowing the researcher to determine how they themselves may be a source of bias during the data collection process by altering the nature of how individuals behave or interact with others in the presence of the researcher. Facial expressions, spoken language, body language, style of dress, age, race, gender, social status, culture, and the researcher’s relationship with the participants may potentially influence either participants’ responses or how the researcher interprets those responses. 33 “Fitting in” by demonstrating an appreciation and understanding of the cultural norms of the population being studied potentially allows the researcher to obtain more open and honest responses from participants. However, if the research participants or topic are too familiar or personal, this may also influence data collection or analysis and interpretation of the results. 33 The possible applications of this section to faculty research with student participants in the context of pharmacy education are obvious, and researcher reflexivity is critical to rigor.

Some researchers using observational methods adopt a strategy of direct field observation, while others play partial or full participant roles in the activity being observed. In both observation scenarios, it is impossible to separate the researcher from the environment, and researcher reflexivity is essential. The pros and cons of observation approach, relative to the research question and study purpose, should be evaluated by the researcher, and the justification for the observational strategy selected should be made clear. 34 Regardless of the researcher’s degree of visibility to the study participants, persistent observation of the targeted sample is critical to the confirmability standard and to achieving data saturation. That is, study conclusions must be clearly grounded in persistent phenomena witnessed during the study, rather than on a fluke event. 28

Researchers acknowledge that observational methodologies are limited by the reality that the researcher carries a bias in determining what is observed, what is recorded, how it is recorded, and how it is transcribed for analysis. A study’s conceptual framework is critical to achieving rigor and quality and provides guidance in developing predetermined notions or plans for what to observe, how to record, and how to minimize the influence of potential bias. 34 Researcher notes should be recorded as soon as possible after the observation event to optimize accuracy. The more detailed and complete the notes, the more accurate and useful they can be in data analysis or in auditing processes for enhancing rigor in the interpretation phase of the study. 34

Triangulation is among the common standards of rigor applied within the qualitative research paradigm. Data triangulation is used to identify convergence of data obtained through multiple data sources and methods (eg, observation field notes and interview transcripts) to avoid or minimize error or bias and optimize accuracy in data collection and analysis processes. 33,35,36

Again, researcher practice in reflexivity throughout research processes is integral to rigor in study design and implementation. Researchers must demonstrate attention to appropriate methods and reflective critique, which are represented in both core elements of the conceptual framework ( Figure 1 ) and Glassick criteria ( Table 1 ). In so doing, the researcher will be well-prepared to justify sampling design and data collection decisions to manuscript reviewers and, ultimately, readers.

Step 3: Data Analysis

In many qualitative studies, data collection runs concurrently with data analysis. Specific standards of rigor are commonly used to ensure trustworthiness and integrity within the data analysis process, including use of computer software, peer review, audit trail, triangulation, and negative case analysis.

Management and analyses of qualitative data from written text, observational field notes, and interview transcriptions may be accomplished using manual methods or the assistance of computer software applications for coding and analysis. When managing very large data sets or complex study designs, computer software can be very helpful to assist researchers in coding, sorting, organizing, and weighting data elements. Software applications can facilitate ease in calculating semi-quantitative descriptive statistics, such as counts of specific events, that can be used as evidence that the researcher’s analysis is based on a representative majority of data collected ( inclusivism ) rather than focusing on selected rarities ( anecdotalism ). Using software to code data can also make it easier to identify deviant cases, detect coding errors, and estimate interrater reliability among multiple coders. 37 While such software helps to manage data, the actual analyses and interpretation still reside with the researcher.

Peer review, another common standard of rigor, is a process by which researchers invite an independent third-party researcher to analyze a detailed audit trail maintained by the study author. The audit trail methodically describes the step-by-step processes and decision-making throughout the study. Review of this audit trail occurs prior to manuscript development and enhances study confirmability. 1,16 The peer reviewer offers a critique of the study methods and validation of the conclusions drawn by the author as a thorough check on researcher bias.

Triangulation also plays a role in data analysis, as the term can also be used to describe how multiple sources of data can be used to confirm or refute interpretation, assertions, themes, and study conclusions. If a theme or theory can be arrived at and validated using multiple sources of data, the result of the study has greater credibility and confirmability. 16,33,36 Should any competing or controversial theories emerge during data collection or analysis, it is vital to the credibility and trustworthiness of the study that the author disclose and explore those negative cases. Negative case analysis refers to actively seeking out and scrutinizing data that do not fit or support the researcher’s interpretation of the data. 16

The use of best practices applying to data collection and data analysis facilitates the full examination of data relative to the study purpose and research question and helps to prevent premature closure of the study. Rather than stopping at the initial identification of literal, first-level assertion statements and themes, authors must progress to interpreting how results relate to, revise, or expand the conceptual framework, or offer an improved theory or model for explaining the study phenomenon of interest. Closing the loop on data collection is critical and is achieved when thorough and valid analysis can be linked back to the conceptual framework, as addressed in the next section.

Step 4: Drawing Valid Conclusions

Lingard and Kennedy 38 succinctly state that the purpose of qualitative research is to deepen one’s understanding of specific perspectives, observations, experiences, or events evidenced through the behaviors or products of individuals and groups as they are situated in specific contexts or circumstances. Conclusions generated from study results should enhance the conceptual framework, or contribute to a new theory or model development, and are most often situated within the discussion and conclusion sections of a manuscript.

The discussion section should include interpretation of the results and recommendations for practice. Interpretations should go beyond first-level results or literal description of observed behaviors, patterns, and themes from analysis. The author’s challenge is to provide a complete and thorough examination and explanation of how specific results relate to each other, contribute to answering the research question, and achieve the primary purpose of the research endeavor. The discussion should “close the loop” by integrating study results and analysis with the original conceptual framework. The discussion section should also provide a parsimonious narrative or graphical explanation and interpretation of study results that enhances understanding of the targeted phenomena.

The conclusion section should provide an overall picture or synopsis of the study, including its important and unique contributions to the field from the perspective of both conceptual and practical significance. The conclusion should also include personal and theoretical perspectives and future directions for research. Together, the discussion and conclusion should include responses to the larger questions of the study’s contributions, such as: So what? Why do these results matter? What next?

The strength of conclusions is dependent upon the extent to which standards of rigor and best practices were demonstrated in design, data collection, data analysis, and interpretation, as described in previous sections of this article. 4,12,17,23,24 Quality and rigor expectations for drawing valid conclusions and generating new theories are reflected in the following essential features of rigor and quality, which include: “Close the loop” to clearly link research questions, study design, data collection and analysis, and interpretation of results. Reflect effective integration of the study results with the conceptual framework and explain results in ways that relate, support, elaborate, and/or challenge conclusions of prior scholarship. Descriptions of new or enhanced frameworks or models are clear and effectively grounded in the study results and conclusions. Practical or theoretical implications are effectively discussed, including guidance for future studies. Limitations and issues of reflexivity and ethics are clearly and explicitly described, including references to actions taken to address these areas. 3,4,12,14

Step 5: Reporting Research Results

Key to quality reporting of qualitative research results are clarity, organization, completeness, accuracy, and conciseness in communicating the results to the reader of the research manuscript. O’Brien and others 4 proposed a standardized framework specifically for reporting qualitative studies known as the Standards for Reporting Qualitative Research (SRQR, Table 2 ). This framework provides detailed explanations of what should be reported in each of 21 sections of a qualitative research manuscript. While the SRQR does not explicitly mention a conceptual framework, the descriptions and table footnote clarification for the introduction and problem statement reflect the essential elements and focus of a conceptual framework. Ultimately, readers of published work determine levels of credibility, trustworthiness, and the like. A manuscript reviewer, the first reader of a study report, has the responsibility and privilege of providing critique and guidance to authors regarding achievement of quality criteria, execution and reporting of standards of rigor, and the extent to which meaningful contributions to thinking and practice in the field are presented. 13,39

An Adaptation of the 21 Elements of O’Brien and Colleagues’ Standards for Reporting Qualitative Research (SRQR) 4

Authors must avoid language heavy with connotations or adjectives that insert the researcher’s opinion into the database or manuscript. 14,40 The researcher should be as neutral and objective as possible in interpreting data and in presenting results. Thick and rich descriptions, where robust descriptive language is used to provide sufficient contextual information, enable the reader to determine credibility, transferability, dependability, and confirmability .

The process of demonstrating the credibility of research is rooted in honest and transparent reporting of how biases and other possible confounders were identified and addressed throughout study processes. Such reporting, first described within the study’s conceptual framework, should be revisited in reporting the work. Confounders may include the researcher’s training and previous experiences, personal connections to the background theory, access to the study population, and funding sources. These elements and processes are best represented in Glassick’s criteria for effective presentation and reflective critique ( Table 1 , criteria 5 and 6). Transferability is communicated, in part, through description of sampling factors such as: geographical location of the study, number and characteristics of participants, and the timeframe of data collection and analysis. 40 Such descriptions also contribute to the credibility of the results and readers’ determination of transfer to their and other contexts. To ensure dependability, the research method must be reported in detail such that the reader can determine proper research practices have been followed and that future researchers can repeat the study. 40 The confirmability of the results is influenced by reducing or at a minimum explaining any researcher influence on the result by applying and meeting standards of rigor such as member checking, triangulation, and peer review. 29,33

In qualitative studies, the researcher is often the primary instrument for data collection. Any researcher biases not adequately addressed or errors in judgement can affect the quality of data and subsequent research results. 33 Thus, due to the creative interpretative and contextually bound nature of qualitative studies, the application of standards of rigor and adherence to systematic processes well-documented in an audit trail are essential. The application of rigor and quality criteria extend beyond the researcher and are also important to effective peer review processes within a study and for scholarly dissemination. The goal of rigor in qualitative research can be described as ensuring that the research design, method, and conclusions are explicit, public, replicable, open to critique, and free of bias. 41 Rigor in the research process and results are achieved when each element of study methodology is systematic and transparent through complete, methodical, and accurate reporting. 33 Beginning the study with a well-developed conceptual framework and active use of both researcher reflexivity and rigorous peer review during study implementation can drive both study rigor and quality.

As the number of published qualitative studies in health professions educational research increases, it is important for our community of health care educators to keep in mind the unique aspects of rigor in qualitative studies presented here. Qualitative researchers should select and apply any of the above referenced study methods and research practices, as appropriate to the research question, to achieve rigor and quality. As in any research paradigm, the goal of quality and rigor in qualitative research is to minimize the risk of bias and maximize the accuracy and credibility of research results. Rigor is best achieved through thoughtful and deliberate planning, diligent and ongoing application of researcher reflexivity, and honest communication between the researcher and the audience regarding the study and its results.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

Reliability vs. Validity in Research | Difference, Types and Examples

Published on July 3, 2019 by Fiona Middleton . Revised on June 22, 2023.

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method , technique. or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure.opt

It’s important to consider reliability and validity when you are creating your research design , planning your methods, and writing up your results, especially in quantitative research . Failing to do so can lead to several types of research bias and seriously affect your work.

Table of contents

Understanding reliability vs validity, how are reliability and validity assessed, how to ensure validity and reliability in your research, where to write about reliability and validity in a thesis, other interesting articles.

Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?

Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.

What is validity?

Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.

If the thermometer shows different temperatures each time, even though you have carefully controlled conditions to ensure the sample’s temperature stays the same, the thermometer is probably malfunctioning, and therefore its measurements are not valid.

However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

Prevent plagiarism. Run a free check.

Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types.

Types of reliability

Different types of reliability can be estimated through various statistical methods.

Types of validity

The validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment ) and external validity (the generalizability of the results).

The reliability and validity of your results depends on creating a strong research design , choosing appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring validity

If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data.

• Choose appropriate methods of measurement

Ensure that your method and measurement technique are high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge.

For example, to collect data on a personality trait, you could use a standardized questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or findings of previous studies, and the questions should be carefully and precisely worded.

• Use appropriate sampling methods to select your subjects

To produce valid and generalizable results, clearly define the population you are researching (e.g., people from a specific age range, geographical location, or profession).  Ensure that you have enough participants and that they are representative of the population. Failing to do so can lead to sampling bias and selection bias .

Ensuring reliability

Reliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable, and reproducible .

• Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations , clearly define how specific behaviors or responses will be counted, and make sure questions are phrased the same way each time. Failing to do so can lead to errors such as omitted variable bias or information bias .

• Standardize the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions, preferably in a properly randomized setting. Failing to do so can lead to a placebo effect , Hawthorne effect , or other demand characteristics . If participants can guess the aims or objectives of a study, they may attempt to act in more socially desirable ways.

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper . Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Middleton, F. (2023, June 22). Reliability vs. Validity in Research | Difference, Types and Examples. Scribbr. Retrieved April 2, 2024, from https://www.scribbr.com/methodology/reliability-vs-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, what is quantitative research | definition, uses & methods, data collection | definition, methods & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Everything Advanced Search

Qualitative Methods in Public Health Research

Find similar titles by category

• Study Protocol
• Open access
• Published: 26 March 2024

The effect of a midwifery continuity of care program on clinical competence of midwifery students and delivery outcomes: a mixed-methods protocol

• Fatemeh Razavinia   ORCID: orcid.org/0000-0002-6827-509X 1 , 2 ,
• Parvin Abedi   ORCID: orcid.org/0000-0002-6980-0693 3 ,
• Mina Iravani   ORCID: orcid.org/0000-0002-8854-1738 4 ,
• Eesa Mohammadi   ORCID: orcid.org/0000-0001-6169-9829 5 ,
• Bahman Cheraghian   ORCID: orcid.org/0000-0001-5446-6998 6 ,
• Shayesteh Jahanfar   ORCID: orcid.org/0000-0001-6149-1067 7 &
• Mahin Najafian   ORCID: orcid.org/0000-0002-6649-3931 8  

BMC Medical Education volume  24 , Article number:  338 ( 2024 ) Cite this article

113 Accesses

Metrics details

The midwifery continuity of care model is one of the care models that have not been evaluated well in some countries including Iran. We aimed to assess the effect of a program based on this model on the clinical competence of midwifery students and delivery outcomes in Ahvaz, Iran.

This sequential embedded mixed-methods study will include a quantitative and a qualitative phase. In the first stage, based on the Iranian midwifery curriculum and review of seminal midwifery texts, a questionnaire will be developed to assess midwifery students’ clinical competence. Then, in the second stage, the quantitative phase (randomized clinical trial) will be conducted to see the effect of continuity of care provided by students on maternal and neonatal outcomes. In the third stage, a qualitative study (conventional content analysis) will be carried out to investigate the students’ and mothers’ perception of continuity of care. Finally, the results of the quantitative and qualitative phases will be integrated.

According to the nature of the study, the findings of this research can be effectively used in providing conventional midwifery services in public centers and in midwifery education.

Trial registration

This study was approved by the Ethics Committee of Ahvaz Jundishapur University of Medical Sciences (IR.AJUMS.REC.1401.460). Also, the study protocol was registered in the Iranian Registry for Randomized Controlled Trials (IRCT20221227056938N1).

Peer Review reports

Providing quality services to pregnant women has been recommended to all countries to achieve the Millennium Development Goals (MDGs) (Goals 3, 4 and 5) [ 1 ]. There are different care methods to maintain maternal and neonatal health during pregnancy and postpartum [ 1 ]. One of these care models is continuity of care that can be provided by a midwife or an obstetrician.

Midwifery continuity of care is a relationship-based care provided by a midwife who can be supported by one to three more midwives. They provide planned care for a woman during pregnancy, labor, birth, and the early postpartum period up to 6 weeks after delivery [ 2 ].

Continuity of midwifery care has become a global effort to enable women to have access to high-quality maternity care and delivery services [ 3 ]. As a result, many service providers today are transitioning to a continuous care model [ 4 ], and they have considered continuous care to be necessary for realizing women's rights [ 5 ]. Also, continuous midwifery care is known as the gold standard in maternity care to achieve excellent results for women [ 5 , 6 ]. In order to strengthen midwifery services to achieve global health goals in 2015, the World Health Organization (WHO) proposed a midwife-led continuous care model [ 7 ].

Countries use different midwifery care models. In Iran, for example, primary health services that are specific to pregnant mothers are provided in public health centers by midwives working in the network system and in compliance with the level of services and the referral system [ 8 ].

In general, midwifery continuous care not only has an important impact on a wide range of health and clinical outcomes for mothers and neonates but also brings about economic consequences for the health system [ 2 , 9 ]. This care model is useful for healthcare professionals as well [ 10 ], and it has improved the job satisfaction of midwives [ 11 ]. The midwife is the main guide in planning, organizing and providing care to a woman from the beginning of pregnancy to the postpartum period [ 12 ]. In 2011, in order to increase job motivation and satisfaction, promote retention of the midwifery workforce [ 13 ], and alleviate the shortage of workforce at the international level [ 14 ], the Nursing and Midwifery Advisory Center recommended using midwifery students (at the bedside and to perform midwifery work) to overcome this problem.

Providing high quality care requires enhancing the clinical competence of the professionals [ 4 ]. There is a close relationship between the concept of patient care quality and clinical competence. Therefore, clinical competence is of unique importance in midwifery practice [ 15 ]. As a result, in order to achieve quality patient care, midwifery professionals need to train students to become workforce with clinical competence in order to provide quality care in the health system. WHO defined clinical competence as a level of performance that demonstrates the effective application of knowledge, skills, and judgment [ 16 ].

A previous study showed that clinical competence of midwives plays an important role in managing the process of providing care, achieving care goals, and improving the quality of midwifery services [ 17 ]. In other words, the graduates of this field must have an acceptable level of clinical and professional skills in performing midwifery duties so that the health of mothers, children, and ultimately the community can be improved.

In Iran, prenatal care and the care during labor, delivery and postpartum are not continuous, and a new health provider may take the responsibility of care at any stage. This fragmented care may negatively affect the pregnancy outcomes and increase the rate of cesarean section [ 18 ]. Furthermore, the results of some studies in Iran indicate that the clinical competence obtained by midwifery students is far from optimal and that they do not acquire the necessary skills and abilities at the end of their studies [ 19 ]. Farrokhi et al. showed that the performance quality of 70% of midwives is average, and only 18.5% of them have good quality performance [ 20 ]. Several factors play a role in acquiring, maintaining and improving clinical competence [ 21 ]. There are a number of solutions that can increase the clinical competence of midwifery students, and one is the use of different care models such as the continuity of care model. The continuity of care model allows students to develop their midwifery knowledge, skills, and values individually [ 22 ]. Despite the strong foundation of midwifery in Iran, midwifery care models have not yet been tested. Some studies have reported that the quality of services provided during pregnancy, delivery and after delivery in Iran is poor to moderate. Also, these studies emphasize the necessity of a paradigm shift for better quality care and greater satisfaction of mothers, and they consider lack of continuity of care as the reason for the increase in unnecessary cesarean sections [ 23 , 24 , 25 ]. Moreover, the lack of qualified and experienced workforce has led to low quality health services, including midwifery care, and an increase in the economic burden of health. In Iran, no study has yet been conducted to investigate the effect of the midwifery continuity of care model on the students’ clinical competence and pregnancy outcomes. Given the importance of this topic, using a mixed-methods study design, we aimed to assess the effect of a midwifery continuity of care program on the clinical competence of midwifery students and pregnancy outcomes in Ahvaz, Iran.

Specific objectives

To determine the effect of midwifery continuity of care program on the clinical competence of midwifery students.

To determine the effect of a midwifery continuity of care program provided by midwifery students on pregnancy outcomes.

To explain the perception of midwifery students and mothers about the use of the midwifery continuity of care program provided by midwifery students.

Methods/design

Study design.

This sequential embedded mixed-methods study will include a quantitative phase and a qualitative one. A mixed (embedded) experimental design involves the collection and analysis of quantitative and qualitative data by the researcher and the integration of the information into an experimental study or intervention trial. This design adds qualitative data to an experiment or intervention to integrate the personal experience of research participants. Therefore, the qualitative data are converted into a secondary source of data embedded before and after the test. Qualitative data is added to the experiment in differrent ways, including: before the experiment, during the experiment, or after the experiment [ 26 , 27 ]. Embedded mixed-methods studies that are qualitative followed by quantitative are used to understand the rationale for the results and receive feedback from participants (to confirm and support the findings of the quantitative studies) [ 27 ]. In the first stage of this study, a questionnaire for assessing midwifery students’ clinical competence will be created based on the midwifery curriculum of Iran and a review of seminal texts of midwifery. Then, the effect of continuity of care provided by midwifery students on maternal and neonatal outcomes will be assessed in a randomized clinical trial. In the third stage, a qualitative study will be carried out to investigate the perception of students and mothers. Finally, the results of the quantitative and qualitative phases will be integrated (Fig.  1 ).

Sequential and embedded mixed-methods design

First stage: questionnaire development

This questionnaire will be developed based on midwifery curriculum and a comprehensive and systematic search (with no time limit) in English and Persian databases (Web of Science, Embase, Scopus, ProQuest, Google scholar, Magiran, SID).

Tool design

There are four steps in tool development:

Choosing a conceptual model to show aspects of clinical competence in the measurement process

Explaining the purpose of the tool

Designing the route map

Developing the tool (use of methods, classification of objects, rules and procedures for scoring tools) [ 28 ].

Answer to the objects

A 1 to 4-point Likert scale will be used for scoring [ 29 ].

Content validity

To ensure the selection of the most important and correct content (necessity of the case), the content validity will be assessed. Also, to ensure that the instrument items are designed in the best way to measure the content, the content validity index will be calculated [ 30 ].

Reliability

Reliability will be evaluated using internal consistency (Cronbach's alpha coefficient ≥ 0.7) and stability (test-re-test ≥ 0.74) by piloting the questionnaire on 20 midwifery students [ 31 ].

Second stage: quantitative phase

A randomized controlled clinical trial will be conducted in this phase of research to examine the effect of the continuous care program of midwifery students on their clinical competence and pregnancy outcomes.

Sample size

According to the study objective and previous study results [ 32 ] with α = 0.01, β = 0.1, p 1  = 0.51 and p 2  = 0.021, the sample size will be n  = 23. Considering a 20% dropout rate, the final sample size will be 58 women (29 women in each group).

Data collection

This phase of the randomized clinical trial will be conducted with the participation of 58 undergraduate midwifery students at their 7th and 8th semesters. The students will be divided randomly to intervention (continuous care) and control (routine care) groups providing care to 58 pregnant women in six health centers and two hospitals (Sina and Razi) in Ahvaz city, southwest of Iran.

The study will begin after receiving the approval of the Ethics Committee of Ahvaz University of Medical Sciences and registering the study in the Iranian Registry for Randomized Clinical Trials. Inclusion criteria will be willingness to participate in the study.

Randomization

To implement the intervention, the students will be divided into two intervention (providing continuous care for pregnant women) and control (providing standard care for pregnant women) groups. Allocating students will be done using permuted block randomization technique with a block size of four and an allocation ratio of 1:1. Five blocks of 4 pieces and 3 blocks of 3 pieces will be extracted randomly using WIN PEPI software. In each block of 4, 2 students will be in control and 2 will be in intervention group. Also, in each block of 3 students, 1 student will be in control and 2 will be in intervention group, and the arrangement of each person is random. To prevent contamination, first the control group will provide routine care, and then the intervention group will conduct continuity of care for pregnant women. Mothers are randomly selected based on the hospital where they will give birth. As a result, Razi Hospital will be the control group and Sina Hospital will be the intervention group.

Intervention

Women who meet the inclusion criteria will be recruited in the study using a non-probability convenience sampling method. Women in the intervention group will be included in the study after their first pregnancy visit (6–10 weeks of gestation) and will receive continuous care by midwifery students. Women in the control group will receive the usual and routine care, and will be included in the study at the time of delivery. They will have a gestational age of more than 37 weeks based on the inclusion criteria of the study. Their delivery will be performed by midwifery students who will follow them up until six weeks after delivery.

At first, the necessary training will be given by the lead researcher (FR) to the students in orientation sessions held for both groups separately. In the intervention group, each midwifery student as the main midwife will be responsible for taking care of two or three pregnant women and will be the back-up midwife for two other pregnant women (under the supervision of other students). The lead researcher will create a group in WhatsApp with the participation of students in the intervention group, and they can communicate with each other and the researcher. Also, the midwifery students will be directly and indirectly under the supervision of a qualified person (lead researcher). Another WhatsApp group will be created for the women of the intervention and control groups (to facilitate communication between the researcher and the women). Two midwifery students will be introduced to each pregnant woman in the intervention group (as a main midwife and a backup midwife). If the main midwife is not available, the woman will be in contact with the backup midwife. The backup student will meet the woman at least once and will be introduced to her.

Instruments

All students and pregnant women participating in this study will complete a demographic questionnaire. A checklist will be provided for collecting data during prenatal care, labor, and delivery.

Also, the midwifery students will complete the clinical competency questionnaire at the beginning and end of the study.

Care will be provided and recorded by the main student according to the pregnancy care protocol. Also, danger signs will be taught to the students according to the national protocol, and emergencies will be handled by the midwifery student under the supervision of the lead researcher. Admission to hospital will be arranged by the student, and all information will be recorded. Pregnancy, labour and delivery, postpartum, and newborn checklist will be completed. Students will complete a demographic and obstetric questionnaire that includes questions about age, education, occupation, gravidity, parity, abortions, live and dead children, last contraceptive method, intended and unintended pregnancies, last menstrual period (LMP), gestational age, date of birth, body mass index (BMI), previous pregnancy and childbirth records, high-risk behavior of the mother and father, current history of special care, test and ultrasound results, and participation in childbirth preparation class. Also, the following data will be recorded in the labor and delivery and post-partum checklist: checking the conditions of labor according to the partograph, length of labor, need for induction and the method used type of delivery, examination of perineal trauma, postpartum bleeding, and examination of the condition of the mother up to 6 weeks after delivery. In addition, the amount of bleeding will be checked visually and by measuring the level of hemoglobin and hematocrit. Apgar score of the newborn will be recorded (in infant checklist) in minutes 1 and 5. Also, the newborn’s hospitalization status, breastfeeding and anthropometric indices will be recorded.

The students in the intervention group will start prenatal care < 20 weeks of gestation. At least five round of prenatal care will be provided by each student according to national guidelines for each pregnant woman. Pregnant women can communicate with their in-charge students in non-emergency cases from 8:00 a.m. to 23:00 p.m. and in emergency cases 24 h a day, all days a week. All reports will be recorded by the students. During labor and delivery, the student and the lead researcher will be present at the mother's bedside. In case of natural vaginal delivery (NVD), delivery will be done by a student midwife under the supervision of the researcher. In case of cesarean delivery (CS), a student will be present at the patient's bedside. Postpartum care will be provided by midwifery students in both groups (intervention and control). Each student will be at the mother's bedside for two hours after delivery. The conditions of labor, delivery, and the neonate will be recorded by the student in the relevant form. Also, the mother will be followed up by telephone for up to 6 weeks after delivery (postpartum). The clinical competency questionnaire will be completed by students before and after the intervention.

Inclusion criteria

Inclusion criteria for midwifery students will be: studying at the seventh and eighth semester and willingness to participate in the study.

Inclusion criteria for service recipients (pregnant women) will be: age 18 – 40 years, Iranian nationality, singleton pregnancy, low risk pregnancy, and gestational age < 20 wks.

Exclusion criteria

Exclusion criteria will be: history of psychiatric disorders, previous caesarean section, use of alcohol and tobacco, or having a disease that requires prenatal care by a specialist.

Primary outcome

Clinical competence of midwifery students.

Secondary outcome

Mode of delivery, length of labor stages, the need to induction, postpartum bleeding first and fifth minute Apgar score, admission of neonate to the neonatal intensive care unit, breastfeeding initiation, and exclusive breastfeeding up to 6 weeks postpartum.

Data analysis

Statistical analyses will be done using SPSS version 26.0 (SPSS, Inc., Chicago, IL, USA). The independent t-test and Chi-square tests will be used for continuous data and categorical data, respectively. ANCOVA test will be used to eliminate the influence of confounding variables. The effect size will be calculated. A 95% confidence interval (CI) and p values will be reported. P -values less than 0.5 will be considered statistically significant.

Third step of research: qualitative study

This phase will be a qualitative study using conventional content analysis.

Purposeful sampling will be used in this study [ 33 ]. Sampling will continue until data saturation [ 34 ], i.e., no new information or data about a class or relationships between classes is revealed.

This phase of the study is a conventional qualitative content analysis [ 35 ] aimed at examining the perceptions of midwifery students and mothers receiving continuous care. The researcher will conduct in-depth, semi-structured interviews with open-ended questions with students and mothers in the group of the continuous care program. All interviews will be done by the lead researcher who is qualified in qualitative research method. The interview will start with a general and open question such as: “Please tell me about your experiences or feelings about participating in the continuous midwifery care program. How did you feel about participating in this program?” Then, in-depth exploratory questions will be asked based on their answers (e.g., what do you mean? Why? Can you elaborate on that? Can you give me an example so I can understand what you mean?). All interviews will be recorded with the participants' consent. Paralinguistic features, such as mood and features of the participants, including tone of voice, facial expressions, and their posture, will be recorded by the researcher during the interview [ 35 ].

The data will be analyzed based on Granheim and Lundman's 2004 content analysis approach [ 36 ].

Interviews will be transcribed at the end of each interview. Data analysis begins with a careful study of all data so that the researcher can immerse herself in the data and gain an overview. Interviews will be transcribed verbatim. Key concepts will be highlighted and codes will be extracted. Then the first interpretations will be made and analyzed. Labels emerge for codes that represent more than one key concept and are usually taken directly from the text and become the initial coding map. Then the codes are placed in the category based on their similarity. Then, definitions will be created for each category, subcategory and code. When reporting findings, examples of each code and data category will be provided [ 35 ].

Inclusion criteria for midwifery students will be: studying at the seventh or eighth semester, willingness to participate in the study.

Inclusion criteria for service recipients (pregnant women) will be: receiving continuous care provided by the student, willingness to participate in the study, and being able to communicate.

The qualitative study and interview data will be analyzed based on the content analysis approach of Granheim and Lundman 2004 [ 36 ] as follows:

Reading and re-reading the interviews after completion of each interview

Selection of the unit of analysis

Determination of semantic units

Classification

Extraction of information content

In the first step, the data is converted into text format. As soon as possible after the interview, the interview will be typed verbatim. Then the whole text will be read several times to get a general understanding of the content of interview. Each meaning unit will be converted into condensed meaning units and then coded. The Codes will be classified into subcategories and categories based on their common characteristics. Finally, the content of the categories will be revealed, taking into account their hidden meaning [ 36 ].

Trustworthiness

Five criteria of will be used to increase data trustworthiness according to Lincoln & Guba [ 37 ]. These include: 1. Credibility, 2. Dependability, 3. Confirmability, 4. Transferability, 5. Authenticity.

Credibility of the data will be ensured by continuous engagement of the researchers with the subject, member checks, and external checks. Dependability will be ensured by relying on the insight of external observers. In order to increase the confirmability, data will be accurately recorded and reported. Also, transferability will be ensured by presenting the research process accurately, clearly and purposefully, which includes purposive sampling and presenting the research results to a number of people with the same profile of the participants who did not participate in the research. Finally, authenticity will be guaranteed by continuous reflection on information, long-term presence of the researcher, interview recording, writing, and reporting of findings.

Combining qualitative and quantitative phases

Data combination will be done using data integration strategies. The integration or combination of data starts from quantitative data analysis. Then qualitative data is collected by interview. In fact, the qualitative study is a secondary source of embedded data in the collection of experimental test data (continuous care) after the quantitative study. In this research, in order to understand the results of the RCT, the views of the participants will be unified in order to get a correct understanding of the intervention (implementation of the continuity of care model by the students) from the mothers' and students' point of view (Fig.  2 ).

Study diagram

Study status

The development of the evaluation tools was made. Also, sampling the quantitative phase of the study and the basic of the program are in process (Table 1 ).

This is the first mixed-methods study to be conducted in Iran investigating the effect of a midwifery continuity of care program on clinical competence of midwifery students and pregnancy outcomes. According to the recommendations of the WHO, midwifery continuity of care should be adopted in order to increase the quality of pregnancy care as well as the satisfaction of pregnant women and service providers [ 7 ]. Contrary to the recommendation of WHO, the continuous care program is neither implemented in Iran's health system nor included in the midwifery curriculum. The results of this study can help health planners and policy makers to implement high quality midwifery care program based on global recommendations.

The study has several strengths. The use of a mixed-methods study design (combination of quantitative and qualitative approaches) in contrast to the separate use of quantitative and qualitative studies provides a better understanding of the research questions [ 38 ]. In embedded design, one type of data collection (quantitative or qualitative) plays a supporting and essential role for another type. As a result, the embedded mixed-methods technique in the qualitative phase after designing the intervention will be used to receive feedback from the participants to confirm and support the findings of quantitative phase [ 39 ]. Also, interviews with mothers and midwifery students in the intervention group can reflect their positive and negative experiences of this program. Considering that Iran's healthcare system lacks continuous midwifery care, the findings of this research can be effectively used in providing conventional midwifery services in public centers and in midwifery education.

Considering that this care model will be implemented for the first time in Iran's midwifery education and healthcare system, there may be two possible limitations in this study: lack of infrastructure and interference with other educational programs.

Availability of data and materials

All the data that will be obtained will be published in the next article after the implementation of the study.

Abbreviations

Body mass index

Cesarean section

Last menstrual period

Millennium Development Goals

Natural vaginal delivery

World Health Organization

Bagheri A, Simbar M, Samimi M, Nahidi F, Majd HA. Exploring the concept of continuous midwifery-led care and its dimensions in the prenatal, perinatal, and postnatal periods in Iran (Kashan). Midwifery. 2017;51:44–52.

Article   Google Scholar  

Cummins A, Coddington R, Fox D, Symon A. Exploring the qualities of midwifery-led continuity of care in Australia (MiLCCA) using the quality maternal and newborn care framework. Women Birth. 2020;33(2):125–34.

Choudhary S, Jelly P, Mahala P. Models of maternity care: a continuity of midwifery care. Int J Reprod Contracept Obstet Gynecol. 2020;9(6):2666–70.

Bradford BF, Wilson AN, Portela A, McConville F, Fernandez Turienzo C, Homer CS. Midwifery continuity of care: a scoping review of where, how, by whom and for whom? PLOS Global Public Health. 2022;2(10):e0000935.

Lettink A, Chaibekava K, Smits L, Langenveld J, van de Laar R, Peeters B, et al. CCT: continuous care trial-a randomized controlled trial of the provision of continuous care during labor by maternity care assistants in the Netherlands. BMC Pregnancy Childbirth. 2020;20(1):1–6.

Chaibekava KV, Scheenen AJ, Lettink A, Smits LJ, Langenveld J, Van De Laar R, et al. Continuous care during labor by maternity care assistants in the Netherlands vs care-as-usual: a randomized controlled trial. Am J Obstet Gynecol MFM. 2023;5(11):101168.

Michel-Schuldt M, McFadden A, Renfrew M, Homer C. The provision of midwife-led care in low-and middle-income countries: an integrative review. Midwifery. 2020;84:102659.

Khosravi S, Babaey F, Abedi P, Kalahroodi ZM, Hajimirzaie SS. Strategies to improve the quality of midwifery care and developing midwife-centered care in Iran: analyzing the attitudes of midwifery experts. BMC Pregnancy Childbirth. 2022;22(1):40.

Donnellan Fernandez R, Scarf V, Devane D, Healey A. Is Midwifery Continuity of Care Cost Effective? Midwifery Continuity of Care. Elsevier; 2019.  p. 137–56.

Aune I, Tysland T, Amalie VS. Norwegian midwives’ experiences of relational continuity of midwifery care in the primary healthcare service: a qualitative descriptive study. Nordic J Nurs Res. 2021;41(1):5–13.

Hanley A, Davis D, Kurz E. Job satisfaction and sustainability of midwives working in caseload models of care: an integrative literature review. Women Birth. 2022;35(4):e397–407.

Grylka-Baeschlin S, Iglesias C, Erdin R, Pehlke-Milde J. Evaluation of a midwifery network to guarantee outpatient postpartum care: a mixed methods study. BMC Health Serv Res. 2020;20:1–12.

Evans J, Taylor J, Browne J, Ferguson S, Atchan M, Maher P, et al. The future in their hands: graduating student midwives’ plans, job satisfaction and the desire to work in midwifery continuity of care. Women Birth. 2020;33(1):e59–66.

Carter J, Sidebotham M, Dietsch E. Prepared and motivated to work in midwifery continuity of care? A descriptive analysis of midwifery students’ perspectives. Women Birth. 2022;35(2):160–71.

Inoue N, Nakao Y, Yoshidome A. Development and validity of an intrapartum self-assessment scale aimed at instilling midwife-led care competencies used at freestanding midwifery units. Int J Environ Res Public Health. 2023;20(3):1859.

Vázquez-Sánchez C, Gigirey L, editors. Design of a scoring rubric for the assessment of clinical competencies in the subject of optometry IV of the degree of optics and optometry-USC. INTED2023 Proceedings. Valencia: IATED; 2023. p. 2224–30. https://doi.org/10.21125/inted.2023.0613 .

Kubota S, Ando M, Murray J, Khambounheuang S, Theppanya K, Nanthavong P, et al. A regulatory gap analysis of midwifery to deliver essential reproductive, maternal, newborn, child and adolescent health services in Lao People’s Democratic Republic. Lancet Reg Health West Pac. 2024;43:1–10. https://doi.org/10.1016/j.lanwpc.2023.100959 .

Pazandeh F, Moridi M, Safari K. Labouring women perspectives on mistreatment during childbirth: a qualitative study. Nursing Ethics. 2023;30(4):513–25. https://doi.org/10.1177/09697330231158732 .

Abbasi A, Bazghaleh M, FadaeeAghdam N, Basirinezhad MH, Tanhan A, Montazeri R, et al. Efficacy of simulated video on test anxiety in objective structured clinical examination among nursing and midwifery students: a quasi-experimental study. Nurs Open. 2023;10(1):165–71.

Farokhi F. Quality assessment of midwives performance in prenatal cares in urban health centers in Mashhad, Iran. Payesh (Health Monitor). 2008;7(3):0.

Google Scholar  

Moradali MR, Hajian S, Majd HA, Rahbar M, Entezarmahdi R. Job Satisfaction and its Related Factors in Midwives Working in the Health Services System in Iran: A Systematic Review. J Midwifery Reprod Health. 2023;11(2):3650–63. https://doi.org/10.22038/jmrh.2023.64824.1890 .

Hainsworth N, Cummins A, Newnham E, Foureur M. Learning through relationships: the transformative learning experience of midwifery continuity of care for students: a qualitative study. Women Birth. 2023;36(4):385–92.

Lazar J. Exploring the experiences of midwives facilitating group antenatal care: City, University of London; 2023.

Adelson P, Fleet J-A, McKellar L. Evaluation of a regional midwifery caseload model of care integrated across five birthing sites in South Australia: women’s experiences and birth outcomes. Women Birth. 2023;36(1):80–8.

Prussing E, Browne G, Dowse E, Hartz D, Cummins A. Implementing midwifery continuity of care models in regional Australia: a constructivist grounded theory study. Women Birth. 2023;36(1):99–107.

Pintubatu J. keterampilan proses sains pada pembelajaran ipas berorientasi outdoor learning siswa SMK Negeri 1 Lolak. Charm Sains: Jurnal Pendidikan Fisika. 2023;4(1):7–12.

Creswell JW. Controversies in mixed methods research. Sage Handbook Qual Res. 2011;4(1):269–84.

Kristensen SB, Clausen A, Skjødt MK, Søndergaard J, Abrahamsen B, Möller S, et al. An enhanced version of FREM (Fracture Risk Evaluation Model) using national administrative health data: analysis protocol for development and validation of a multivariable prediction model. Diagn Progn Res. 2023;7(1):19.

Alabi AT, Jelili MO. Clarifying likert scale misconceptions for improved application in urban studies. Qual Quant. 2023;57(2):1337–50.

Roopashree MR. A pragmatic approach for the calculation content validity indices: a study on validation of training tool for pre and post-test questionnaire for the health care sector. QAI J Healthc Qual Patient Saf. 2023;4(1):17–23.

Martyushev NV, Malozyomov BV, Sorokova SN, Efremenkov EA, Valuev DV, Qi M. Review models and methods for determining and predicting the reliability of technical systems and transport. Mathematics. 2023;11(15):3317.

Hildingsson I, Karlström A, Haines H, Johansson M. Swedish women’s interest in models of midwifery care–Time to consider the system? A prospective longitudinal survey. Sex Reprod Healthc. 2016;7:27–32.

Creswell JW, Creswell JD. Research design: Qualitative, quantitative, and mixed methods approaches. 3rd ed. SAGE Publications Inc.; 2017.

Obilor EI. Convenience and purposive sampling techniques: are they the same. Int J Innov Soc Sci Educ Res. 2023;11(1):1–7.

Serafini F, Reid SF. Multimodal content analysis: expanding analytical approaches to content analysis. Vis Commun. 2023;22(4):623–49.

Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12.

Lincoln YS, Guba EG. Naturalistic inquiry. SAGE Publications; 1985.

Matović N, Ovesni K. Interaction of quantitative and qualitative methodology in mixed methods research: integration and/or combination. Int J Social Res Methodol. 2023;26(1):51–65.

Creswell JW, Clark VLP. Designing and conducting mixed methods research. SAGE Publication, Inc; 2017.

Download references

The study was funded by Ahvaz Jundishapur University of Medical Sciences.

Author information

Authors and affiliations.

Midwifery Department, Reproductive Health Promotion Research Center, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

Fatemeh Razavinia

Midwifery Department, Menopause Andropause Research Center, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

Midwifery Department, Menopause Andropause Research Center, Ahvaz Jundisahpur University of Medical Sciences, Golestan BLvd, Ahvaz, Iran

Parvin Abedi

Reproductive Health Promotion Research Center, Midwifery Department, Nursing and Midwifery School, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

Mina Iravani

Department of Nursing, Faculty of Medical Sciences, Tarbiat Modares University, Tehran, Iran

Eesa Mohammadi

Alimentary Tract Research Center, Clinical Sciences Research Institute, Department of Biostatistics and Epidemiology, School of Public Health, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

Bahman Cheraghian

MPH Program, Department of Public Health and Community Medicine, Tufts University School of Medicine, Boston, USA

Shayesteh Jahanfar

Department of Obstetrics and Gynecology, School of Medicine, Ahvaz Jundishapur University of Medical Sciences, Ahvaz, Iran

Mahin Najafian

You can also search for this author in PubMed   Google Scholar

Contributions

FR, PA, MI, EM, BCh, ShJ and MN conceptualized the study. FR will collect the data. FR drafted the protocol. PA revised the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Parvin Abedi .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the Ethics Committee of Ahvaz Jundishapur University of Medical Sciences (IR.AJUMS.REC.1401.460). Also, the study protocol was registered in the Iranian Registry for Randomized Controlled Trials (IRCT20221227056938N1). Informed consent will be obtained from all participants. The study’s findings will be shared via the publishing of peer-reviewed articles, talks at scientific conferences and meetings with related teams.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Razavinia, F., Abedi, P., Iravani, M. et al. The effect of a midwifery continuity of care program on clinical competence of midwifery students and delivery outcomes: a mixed-methods protocol. BMC Med Educ 24 , 338 (2024). https://doi.org/10.1186/s12909-024-05321-5

Download citation

Received : 28 October 2023

Accepted : 15 March 2024

Published : 26 March 2024

DOI : https://doi.org/10.1186/s12909-024-05321-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Continuity of care
• Clinical competence
• Mixed-methods
• Midwifery students
• Pregnancy outcomes

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 01 April 2024

Midwives’ lived experiences of caring for women with mobility disabilities during pregnancy, labour and puerperium in Eswatini: a qualitative study

• Annie M. Temane 1 ,
• Fortunate N. Magagula 2 &
• Anna G. W. Nolte 1  

BMC Women's Health volume  24 , Article number:  207 ( 2024 ) Cite this article

Metrics details

Midwives encounter various difficulties while aiming to achieve excellence in providing maternity care to women with mobility disabilities. The study aimed to explore and describe midwives’ experiences of caring for women with mobility disabilities during pregnancy, labour and puerperium in Eswatini.

A qualitative, exploratory, descriptive, contextual research design with a phenomenological approach was followed. Twelve midwives working in maternal health facilities in the Hhohho and Manzini regions in Eswatini were interviewed. Purposive sampling was used to select midwives to participate in the research. In-depth phenomenological interviews were conducted, and Giorgi’s descriptive phenomenological method was used for data analysis.

Three themes emerged from the data analysis: midwives experienced physical and emotional strain in providing maternity care to women with mobility disabilities, they experienced frustration due to the lack of equipment to meet the needs of women with mobility disabilities, and they faced challenges in providing support and holistic care to women with mobility disabilities during pregnancy, labour and puerperium.

Conclusions

Midwives experienced challenges caring for women with mobility disabilities during pregnancy, labour and the puerperium in Eswatini. There is a need to develop and empower midwives with the knowledge and skill to implement guidelines and enact protocols. Moreover, equipment and infrastructure are required to facilitate support and holistic maternity care for women with mobility disabilities.

Peer Review reports

Globally, few studies have focused on midwives’ views of providing maternity care to women with mobility disabilities during pregnancy, labour and the puerperium [ 1 ]. In The Disabled World [ 2 ], the World Health Organisation (WHO) defines ‘disability’ as an umbrella term covering impairments, activity limitations, and participation restrictions. Furthermore, the WHO defines an ‘impairment’ as a problem in bodily function or structure; an ‘activity limitation’ as a difficulty encountered by an individual in executing a task or action; and ‘participation restriction’ as a problem experienced by an individual in various life situations [ 2 ]. In this study, mobility disabilities refer to an impairment in the functioning of the upper and lower extremities as experienced by women during pregnancy, labour and the puerperium.

Midwives, as frontline workers in the delivery of maternity care [ 3 ] responsible for the lives of the mother and the baby, are accountable for providing competent and holistic care for women during pregnancy, labour and puerperium. As part of healthcare provision, midwives play an important role in ensuring that every woman, including women with mobility disabilities, receives the best maternity care during pregnancy, labour and puerperium. Moridi et al. [ 4 ] state that women with mobility disabilities are entitled to feel safe, respected and well cared for by midwives, who must be sufficiently prepared to care for these women.

According to the Global Population Report, [ 5 ] more than one billion people have some form of disability. Eswatini is classified as a middle-income setting in the southern African region, measuring 17 000 square kilometres with a population of 1 093 238. Of the population, 76.2% reside in rural areas (833 472), and 23.8% (259 766) reside in urban areas [ 6 ]. The economy is largely agricultural as most industries manufacture agricultural products [ 7 ]. Of the Eswatini population, 146 554 (13%) live with disabilities, with most being women (87 258; 16%), 22,871 (14.1%) and 26,270 (14.3%) of them reside in the Hhohho and Manzini regions respectively [ 8 ]. 15% (125 545) of people with disabilities live in rural areas, and 85% of the disabled population is unemployed [ 8 ], which means most of these individuals are economically disadvantaged. Furthermore, according to the Eswatini Central Statistics Office, 8 26.5% of people with disabilities have a mobility (walking) disability, with 63.5% of these being women.

Midwives may encounter difficulties while aiming to achieve excellence in providing maternity care to women with mobility disabilities in what may be challenging circumstances [ 9 ]. The WHO [ 10 ] claims people with disabilities do not receive the health services they need and are thus likely to find healthcare providers have inadequate skills. Lawler et al. [ 11 ] argue that ineffective interactions and poor communication with women needing care, particularly among health professionals engaged in providing maternity services, limit these women’s opportunities to participate in decision-making processes during pregnancy, childbirth, and postpartum care. According to the University of Johannesburg, [ 12 ] the midwife, together with the mother, have to engage collaboratively in order to come up with opportunities to promote health while removing any challenges that could impede the achievement thereof.

Walsh-Gallagher et al. [ 13 ] postulate that healthcare professionals tend to view women with disabilities as liabilities and regard them as high risk; they often exclude them from the individualised plan of care, which leads to an increase in these women’s fears about their maternity care. These challenges frequently result in health disparities and prevent women with mobility disabilities from receiving optimal maternity care. By exploring midwives’ experiences of this phenomenon, guidelines for support can be developed to extend available knowledge on maternity care for women with mobility disabilities during pregnancy, labour and puerperium.

Study design

The aim of the study was to explore and describe midwives’ experiences of caring for women with mobility disabilities during pregnancy, labour and puerperium in the Hhohho and Manzini regions of Eswatini. A qualitative, [ 14 ] exploratory, [ 15 ] descriptive, [ 16 ] contextual [ 17 ] research design with a phenomenological approach [ 18 ] was applied for this study to gain insight and understanding of the research phenomenon [ 19 ]. The phenomenon under study was midwives’ lived experiences caring for women with mobility disabilities during pregnancy, labour and puerperium. The participants were approached face-to-face to participate in the study. The researchers followed the Consolidated Criteria for Reporting Qualitative Research (COREQ) to report on this qualitative study [ 20 ].

The setting for the study was the Hhohho and Manzini regions of Eswatini. The researcher collected data at the site where participants experienced the phenomenon, as emphasised by Yildiz, [ 21 ] within the context in which they were comfortable to be interviewed [ 22 ]. This setting included maternal health facilities in hospitals and public health units.

Population and sampling

The study’s population comprised midwives working in maternal health facilities in hospitals and public health units, that is, one referral hospital and one public health unit in the Hhohho region and two referral hospitals and one public health unit in the Manzini region of Eswatini. Purposive sampling was used to select midwives to participate in the study; [ 16 ] 12 midwives from both regions were included. The midwives were between the ages of 35 and 55, and all midwives were black in race and identified as females. The years of experience in the field ranged between 5 and 15 years. The criteria for inclusion were midwives who had provided maternity care to women with mobility disabilities during pregnancy, labour and puerperium for a period of not more than two to three years, willing to participate in the study. The sample size was determined by repetitions of key statements about the research phenomenon during data collection, termed data saturation [ 23 ]. None of the participants refused to participate in the study.

Table  1 summarises the participants’ demographic characteristics.

Data collection

In-depth phenomenological, face-to-face, individual interviews were conducted to collect data [ 17 ]. The researcher who was a Midwifery lecturer held a Master’s Degree in Maternal and Neonatal science at the time of the study requested approval from the Unit manager to seek permission from the midwives to take part in the study. The midwives were given an information letter which included objectives of the study and the reasons for conducting the study. After recruiting midwives and obtaining their written consent to participate in the study and permission to audio-record the interviews, the researcher set up appointments with them for the interviews, and the data collection process commenced. The central question posed to participants was: How was it for you to care for a woman with a mobility disability during pregnancy, labour and puerperium? A pilot of the tool was performed on the first participant who met the inclusion criteria and possessed the same characteristics as those of the study sample. The pre-testing question yielded positive results, the participant responded to the question asked and there was no need to rephrase it or further test it.

The interviews were conducted from March 2019 to July 2019 and lasted 30–45 min. The researcher conducted interviews until the data became redundant and repetitive, reflecting that saturation had been reached, in congruence with Fouché et al. [ 25 ] In addition, field notes were recorded in a notebook after each in-depth phenomenological interview. No repeat interviews were held. The researcher ensured bracketing by omitting any perceptions from her past experiences that were likely to influence her interpretation of the research findings.

Data analysis

Before data analysis commenced, data were organised in computer files after being transcribed and translated into narrative form. Data from each participant were coded and stored in the relevant file and kept in a safe place; only the researcher could access the information. Back-up copies were made of all the data, and the master copies were stored in a safe to which only the researcher had access.

Data collection and analysis occurred concurrently. The researcher was guided by Giorgi et al.’s [ 26 ] five-step method of data analysis. This entailed the researcher reading all the transcribed data and the entire ‘naïve description’ provided by the participants during the interviews. The demarcation of ‘meaning units’ within narratives followed. In addition, the researcher marked where meaning shifts occurred and transformed meaning units into descriptive expressions. The researcher laid out the general structure of midwives’ experiences. Moreover, an independent coder was provided with the raw data (after signing a confidentiality agreement) to analyse the findings. The researcher and independent coder analysed the data separately and met for a consensus discussion. Both agreed on all the units of analysis, with an inter-coder reliability of 100%.

Measures of trustworthiness

The research was informed by Guba and Lincoln’s [ 27 ] model in relation to credibility, transferability, dependability and confirmability. For credibility, the researcher ensured prolonged engagement in the field [ 28 ], peer debriefing, [ 29 ] member checking, and an external auditor was used [ 25 ]. The study was also presented at a national conference. Transferability refers to the ability to extend the findings of one’s study to comparable environments or participants, as stated by Pitney et al. [ 30 ] The researcher ensured the study’s transferability by providing a richly documented account and in-depth description of all aspects and processes of the study protocol. Data saturation also confirmed transferability [ 23 ]. Dependability is evident in a study when other researchers are able to follow the researcher’s decision trail [ 31 ]. The researcher ensured dependability by densely describing the research process in congruence with Fouché et al.’s [ 25 ] guidelines, so that other researchers can follow similar steps of the same research methodology. Confirmability occurs when the research is judged by the way in which the findings and conclusions achieve their aim and are not the result of the researcher’s prior assumptions and preconceptions [ 32 ]. The researcher ensured this by remaining true to the research process through reflexivity and not compromising the research process in any way [ 28 ]. In addition, the researcher engaged an independent coder and provided a chain of evidence of the entire research process to enable an audit. Therefore, all forms of collected data, including raw data, reflexive journals, [ 29 ] notes and transcriptions, were recorded.

Ethical clearance to conduct this study was obtained from the University of Johannesburg Faculty of Health Sciences Higher Degrees Committee (ref. no. HDC-01-50-2018), University of Johannesburg Faculty of Health Research Ethics Committee (ref. no. REC-01-82-2018), and the Eswatini National Health Research Review Board (ref. no. NHRRB982/2018). The researcher applied and adhered to the four principles to be considered when conducting research: autonomy, beneficence, non-maleficence and justice [ 33 ]. Autonomy was adhered to by affording the participants the right to choose to participate in the study and by signing a written informed consent form a week after it was given to them before the interviews commenced. Beneficence was ensured through doing good and doing no harm to participants by prioritising the participants’ interests above those of the researcher, and did not engage in any practice that jeopardised their rights. Non-maleficence was observed by eradicating any possible harmful risks in the study; the researcher ensured the safety of the participants by conducting interviews in a familiar, private environment where they felt free and safe from harm. Furthermore, justice was observed by treating all participants equally regardless of their biographical, social and economic status.

Three themes and categories emerged from the data analysis. Table  2 summarises the themes and categories of midwives’ lived experiences caring for women with mobility disabilities during pregnancy, labour and puerperium in Eswatini.

Theme 1: physical and emotional efforts required from midwives to provide maternity care to women with mobility disabilities

Category 1.1: midwives experienced that woman with mobility disabilities needed assistance getting onto the bed during labour and delivery.

According to the participants, caring for women with mobility disabilities weighed heavily on them physically as they were required to assist the women onto delivery beds, which were too high for the women to climb up on their own:

“The beds are too high, they need to be adjustable…unless you change her to another room, we only have one in the other room…but to be honest she delivered on the same high bed with the help…It’s uncomfortable even with me who is normal, how about someone who has a disability? Getting the woman onto the bed is also uncomfortable for us we end up having pain on our backs.” (M3) . “The challenge is that I couldn’t help her to climb on to the bed, because I needed someone to assist when she came for postnatal care as she was even carrying 3 babies, I didn’t know what to do…I eventually went out and asked for assistance from my colleague…” (M10) . “I believe that the equipment should accommodate the women with disability, however, ours is not accommodative to the women…there are no special delivery beds, specifically designed for them because in my opinion the beds have to be shorter so they can be able to get on to them easily…yes so that they can be able to climb on the beds” (M1) .

Category 1.2: midwives experienced challenges in manoeuvring women with mobility disabilities during labour

Midwives reported it was difficult to perform some procedures while progressing these women during labour and delivery. This situation called for some adjustment and improvisation on their part, and they were unsure if it was the right thing to do.

“Even though she was a bit uncomfortable and anxious because the leg was just straight and could not bend, I reassured her…She had to remove the artificial leg and remain with the stump. I placed her on the lithotomy position. With the other hand she had to hold on to the ankle of the normal foot, even though it was awkward and difficult to manoeuvre, she managed to deliver the baby.” (M1) . “Luckily for us, she didn’t sustain a tear and we were saved from suturing her cause we foresaw difficulties as how we could have done it as she couldn’t open her thighs well due to the disability…yes I had to get a partner to assist, since she couldn’t even open her thighs. She also couldn’t cooperate possibly because of the pain that is also more reason I asked for my colleague to assist.” (M6) . “…yes…let me make an example, in my case she had a fracture, even if the pelvis was gynaecoid, there were problems of finding the right position for her during delivery, when she had to push the baby out…” (M8) . “The one that I saw did not have one leg. She had come for her postnatal care. We assisted and her on the couch, with my colleague. Since she couldn’t keep her legs open, I asked my colleague to keep one of her legs open whilst I examined her.” (M12) .

Category 1.3: midwives experienced anxiety and the need to exercise patience when caring for women with mobility disabilities

The participants experienced an emotional and psychological burden when caring for women with mobility disabilities. They felt unqualified and foresaw difficulties that triggered anxiety, which led to them not knowing what to do and how to handle these women.

“It was during labour…the woman was limping the woman she was on crutches. The moment she came into the ward I am a human being I just felt sorry for her kutsi (as to) how is she going to take care of the baby, and the hand was somehow deformed.” (M3) . “At first its emotionally draining as an individual you cause you start sympathising…(other midwife chips in)…yes you even find yourself saying things just because you pity her, and in the process they get hurt.” (M6) . “It came as a shock and it was my first experience, it came as a shock as to how I was going to help her as even my experience was limited in that area.” (M7) . “As I was taking care of her it became necessary for me to put myself into her shoes and to bear with her considering her situation….When you see her for the first time you would pity her yet she is now used to it.” (M1) .

Theme 2: lack of equipment to meet the needs of women with mobility disabilities

Category 2.1: midwives reported a lack of special beds and infrastructure to meet the needs of women with mobility disabilities.

Midwives reported their frustration at the lack of sufficient equipment like special beds and examination tables, tailored for women with mobility disabilities. It was a challenge to provide maternity care for women without this equipment.

“I believe that the infrastructure and equipment should accommodate the women with mobility disability, however, ours is not accommodative to the women…Usually we don’t have the prenatal ward in the maternity, most women who come in the latent phase have to ambulate, or go to the waiting huts and come back when the labour pains are stronger…There are no special delivery beds, specifically designed for them because in my opinion the beds have to be shorter so they can be able to get on to them easily. We do not even have toilets meant for them.” (M1) . “I was anxious as to how was she going to push how to push cause we do not have the right beds when it was time for pushing I asked for assistance…” (M2) . “The challenge is that I couldn’t help her to climb on to the bed, because I needed someone to assist when she came for postnatal care…the beds need to be adjustable so that they are able to be pushed lower for the mother to move from wheel chair to the bed and we pull the bed up again to examine her.” (M11) .

Theme 3: challenges in providing holistic care to women with mobility disabilities during pregnancy, labour, and puerperium

Category 3.1: midwives reported a lack of guidelines and protocols in caring holistically for women with mobility disabilities.

Midwives emphasised a lack of guidelines, protocols and knowledge about caring holistically for women with mobility disabilities. This resulted in everyone making their own decisions and doing as they saw fit in caring for these women:

“I think during antenatal care they (the women with mobility disabilities) need to be prepared for labour cause for others the pain is extraordinary, apart from the pain threshold, they also face self-esteem issues, they are looked down upon…I only saw that she was disabled during assessment cause nothing was recorded on the antenatal care card.” (M2) . “I was not aware of the disability at first, I only discovered when she was pushing…she was admitted and progressed by another midwife, I only attended to her when she was pushing… there was nothing written on the nurse’s notes/ handover notes about her disability.” (M5) . “There is no normal practice for a woman with mobility disability when they come and they are in labour, I usually admit regardless of the stage of labour or dilatation…It is not a protocol, it’s a midwife’s prerogative.” (M1) . “We assess and come up with our own discretion even in terms of admitting them (women with mobility disability). Some midwives will admit them regardless of the stage of labour and disregard the protocol that women who come into labour have to ambulate if they are in the latent phase.” (M8) . “There is one that came the past 3 days she has 3 children now and we just scheduled her for c/section because we know that she has been having c/section since she started. Just from looking at the way she walked, we could tell that she couldn’t deliver normally.” (M9) .

Category 3.2: midwives experienced challenges in allowing significant others to support women with mobility disabilities during labour and delivery

Consequent to the challenges in providing holistic care to women with mobility disabilities, midwives experienced challenges in allowing significant others to support these women during labour and delivery.

“It can depend on the patients themselves, they should decide and we need to be flexible for it to happen…as you can see our labour room also has the issue of privacy…we would need to restructure cause we have beds for 5 or more women in labour room…and then bringing someone from outside could be tricky” (M6) . “Maybe…not sure though, that they can bring their relatives, but maybe, considering staffing limitation…also the issue of discrimination and privacy, they (the women with disabilities) might feel we discriminate against them because they are disabled we now treat them differently.” (M7) . “Maybe if she can (bring her relative) but that’s not necessary, because I can always ask my colleague to assist, unless there is no one…” (M12) .

Childbirth is a special experience that requires a personal connection between the midwife and the woman giving birth, characterised by successful communication and respect [ 34 ]. However, the themes identified in the study indicated that midwives experienced challenges caring for women with mobility disabilities during pregnancy, labour and puerperium based on their limited capacity and preparedness, and lack of protocols to care for these women. They also reported a lack of supportive equipment for women with mobility disabilities. This posed a challenge for them in attending to these women’s specific needs, and they did not always know how to handle the situation appropriately.

One of the themes centred on midwives’ experiences of the physical and emotional efforts required of them to provide maternity care to women with mobility disabilities. They explained women with mobility disabilities required assistance getting onto the bed during labour and delivery, and more manoeuvring was expected of them (as midwives) as they had to adjust their performance and some procedures. The midwives also reported challenges in providing holistic care to women with mobility disabilities during pregnancy, labour and puerperium. Konig-Bachmann et al. [ 35 ] reiterate that caring for women with disabilities requires a level of flexibility, adaptation beyond routine procedures, and demands a high degree of improvisation from healthcare providers to ensure high-quality care. Morrison et al. [ 36 ] also found that healthcare providers reported difficulties with equipment when providing healthcare for women with physical disabilities; particularly the beds being too high for them to access. Smeltzer et al. [ 37 ] similarly allude to the importance of educating and training clinicians to equip them with knowledge and technical skills to provide more effective care to women with physical disabilities.

The midwives also shared that labour and deliveries were further complicated by some women with mobility disabilities not being able to cooperate due to the pain they experienced; others could not change position due to their disability. In a study by Sonalkar et al., [ 38 ] healthcare providers described the gynaecologic examination as challenging to complete as it required patience and the ability to be adaptable to different methods and positioning. Similarly, Konig-Bachmann et al. [ 35 ] indicate that in order to provide high-quality care for women with disabilities, healthcare providers need to exercise strong flexibility, adapt beyond routine procedures, and engage in a high degree of improvisation. Byrnes and Hickey [ 39 ] concur with this study’s findings and state that due to mobility restrictions, it may be difficult to assess the fundal height and foetal growth in women with physical disabilities.

Some midwives reported their caregiving role was emotionally draining as they felt sorry and pitied the women with mobility disabilities; thus, they needed to show compassion and reassure them. According to Mgwili et al., [ 40 ] psychoanalytic thinkers associate pity among staff members upon first contact with a physically disabled person as being instigated by personal feelings, stimulated by the disability. The midwives in this study stated they needed to be more patient and adjust their approach to caring for these women. Tarasoff [ 41 ] and Schildberger et al. [ 42 ] reiterated that healthcare providers seemed uncomfortable with women’s disability, consequently failing to offer needed support. According to Sonalkar et al., [ 38 ] healthcare providers reported there would be less fear and concern about hurting women with disabilities if midwives had increased training. Similarly, Mitra et al. [ 43 ] mentioned that healthcare providers had a general lack of confidence in their ability to provide adequate maternity care for women with physical disabilities.

Another theme was midwives’ challenges in providing competent and quality care for women with mobility disabilities due to a lack of equipment, including special beds and examination tables to meet these women’s needs. The examination, labour and delivery beds were too high and could not be adjusted for the women to get on by themselves, or even with the assistance of a midwife. In addition, the midwives reported there was no prenatal ward or waiting huts where they could place these women during the latent phase of labour. The midwives further emphasised there were no special toilets for women with mobility disabilities, which made it hazardous and difficult for them. Mitra et al. [ 43 ] concur on the barriers to providing maternity care to women with physical disabilities presented from health professionals’ perspectives. The authors indicated that participants from their study reported inaccessible equipment, including examination tables, as a barrier, making it more difficult and time-consuming to care for women with physical disabilities. In addition, Sonalkar et al. [ 38 ] said healthcare providers shared their concern about the lack of adjustable examination tables and transfer equipment, thus presenting a barrier to equitable care for women with disabilities.

Midwives further reported a lack of guidelines and protocols. This resulted in everyone making their own decisions and doing as they saw fit in caring for these women, and, in most instances, not recording the disability at all during antenatal care and admission into labour records. They often only discovered that the woman had a mobility disability at a later stage, when they were in labour. Sonalkar et al. [ 38 ] reported that healthcare providers felt frustrated and overwhelmed by the uncertainty of whether they made the correct decisions when caring for women with physical disabilities due to the lack of guidelines forcing them to use their own judgement. Mitra et al. [ 43 ] determined that most healthcare providers reported a lack of maternity practice guidelines for women with physical disabilities. Also, healthcare providers highlighted the importance of learning about disabilities and having a better understanding of a condition, particularly if it is likely to be exacerbated during pregnancy [ 44 ]. The need to make and read the notes on these women’s antenatal care cards or reports was emphasised.

Due to the lack of clear guidelines and protocols in caring for women with mobility disabilities, the midwives reported they sometimes admitted the woman into the labour ward regardless of the stage of labour, while other midwives did not and wanted them to walk around and come back for admission once they are in the active phase of labour. Furthermore, the midwives explained they often referred these women for caesarean sections right away, regardless of whether the woman could deliver normally due to mere panic from just seeing the disability or based on a previous record of surgery. Smeltzer et al. [ 45 ] researched obstetric clinicians’ experiences and educational preparation in caring for pregnant women with physical disabilities, and they agree on the lack of knowledge among health professionals caring for women with mobility disability.

Devkota et al. [ 46 ] also agree regarding midwives’ inefficiency in providing quality care for women with mobility disabilities. They claim healthcare providers often struggle to understand women with disabilities’ needs as they are not formally trained to provide services to this population. These healthcare providers were found to be undertrained in specific skills that would equip them to provide better and more targeted services for women with disabilities.

Consequent to the challenges in providing holistic care to women with mobility disabilities during pregnancy, labour and puerperium, midwives experienced challenges in allowing significant others to support these women. They reported that as much as they needed assistance caring for these women, and as much as the women would prefer to have their family members or significant others assisting them, this is not possible due to the lack of privacy, especially in public health facilities. Walsh-Gallager et al.’s [ 13 ] study on the ambiguity of disabled women’s experiences of pregnancy, childbirth and motherhood resonate with this study’s findings. The authors reported that women with disabilities’ partners were denied access or had their visits curtailed on several occasions due to inflexible hospital visiting policies. Redshaw et al. [ 47 ] reiterated the same in their study; disabled women were less likely to say their companion or partner was welcome to visit, let alone provide any form of assistance. In addition, a study by Bassoumah and Mohammad [ 48 ] reported that women with disabilities were denied their spouses’ support while receiving maternity care. Byrnes and Hickey [ 39 ] also concur that every effort should be made to allow women with disabilities who are in labour to receive support from significant others, and they should be active partners in the labour process.

Limitations

The study was limited to two of the four regions of Eswatini, namely Hhohho and Manzini; hence, the results could not be generalised for the whole country. The study also only focused on mobility disabilities due to time constraints and limited funds. Future research could be conducted to cover all other forms of disabilities.

This study focused on midwives’ lived experiences caring for women with mobility disabilities during pregnancy, labour and puerperium in Eswatini. In-depth phenomenological interviews were conducted, the findings were analysed, and themes were established. The findings illustrate that midwives experienced challenges caring for women with mobility disabilities during pregnancy, labour and puerperium in Eswatini. There is a need to develop and implement guidelines to empower midwives with knowledge and skill to provide support and holistic maternity care, and enact protocols. They should also have access to appropriate equipment and infrastructure specifically tailored towards promoting optimal health for women with mobility disabilities.

Data availability

The data analysed is available from the corresponding author upon reasonable request.

González-Timoneda A, Hernández Hernández V, Pardo Moya S, Alfaro Blazquez R. Experiences and attitudes of midwives during the birth of a pregnant woman with COVID-19 infection: a qualitative study. Women Birth. 2021;34(5):467.

Article   Google Scholar  

Disabled World. Definitions of disability [home page on the internet]. C2009 [updated 2021; cited 2023 July 26]. Available from: https://disabled-world.com/definitions/disability-definitions.php .

Aune I, Tysland T, Vollheim SA. Norwegian midwives’ experiences of relational continuity of midwifery care in the primary health care service: a qualitative descriptive study. Nordic J Nurs Res. 2021;4(1):5–13.

Moridi M, Pazandeh F, Hajian S, Potrata B. Midwives’ perspectives of respectful maternity care during childbirth: a qualitative study. PLoS ONE. 2020;15(3):1–12. https://doi.org/10.1371/journal.pone.0229941 .

Article   CAS   Google Scholar  

United Nations. Background: International Day of Persons with Disabilities. [homepage on the internet]. c2022 [updated 2022 December 1; cited 2023 July 20]. Available from: https://un.org/en/observances/day-of-persons-with-disabilities/background .

Central Statistics Office. Population and housing census: 2017. Volume 3. Mbabane: Swaziland Government Printing Office; 2019a.

Google Scholar  

Central Statistics Office. National accounts estimates. Mbabane: Swaziland Government Printing Office; 2018.

Central Statistics Office. Population and housing census: 2017. Volume 6. Mbabane: Swaziland Government Printing Office; 2019b.

Magqadiyane S. Experiences of midwives for caring un-booked pregnant mothers in a maternity unit at a district hospital in the Eastern Cape Province. Advances in reproductive sciences [serial online]. 2020. [cited 2021 August 5];8:186–200. https://doi.org/10.4236/arsci.2020.84016 .

World Health Organisation (WHO). Global report on health equity for persons with disabilities. [homepage on the internet]. c2022 [updated 2022 December 2; cited 2023 July 20]. Available from: https://who.int/health-topics/disability#tab=tab_1 .

Lawler D, Lalor J, Begley C. Access to maternity services for women with physical disability: a systematic review of literature. Int J Childbirth. 2013;3(4):203–17.

University of Johannesburg. Department of nursing paradigm. Johannesburg: University of Johannesburg;2017.

Walsh-Gallagher D, McConkey R, Sinclair M, Clarke R. Normalising birth for women with a disability: the challenges facing practitioners. Midwifery. 2013;29:294–9.

Article   PubMed   Google Scholar  

Silverman D, editor. Qualitative research. 5th ed. Los Angeles: SAGE; 2021.

Nassaji H. Good qualitative research. Language Teaching Research [serial online]. 2020. [cited 2021 August 6];24(4):427–431. Available from: https://journals.sagepub.com/doi/pdf/10.1177/1362168820941288 .

Doyle L, McCabe C, Keogh B, Brady A, McCann M. An overview of the qualitative design within nursing research. Journal of Research in Nursing [serial online]. 2020. [cited 2021 August 6];25(5):444–446. Available from: https://journals.sagepub.com/doi/pdf/10.1177/1744987119880234 .

Hennink M, Hutter I, Bailey A. Qualitative research methods. 2nd ed. London: SAGE; 2020.

Frechette J, Bitzas V, Aubry M, Kilpatrick K, Lavoie-Tremblay M. Capturing lived experience: methodological considerations for interpretive phenomenological inquiry. Int J Qual Meth. 2022;19:2–11.

Flick U. The SAGE handbook of qualitative research design. London: SAGE; 2022.

Book   Google Scholar  

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042 .

Yildiz A. A discussion on accurate and effective data collection for qualitative research. J Curr Researches Educational Stud. 2020;10(2):17–24.

Papakitsou V. Qualitative research: narrative approaches in sciences. Dialogues Clin Neurosci Mental Health. 2020;3(1):63–70.

Johnson JL, Adkins D, Chauvin S. Qualitative research in pharmacy education: a review of the quality indicators of rigor in qualitative research. Am J Pharm Educ. 2020;84(1):138–46.

Magagula T. The guidelines of maternity care of women with mobility disabilities in the Hhohho and Manzini regions: Eswatini [unpublished thesis]. University of Johannesburg, Johannesburg; 2021.

Fouché CB, Strydom H, Roestenburg WJH, editors. Research at grassroots for social sciences and human services professions. 5th ed. Pretoria: Van Schaik; 2021.

Giorgi A, Giorgi B, Morley J. The descriptive phenomenological psychological method. In: The SAGE handbook of qualitative research in psychology. 2nd edition. Los Angeles: SAGE; 2017.

Guba EG, Lincoln YS. Fourth generation evaluation. Newbury Park, CA: SAGE; 1989.

Rose J, Johnson W. Contextualising reliability and validity in qualitative research: toward more rigorous and trustworthy qualitative social science in leisure research. J Leisure Res. 2020;1:10–3. https://doi.org/10.1080/00222216.2020.1722042 .

Creswell JW, Creswell JD. Research Design: qualitative, quantitative and mixed methods approaches. 5th Ed. California: SAGE; 2018.

Pitney WA, Parker J, Singe SM, Potteiger K. Qualitative research in health professions. Thorofare: SLACK Incorporated; 2020.

Leavy P, editor. The Oxford handbook of qualitative research. New York: Oxford University Press; 2020.

Kyngäs H, Mikkonen K, Kääriäinen M, editors. The application of content analysis in nursing science research. 2020. [cited 2022 April 27]. Available from: https://dl1tarjomac.ir/nursing-ebooks/TPC202203.pdf .

Dhai A, McQuoid-Mason DJ. Bioethics, human rights and health law: principles and practice. Cape Town: Juta; 2020.

Hallam J, Howard C, Locke A, Thomas M. Communicating choice: an exploration of mothers’ experiences of birth. J Reprod Infant Psyc. 2016;34(2):175–84.

König-Bachmann M, Zenzmaier C, Schildberger B. Health professionals’ views on maternity care for women with physical disabilities: a qualitative study. BMC Health Serv Res. 2019;19(551):1–11.

Morrison J, Basnet M, Buthathoki B, et al. Disabled women’s maternal and newborn health care in rural Nepal: a qualitative study. Midwifery. 2014;30:1132–9.

Article   PubMed   PubMed Central   Google Scholar  

Smeltzer S, Wint A, Ecker J, Iezzoni L. Labor, delivery, and anaesthesia experiences of women with physical disability. Birth. 2017;44(4):315–24.

Sonalkar S, Chavez V, McClusky J, Hunter TA, Mollen CJ. Gynaecologic care for women with physical disabilities: a qualitative study of patients and providers. Women Health Iss. 2020;30(2):136–41.

Byrnes L, Hickey M. Perinatal care for women with disabilities: clinical considerations. J Nurse Practitioners. 2016;12(8):506–7.

Mgwili VN, Watermayer B. Physically disabled women and discrimination in reproductive health care: Psychoanalytic reflections. In: Disability and Social Change: A South African agenda [serial online]. 2006. [cited 2020 June 01]. Available from: https://www.hsrcpress.ac.za .

Tarasoff LA. Improving perinatal care for women with physical disabilities [Abstract]. J Obstet Gynaecol Can. 2016;38(5):501.

Schildberger B, Zenzmaier C, König-Bachmann M. Experiences of Austrian mothers with mobility or sensory impairments during pregnancy, childbirth and the puerperium: a qualitative study. BMC Pregnancy Childb. 2017;17(201):1–11.

Mitra M, Akobirshoev I, Moring N, et al. Access to and satisfaction with prenatal care among pregnant women with physical disabilities: findings from a national survey. J Womens Health. 2017;26(12):1356–63.

Hall J, Hundley V, Collins B, Ireland J. Dignity and respect during pregnancy and childbirth: a survey of experience of disabled women. BMC Pregnancy Childb. 2018;18(328):1–13.

Smeltzer S, Mitra M, Long-Bellil L, Iezzoni L, Smith L. Obstetric clinicians’ experiences and educational preparation for caring for pregnant women with physical disabilities: a qualitative study. Disabil Health J. 2018;11(1):8–13.

Devkota HR, Murray EA, Kett M, Groce N. Health care provider’s attitude towards disability and experience of women with disabilities in the use of maternal healthcare service in rural Nepal. Reprod Health. 2017;14(79):1–14.

Redshaw M, Malouf R, Gao H, Gray R. Women with disability: the experience of maternity care during pregnancy, labour and birth and the postnatal period. BMC Pregnancy Childb. 2013;13(174):1–14.

Bassoumah B, Mohammed A. The socio-cultural challenges to maternal and neonatal care: the views of women with disabilities receiving maternity care in the Chereponi district of Northern Ghana. Sci Afr. 2020;7:1–10.

Download references

Acknowledgements

The authors would like to acknowledge the midwives in the Hhohho and Manzini regions of Eswatini who participated in the study and provided their own experiences of providing maternity care to women with mobility disabilities during pregnancy, labour and puerperium.

The research received funding from the University of Johannesburg Postgraduate Supervisor-linked Bursary.

Author information

Authors and affiliations.

Health Sciences, University of Johannesburg, Johannesburg, South Africa

Annie M. Temane & Anna G. W. Nolte

Mother and Child Nursing, University of Eswatini, Kwaluseni, Eswatini

Fortunate N. Magagula

You can also search for this author in PubMed   Google Scholar

Contributions

F.N.M conducted the research and wrote the manuscript. A.M.T supervised, reviewed, and finalised the manuscript. A.G.W.N co-supervised the study and edited the manuscript for final submission.

Corresponding author

Correspondence to Annie M. Temane .

Ethics declarations

Ethical approval and consent to participate.

Ethical clearance to conduct this study was obtained from the University of Johannesburg Faculty of Health Sciences Higher Degrees Committee (ref. no. HDC-01-50-2018), University of Johannesburg Faculty of Health Research Ethics Committee (ref. no. REC-01-82-2018) and the Eswatini National Health Research Review Board (ref. no. NHRRB982/2018). Participation in this study was voluntary, and informed consent was obtained from participants before the interviews commenced.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Temane, A.M., Magagula, F.N. & Nolte, A.G.W. Midwives’ lived experiences of caring for women with mobility disabilities during pregnancy, labour and puerperium in Eswatini: a qualitative study. BMC Women's Health 24 , 207 (2024). https://doi.org/10.1186/s12905-024-03032-z

Download citation

Received : 18 August 2023

Accepted : 18 March 2024

Published : 01 April 2024

DOI : https://doi.org/10.1186/s12905-024-03032-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Experiences
• Maternity care
• Women with mobility disabilities
• Labour and the puerperium

BMC Women's Health

ISSN: 1472-6874

• Submission enquiries: [email protected]
• General enquiries: [email protected]

This paper is in the following e-collection/theme issue:

Published on 29.3.2024 in Vol 11 (2024)

Developing and Implementing a Web-Based Branching Logic Survey to Support Psychiatric Crisis Evaluations of Individuals With Developmental Disabilities: Qualitative Study and Evaluation of Validity

Authors of this article:

Original Paper

• Deborah A Bilder 1 , MD   ; 
• Mariah Mthembu 2 , MD   ; 
• Whitney Worsham 2 , MD   ; 
• Patricia Aguayo 1 , MPH, MD   ; 
• Jacob R Knight 2 , BS   ; 
• Steven W Deng 2 , HBS   ; 
• Tejinder P Singh 2 , BDS, MDS, MPH   ; 
• John Davis 3 , PhD  

1 University of Utah Huntsman Mental Health Institute, Salt Lake City, UT, United States

2 University of Utah School of Medicine, Salt Lake City, UT, United States

3 University of Utah, Salt Lake City, UT, United States

Corresponding Author:

Deborah A Bilder, MD

University of Utah Huntsman Mental Health Institute

383 Colorow Drive, Room 360

Salt Lake City, UT, 84108

United States

Phone: 1 801 410 0852

Email: [email protected]

Background: Individuals with developmental disabilities (DD) experience increased rates of emotional and behavioral crises that necessitate assessment and intervention. Psychiatric disorders can contribute to crises; however, screening measures developed for the general population are inadequate for those with DD. Medical conditions can exacerbate crises and merit evaluation. Screening tools using checklist formats, even when designed for DD, are too limited in depth and scope for crisis assessments. The Sources of Distress survey implements a web-based branching logic format to screen for common psychiatric and medical conditions experienced by individuals with DD by querying caregiver knowledge and observations.

Objective: This paper aims to (1) describe the initial survey development, (2) report on focus group and expert review processes and findings, and (3) present results from the survey’s clinical implementation and evaluation of validity.

Methods: Sources of Distress was reviewed by focus groups and clinical experts; this feedback informed survey revisions. The survey was subsequently implemented in clinical settings to augment providers’ psychiatric and medical history taking. Informal and formal consults followed the completion of Sources of Distress for a subset of individuals. A records review was performed to identify working diagnoses established during these consults.

Results: Focus group members (n=17) expressed positive feedback overall about the survey’s content and provided specific recommendations to add categories and items. The survey was completed for 231 individuals with DD in the clinical setting (n=161, 69.7% men and boys; mean age 17.7, SD 10.3; range 2-65 years). Consults were performed for 149 individuals (n=102, 68.5% men and boys; mean age 18.9, SD 10.9 years), generating working diagnoses to compare survey screening results. Sources of Distress accuracy rates were 91% (95% CI 85%-95%) for posttraumatic stress disorder, 87% (95% CI 81%-92%) for anxiety, 87% (95% CI 81%-92%) for episodic expansive mood and bipolar disorder, 82% (95% CI 75%-87%) for psychotic disorder, 79% (95% CI 71%-85%) for unipolar depression, and 76% (95% CI 69%-82%) for attention-deficit/hyperactivity disorder. While no specific survey items or screening algorithm existed for unspecified mood disorder and disruptive mood dysregulation disorder, these conditions were caregiver-reported and working diagnoses for 11.7% (27/231) and 16.8% (25/149) of individuals, respectively.

Conclusions: Caregivers described Sources of Distress as an acceptable tool for sharing their knowledge and insights about individuals with DD who present in crisis. As a screening tool, this survey demonstrates good accuracy. However, better differentiation among mood disorders is needed, including the addition of items and screening algorithm for unspecified mood disorder and disruptive mood dysregulation disorder. Additional validation efforts are necessary to include a more geographically diverse population and reevaluate mood disorder differentiation. Future study is merited to investigate the survey’s impact on the psychiatric and medical management of distress in individuals with DD.

Introduction

Individuals with developmental disabilities (DD) such as autism and intellectual disability (ID) experience mental health crises more frequently than the general population [ 1 , 2 ]. A broad range of psychiatric and medical conditions can contribute to the agitation, aggression, and self-injury that often characterize these crises [ 3 - 10 ]. Rates of anxiety (20%-77%), depression (10%-20%), expansive mood and bipolar disorder (5%-11%), and psychosis (5%-10%) among individuals with autism exceed those in neurotypical individuals [ 11 - 19 ]. Elevated rates of psychiatric disorders have also been identified in individuals with ID, notably for unspecified psychosis (4.8%), schizophrenia (3.9%), and bipolar disorder (8%) [ 20 - 22 ]. A history of trauma or abuse should also be considered in individuals with DD presenting in crisis [ 23 ].

When psychiatric and medical conditions are recognized as factors contributing to a person’s mental health crisis, clear long-term treatment targets emerge. Nevertheless, for those with DD, co-occurring medical and psychiatric conditions are often unrecognized, leaving them vulnerable to experiencing diagnostic overshadowing. Diagnostic overshadowing occurs when disruptive behaviors in individuals with DD are attributed to their disability without consideration of other potential medical or psychiatric conditions that could contribute to their behavioral presentation [ 24 ].

Self-, parent-, and caregiver-report mental health questionnaires provide an efficient means of screening for common psychiatric conditions in the neurotypical population. However, for those with DD, self-report questionnaires may be impeded by communication deficits or a limited capacity to reflect on internal experiences. Parent- and caregiver-report questionnaires normed in typically developing children may also provide inadequate mental health screening for those with ID because they often include items that are inapplicable to children with minimal language ability, exclude severe conditions that disproportionately affect children with DD (eg, mania and psychosis), and overlook the individualized manner in which psychiatric symptoms manifest in this population [ 20 , 25 - 27 ].

The American Psychiatric Association and the National Association for the Dually Diagnosed published the Diagnostic Manual–Intellectual Disability in 2007, and subsequently, in 2016, the second edition ( Diagnostic Manual–Intellectual Disability–Second Edition ; DM-ID-2 ) [ 28 , 29 ]. These texts adapt the Diagnostic and Statistical Manual of Mental Disorders criteria to reflect their presentation in individuals with ID. The Psychopathology Instrument for Mentally Retarded Adults and the Psychiatric Assessment Schedule for Adults with Developmental Disabilities (PAS-ADD) operationalize adapted diagnostic criteria into structured interviews to provide a framework through which to identify psychiatric conditions in this population [ 30 , 31 ]. These interviews are quite lengthy and require training to administer. Even as an abbreviated semistructured interview, the Mini PAS-ADD Clinical Interview takes approximately 45 minutes to complete [ 32 ]. Existing parent- and caregiver-report psychiatric screening tools for individuals with ID create a more efficient and practical means of collecting information [ 33 - 36 ]; yet, the checklist format of parent- and caregiver-report questionnaires limits depth and scope, both of which are necessary when evaluating crises in a population with complex medical and mental health needs. In addition, there is a great need for the inclusion of items that query symptoms of common medical conditions (eg, epilepsy, gastrointestinal disorders, and poor dentition) that manifest with agitation and aggression and occur more frequently in individuals with DD [ 3 , 37 , 38 ].

Sources of Distress is a survey developed for parents and caregivers (hereinafter collectively referred to as caregivers ) that uses a web-based branching logic format to screen for mental health and medical conditions among individuals with DD who present in crisis. This tool informs the care of individuals experiencing distress and is intended for use when the severity or persistence of disruptive behavior prompts the consideration of medication intervention. Screening information endorsed by caregivers is organized into relevant psychiatric and medical categories within a report. This report ( Multimedia Appendix 1 [ 39 ]) is developed for the caregiver and can subsequently facilitate their shared decision-making process with health care providers as specific underlying conditions are evaluated. Sources of Distress aims to minimize diagnostic overshadowing and optimize the ability of the caregiver and the provider to recognize the presence of psychiatric and medical conditions that merit targeted intervention. The web-based branching logic format is adaptive in nature—optimizing caregiver and health care provider convenience and efficiency and minimizing caregiver burden for survey completion [ 40 ].

This paper aims to (1) describe the initial development of Sources of Distress; (2) report on the findings from focus group evaluations and expert reviews and indicate how this feedback shaped the subsequent version of the survey; and (3) present the results from the evaluation of validity for Sources of Distress after its implementation in the clinical setting. The Methods and Results sections are divided into 3 subsections (apart from the Ethical Considerations section in Methods ) corresponding to the development, initial evaluation, and clinical implementation phases of Sources of Distress.

Ethical Considerations

The University of Utah Institutional Review Board approved focus group activities for Sources of Distress content validation (IRB_00111975). Focus group participants provided informed consent and received compensation for their time in the form of an Amazon gift card worth US $50. The University of Utah Institutional Review Board approved with a waiver of consent for the retrospective records review, data collection, and subsequent deidentified data analysis for individuals for whom Sources of Distress was completed as part of their clinical care (IRB_00170868).

Early Survey Development

Funding for the development of Sources of Distress was provided by the Autism Council of Utah based in Murray, Utah, United States [ 41 ]. The development team comprised a triple board physician (pediatrics, general psychiatry, and child and adolescent psychiatry), an educational psychologist, a medical student, and a business consultant grandparent of a child with autism and ID. In the initial development phase, Sources of Distress was built in Qualtrics (Qualtrics International Inc) using a branching logic format to approximate the history-taking component of a DD psychiatric evaluation. This evaluation queries psychiatric symptom clusters, physical complaints, and psychiatric medical history to support the development of a diagnostic impression for which treatment recommendations could be made.

Multiple expert opinion sources were reviewed to identify pertinent screening categories and corresponding items to include in Sources of Distress. The expert sources included published literature, the DM-ID-2 , the Mini PAS-ADD Clinical Interview, and the screening interview for the Kiddie Schedule for Affective Disorders and Schizophrenia–Present and Lifetime (a semistructured psychiatric diagnostic interview for children and adolescents) [ 28 , 32 , 42 ]. As Sources of Distress is intended for use in the context of distress, the presence of at least 1 manifestation of a behavioral or emotional crisis must be endorsed to initiate survey questions.

Initial Survey Evaluation

Focus group evaluation.

In 2018 and early 2019, focus group participants were recruited from (1) a university-based outpatient program that provides medical and psychiatric care for individuals with DD across the lifespan and (2) the Autism Council of Utah (a community stakeholder organization for individuals and families affected by autism). Six focus groups were conducted that consisted collectively of parents (6/17, 35%), professional caregivers (6/17, 35%), and adults with both DD and the ability to provide verbal feedback (5/17, 29%). Participants completed Sources of Distress before attending the focus group and reported on specific items, missing items, item wording, and attribution of items to corresponding conditions. Interviews and discussions were transcribed and analyzed following the framework analysis of Ritchie and Spencer [ 43 ]. Inductive reasoning and the constant comparative method put forth by Strauss and Corbin [ 44 ] were used to compare statements by parents, professional caregivers, and individuals with disability within and across focus groups.

Expert Review Evaluation

Revisions were made to Sources of Distress based on focus group feedback. Experts reviewed the revised survey version, and additional changes were made. The experts included a pediatrician and 2 child psychiatrists, all with national recognition for their clinical and research work in DD.

Clinical Implementation

Sources of Distress was implemented in various clinical settings to augment the clinical history-taking process—outpatient (primary care, neurology, developmental pediatrics, and psychiatry), emergency department, psychiatric inpatient, and residential care. Caregivers were given a link to the survey when their health care provider identified the need for expert support in managing severe agitation and aggression. All caregivers (231/231, 100%) completed the survey outside of the clinical setting. An informal or formal consult followed survey completion for a subset of individuals. In August 2020, the survey was transitioned from the Qualtrics platform to the REDCap (Research Electronic Data Capture; Vanderbilt University) platform to automate the Sources of Distress report generation using the custom template engine [ 45 ]. This external REDCap module was developed and has been maintained by the Integrated Research Informatics Services of British Columbia Children’s Hospital Research Institute [ 46 ].

Survey Data Collection

Sources of Distress responses were collected from its first use in a clinical setting from February 2019 through June 2022. The following information was obtained: respondent type, individual characteristics, caregiver-reported diagnoses, current medications, distress manifestations, psychiatric symptoms, and medical symptoms, conditions, or concerns. When multiple caregivers reported on the same individual, responses were used from the caregiver closest to where the individual lived (eg, parent for a child living at home and professional caregiver for an individual living in a residential setting). Psychotropic medications were organized within the following mutually exclusive categories: antipsychotics, antidepressants, non-antidepressant anxiolytics, anticonvulsants, lithium, alpha-2 agonists, stimulants, and atomoxetine.

A medical decision-making support consultation took place after survey completion as either an informal or a formal consult for a subset of individuals. This consult was conducted by a clinical team led by the triple board physician member of the survey’s development team. The consult team used DM-ID-2 criteria as the basis for establishing psychiatric diagnoses. At a minimum (as an informal consult), the consult involved a discussion between a DD clinical expert and the referring provider. This discussion resulted in a collective determination of working diagnoses and treatment plan. A formal consult included the additional components of medical records review, caregiver interview, and direct participant evaluation. Psychiatric diagnoses that were not reported in the survey but discussed by the provider or documented in the medical record were included among preexisting diagnoses.

Working diagnoses were abstracted from formal and informal consult documentation and served as the standard to define true case status.

Mood Disorder Classification

The presence of a mood disorder among preexisting and working diagnoses was classified into mutually exclusive categories such that there was no overlap among individuals across mood disorder categories to allow for direct comparisons across preexisting diagnoses, survey screening status results, and working diagnoses. The following mood disorder classification hierarchy was used from highest to lowest: (1) episodic expansive mood, hypomania, mania, and bipolar disorder, hereafter collectively referred to as bipolar disorder , (2) disruptive mood dysregulation disorder (DMDD) and unspecified mood disorder, and (3) unipolar depression. If an individual had a diagnosis of bipolar disorder, regardless of what other mood disorder diagnosis was reported or identified, their mood disorder classification would be bipolar disorder. An individual was only classified with unipolar depression if (1) they had a depression diagnosis and (2) they had no other mood disorder diagnosis.

Statistical Analyses and Evaluation of Validity

Descriptive statistics and chi-square tests were conducted in SPSS (version 28.0; IBM Corp) with an α of .05 selected to assess statistical significance. Differences between surveys with an accompanying consult and those without were measured. Positive predictive value (PPV), negative predictive value (NPV), and accuracy rates were calculated for (1) preexisting diagnoses and (2) survey screening results with working diagnoses used as the determinant of true case status. We calculated 95% CIs for the binomial distribution of accuracy rates.

Table 1 lists the modules and corresponding items initially selected as the categories, characteristics, and symptoms to be queried by Sources of Distress. The initial version of the survey included scoring algorithms to determine positive screen status for the following conditions: anxiety, unipolar depression, bipolar disorder, psychosis, and attention-deficit/hyperactivity disorder (ADHD).

a ADHD: attention-deficit/hyperactivity disorder.

b Module added in response to focus group feedback.

c N/A: not applicable.

d IUD: intrauterine device.

Focus Group Feedback

During the focus groups, 3 main themes emerged in this analysis.

• Theme A: respondents gave overall positive feedback regarding existing content and specific feedback regarding areas where there was room to expand content. Table 1 describes the modules and items added in response to this feedback. Notably, a posttraumatic stress disorder (PTSD) module was added along with a PTSD scoring algorithm to determine positive screen status.
• Theme B: most of the respondents (15/17, 88%) agreed that the symptoms queried matched their understanding of the psychiatric and medical conditions to which they are attributed.
• Theme C: all participant groups reported positive acceptability of the branching logic format and time required to complete the measure.

Expert Review

Overall, the expert review supported the Sources of Distress categories and respective items attributed to each condition. One expert recommended adding items that query gender and replacing sex as the basis for pronoun selection within the tool and its report. This expert also suggested that the report include screening results for each psychiatric condition. The former recommendations were implemented when Sources of Distress was transitioned to the REDCap platform. The latter recommendation was deferred until after screening algorithms are validated in a clinical setting.

Sample Characteristics

Surveys (N=264) were completed by parents or guardians (n=200, 75.8%), professional caregivers (n=43, 16.3%), and other caregivers (n=21, 8%) of 231 individuals (n=161, 69.7% men and boys; n=69, 29.9% women and girls; and n=1, 0.4% other; mean age 17.7, SD 10.3; range 2-65 years). Informal (n=62, 41.6%) and formal (n=87, 58.4%) consults were performed for 149 individuals collectively. Table 2 presents sample characteristics, the manifestations of distress, and a comparison between individuals with a consult and those without.

a Includes informal and formal consults.

b One participant reported other as gender: no participants reported non-binary as gender.

c When multiple caregivers completed Sources of Distress, the report from the caregiver with whom the participant spends the most time was used in this table.

d ID: intellectual disability.

e Genetic syndrome includes some individuals who also populate the autism or ID categories.

Preexisting Psychiatric Diagnoses

The presence of at least 1 preexisting psychiatric diagnosis was reported in 65.4% (151/231) of the individuals. Individuals who received a consult compared to those without a consult were more likely to have a caregiver-reported history of psychotic disorder (14/149, 9.4% vs 1/82, 1%; P =.02; Table 3 ).

b Medical conditions perceived by the caregiver as contributing to the current presentation of distress.

c Analysis for menstrual concerns restricted to female patients aged >12.

d Unipolar depression, unspecified mood disorder and disruptive mood dysregulation disorder, and episodic expansive mood and bipolar disorder are mutually exclusive categories.

e DMDD: disruptive mood dysregulation disorder.

f Preexisting diagnosis of obsessive-compulsive disorder is included within the anxiety disorder category.

g PTSD: posttraumatic stress disorder.

h ADHD: attention-deficit/hyperactivity disorder.

Caregiver-Reported Medical Conditions

Table 3 describes medical conditions reported by caregivers. Caregivers of 73.2% (169/231) of the individuals identified at least 1 physical concern that they perceived as contributing to distress. The most common conditions were gastrointestinal concerns (119/231, 51.5%), menstrual concerns (21/48, 44% of female patients aged >12 y), seasonal allergies (47/231, 20.3%), and seizures (45/231, 19.5%).

Psychiatric Screening Results

Table 3 lists the frequency of positive psychiatric screening results. All but 2% (5/231) of the individuals screened positive for a psychiatric condition, with a mean of 2.8 (SD 1.1; range 0-5) conditions per individual. Of those who were classified as having bipolar disorder, 89% (78/88) screened positive for a recent depressive episode. Positive screen status for psychiatric conditions were similar between those with a consult and those without, except in the case of psychosis (52/149, 34.9% vs 15/82, 18%; P =.008).

Psychotropic Medication Use

Table 4 reports on the frequency of medication use. Most of the individuals (194/231, 84%) were taking psychotropic medication, and the majority were receiving antipsychotics (142/231, 61.5%) and antidepressants (129/231, 55.8%).

a Selective serotonin reuptake inhibitors, duloxetine, tricyclics, mirtazapine, and trazodone were included exclusively within the antidepressant category.

b Benzodiazapines, buspirone, hydroxyzine, beta-blockers, and prazosin were included exclusively within the anxiolytic category.

c Anticonvulsant medication use in the absence of a reported seizure history.

Working Psychiatric Diagnoses

Of the 149 individuals who received a consult, 148 (99.3%) were diagnosed with at least 1 psychiatric condition with a mean of 2.7 (SD 1.0; range 0-5) diagnoses per individual. The conditions identified were anxiety (129/149, 86.6%), ADHD (84/149, 56.4%), bipolar disorder (67/149, 45%), unipolar depression (33/149, 22.1%), PTSD (35/149, 23.5%), and psychosis (31/149, 20.8%). Furthermore, 25 (16.8%) of the 149 individuals were diagnosed with either unspecified mood disorder or DMDD. Nearly all individuals identified with psychosis (29/31, 94%) had a co-occurring mood disorder diagnosis: bipolar disorder (22/31, 71%), unipolar depression (5/31, 16%), and unspecified mood disorder or DMDD (2/31, 6%).

Evaluation of Validity

Sources of Distress accuracy rates ranged from 76% (95% CI 69%-82%) for ADHD to 91% (95% CI 85%-95%) for PTSD and exceeded those of preexisting diagnoses, except in the case of psychosis, for which the accuracy rates were equivocal (82%, 95% CI 75%-87%; Table 5 ). The survey demonstrated higher NPVs (81%-98%) than PPVs (51%-78%) for all conditions, with the exceptions of anxiety (53% and 92%, respectively) and episodic expansive mood bipolar disorder (85% and 90%, respectively). Low PPVs were notable for depression (51%) and psychosis (54%).

a Preexisting diagnoses included caregiver-reported diagnoses in Sources of Distress and diagnoses in the medical record before survey completion.

e Depression, episodic expansive mood and bipolar disorder, and disruptive mood dysregulation disorder and unspecified mood disorder are mutually exclusive categories. There is no Sources of Distress screening algorithm for disruptive mood dysregulation disorder or unspecified mood disorder.

f Preexisting and working diagnoses included schizoaffective disorder when hypomanic, manic, or mixed episode was specified.

g DMDD: disruptive mood dysregulation disorder.

h N/A: not applicable.

i Preexisting and working diagnoses of anxiety disorder and obsessive-compulsive disorder are combined to coincide with anxiety disorder screening status.

j Preexisting and working diagnoses were schizophrenia, schizoaffective disorder, unspecified psychotic disorder, and psychotic features associated with a mood disorder.

Exploration of Mood Disorder Categories

Figure 1 demonstrates the distribution of mood disorder diagnoses among individuals based on (1) preexisting mood disorder diagnosis and (2) Sources of Distress mood disorder screening status. The majority of the individuals (18/25, 72%) who received a working diagnosis of unspecified mood disorder and DMDD screened positive for either unipolar depression or bipolar disorder.

Principal Findings

The focus group feedback indicates that Sources of Distress provides an acceptable means for caregivers to share their knowledge and insights about individuals with DD who present in crisis. As a screening tool, this survey demonstrates good accuracy, although additional work is needed to differentiate among mood disorders. The purpose of this survey is to screen individuals with DD for mental health and common medical concerns in health care settings when they present in crisis. By querying what symptom clusters and physical conditions coincide with their patient’s crisis, providers can direct their evaluation toward specific psychiatric and medical conditions that have established treatment protocols in the general population. This approach aims to reduce diagnostic overshadowing and improve medical decision-making surrounding the management of agitation and aggression in individuals with DD. Focus group participants validated the survey content and provided recommendations that prompted the inclusion of additional modules and items. Despite its length (ie, 15-20 min), participants reported positive acceptability of the survey’s format and duration. This feedback may reflect the convenience of completing a web-based survey at home versus in the medical setting and highlights caregivers’ motivation toward understanding potential factors contributing to the person’s distress. After incorporating caregiver recommendations, Sources of Distress content was also reviewed and supported by clinical and research experts.

Caregivers of most of the individuals (200/231, 86.6%) identified agitation as a presenting concern. The Food and Drug Administration has approved short-term antipsychotic medication for treating irritability in individuals with autism [ 47 ]; 61.5% (142/231) of the individuals were taking antipsychotics at the time of presenting in crisis. This frequency exceeds previously reported estimates of antipsychotic use in the population with DD (ie, 10%-48%) and reflects the high acuity and potentially treatment-resistant nature of individuals for whom the survey was completed [ 48 , 49 ]. This study group’s acuity is further supported by the high frequency in which severe mental health conditions were diagnosed in those receiving a consultation (eg, bipolar disorder and psychosis).

Anxiety was the most common condition to screen positive (201/231, 87%) and be established as a working diagnosis (129/149, 86.6%). These rates exceeded measured anxiety prevalence rates in the population with DD (ie, 20%-77%), indicating a higher propensity toward experiencing anxiety among those presenting in crisis [ 12 , 15 , 16 ]. As a precipitant of distress, prior studies have identified aggression, disruptive behavior, sleep disturbance, and self-injurious behavior as symptoms of anxiety among individuals with DD [ 4 , 8 , 50 ]. To reduce overclassification among individuals whose autism core features overlap with some anxiety symptoms [ 51 ], the Sources of Distress anxiety scoring algorithm was set at a higher threshold than the generalized anxiety disorder criteria described in DM-ID-2 . The survey’s low NPV (53%) and high PPV (92%) for anxiety likely reflect this adaption.

Sources of Distress captured well the presence of a mood disturbance; however, the type of mood disorder was not. Study results report a diagnosis frequency of 16.8% (25/149) for unspecified mood disorder and DMDD and indicate the need to add items and a screening algorithm for this condition. The low PPV (51%) for depression primarily resulted from individuals screening positive for depression who were subsequently diagnosed with unspecified mood disorder and DMDD. The DM-ID-2 , survey data, and records review will inform new items and algorithm development as well as revisions for the depression screening algorithm. In the interim, the Sources of Distress report will replace the “depression” category label with “depression and unspecified mood disorder” to broaden the range of conditions which it currently captures.

Caregivers of the majority of the individuals (169/231, 73.2%) identified at least 1 physical concern that they perceived as contributing to distress. As agitation may be one of the few visible indicators of pain in an individual with limited expressive language ability and DD, sources of pain should be considered when unexplained agitation is present [ 3 , 9 , 52 ]. Limited access to medical care by the population with DD further reduces the likelihood that pain and other underlying physical causes of agitation are recognized [ 53 ]. Through Sources of Distress, caregivers demonstrated their ability to provide meaningful insight into the potential presence of physical discomfort. This attention was directed most frequently to gastrointestinal, menstrual, dental, and seizure concerns.

Limitations

The generalizability of study results is limited to the geographic, racial, and ethnic diversity of Utah. While survey access requires internet or smartphone access, it has been completed by parents without this access through the assistance of state-sponsored support coordinators and medical assistants. Sources of Distress has a Spanish translation available ( Causas de Aflicción ); however, these data were not included because its content has not yet been validated by Spanish-speaking caregivers and individuals who are affected. The expert leading the consult team was a member of the survey’s development team, which introduces the inherent bias of evaluating for the presence of mental health conditions through the lens of DM-ID-2 criteria on which survey components were also based. While the DM-ID-2 is well recognized and accepted in the ID provider community, few autism specialty providers are familiar with its use.

Future Directions

Edits and additions to mood disorder items and scoring algorithms are being made to improve differentiation across mood disorders. Branching logic that incorporates the individual’s language ability has recently been added to the psychosis module to improve question clarity and scoring algorithm accuracy. The most updated version of the Sources of Distress can be accessed through the Utah Department of Health and Human Services Autism Systems Development Program webpage [ 39 ]. Reevaluation of the survey’s PPVs, NPVs, and accuracy will follow the completion of these changes. Additional studies of this survey are needed to measure its acceptability and validity in clinical settings outside of Utah and by other DD specialty providers. REDCap has also demonstrated capacity to integrate digital mental health screening results into electronic medical records, significantly improving provider adoption of the screening tools [ 54 ]. The integration of Sources of Distress into electronic medical records could further enhance its impact on provider efficiency. This survey has already been used during medical evaluations to facilitate the consideration of potential discordance between medications prescribed and conditions present [ 55 ]. Prospective studies are merited to determine the survey’s impact on treatment approaches, hospital and emergency department use, and outcomes for individuals with DD who experience crisis.

Conclusions

Individuals with DD presenting in crisis experience high rates of psychiatric disorders and medical concerns that may contribute to, or manifest as, distress. Sources of Distress is a valuable screening tool for psychiatric and medical conditions that commonly accompany treatment-resistant agitation in individuals with DD. When systematically queried, caregivers’ knowledge provides essential information to minimize diagnostic overshadowing and support an evaluation focused on the individual rather than their disability when persistent agitation is assessed in the population with DD.

Acknowledgments

The authors thank the individuals with developmental disabilities and their parents and caregivers whose support and participation shaped the development, validation, and piloting of the Sources of Distress. In particular, the authors appreciate the insight and assistance provided by Jaye Olafson. The Autism Council of Utah funded the development of this survey, and access to REDCap (Research Electronic Data Capture) is supported through UM1TR004409 NCATS/NIH.

Conflicts of Interest

The Sources of Distress survey is copyrighted by the University of Utah; DAB, JD, and WW are listed as inventors. DAB consults for BioMarin Pharmaceutical Inc, Encoded Therapeutics, Taysha Gene Therapies, and Synlogic Therapeutics and attended an advisory board meeting for Sanofi. All other authors declare no other conflicts of interest.

A sample of the Sources of Distress report.

• Durbin A, Balogh R, Lin E, Wilton AS, Selick A, Dobranowski KM, et al. Repeat emergency department visits for individuals with intellectual and developmental disabilities and psychiatric disorders. Am J Intellect Dev Disabil. May 2019;124(3):206-219. [ CrossRef ] [ Medline ]
• Lunsky Y, Lin E, Balogh R, Klein-Geltink J, Wilton AS, Kurdyak P. Emergency department visits and use of outpatient physician services by adults with developmental disability and psychiatric disorder. Can J Psychiatry. Oct 01, 2012;57(10):601-607. [ CrossRef ] [ Medline ]
• Antonacci DJ, Manuel C, Davis E. Diagnosis and treatment of aggression in individuals with developmental disabilities. Psychiatr Q. Sep 23, 2008;79(3):225-247. [ CrossRef ] [ Medline ]
• Kim JA, Szatmari P, Bryson SE, Streiner DL, Wilson FJ. The prevalence of anxiety and mood problems among children with Autism and Asperger syndrome. Autism. Jun 30, 2016;4(2):117-132. [ CrossRef ]
• Kishore MT, Nizamie SH, Nizamie A. The behavioural profile of psychiatric disorders in persons with intellectual disability. J Intellect Disabil Res. Nov 06, 2005;49(Pt 11):852-857. [ CrossRef ] [ Medline ]
• Lowe K, Allen D, Jones E, Brophy S, Moore K, James W. Challenging behaviours: prevalence and topographies. J Intellect Disabil Res. Aug 09, 2007;51(Pt 8):625-636. [ CrossRef ] [ Medline ]
• McGuire K, Fung LK, Hagopian L, Vasa RA, Mahajan R, Bernal P, et al. Irritability and problem behavior in autism spectrum disorder: a practice pathway for pediatric primary care. Pediatrics. Feb 2016;137 Suppl 2:S136-S148. [ CrossRef ] [ Medline ]
• Oliver C, Licence L, Richards C. Self-injurious behaviour in people with intellectual disability and autism spectrum disorder. Curr Opin Psychiatry. Mar 2017;30(2):97-101. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Regnard C, Reynolds J, Watson B, Matthews D, Gibson L, Clarke C. Understanding distress in people with severe communication difficulties: developing and assessing the Disability Distress Assessment Tool (DisDAT). J Intellect Disabil Res. Apr 29, 2007;51(Pt 4):277-292. [ CrossRef ] [ Medline ]
• Rojahn J, Matson JL, Naglieri JA, Mayville E. Relationships between psychiatric conditions and behavior problems among adults with mental retardation. Am J Ment Retard. Jan 2004;109(1):21-33. [ CrossRef ] [ Medline ]
• Brookman-Frazee L, Stadnick N, Chlebowski C, Baker-Ericzén M, Ganger W. Characterizing psychiatric comorbidity in children with autism spectrum disorder receiving publicly funded mental health services. Autism. Nov 15, 2018;22(8):938-952. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Buck TR, Viskochil J, Farley M, Coon H, McMahon WM, Morgan J, et al. Psychiatric comorbidity and medication use in adults with autism spectrum disorder. J Autism Dev Disord. Dec 24, 2014;44(12):3063-3071. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eaves LC, Ho HH. Young adult outcome of autism spectrum disorders. J Autism Dev Disord. Apr 1, 2008;38(4):739-747. [ CrossRef ] [ Medline ]
• Leyfer OT, Folstein SE, Bacalman S, Davis NO, Dinh E, Morgan J, et al. Comorbid psychiatric disorders in children with autism: interview development and rates of disorders. J Autism Dev Disord. Oct 15, 2006;36(7):849-861. [ CrossRef ] [ Medline ]
• Lai MC, Kassee C, Besney R, Bonato S, Hull L, Mandy W, et al. Prevalence of co-occurring mental health diagnoses in the autism population: a systematic review and meta-analysis. Lancet Psychiatry. Oct 2019;6(10):819-829. [ CrossRef ] [ Medline ]
• Lecavalier L, McCracken CE, Aman MG, McDougle CJ, McCracken JT, Tierney E, et al. An exploration of concomitant psychiatric disorders in children with autism spectrum disorder. Compr Psychiatry. Jan 2019;88:57-64. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morgan CN, Roy M, Chance P. Psychiatric comorbidity and medication use in autism: a community survey. Psychiatr Bull. Jan 02, 2018;27(10):378-381. [ CrossRef ]
• Simonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. J Am Acad Child Adolesc Psychiatry. Aug 2008;47(8):921-929. [ CrossRef ] [ Medline ]
• Varcin KJ, Herniman SE, Lin A, Chen Y, Perry Y, Pugh C, et al. Occurrence of psychosis and bipolar disorder in adults with autism: a systematic review and meta-analysis. Neurosci Biobehav Rev. Mar 2022;134:104543. [ CrossRef ] [ Medline ]
• Mazza MG, Rossetti A, Crespi G, Clerici M. Prevalence of co-occurring psychiatric disorders in adults and adolescents with intellectual disability: a systematic review and meta-analysis. J Appl Res Intellect Disabil. Mar 2020;33(2):126-138. [ CrossRef ] [ Medline ]
• Platt JM, Keyes KM, McLaughlin KA, Kaufman AS. Intellectual disability and mental disorders in a US population representative sample of adolescents. Psychol Med. Apr 2019;49(6):952-961. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Munir KM. The co-occurrence of mental disorders in children and adolescents with intellectual disability/intellectual developmental disorder. Curr Opin Psychiatry. Mar 2016;29(2):95-102. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Smit MJ, Scheffers M, Emck C, van Busschbach JT, Beek PJ. Clinical characteristics of individuals with intellectual disability who have experienced sexual abuse. An overview of the literature. Res Dev Disabil. Dec 2019;95:103513. [ CrossRef ] [ Medline ]
• Jamieson D, Mason J. Investigating the existence of the diagnostic overshadowing bias in Australia. J Ment Health Res Intellect Disabil. Apr 22, 2019;12(1-2):58-70. [ CrossRef ]
• Joshi G, Biederman J, Petty C, Goldin RL, Furtak SL, Wozniak J. Examining the comorbidity of bipolar disorder and autism spectrum disorders: a large controlled analysis of phenotypic and familial correlates in a referred population of youth with bipolar I disorder with and without autism spectrum disorders. J Clin Psychiatry. Jun 2013;74(6):578-586. [ CrossRef ] [ Medline ]
• Rapoport J, Chavez A, Greenstein D, Addington A, Gogtay N. Autism spectrum disorders and childhood-onset schizophrenia: clinical and biological contributions to a relation revisited. J Am Acad Child Adolesc Psychiatry. Jan 2009;48(1):10-18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bertelli M, Scuticchio D, Ferrandi A, Lassi S, Mango F, Ciavatta C, et al. Reliability and validity of the SPAID-G checklist for detecting psychiatric disorders in adults with intellectual disability. Res Dev Disabil. 2012;33(2):382-390. [ CrossRef ] [ Medline ]
• Fletcher RJ, Barnhill J, Cooper SA. Diagnostic Manual—Intellectual Disability 2 (DM-ID): A Textbook of Diagnosis of Mental Disorders in Persons with Intellectual Disability. Washington, DC. NADD Press; 2016.
• Fletcher R, Loschen E, Stavrakiki C, Kingston MF. Diagnostic Manual - Intellectual Disability: A Textbook of Diagnosis of Mental Disorders in Persons With Intellectual Disability. New York, NY. NADD Press; 2007.
• Moss S, Patel P, Prosser H, Goldberg D, Simpson N, Rowe S, et al. Psychiatric morbidity in older people with moderate and severe learning disability. I: development and reliability of the patient interview (PAS-ADD). Br J Psychiatry. Oct 1993;163:471-480. [ CrossRef ] [ Medline ]
• Matson JL, Kazdin AE, Senatore V. Psychometric properties of the psychopathology instrument for mentally retarded adults. Appl Res Ment Retard. 1984;5(1):81-89. [ CrossRef ] [ Medline ]
• Devine M, Taggart L, McLornian P. Screening for mental health problems in adults with learning disabilities using the Mini PAS‐ADD Interview. Br J Learn Disabil. Nov 04, 2010;38(4):252-258. [ CrossRef ]
• Moss S, Prosser H, Costello H, Simpson N, Patel P, Rowe S, et al. Reliability and validity of the PAS-ADD Checklist for detecting psychiatric disorders in adults with intellectual disability. J Intellect Disabil Res. Apr 1998;42 ( Pt 2):173-183. [ CrossRef ] [ Medline ]
• Sevin JA, Matson JL, Williams D, Kirkpatrick-Sanchez S. Reliability of emotional problems with the diagnostic assessment for the Severely Handicapped (DASH). Br J Clin Psychol. Feb 1995;34(1):93-94. [ CrossRef ] [ Medline ]
• Myrbakk E, von Tetzchner S. Screening individuals with intellectual disability for psychiatric disorders: comparison of four measures. Am J Ment Retard. Jan 2008;113(1):54-70. [ CrossRef ] [ Medline ]
• Taylor JL, Hatton C, Dixon L, Douglas C. Screening for psychiatric symptoms: PAS-ADD Checklist norms for adults with intellectual disabilities. J Intellect Disabil Res. Jan 2004;48(1):37-41. [ CrossRef ] [ Medline ]
• Marcangelo MJ, Ovsiew F. Psychiatric aspects of epilepsy. Psychiatr Clin North Am. Dec 2007;30(4):781-802. [ CrossRef ] [ Medline ]
• Périsse D, Amiet C, Consoli A, Thorel MV, Gourfinkel-An I, Bodeau N, et al. Risk factors of acute behavioral regression in psychiatrically hospitalized adolescents with autism. J Can Acad Child Adolesc Psychiatry. May 2010;19(2):100-108. [ FREE Full text ] [ Medline ]
• Bilder DA. Sources of distress - Autism Systems Development. Utah Department of Health and Human Services. URL: https://familyhealth.utah.gov/cshcn/asd/# [accessed 2024-03-11]
• Martin-Key NA, Spadaro B, Funnell E, Barker EJ, Schei TS, Tomasik J, et al. The current state and validity of digital assessment tools for psychiatry: systematic review. JMIR Ment Health. Mar 30, 2022;9(3):e32824. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Home page. Autism Council of Utah. URL: https://www.autismcouncilofutah.org/ [accessed 2024-02-11]
• Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, et al. Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. Jul 1997;36(7):980-988. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess B, editors. Analyzing Qualitative Data. London, UK. Routledge; 1994.
• Strauss A, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 2nd edition. Thousand Oaks, CA. Sage Publications; 1998.
• Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. REDCap Consortium. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. Jul 2019;95:103208. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Custom template engine. GitHub. URL: https://github.com/BCCHR-IT/custom-template-engine, [accessed 2024-02-18]
• LeClerc S, Easley D. Pharmacological therapies for autism spectrum disorder: a review. P T. Jun 2015;40(6):389-397. [ FREE Full text ] [ Medline ]
• Downs J, Hotopf M, Ford T, Simonoff E, Jackson RG, Shetty H, et al. Clinical predictors of antipsychotic use in children and adolescents with autism spectrum disorders: a historical open cohort study using electronic health records. Eur Child Adolesc Psychiatry. Jun 15, 2016;25(6):649-658. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Esbensen AJ, Greenberg JS, Seltzer MM, Aman MG. A longitudinal investigation of psychotropic and non-psychotropic medication use among adolescents and adults with autism spectrum disorders. J Autism Dev Disord. Sep 12, 2009;39(9):1339-1349. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shui AM, Katz T, Malow BA, Mazurek MO. Predicting sleep problems in children with autism spectrum disorders. Res Dev Disabil. Dec 2018;83:270-279. [ CrossRef ] [ Medline ]
• Wood JJ, Gadow KD. Exploring the nature and function of anxiety in youth with autism spectrum disorders. Clin Psychol Sci Pract. Dec 2010;17(4):281-292. [ CrossRef ]
• Gössler A, Schalamon J, Huber-Zeyringer A, Höllwarth ME. Gastroesophageal reflux and behavior in neurologically impaired children. J Pediatr Surg. Sep 2007;42(9):1486-1490. [ CrossRef ] [ Medline ]
• Matson JL, Rivet TT. Characteristics of challenging behaviours in adults with autistic disorder, PDD-NOS, and intellectual disability. J Intellect Dev Disabil. Dec 2008;33(4):323-329. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hawley S, Yu J, Bogetic N, Potapova N, Wakefield C, Thompson M, et al. Digitization of measurement-based care pathways in mental health through REDCap and electronic health record integration: development and usability study. J Med Internet Res. May 20, 2021;23(5):e25656. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bartel RL, Knight JR, Worsham W, Bilder D. Discordance between psychiatric diagnoses and medication use in children and adults with autism presenting in crisis. Focus (forthcoming). 2024;22 [ CrossRef ]

Abbreviations

Edited by J Torous; submitted 17.07.23; peer-reviewed by H Onyeaka, A AL-Asadi; comments to author 04.02.24; revised version received 18.02.24; accepted 20.02.24; published 29.03.24.

©Deborah A Bilder, Mariah Mthembu, Whitney Worsham, Patricia Aguayo, Jacob R Knight, Steven W Deng, Tejinder P Singh, John Davis. Originally published in JMIR Mental Health (https://mental.jmir.org), 29.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on https://mental.jmir.org/, as well as this copyright and license information must be included.