examples of qualitative research papers

18 Qualitative Research Examples

Qualitative research is an approach to scientific research that involves using observation to gather and analyze non-numerical, in-depth, and well-contextualized datasets.

It serves as an integral part of academic, professional, and even daily decision-making processes (Baxter & Jack, 2008).

Methods of qualitative research encompass a wide range of techniques, from in-depth personal encounters, like ethnographies (studying cultures in-depth) and autoethnographies (examining one’s own cultural experiences), to collection of diverse perspectives on topics through methods like interviewing focus groups (gatherings of individuals to discuss specific topics).

Qualitative Research Examples

1. ethnography.

Definition: Ethnography is a qualitative research design aimed at exploring cultural phenomena. Rooted in the discipline of anthropology , this research approach investigates the social interactions, behaviors, and perceptions within groups, communities, or organizations.

Ethnographic research is characterized by extended observation of the group, often through direct participation, in the participants’ environment. An ethnographer typically lives with the study group for extended periods, intricately observing their everyday lives (Khan, 2014).

It aims to present a complete, detailed and accurate picture of the observed social life, rituals, symbols, and values from the perspective of the study group.

Example of Ethnographic Research

Title: “ The Everyday Lives of Men: An Ethnographic Investigation of Young Adult Male Identity “

Citation: Evans, J. (2010). The Everyday Lives of Men: An Ethnographic Investigation of Young Adult Male Identity. Peter Lang.

Overview: This study by Evans (2010) provides a rich narrative of young adult male identity as experienced in everyday life. The author immersed himself among a group of young men, participating in their activities and cultivating a deep understanding of their lifestyle, values, and motivations. This research exemplified the ethnographic approach, revealing complexities of the subjects’ identities and societal roles, which could hardly be accessed through other qualitative research designs.

Read my Full Guide on Ethnography Here

2. Autoethnography

Definition: Autoethnography is an approach to qualitative research where the researcher uses their own personal experiences to extend the understanding of a certain group, culture, or setting. Essentially, it allows for the exploration of self within the context of social phenomena.

Unlike traditional ethnography, which focuses on the study of others, autoethnography turns the ethnographic gaze inward, allowing the researcher to use their personal experiences within a culture as rich qualitative data (Durham, 2019).

The objective is to critically appraise one’s personal experiences as they navigate and negotiate cultural, political, and social meanings. The researcher becomes both the observer and the participant, intertwining personal and cultural experiences in the research.

Example of Autoethnographic Research

Title: “ A Day In The Life Of An NHS Nurse “

Citation: Osben, J. (2019). A day in the life of a NHS nurse in 21st Century Britain: An auto-ethnography. The Journal of Autoethnography for Health & Social Care. 1(1).

Overview: This study presents an autoethnography of a day in the life of an NHS nurse (who, of course, is also the researcher). The author uses the research to achieve reflexivity, with the researcher concluding: “Scrutinising my practice and situating it within a wider contextual backdrop has compelled me to significantly increase my level of scrutiny into the driving forces that influence my practice.”

Read my Full Guide on Autoethnography Here

3. Semi-Structured Interviews

Definition: Semi-structured interviews stand as one of the most frequently used methods in qualitative research. These interviews are planned and utilize a set of pre-established questions, but also allow for the interviewer to steer the conversation in other directions based on the responses given by the interviewee.

In semi-structured interviews, the interviewer prepares a guide that outlines the focal points of the discussion. However, the interview is flexible, allowing for more in-depth probing if the interviewer deems it necessary (Qu, & Dumay, 2011). This style of interviewing strikes a balance between structured ones which might limit the discussion, and unstructured ones, which could lack focus.

Example of Semi-Structured Interview Research

Title: “ Factors influencing adherence to cancer treatment in older adults with cancer: a systematic review “

Citation: Puts, M., et al. (2014). Factors influencing adherence to cancer treatment in older adults with cancer: a systematic review. Annals of oncology, 25 (3), 564-577.

Overview: Puts et al. (2014) executed an extensive systematic review in which they conducted semi-structured interviews with older adults suffering from cancer to examine the factors influencing their adherence to cancer treatment. The findings suggested that various factors, including side effects, faith in healthcare professionals, and social support have substantial impacts on treatment adherence. This research demonstrates how semi-structured interviews can provide rich and profound insights into the subjective experiences of patients.

4. Focus Groups

Definition: Focus groups are a qualitative research method that involves organized discussion with a selected group of individuals to gain their perspectives on a specific concept, product, or phenomenon. Typically, these discussions are guided by a moderator.

During a focus group session, the moderator has a list of questions or topics to discuss, and participants are encouraged to interact with each other (Morgan, 2010). This interactivity can stimulate more information and provide a broader understanding of the issue under scrutiny. The open format allows participants to ask questions and respond freely, offering invaluable insights into attitudes, experiences, and group norms.

Example of Focus Group Research

Title: “ Perspectives of Older Adults on Aging Well: A Focus Group Study “

Citation: Halaweh, H., Dahlin-Ivanoff, S., Svantesson, U., & Willén, C. (2018). Perspectives of older adults on aging well: a focus group study. Journal of aging research .

Overview: This study aimed to explore what older adults (aged 60 years and older) perceived to be ‘aging well’. The researchers identified three major themes from their focus group interviews: a sense of well-being, having good physical health, and preserving good mental health. The findings highlight the importance of factors such as positive emotions, social engagement, physical activity, healthy eating habits, and maintaining independence in promoting aging well among older adults.

5. Phenomenology

Definition: Phenomenology, a qualitative research method, involves the examination of lived experiences to gain an in-depth understanding of the essence or underlying meanings of a phenomenon.

The focus of phenomenology lies in meticulously describing participants’ conscious experiences related to the chosen phenomenon (Padilla-Díaz, 2015).

In a phenomenological study, the researcher collects detailed, first-hand perspectives of the participants, typically via in-depth interviews, and then uses various strategies to interpret and structure these experiences, ultimately revealing essential themes (Creswell, 2013). This approach focuses on the perspective of individuals experiencing the phenomenon, seeking to explore, clarify, and understand the meanings they attach to those experiences.

Example of Phenomenology Research

Title: “ A phenomenological approach to experiences with technology: current state, promise, and future directions for research ”

Citation: Cilesiz, S. (2011). A phenomenological approach to experiences with technology: Current state, promise, and future directions for research. Educational Technology Research and Development, 59 , 487-510.

Overview: A phenomenological approach to experiences with technology by Sebnem Cilesiz represents a good starting point for formulating a phenomenological study. With its focus on the ‘essence of experience’, this piece presents methodological, reliability, validity, and data analysis techniques that phenomenologists use to explain how people experience technology in their everyday lives.

6. Grounded Theory

Definition: Grounded theory is a systematic methodology in qualitative research that typically applies inductive reasoning . The primary aim is to develop a theoretical explanation or framework for a process, action, or interaction grounded in, and arising from, empirical data (Birks & Mills, 2015).

In grounded theory, data collection and analysis work together in a recursive process. The researcher collects data, analyses it, and then collects more data based on the evolving understanding of the research context. This ongoing process continues until a comprehensive theory that represents the data and the associated phenomenon emerges – a point known as theoretical saturation (Charmaz, 2014).

Example of Grounded Theory Research

Title: “ Student Engagement in High School Classrooms from the Perspective of Flow Theory “

Citation: Shernoff, D. J., Csikszentmihalyi, M., Shneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18 (2), 158–176.

Overview: Shernoff and colleagues (2003) used grounded theory to explore student engagement in high school classrooms. The researchers collected data through student self-reports, interviews, and observations. Key findings revealed that academic challenge, student autonomy, and teacher support emerged as the most significant factors influencing students’ engagement, demonstrating how grounded theory can illuminate complex dynamics within real-world contexts.

7. Narrative Research

Definition: Narrative research is a qualitative research method dedicated to storytelling and understanding how individuals experience the world. It focuses on studying an individual’s life and experiences as narrated by that individual (Polkinghorne, 2013).

In narrative research, the researcher collects data through methods such as interviews, observations , and document analysis. The emphasis is on the stories told by participants – narratives that reflect their experiences, thoughts, and feelings.

These stories are then interpreted by the researcher, who attempts to understand the meaning the participant attributes to these experiences (Josselson, 2011).

Example of Narrative Research

Title: “Narrative Structures and the Language of the Self”

Citation: McAdams, D. P., Josselson, R., & Lieblich, A. (2006). Identity and story: Creating self in narrative . American Psychological Association.

Overview: In this innovative study, McAdams et al. (2006) employed narrative research to explore how individuals construct their identities through the stories they tell about themselves. By examining personal narratives, the researchers discerned patterns associated with characters, motivations, conflicts, and resolutions, contributing valuable insights about the relationship between narrative and individual identity.

8. Case Study Research

Definition: Case study research is a qualitative research method that involves an in-depth investigation of a single instance or event: a case. These ‘cases’ can range from individuals, groups, or entities to specific projects, programs, or strategies (Creswell, 2013).

The case study method typically uses multiple sources of information for comprehensive contextual analysis. It aims to explore and understand the complexity and uniqueness of a particular case in a real-world context (Merriam & Tisdell, 2015). This investigation could result in a detailed description of the case, a process for its development, or an exploration of a related issue or problem.

Example of Case Study Research

Title: “ Teacher’s Role in Fostering Preschoolers’ Computational Thinking: An Exploratory Case Study “

Citation: Wang, X. C., Choi, Y., Benson, K., Eggleston, C., & Weber, D. (2021). Teacher’s role in fostering preschoolers’ computational thinking: An exploratory case study. Early Education and Development , 32 (1), 26-48.

Overview: This study investigates the role of teachers in promoting computational thinking skills in preschoolers. The study utilized a qualitative case study methodology to examine the computational thinking scaffolding strategies employed by a teacher interacting with three preschoolers in a small group setting. The findings highlight the importance of teachers’ guidance in fostering computational thinking practices such as problem reformulation/decomposition, systematic testing, and debugging.

Read about some Famous Case Studies in Psychology Here

9. Participant Observation

Definition: Participant observation has the researcher immerse themselves in a group or community setting to observe the behavior of its members. It is similar to ethnography, but generally, the researcher isn’t embedded for a long period of time.

The researcher, being a participant, engages in daily activities, interactions, and events as a way of conducting a detailed study of a particular social phenomenon (Kawulich, 2005).

The method involves long-term engagement in the field, maintaining detailed records of observed events, informal interviews, direct participation, and reflexivity. This approach allows for a holistic view of the participants’ lived experiences, behaviours, and interactions within their everyday environment (Dewalt, 2011).

Example of Participant Observation Research

Title: Conflict in the boardroom: a participant observation study of supervisory board dynamics

Citation: Heemskerk, E. M., Heemskerk, K., & Wats, M. M. (2017). Conflict in the boardroom: a participant observation study of supervisory board dynamics. Journal of Management & Governance , 21 , 233-263.

Overview: This study examined how conflicts within corporate boards affect their performance. The researchers used a participant observation method, where they actively engaged with 11 supervisory boards and observed their dynamics. They found that having a shared understanding of the board’s role called a common framework, improved performance by reducing relationship conflicts, encouraging task conflicts, and minimizing conflicts between the board and CEO.

10. Non-Participant Observation

Definition: Non-participant observation is a qualitative research method in which the researcher observes the phenomena of interest without actively participating in the situation, setting, or community being studied.

This method allows the researcher to maintain a position of distance, as they are solely an observer and not a participant in the activities being observed (Kawulich, 2005).

During non-participant observation, the researcher typically records field notes on the actions, interactions, and behaviors observed , focusing on specific aspects of the situation deemed relevant to the research question.

This could include verbal and nonverbal communication , activities, interactions, and environmental contexts (Angrosino, 2007). They could also use video or audio recordings or other methods to collect data.

Example of Non-Participant Observation Research

Title: Mental Health Nurses’ attitudes towards mental illness and recovery-oriented practice in acute inpatient psychiatric units: A non-participant observation study

Citation: Sreeram, A., Cross, W. M., & Townsin, L. (2023). Mental Health Nurses’ attitudes towards mental illness and recovery‐oriented practice in acute inpatient psychiatric units: A non‐participant observation study. International Journal of Mental Health Nursing .

Overview: This study investigated the attitudes of mental health nurses towards mental illness and recovery-oriented practice in acute inpatient psychiatric units. The researchers used a non-participant observation method, meaning they observed the nurses without directly participating in their activities. The findings shed light on the nurses’ perspectives and behaviors, providing valuable insights into their attitudes toward mental health and recovery-focused care in these settings.

11. Content Analysis

Definition: Content Analysis involves scrutinizing textual, visual, or spoken content to categorize and quantify information. The goal is to identify patterns, themes, biases, or other characteristics (Hsieh & Shannon, 2005).

Content Analysis is widely used in various disciplines for a multitude of purposes. Researchers typically use this method to distill large amounts of unstructured data, like interview transcripts, newspaper articles, or social media posts, into manageable and meaningful chunks.

When wielded appropriately, Content Analysis can illuminate the density and frequency of certain themes within a dataset, provide insights into how specific terms or concepts are applied contextually, and offer inferences about the meanings of their content and use (Duriau, Reger, & Pfarrer, 2007).

Example of Content Analysis

Title: Framing European politics: A content analysis of press and television news .

Citation: Semetko, H. A., & Valkenburg, P. M. (2000). Framing European politics: A content analysis of press and television news. Journal of Communication, 50 (2), 93-109.

Overview: This study analyzed press and television news articles about European politics using a method called content analysis. The researchers examined the prevalence of different “frames” in the news, which are ways of presenting information to shape audience perceptions. They found that the most common frames were attribution of responsibility, conflict, economic consequences, human interest, and morality.

Read my Full Guide on Content Analysis Here

12. Discourse Analysis

Definition: Discourse Analysis, a qualitative research method, interprets the meanings, functions, and coherence of certain languages in context.

Discourse analysis is typically understood through social constructionism, critical theory , and poststructuralism and used for understanding how language constructs social concepts (Cheek, 2004).

Discourse Analysis offers great breadth, providing tools to examine spoken or written language, often beyond the level of the sentence. It enables researchers to scrutinize how text and talk articulate social and political interactions and hierarchies.

Insight can be garnered from different conversations, institutional text, and media coverage to understand how topics are addressed or framed within a specific social context (Jorgensen & Phillips, 2002).

Example of Discourse Analysis

Title: The construction of teacher identities in educational policy documents: A critical discourse analysis

Citation: Thomas, S. (2005). The construction of teacher identities in educational policy documents: A critical discourse analysis. Critical Studies in Education, 46 (2), 25-44.

Overview: The author examines how an education policy in one state of Australia positions teacher professionalism and teacher identities. While there are competing discourses about professional identity, the policy framework privileges a  narrative that frames the ‘good’ teacher as one that accepts ever-tightening control and regulation over their professional practice.

Read my Full Guide on Discourse Analysis Here

13. Action Research

Definition: Action Research is a qualitative research technique that is employed to bring about change while simultaneously studying the process and results of that change.

This method involves a cyclical process of fact-finding, action, evaluation, and reflection (Greenwood & Levin, 2016).

Typically, Action Research is used in the fields of education, social sciences , and community development. The process isn’t just about resolving an issue but also developing knowledge that can be used in the future to address similar or related problems.

The researcher plays an active role in the research process, which is normally broken down into four steps: 

• developing a plan to improve what is currently being done
• implementing the plan
• observing the effects of the plan, and
• reflecting upon these effects (Smith, 2010).

Example of Action Research

Title: Using Digital Sandbox Gaming to Improve Creativity Within Boys’ Writing

Citation: Ellison, M., & Drew, C. (2020). Using digital sandbox gaming to improve creativity within boys’ writing. Journal of Research in Childhood Education , 34 (2), 277-287.

Overview: This was a research study one of my research students completed in his own classroom under my supervision. He implemented a digital game-based approach to literacy teaching with boys and interviewed his students to see if the use of games as stimuli for storytelling helped draw them into the learning experience.

Read my Full Guide on Action Research Here

14. Semiotic Analysis

Definition: Semiotic Analysis is a qualitative method of research that interprets signs and symbols in communication to understand sociocultural phenomena. It stems from semiotics, the study of signs and symbols and their use or interpretation (Chandler, 2017).

In a Semiotic Analysis, signs (anything that represents something else) are interpreted based on their significance and the role they play in representing ideas.

This type of research often involves the examination of images, sounds, and word choice to uncover the embedded sociocultural meanings. For example, an advertisement for a car might be studied to learn more about societal views on masculinity or success (Berger, 2010).

Example of Semiotic Research

Title: Shielding the learned body: a semiotic analysis of school badges in New South Wales, Australia

Citation: Symes, C. (2023). Shielding the learned body: a semiotic analysis of school badges in New South Wales, Australia. Semiotica , 2023 (250), 167-190.

Overview: This study examines school badges in New South Wales, Australia, and explores their significance through a semiotic analysis. The badges, which are part of the school’s visual identity, are seen as symbolic representations that convey meanings. The analysis reveals that these badges often draw on heraldic models, incorporating elements like colors, names, motifs, and mottoes that reflect local culture and history, thus connecting students to their national identity. Additionally, the study highlights how some schools have shifted from traditional badges to modern logos and slogans, reflecting a more business-oriented approach.

15. Qualitative Longitudinal Studies

Definition: Qualitative Longitudinal Studies are a research method that involves repeated observation of the same items over an extended period of time.

Unlike a snapshot perspective, this method aims to piece together individual histories and examine the influences and impacts of change (Neale, 2019).

Qualitative Longitudinal Studies provide an in-depth understanding of change as it happens, including changes in people’s lives, their perceptions, and their behaviors.

For instance, this method could be used to follow a group of students through their schooling years to understand the evolution of their learning behaviors and attitudes towards education (Saldaña, 2003).

Example of Qualitative Longitudinal Research

Title: Patient and caregiver perspectives on managing pain in advanced cancer: a qualitative longitudinal study

Citation: Hackett, J., Godfrey, M., & Bennett, M. I. (2016). Patient and caregiver perspectives on managing pain in advanced cancer: a qualitative longitudinal study.  Palliative medicine ,  30 (8), 711-719.

Overview: This article examines how patients and their caregivers manage pain in advanced cancer through a qualitative longitudinal study. The researchers interviewed patients and caregivers at two different time points and collected audio diaries to gain insights into their experiences, making this study longitudinal.

Read my Full Guide on Longitudinal Research Here

16. Open-Ended Surveys

Definition: Open-Ended Surveys are a type of qualitative research method where respondents provide answers in their own words. Unlike closed-ended surveys, which limit responses to predefined options, open-ended surveys allow for expansive and unsolicited explanations (Fink, 2013).

Open-ended surveys are commonly used in a range of fields, from market research to social studies. As they don’t force respondents into predefined response categories, these surveys help to draw out rich, detailed data that might uncover new variables or ideas.

For example, an open-ended survey might be used to understand customer opinions about a new product or service (Lavrakas, 2008).

Contrast this to a quantitative closed-ended survey, like a Likert scale, which could theoretically help us to come up with generalizable data but is restricted by the questions on the questionnaire, meaning new and surprising data and insights can’t emerge from the survey results in the same way.

Example of Open-Ended Survey Research

Title: Advantages and disadvantages of technology in relationships: Findings from an open-ended survey

Citation: Hertlein, K. M., & Ancheta, K. (2014). Advantages and disadvantages of technology in relationships: Findings from an open-ended survey.  The Qualitative Report ,  19 (11), 1-11.

Overview: This article examines the advantages and disadvantages of technology in couple relationships through an open-ended survey method. Researchers analyzed responses from 410 undergraduate students to understand how technology affects relationships. They found that technology can contribute to relationship development, management, and enhancement, but it can also create challenges such as distancing, lack of clarity, and impaired trust.

17. Naturalistic Observation

Definition: Naturalistic Observation is a type of qualitative research method that involves observing individuals in their natural environments without interference or manipulation by the researcher.

Naturalistic observation is often used when conducting research on behaviors that cannot be controlled or manipulated in a laboratory setting (Kawulich, 2005).

It is frequently used in the fields of psychology, sociology, and anthropology. For instance, to understand the social dynamics in a schoolyard, a researcher could spend time observing the children interact during their recess, noting their behaviors, interactions, and conflicts without imposing their presence on the children’s activities (Forsyth, 2010).

Example of Naturalistic Observation Research

Title: Dispositional mindfulness in daily life: A naturalistic observation study

Citation: Kaplan, D. M., Raison, C. L., Milek, A., Tackman, A. M., Pace, T. W., & Mehl, M. R. (2018). Dispositional mindfulness in daily life: A naturalistic observation study. PloS one , 13 (11), e0206029.

Overview: In this study, researchers conducted two studies: one exploring assumptions about mindfulness and behavior, and the other using naturalistic observation to examine actual behavioral manifestations of mindfulness. They found that trait mindfulness is associated with a heightened perceptual focus in conversations, suggesting that being mindful is expressed primarily through sharpened attention rather than observable behavioral or social differences.

Read my Full Guide on Naturalistic Observation Here

18. Photo-Elicitation

Definition: Photo-elicitation utilizes photographs as a means to trigger discussions and evoke responses during interviews. This strategy aids in bringing out topics of discussion that may not emerge through verbal prompting alone (Harper, 2002).

Traditionally, Photo-Elicitation has been useful in various fields such as education, psychology, and sociology. The method involves the researcher or participants taking photographs, which are then used as prompts for discussion.

For instance, a researcher studying urban environmental issues might invite participants to photograph areas in their neighborhood that they perceive as environmentally detrimental, and then discuss each photo in depth (Clark-Ibáñez, 2004).

Example of Photo-Elicitation Research

Title: Early adolescent food routines: A photo-elicitation study

Citation: Green, E. M., Spivak, C., & Dollahite, J. S. (2021). Early adolescent food routines: A photo-elicitation study. Appetite, 158 .

Overview: This study focused on early adolescents (ages 10-14) and their food routines. Researchers conducted in-depth interviews using a photo-elicitation approach, where participants took photos related to their food choices and experiences. Through analysis, the study identified various routines and three main themes: family, settings, and meals/foods consumed, revealing how early adolescents view and are influenced by their eating routines.

Features of Qualitative Research

Qualitative research is a research method focused on understanding the meaning individuals or groups attribute to a social or human problem (Creswell, 2013).

Some key features of this method include:

• Naturalistic Inquiry: Qualitative research happens in the natural setting of the phenomena, aiming to understand “real world” situations (Patton, 2015). This immersion in the field or subject allows the researcher to gather a deep understanding of the subject matter.
• Emphasis on Process: It aims to understand how events unfold over time rather than focusing solely on outcomes (Merriam & Tisdell, 2015). The process-oriented nature of qualitative research allows researchers to investigate sequences, timing, and changes.
• Interpretive: It involves interpreting and making sense of phenomena in terms of the meanings people assign to them (Denzin & Lincoln, 2011). This interpretive element allows for rich, nuanced insights into human behavior and experiences.
• Holistic Perspective: Qualitative research seeks to understand the whole phenomenon rather than focusing on individual components (Creswell, 2013). It emphasizes the complex interplay of factors, providing a richer, more nuanced view of the research subject.
• Prioritizes Depth over Breadth: Qualitative research favors depth of understanding over breadth, typically involving a smaller but more focused sample size (Hennink, Hutter, & Bailey, 2020). This enables detailed exploration of the phenomena of interest, often leading to rich and complex data.

Qualitative vs Quantitative Research

Qualitative research centers on exploring and understanding the meaning individuals or groups attribute to a social or human problem (Creswell, 2013).

It involves an in-depth approach to the subject matter, aiming to capture the richness and complexity of human experience.

Examples include conducting interviews, observing behaviors, or analyzing text and images.

There are strengths inherent in this approach. In its focus on understanding subjective experiences and interpretations, qualitative research can yield rich and detailed data that quantitative research may overlook (Denzin & Lincoln, 2011).

Additionally, qualitative research is adaptive, allowing the researcher to respond to new directions and insights as they emerge during the research process.

However, there are also limitations. Because of the interpretive nature of this research, findings may not be generalizable to a broader population (Marshall & Rossman, 2014). Well-designed quantitative research, on the other hand, can be generalizable.

Moreover, the reliability and validity of qualitative data can be challenging to establish due to its subjective nature, unlike quantitative research, which is ideally more objective.

Compare Qualitative and Quantitative Research Methodologies in This Guide Here

In conclusion, qualitative research methods provide distinctive ways to explore social phenomena and understand nuances that quantitative approaches might overlook. Each method, from Ethnography to Photo-Elicitation, presents its strengths and weaknesses but they all offer valuable means of investigating complex, real-world situations. The goal for the researcher is not to find a definitive tool, but to employ the method best suited for their research questions and the context at hand (Almalki, 2016). Above all, these methods underscore the richness of human experience and deepen our understanding of the world around us.

Angrosino, M. (2007). Doing ethnographic and observational research. Sage Publications.

Areni, C. S., & Kim, D. (1994). The influence of in-store lighting on consumers’ examination of merchandise in a wine store. International Journal of Research in Marketing, 11 (2), 117-125.

Barker, C., Pistrang, N., & Elliott, R. (2016). Research Methods in Clinical Psychology: An Introduction for Students and Practitioners. John Wiley & Sons.

Baxter, P. & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13 (4), 544-559.

Berger, A. A. (2010). The Objects of Affection: Semiotics and Consumer Culture. Palgrave Macmillan.

Bevan, M. T. (2014). A method of phenomenological interviewing. Qualitative health research, 24 (1), 136-144.

Birks, M., & Mills, J. (2015). Grounded theory: A practical guide . Sage Publications.

Bryman, A. (2015) . The SAGE Handbook of Qualitative Research. Sage Publications.

Chandler, D. (2017). Semiotics: The Basics. Routledge.

Charmaz, K. (2014). Constructing grounded theory. Sage Publications.

Cheek, J. (2004). At the margins? Discourse analysis and qualitative research. Qualitative Health Research, 14(8), 1140-1150.

Clark-Ibáñez, M. (2004). Framing the social world with photo-elicitation interviews. American Behavioral Scientist, 47(12), 1507-1527.

Creswell, J. W. (2013). Research Design: Qualitative, Quantitative and Mixed Methods Approaches. Sage Publications.

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011). The case study approach. BMC Medical Research Methodology, 11(100), 1-9.

Denzin, N. K., & Lincoln, Y. S. (2011). The Sage Handbook of Qualitative Research. Sage.

Dewalt, K. M., & Dewalt, B. R. (2011). Participant observation: A guide for fieldworkers. Rowman Altamira.

Doody, O., Slevin, E., & Taggart, L. (2013). Focus group interviews in nursing research: part 1. British Journal of Nursing, 22(1), 16-19.

Durham, A. (2019). Autoethnography. In P. Atkinson (Ed.), Qualitative Research Methods. Oxford University Press.

Duriau, V. J., Reger, R. K., & Pfarrer, M. D. (2007). A content analysis of the content analysis literature in organization studies: Research themes, data sources, and methodological refinements. Organizational Research Methods, 10(1), 5-34.

Evans, J. (2010). The Everyday Lives of Men: An Ethnographic Investigation of Young Adult Male Identity. Peter Lang.

Farrall, S. (2006). What is qualitative longitudinal research? Papers in Social Research Methods, Qualitative Series, No.11, London School of Economics, Methodology Institute.

Fielding, J., & Fielding, N. (2008). Synergy and synthesis: integrating qualitative and quantitative data. The SAGE handbook of social research methods, 555-571.

Fink, A. (2013). How to conduct surveys: A step-by-step guide . SAGE.

Forsyth, D. R. (2010). Group Dynamics . Wadsworth Cengage Learning.

Fugard, A. J. B., & Potts, H. W. W. (2015). Supporting thinking on sample sizes for thematic analyses: A quantitative tool. International Journal of Social Research Methodology, 18 (6), 669–684.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Aldine de Gruyter.

Gray, J. R., Grove, S. K., & Sutherland, S. (2017). Burns and Grove’s the Practice of Nursing Research E-Book: Appraisal, Synthesis, and Generation of Evidence. Elsevier Health Sciences.

Greenwood, D. J., & Levin, M. (2016). Introduction to action research: Social research for social change. SAGE.

Harper, D. (2002). Talking about pictures: A case for photo elicitation. Visual Studies, 17 (1), 13-26.

Heinonen, T. (2012). Making Sense of the Social: Human Sciences and the Narrative Turn. Rozenberg Publishers.

Heisley, D. D., & Levy, S. J. (1991). Autodriving: A photoelicitation technique. Journal of Consumer Research, 18 (3), 257-272.

Hennink, M. M., Hutter, I., & Bailey, A. (2020). Qualitative Research Methods . SAGE Publications Ltd.

Hsieh, H. F., & Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. Qualitative Health Research, 15 (9), 1277–1288.

Jorgensen, D. L. (2015). Participant Observation. In Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource. John Wiley & Sons, Inc.

Jorgensen, M., & Phillips, L. (2002). Discourse Analysis as Theory and Method . SAGE.

Josselson, R. (2011). Narrative research: Constructing, deconstructing, and reconstructing story. In Five ways of doing qualitative analysis . Guilford Press.

Kawulich, B. B. (2005). Participant observation as a data collection method. Forum: Qualitative Social Research, 6 (2).

Khan, S. (2014). Qualitative Research Method: Grounded Theory. Journal of Basic and Clinical Pharmacy, 5 (4), 86-88.

Koshy, E., Koshy, V., & Waterman, H. (2010). Action Research in Healthcare . SAGE.

Krippendorff, K. (2013). Content Analysis: An Introduction to its Methodology. SAGE.

Lannon, J., & Cooper, P. (2012). Humanistic Advertising: A Holistic Cultural Perspective. International Journal of Advertising, 15 (2), 97–111.

Lavrakas, P. J. (2008). Encyclopedia of survey research methods. SAGE Publications.

Lieblich, A., Tuval-Mashiach, R., & Zilber, T. (2008). Narrative research: Reading, analysis and interpretation. Sage Publications.

Mackey, A., & Gass, S. M. (2015). Second language research: Methodology and design. Routledge.

Marshall, C., & Rossman, G. B. (2014). Designing qualitative research. Sage publications.

McAdams, D. P., Josselson, R., & Lieblich, A. (2006). Identity and story: Creating self in narrative. American Psychological Association.

Merriam, S. B., & Tisdell, E. J. (2015). Qualitative Research: A Guide to Design and Implementation. Jossey-Bass.

Mick, D. G. (1986). Consumer Research and Semiotics: Exploring the Morphology of Signs, Symbols, and Significance. Journal of Consumer Research, 13 (2), 196-213.

Morgan, D. L. (2010). Focus groups as qualitative research. Sage Publications.

Mulhall, A. (2003). In the field: notes on observation in qualitative research. Journal of Advanced Nursing, 41 (3), 306-313.

Neale, B. (2019). What is Qualitative Longitudinal Research? Bloomsbury Publishing.

Nolan, L. B., & Renderos, T. B. (2012). A focus group study on the influence of fatalism and religiosity on cancer risk perceptions in rural, eastern North Carolina. Journal of religion and health, 51 (1), 91-104.

Padilla-Díaz, M. (2015). Phenomenology in educational qualitative research: Philosophy as science or philosophical science? International Journal of Educational Excellence, 1 (2), 101-110.

Parker, I. (2014). Discourse dynamics: Critical analysis for social and individual psychology . Routledge.

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Sage Publications.

Polkinghorne, D. E. (2013). Narrative configuration in qualitative analysis. In Life history and narrative. Routledge.

Puts, M. T., Tapscott, B., Fitch, M., Howell, D., Monette, J., Wan-Chow-Wah, D., Krzyzanowska, M., Leighl, N. B., Springall, E., & Alibhai, S. (2014). Factors influencing adherence to cancer treatment in older adults with cancer: a systematic review. Annals of oncology, 25 (3), 564-577.

Qu, S. Q., & Dumay, J. (2011). The qualitative research interview . Qualitative research in accounting & management.

Ali, J., & Bhaskar, S. B. (2016). Basic statistical tools in research and data analysis. Indian Journal of Anaesthesia, 60 (9), 662–669.

Rosenbaum, M. S. (2017). Exploring the social supportive role of third places in consumers’ lives. Journal of Service Research, 20 (1), 26-42.

Saldaña, J. (2003). Longitudinal Qualitative Research: Analyzing Change Through Time . AltaMira Press.

Saldaña, J. (2014). The Coding Manual for Qualitative Researchers. SAGE.

Shernoff, D. J., Csikszentmihalyi, M., Shneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18 (2), 158-176.

Smith, J. A. (2015). Qualitative Psychology: A Practical Guide to Research Methods . Sage Publications.

Smith, M. K. (2010). Action Research. The encyclopedia of informal education.

Sue, V. M., & Ritter, L. A. (2012). Conducting online surveys . SAGE Publications.

Van Auken, P. M., Frisvoll, S. J., & Stewart, S. I. (2010). Visualising community: using participant-driven photo-elicitation for research and application. Local Environment, 15 (4), 373-388.

Van Voorhis, F. L., & Morgan, B. L. (2007). Understanding Power and Rules of Thumb for Determining Sample Sizes. Tutorials in Quantitative Methods for Psychology, 3 (2), 43–50.

Wodak, R., & Meyer, M. (2015). Methods of Critical Discourse Analysis . SAGE.

Zuber-Skerritt, O. (2018). Action research for developing educational theories and practices . Routledge.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 5 Top Tips for Succeeding at University
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 50 Durable Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 100 Consumer Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 30 Globalization Pros and Cons

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

• Open access
• Published: 27 May 2020

How to use and assess qualitative research methods

• Loraine Busetto   ORCID: orcid.org/0000-0002-9228-7875 1 ,
• Wolfgang Wick 1 , 2 &
• Christoph Gumbinger 1  

Neurological Research and Practice volume  2 , Article number:  14 ( 2020 ) Cite this article

684k Accesses

268 Citations

89 Altmetric

Metrics details

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 , 8 , 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 , 10 , 11 , 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 , 25 , 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 , 32 , 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 , 38 , 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Availability of data and materials

Not applicable.

Abbreviations

Endovascular treatment

Randomised Controlled Trial

Standard Operating Procedure

Standards for Reporting Qualitative Research

Philipsen, H., & Vernooij-Dassen, M. (2007). Kwalitatief onderzoek: nuttig, onmisbaar en uitdagend. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Qualitative research: useful, indispensable and challenging. In: Qualitative research: Practical methods for medical practice (pp. 5–12). Houten: Bohn Stafleu van Loghum.

Chapter   Google Scholar  

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches . London: Sage.

Kelly, J., Dwyer, J., Willis, E., & Pekarsky, B. (2014). Travelling to the city for hospital care: Access factors in country aboriginal patient journeys. Australian Journal of Rural Health, 22 (3), 109–113.

Article   Google Scholar  

Nilsen, P., Ståhl, C., Roback, K., & Cairney, P. (2013). Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Science, 8 (1), 1–12.

Howick J, Chalmers I, Glasziou, P., Greenhalgh, T., Heneghan, C., Liberati, A., Moschetti, I., Phillips, B., & Thornton, H. (2011). The 2011 Oxford CEBM evidence levels of evidence (introductory document) . Oxford Center for Evidence Based Medicine. https://www.cebm.net/2011/06/2011-oxford-cebm-levels-evidence-introductory-document/ .

Eakin, J. M. (2016). Educating critical qualitative health researchers in the land of the randomized controlled trial. Qualitative Inquiry, 22 (2), 107–118.

May, A., & Mathijssen, J. (2015). Alternatieven voor RCT bij de evaluatie van effectiviteit van interventies!? Eindrapportage. In Alternatives for RCTs in the evaluation of effectiveness of interventions!? Final report .

Google Scholar  

Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299 (10), 1182–1184.

Article   CAS   Google Scholar  

Christ, T. W. (2014). Scientific-based research and randomized controlled trials, the “gold” standard? Alternative paradigms and mixed methodologies. Qualitative Inquiry, 20 (1), 72–80.

Lamont, T., Barber, N., Jd, P., Fulop, N., Garfield-Birkbeck, S., Lilford, R., Mear, L., Raine, R., & Fitzpatrick, R. (2016). New approaches to evaluating complex health and care systems. BMJ, 352:i154.

Drabble, S. J., & O’Cathain, A. (2015). Moving from Randomized Controlled Trials to Mixed Methods Intervention Evaluation. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 406–425). London: Oxford University Press.

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science : IS, 8 , 117.

Hak, T. (2007). Waarnemingsmethoden in kwalitatief onderzoek. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Observation methods in qualitative research] (pp. 13–25). Houten: Bohn Stafleu van Loghum.

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36 , 717–732.

Yanow, D. (2000). Conducting interpretive policy analysis (Vol. 47). Thousand Oaks: Sage University Papers Series on Qualitative Research Methods.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

van der Geest, S. (2006). Participeren in ziekte en zorg: meer over kwalitatief onderzoek. Huisarts en Wetenschap, 49 (4), 283–287.

Hijmans, E., & Kuyper, M. (2007). Het halfopen interview als onderzoeksmethode. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [The half-open interview as research method (pp. 43–51). Houten: Bohn Stafleu van Loghum.

Jansen, H. (2007). Systematiek en toepassing van de kwalitatieve survey. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Systematics and implementation of the qualitative survey (pp. 27–41). Houten: Bohn Stafleu van Loghum.

Pv, R., & Peremans, L. (2007). Exploreren met focusgroepgesprekken: de ‘stem’ van de groep onder de loep. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Exploring with focus group conversations: the “voice” of the group under the magnifying glass (pp. 53–64). Houten: Bohn Stafleu van Loghum.

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41 (5), 545–547.

Boeije H: Analyseren in kwalitatief onderzoek: Denken en doen, [Analysis in qualitative research: Thinking and doing] vol. Den Haag Boom Lemma uitgevers; 2012.

Hunter, A., & Brewer, J. (2015). Designing Multimethod Research. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 185–205). London: Oxford University Press.

Archibald, M. M., Radil, A. I., Zhang, X., & Hanson, W. E. (2015). Current mixed methods practices in qualitative research: A content analysis of leading journals. International Journal of Qualitative Methods, 14 (2), 5–33.

Creswell, J. W., & Plano Clark, V. L. (2011). Choosing a Mixed Methods Design. In Designing and Conducting Mixed Methods Research . Thousand Oaks: SAGE Publications.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320 (7226), 50–52.

O'Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine : Journal of the Association of American Medical Colleges, 89 (9), 1245–1251.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52 (4), 1893–1907.

Moser, A., & Korstjens, I. (2018). Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. European Journal of General Practice, 24 (1), 9–18.

Marlett, N., Shklarov, S., Marshall, D., Santana, M. J., & Wasylak, T. (2015). Building new roles and relationships in research: A model of patient engagement research. Quality of Life Research : an international journal of quality of life aspects of treatment, care and rehabilitation, 24 (5), 1057–1067.

Demian, M. N., Lam, N. N., Mac-Way, F., Sapir-Pichhadze, R., & Fernandez, N. (2017). Opportunities for engaging patients in kidney research. Canadian Journal of Kidney Health and Disease, 4 , 2054358117703070–2054358117703070.

Noyes, J., McLaughlin, L., Morgan, K., Roberts, A., Stephens, M., Bourne, J., Houlston, M., Houlston, J., Thomas, S., Rhys, R. G., et al. (2019). Designing a co-productive study to overcome known methodological challenges in organ donation research with bereaved family members. Health Expectations . 22(4):824–35.

Piil, K., Jarden, M., & Pii, K. H. (2019). Research agenda for life-threatening cancer. European Journal Cancer Care (Engl), 28 (1), e12935.

Hofmann, D., Ibrahim, F., Rose, D., Scott, D. L., Cope, A., Wykes, T., & Lempp, H. (2015). Expectations of new treatment in rheumatoid arthritis: Developing a patient-generated questionnaire. Health Expectations : an international journal of public participation in health care and health policy, 18 (5), 995–1008.

Jun, M., Manns, B., Laupacis, A., Manns, L., Rehal, B., Crowe, S., & Hemmelgarn, B. R. (2015). Assessing the extent to which current clinical research is consistent with patient priorities: A scoping review using a case study in patients on or nearing dialysis. Canadian Journal of Kidney Health and Disease, 2 , 35.

Elsie Baker, S., & Edwards, R. (2012). How many qualitative interviews is enough? In National Centre for Research Methods Review Paper . National Centre for Research Methods. http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf .

Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18 (2), 179–183.

Sim, J., Saunders, B., Waterfield, J., & Kingstone, T. (2018). Can sample size in qualitative research be determined a priori? International Journal of Social Research Methodology, 21 (5), 619–634.

Download references

Acknowledgements

no external funding.

Author information

Authors and affiliations.

Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Loraine Busetto, Wolfgang Wick & Christoph Gumbinger

Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Wolfgang Wick

You can also search for this author in PubMed   Google Scholar

Contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

Corresponding author

Correspondence to Loraine Busetto .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Busetto, L., Wick, W. & Gumbinger, C. How to use and assess qualitative research methods. Neurol. Res. Pract. 2 , 14 (2020). https://doi.org/10.1186/s42466-020-00059-z

Download citation

Received : 30 January 2020

Accepted : 22 April 2020

Published : 27 May 2020

DOI : https://doi.org/10.1186/s42466-020-00059-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Mixed methods
• Quality assessment

Neurological Research and Practice

ISSN: 2524-3489

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Qualitative Methods
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The word qualitative implies an emphasis on the qualities of entities and on processes and meanings that are not experimentally examined or measured [if measured at all] in terms of quantity, amount, intensity, or frequency. Qualitative researchers stress the socially constructed nature of reality, the intimate relationship between the researcher and what is studied, and the situational constraints that shape inquiry. Such researchers emphasize the value-laden nature of inquiry. They seek answers to questions that stress how social experience is created and given meaning. In contrast, quantitative studies emphasize the measurement and analysis of causal relationships between variables, not processes. Qualitative forms of inquiry are considered by many social and behavioral scientists to be as much a perspective on how to approach investigating a research problem as it is a method.

Denzin, Norman. K. and Yvonna S. Lincoln. “Introduction: The Discipline and Practice of Qualitative Research.” In The Sage Handbook of Qualitative Research . Norman. K. Denzin and Yvonna S. Lincoln, eds. 3 rd edition. (Thousand Oaks, CA: Sage, 2005), p. 10.

Characteristics of Qualitative Research

Below are the three key elements that define a qualitative research study and the applied forms each take in the investigation of a research problem.

• Naturalistic -- refers to studying real-world situations as they unfold naturally; non-manipulative and non-controlling; the researcher is open to whatever emerges [i.e., there is a lack of predetermined constraints on findings].
• Emergent -- acceptance of adapting inquiry as understanding deepens and/or situations change; the researcher avoids rigid designs that eliminate responding to opportunities to pursue new paths of discovery as they emerge.
• Purposeful -- cases for study [e.g., people, organizations, communities, cultures, events, critical incidences] are selected because they are “information rich” and illuminative. That is, they offer useful manifestations of the phenomenon of interest; sampling is aimed at insight about the phenomenon, not empirical generalization derived from a sample and applied to a population.

The Collection of Data

• Data -- observations yield a detailed, "thick description" [in-depth understanding]; interviews capture direct quotations about people’s personal perspectives and lived experiences; often derived from carefully conducted case studies and review of material culture.
• Personal experience and engagement -- researcher has direct contact with and gets close to the people, situation, and phenomenon under investigation; the researcher’s personal experiences and insights are an important part of the inquiry and critical to understanding the phenomenon.
• Empathic neutrality -- an empathic stance in working with study respondents seeks vicarious understanding without judgment [neutrality] by showing openness, sensitivity, respect, awareness, and responsiveness; in observation, it means being fully present [mindfulness].
• Dynamic systems -- there is attention to process; assumes change is ongoing, whether the focus is on an individual, an organization, a community, or an entire culture, therefore, the researcher is mindful of and attentive to system and situational dynamics.

The Analysis

• Unique case orientation -- assumes that each case is special and unique; the first level of analysis is being true to, respecting, and capturing the details of the individual cases being studied; cross-case analysis follows from and depends upon the quality of individual case studies.
• Inductive analysis -- immersion in the details and specifics of the data to discover important patterns, themes, and inter-relationships; begins by exploring, then confirming findings, guided by analytical principles rather than rules.
• Holistic perspective -- the whole phenomenon under study is understood as a complex system that is more than the sum of its parts; the focus is on complex interdependencies and system dynamics that cannot be reduced in any meaningful way to linear, cause and effect relationships and/or a few discrete variables.
• Context sensitive -- places findings in a social, historical, and temporal context; researcher is careful about [even dubious of] the possibility or meaningfulness of generalizations across time and space; emphasizes careful comparative case study analysis and extrapolating patterns for possible transferability and adaptation in new settings.
• Voice, perspective, and reflexivity -- the qualitative methodologist owns and is reflective about her or his own voice and perspective; a credible voice conveys authenticity and trustworthiness; complete objectivity being impossible and pure subjectivity undermining credibility, the researcher's focus reflects a balance between understanding and depicting the world authentically in all its complexity and of being self-analytical, politically aware, and reflexive in consciousness.

Berg, Bruce Lawrence. Qualitative Research Methods for the Social Sciences . 8th edition. Boston, MA: Allyn and Bacon, 2012; Denzin, Norman. K. and Yvonna S. Lincoln. Handbook of Qualitative Research . 2nd edition. Thousand Oaks, CA: Sage, 2000; Marshall, Catherine and Gretchen B. Rossman. Designing Qualitative Research . 2nd ed. Thousand Oaks, CA: Sage Publications, 1995; Merriam, Sharan B. Qualitative Research: A Guide to Design and Implementation . San Francisco, CA: Jossey-Bass, 2009.

Basic Research Design for Qualitative Studies

Unlike positivist or experimental research that utilizes a linear and one-directional sequence of design steps, there is considerable variation in how a qualitative research study is organized. In general, qualitative researchers attempt to describe and interpret human behavior based primarily on the words of selected individuals [a.k.a., “informants” or “respondents”] and/or through the interpretation of their material culture or occupied space. There is a reflexive process underpinning every stage of a qualitative study to ensure that researcher biases, presuppositions, and interpretations are clearly evident, thus ensuring that the reader is better able to interpret the overall validity of the research. According to Maxwell (2009), there are five, not necessarily ordered or sequential, components in qualitative research designs. How they are presented depends upon the research philosophy and theoretical framework of the study, the methods chosen, and the general assumptions underpinning the study. Goals Describe the central research problem being addressed but avoid describing any anticipated outcomes. Questions to ask yourself are: Why is your study worth doing? What issues do you want to clarify, and what practices and policies do you want it to influence? Why do you want to conduct this study, and why should the reader care about the results? Conceptual Framework Questions to ask yourself are: What do you think is going on with the issues, settings, or people you plan to study? What theories, beliefs, and prior research findings will guide or inform your research, and what literature, preliminary studies, and personal experiences will you draw upon for understanding the people or issues you are studying? Note to not only report the results of other studies in your review of the literature, but note the methods used as well. If appropriate, describe why earlier studies using quantitative methods were inadequate in addressing the research problem. Research Questions Usually there is a research problem that frames your qualitative study and that influences your decision about what methods to use, but qualitative designs generally lack an accompanying hypothesis or set of assumptions because the findings are emergent and unpredictable. In this context, more specific research questions are generally the result of an interactive design process rather than the starting point for that process. Questions to ask yourself are: What do you specifically want to learn or understand by conducting this study? What do you not know about the things you are studying that you want to learn? What questions will your research attempt to answer, and how are these questions related to one another? Methods Structured approaches to applying a method or methods to your study help to ensure that there is comparability of data across sources and researchers and, thus, they can be useful in answering questions that deal with differences between phenomena and the explanation for these differences [variance questions]. An unstructured approach allows the researcher to focus on the particular phenomena studied. This facilitates an understanding of the processes that led to specific outcomes, trading generalizability and comparability for internal validity and contextual and evaluative understanding. Questions to ask yourself are: What will you actually do in conducting this study? What approaches and techniques will you use to collect and analyze your data, and how do these constitute an integrated strategy? Validity In contrast to quantitative studies where the goal is to design, in advance, “controls” such as formal comparisons, sampling strategies, or statistical manipulations to address anticipated and unanticipated threats to validity, qualitative researchers must attempt to rule out most threats to validity after the research has begun by relying on evidence collected during the research process itself in order to effectively argue that any alternative explanations for a phenomenon are implausible. Questions to ask yourself are: How might your results and conclusions be wrong? What are the plausible alternative interpretations and validity threats to these, and how will you deal with these? How can the data that you have, or that you could potentially collect, support or challenge your ideas about what’s going on? Why should we believe your results? Conclusion Although Maxwell does not mention a conclusion as one of the components of a qualitative research design, you should formally conclude your study. Briefly reiterate the goals of your study and the ways in which your research addressed them. Discuss the benefits of your study and how stakeholders can use your results. Also, note the limitations of your study and, if appropriate, place them in the context of areas in need of further research.

Chenail, Ronald J. Introduction to Qualitative Research Design. Nova Southeastern University; Heath, A. W. The Proposal in Qualitative Research. The Qualitative Report 3 (March 1997); Marshall, Catherine and Gretchen B. Rossman. Designing Qualitative Research . 3rd edition. Thousand Oaks, CA: Sage, 1999; Maxwell, Joseph A. "Designing a Qualitative Study." In The SAGE Handbook of Applied Social Research Methods . Leonard Bickman and Debra J. Rog, eds. 2nd ed. (Thousand Oaks, CA: Sage, 2009), p. 214-253; Qualitative Research Methods. Writing@CSU. Colorado State University; Yin, Robert K. Qualitative Research from Start to Finish . 2nd edition. New York: Guilford, 2015.

Strengths of Using Qualitative Methods

The advantage of using qualitative methods is that they generate rich, detailed data that leave the participants' perspectives intact and provide multiple contexts for understanding the phenomenon under study. In this way, qualitative research can be used to vividly demonstrate phenomena or to conduct cross-case comparisons and analysis of individuals or groups.

Among the specific strengths of using qualitative methods to study social science research problems is the ability to:

• Obtain a more realistic view of the lived world that cannot be understood or experienced in numerical data and statistical analysis;
• Provide the researcher with the perspective of the participants of the study through immersion in a culture or situation and as a result of direct interaction with them;
• Allow the researcher to describe existing phenomena and current situations;
• Develop flexible ways to perform data collection, subsequent analysis, and interpretation of collected information;
• Yield results that can be helpful in pioneering new ways of understanding;
• Respond to changes that occur while conducting the study ]e.g., extended fieldwork or observation] and offer the flexibility to shift the focus of the research as a result;
• Provide a holistic view of the phenomena under investigation;
• Respond to local situations, conditions, and needs of participants;
• Interact with the research subjects in their own language and on their own terms; and,
• Create a descriptive capability based on primary and unstructured data.

Anderson, Claire. “Presenting and Evaluating Qualitative Research.” American Journal of Pharmaceutical Education 74 (2010): 1-7; Denzin, Norman. K. and Yvonna S. Lincoln. Handbook of Qualitative Research . 2nd edition. Thousand Oaks, CA: Sage, 2000; Merriam, Sharan B. Qualitative Research: A Guide to Design and Implementation . San Francisco, CA: Jossey-Bass, 2009.

Limitations of Using Qualitative Methods

It is very much true that most of the limitations you find in using qualitative research techniques also reflect their inherent strengths . For example, small sample sizes help you investigate research problems in a comprehensive and in-depth manner. However, small sample sizes undermine opportunities to draw useful generalizations from, or to make broad policy recommendations based upon, the findings. Additionally, as the primary instrument of investigation, qualitative researchers are often embedded in the cultures and experiences of others. However, cultural embeddedness increases the opportunity for bias generated from conscious or unconscious assumptions about the study setting to enter into how data is gathered, interpreted, and reported.

Some specific limitations associated with using qualitative methods to study research problems in the social sciences include the following:

• Drifting away from the original objectives of the study in response to the changing nature of the context under which the research is conducted;
• Arriving at different conclusions based on the same information depending on the personal characteristics of the researcher;
• Replication of a study is very difficult;
• Research using human subjects increases the chance of ethical dilemmas that undermine the overall validity of the study;
• An inability to investigate causality between different research phenomena;
• Difficulty in explaining differences in the quality and quantity of information obtained from different respondents and arriving at different, non-consistent conclusions;
• Data gathering and analysis is often time consuming and/or expensive;
• Requires a high level of experience from the researcher to obtain the targeted information from the respondent;
• May lack consistency and reliability because the researcher can employ different probing techniques and the respondent can choose to tell some particular stories and ignore others; and,
• Generation of a significant amount of data that cannot be randomized into manageable parts for analysis.

Research Tip

Human Subject Research and Institutional Review Board Approval

Almost every socio-behavioral study requires you to submit your proposed research plan to an Institutional Review Board. The role of the Board is to evaluate your research proposal and determine whether it will be conducted ethically and under the regulations, institutional polices, and Code of Ethics set forth by the university. The purpose of the review is to protect the rights and welfare of individuals participating in your study. The review is intended to ensure equitable selection of respondents, that you have met the requirements for obtaining informed consent , that there is clear assessment and minimization of risks to participants and to the university [read: no lawsuits!], and that privacy and confidentiality are maintained throughout the research process and beyond. Go to the USC IRB website for detailed information and templates of forms you need to submit before you can proceed. If you are  unsure whether your study is subject to IRB review, consult with your professor or academic advisor.

Chenail, Ronald J. Introduction to Qualitative Research Design. Nova Southeastern University; Labaree, Robert V. "Working Successfully with Your Institutional Review Board: Practical Advice for Academic Librarians." College and Research Libraries News 71 (April 2010): 190-193.

Another Research Tip

Finding Examples of How to Apply Different Types of Research Methods

SAGE publications is a major publisher of studies about how to design and conduct research in the social and behavioral sciences. Their SAGE Research Methods Online and Cases database includes contents from books, articles, encyclopedias, handbooks, and videos covering social science research design and methods including the complete Little Green Book Series of Quantitative Applications in the Social Sciences and the Little Blue Book Series of Qualitative Research techniques. The database also includes case studies outlining the research methods used in real research projects. This is an excellent source for finding definitions of key terms and descriptions of research design and practice, techniques of data gathering, analysis, and reporting, and information about theories of research [e.g., grounded theory]. The database covers both qualitative and quantitative research methods as well as mixed methods approaches to conducting research.

SAGE Research Methods Online and Cases

NOTE :  For a list of online communities, research centers, indispensable learning resources, and personal websites of leading qualitative researchers, GO HERE .

For a list of scholarly journals devoted to the study and application of qualitative research methods, GO HERE .

• << Previous: 6. The Methodology
• Next: Quantitative Methods >>
• Last Updated: Mar 21, 2024 9:59 AM
• URL: https://libguides.usc.edu/writingguide

Research Paper Guide

Qualitative Research Method

Qualitative Research - Methods, Types, and Examples

16 min read

People also read

Research Paper Writing - A Step by Step Guide

Research Paper Examples - Free Sample Papers for Different Formats!

Guide to Creating Effective Research Paper Outline

Interesting Research Paper Topics for 2024

Research Proposal Writing - A Step-by-Step Guide

How to Start a Research Paper - 7 Easy Steps

How to Write an Abstract for a Research Paper - A Step by Step Guide

Writing a Literature Review For a Research Paper - A Comprehensive Guide

8 Types of Qualitative Research - Overview & Examples

Qualitative vs Quantitative Research - Learning the Basics

Psychology Research Topics - 220+ Ideas

Learn How to Write a Hypothesis in a Research Paper: Examples and Tips!

20+ Types of Research With Examples - A Detailed Guide

Understanding Quantitative Research - Types & Data Collection Techniques

230+ Sociology Research Topics & Ideas for Students

How to Cite a Research Paper - A Complete Guide

Excellent History Research Paper Topics- 300+ Ideas

A Guide on Writing the Method Section of a Research Paper - Examples & Tips

How To Write an Introduction Paragraph For a Research Paper: Learn with Examples

Crafting a Winning Research Paper Title: A Complete Guide

Writing a Research Paper Conclusion - Step-by-Step Guide

Writing a Thesis For a Research Paper - A Comprehensive Guide

How To Write A Discussion For A Research Paper | Examples & Tips

How To Write The Results Section of A Research Paper | Steps & Examples

Writing a Problem Statement for a Research Paper - A Comprehensive Guide

Finding Sources For a Research Paper: A Complete Guide

A Guide on How to Edit a Research Paper

There are various methods for conducting scientific research. The two broad approaches to data collection include qualitative and quantitative research methods. 

However, it is not easy to decide which one to choose while writing a research paper .

If you know the basic difference between both methods, you will produce a well-written and structured paper. 

In this blog, we have explored what is qualitative research, its nature, purpose, and methods of data collection. By reading this, students can gain a good understanding of qualitative research, enhancing their ability to conduct in-depth studies. 

So keep reading!

• 1. What is Qualitative Research - Definition
• 2. Qualitative Research Methods
• 3. Types of Qualitative Research
• 4. Steps in Conducting Qualitative Research
• 5. Qualitative Research vs Quantitative Research
• 6. Qualitative Research Topics
• 7. Qualitative Research Examples

What is Qualitative Research - Definition

Qualitative research is a research methodology that aims to explore and understand the complexities of human behavior, emotions, and experiences through non-numerical data.

Unlike quantitative research, which deals in numbers and statistics, qualitative research is all about revealing the stories, and perspectives that make us uniquely human.

Let's dive deeper and discover why it's a powerful tool in the researcher's arsenal.

Purpose of Qualitative Research Design

Qualitative research simplifies the understanding of complex human behavior and experiences. Its purpose is to:

• Explore Complex Phenomena: Qualitative research allows us to delve deep into intricate human experiences and behaviors.
• Understand Motivations: It helps uncover the 'whys' behind actions, shedding light on underlying motivations.
• Capture Richness: By collecting narratives and stories, qualitative research captures the richness of human life.
• Generate Hypotheses: It often serves as a foundation for hypothesis generation in further quantitative studies.
• Inform Decision-Making: Qualitative findings guide decisions in fields like psychology, sociology, and market research.
• Contextualize Quantitative Data: It provides context to quantitative data, explaining the 'how' and 'why' behind the numbers.

Characteristics of Qualitative Research

The following are the main characteristics of qualitative research.

• The real-world setting is the first important characteristic. It involves various qualitative research methods to study the behavior of participants.
• Researchers play an essential role in choosing a method and making a plan for conducting research.
• All qualitative approaches have their significance and are used for different scenarios.
• Qualitative research questions are beneficial for complex reasoning to get the right results.
• It is also used to explain the outcome of quantitative research methods.
• The role of participants is essential as it brings meaning to the study.
• Qualitative research is flexible and can be changed at any stage of the research work.
• It also describes the research problem by developing a complex cause-and-effect relationship between the variables. 
• Data analysis in qualitative research is an ongoing process.
• Conclusions can be drawn based on the outcomes of the research process.
• Participants are selected from a particular and relevant group.

Qualitative Research Methods

A detailed description of the major qualitative approaches to collecting data is given below.

In-depth Interview

In-depth interviews involve one-on-one conversations to gather detailed information about a specific topic. This method allows researchers to explore participants' motivations, inspirations, and body language.

Interviews can be conducted face-to-face, via email, or over the phone for flexibility.

Focus Groups

Focus groups consist of small group discussions (5-15 participants) on specific topics, ideal for 'what,' 'why,' and 'how' questions about society and the environment. They can be conducted in-person or online, offering versatility in data collection.

Direct Observation

Direct observation collects subjective data through the five senses without interference. It focuses on characteristics, not measurements, often in public settings where privacy isn't a concern.

Open-Ended Surveys

Open-ended surveys use structured or unstructured questions to collect information on respondents' opinions and beliefs, providing insights into their perspectives.

Participant Observation

Participant observation involves researchers actively participating in events while observing people in natural settings, offering firsthand experience and insights.

Literature Review

The literature review method interprets words and images from published works to analyze social life. It examines word usage in context to draw inferences and identify meanings.

Types of Qualitative Research

The following is a comprehensive overview of the types of qualitative research methods.

The case study research method has now become the most valuable method of conducting research. It has evolved in recent years and is used to explain an entity in detail.

Moreover, it also involves a thorough understanding of different types of data sources. These include interviews, documents, reports, and observations.  Mainly, this research type is used in different areas like education, social sciences, etc.

Ethnographic Research

The ethnographic research method is the most familiar and in-depth observational method. It focuses on people and their behaviors in the natural environment.

Here, a researcher needs to adapt to the environment and society of the target audience to conduct better research. It helps to get a first-hand experience of the natural setting, including the customs, traditions, and culture of the subjects.

This type of research is a challenging and time-consuming process as it can last from days to years. However, geographical constraints can be an issue while collecting data.

Grounded Theory

While other methods discuss and focus on an event or activity. The grounded theory method deeply looks into the explanation and the main theory behind the event.

It requires the researcher to observe the interviews and documents to build a theory. Moreover, it usually starts with a question or collection of data.  However, the sample sizes in this method are usually larger than in other methods. 

Phenomenological Method

This type is used in the description of an event, phenomenon, and activity. Here, methods like interviews, reading documents, visiting places, and watching videos are used.

This will help to add new insights to the existing data analysis by checking its reliability and validity.

Check out the video to learn more about the phenomenological method of qualitative research!

Narrative Method 

The narrative method is used to gather data from subjects through interviews or documents. Later, the gathered information is used to derive answers and suggestions for future research. 

Historical Method

The historical method involves the examination of past events to draw conclusions and predictions about the future. The steps included in the method are formulating a plan, gathering data, and analyzing the sources. 

Steps in Conducting Qualitative Research

Conducting qualitative research is a systematic process that involves several key steps to ensure the collection of meaningful data.

Here's a chronological guide to conducting qualitative research:

Step 1: Define Research Objectives

Begin by clearly defining the research objectives and questions. What do you want to learn, explore, or understand through your qualitative research? This step sets the direction for your study.

Step 2: Select a Research Design

Choose an appropriate research design based on your objectives. Common designs include case studies, ethnography, grounded theory, or phenomenology. The design informs your data collection and analysis methods.

Step 3: Sampling Methods

Decide on your sampling strategy. Will you use purposive sampling to select specific participants who are most relevant to your research question? Or will you employ snowball sampling to find participants through referrals?

Step 4: Data Collection Techniques

Determine the data collection techniques that align with your research design. Depending on your approach, this may involve conducting in-depth interviews, facilitating focus groups, observing participants, or analyzing existing documents and content.

Step 5: Plan Interviews and Questions

If conducting interviews, create interview guides with open-ended questions. These questions should allow participants to share their thoughts, experiences, and perspectives freely. Ensure that questions are related to your research objectives.

Step 6: Conducting Data Collection

Collect data according to your chosen methods. For interviews, arrange and conduct interviews with participants, ensuring a comfortable and open environment. If using other techniques, follow the procedures outlined in your research design.

Step 7: Data Recording and Management

Record data meticulously. This may involve audio or video recordings, note-taking, or transcribing interviews. Organize and store data securely to maintain confidentiality.

Step 8: Data Analysis

Qualitative data can be in the form of interviews, transcripts, surveys, videos, audio, etc. The steps involved in qualitative data analysis are given below.   

• Organize the Data: This can be done by transcribing interviews or making detailed notes.
• Review the Data: Examine the data, ideas, and patterns.
• Establish a Data Coding System: Generate a set of codes that you can apply to classify your data.
• Assign Codes to the Data: For qualitative survey analysis, create codes, and add them to your system.
• Identify Themes: Link the codes together into cohesive themes.

Similarly, the following are different approaches to analyzing qualitative data. 

• Content Analysis – It is used to categorize common words and ideas.
• Thematic Analysis – thematic analysis in qualitative research is used to identify and interpret different themes and patterns.
• Textual Analysis – This type of analysis is used to examine the structure, content, and design of text.
• Discourse Analysis – It is used to study how a language is used to achieve specific results.

Step 9: Validity and Reliability

Ensure the validity and reliability of your findings. Consistently apply your chosen analysis methods and cross-check interpretations with colleagues or participants to validate your results.

Step 10: Ethical Considerations

Throughout the research process, uphold ethical principles. Protect the privacy and anonymity of participants, obtain informed consent, and address any ethical concerns that may arise.

Qualitative Research vs Quantitative Research

Qualitative and quantitative research are two distinct approaches to conducting research. Here are the main differences between qualitative vs. quantitative research.

Looking for a more detailed comparison between these 2 types of research? Check out our qualitative vs. quantitative research blog.

Qualitative Research Topics

To write an amazing qualitative research paper, here are some interesting topics for you.

• The Impact of Parental Involvement on Children's Education
• Social Isolation and Loneliness Among the Elderly
• Factors Influencing Consumer Choices in Sustainable Fashion
• Coping Mechanisms for Stress Among College Students
• Experiences of Immigrant Workers in Low-Wage Jobs
• The Role of Music in Expressing Emotions and Well-being
• Perceptions of Mental Health Stigma in Ethnic Communities
• Exploring the Transition to Parenthood: Challenges and Joys
• How Cultural Differences Influence Conflict Resolution Styles
• The Influence of Family Dynamics on Eating Habits and Nutrition in Children

We have also compiled a list of research paper topics in case you need more unique ideas.

Qualitative Research Examples

Check out the examples of qualitative research to get a better idea of writing a qualitative research study.

Qualitative Research Example

Qualitative Research Paper Sample

Qualitative Research Limitations

The following discussed are the qualitative research limitations. 

• The qualitative research data involve fewer expenses and time. 
• It does not have large-scale data.
• It requires a lot of time to manage, gather, and analyze data.
• It is not possible to verify the results as it is open-ended research. 
• It is difficult to analyze the credibility and validity of data because of its subjective nature.
• Expert knowledge of the area is necessary to understand the collected information.

In Conclusion, the qualitative research method shows the idea and perception of your targeted audience. However, not every student is able to choose the right approach while writing a research paper. It requires a thorough understanding of both qualitative research and quantitative research methods.

This is where the professional help from  MyPerfectWords.com comes in. We are a legit paper writing service that provides reliable help with your academic assignments. 

Contact our customer support and place " write my research paper " order today!

Frequently Asked Questions

What are the two methods in research study.

There are two types of studies that involve observing people during a study, participant observation and non-participant observation. 

Why is qualitative research better?

Because qualitative research includes the ability to gain unique insights through deep exploration. Survey respondents are able to disclose their experiences, thoughts, and feelings without constraint or influence from an outside source. 

Nova Allison is a Digital Content Strategist with over eight years of experience. Nova has also worked as a technical and scientific writer. She is majorly involved in developing and reviewing online content plans that engage and resonate with audiences. Nova has a passion for writing that engages and informs her readers.

Paper Due? Why Suffer? That’s our Job!

Keep reading

Join thousands of product people at Insight Out Conf on April 11. Register free.

Insights hub solutions

Analyze data

Uncover deep customer insights with fast, powerful features, store insights, curate and manage insights in one searchable platform, scale research, unlock the potential of customer insights at enterprise scale.

Featured reads

Tips and tricks

Make magic with your customer data in Dovetail

Four ways Dovetail helps Product Managers master continuous product discovery

Product updates

Dovetail retro: our biggest releases from the past year

Events and videos

© Dovetail Research Pty. Ltd.

Qualitative research: methods and examples

Last updated

13 April 2023

Reviewed by

Qualitative research involves gathering and evaluating non-numerical information to comprehend concepts, perspectives, and experiences. It’s also helpful for obtaining in-depth insights into a certain subject or generating new research ideas. 

As a result, qualitative research is practical if you want to try anything new or produce new ideas.

There are various ways you can conduct qualitative research. In this article, you'll learn more about qualitative research methodologies, including when you should use them.

Make research less tedious

Dovetail streamlines research to help you uncover and share actionable insights

• What is qualitative research?

Qualitative research is a broad term describing various research types that rely on asking open-ended questions. Qualitative research investigates “how” or “why” certain phenomena occur. It is about discovering the inherent nature of something.

The primary objective of qualitative research is to understand an individual's ideas, points of view, and feelings. In this way, collecting in-depth knowledge of a specific topic is possible. Knowing your audience's feelings about a particular subject is important for making reasonable research conclusions.

Unlike quantitative research , this approach does not involve collecting numerical, objective data for statistical analysis. Qualitative research is used extensively in education, sociology, health science, history, and anthropology.

• Types of qualitative research methodology

Typically, qualitative research aims at uncovering the attitudes and behavior of the target audience concerning a specific topic. For example,  “How would you describe your experience as a new Dovetail user?”

Some of the methods for conducting qualitative analysis include:

Focus groups

Hosting a focus group is a popular qualitative research method. It involves obtaining qualitative data from a limited sample of participants. In a moderated version of a focus group, the moderator asks participants a series of predefined questions. They aim to interact and build a group discussion that reveals their preferences, candid thoughts, and experiences.

Unmoderated, online focus groups are increasingly popular because they eliminate the need to interact with people face to face.

Focus groups can be more cost-effective than 1:1 interviews or studying a group in a natural setting and reporting one’s observations.

Focus groups make it possible to gather multiple points of view quickly and efficiently, making them an excellent choice for testing new concepts or conducting market research on a new product.

However, there are some potential drawbacks to this method. It may be unsuitable for sensitive or controversial topics. Participants might be reluctant to disclose their true feelings or respond falsely to conform to what they believe is the socially acceptable answer (known as response bias).

Case study research

A case study is an in-depth evaluation of a specific person, incident, organization, or society. This type of qualitative research has evolved into a broadly applied research method in education, law, business, and the social sciences.

Even though case study research may appear challenging to implement, it is one of the most direct research methods. It requires detailed analysis, broad-ranging data collection methodologies, and a degree of existing knowledge about the subject area under investigation.

Historical model

The historical approach is a distinct research method that deeply examines previous events to better understand the present and forecast future occurrences of the same phenomena. Its primary goal is to evaluate the impacts of history on the present and hence discover comparable patterns in the present to predict future outcomes.

Oral history

This qualitative data collection method involves gathering verbal testimonials from individuals about their personal experiences. It is widely used in historical disciplines to offer counterpoints to established historical facts and narratives. The most common methods of gathering oral history are audio recordings, analysis of auto-biographical text, videos, and interviews.

Qualitative observation

One of the most fundamental, oldest research methods, qualitative observation , is the process through which a researcher collects data using their senses of sight, smell, hearing, etc. It is used to observe the properties of the subject being studied. For example, “What does it look like?” As research methods go, it is subjective and depends on researchers’ first-hand experiences to obtain information, so it is prone to bias. However, it is an excellent way to start a broad line of inquiry like, “What is going on here?”

Record keeping and review

Record keeping uses existing documents and relevant data sources that can be employed for future studies. It is equivalent to visiting the library and going through publications or any other reference material to gather important facts that will likely be used in the research.

Grounded theory approach

The grounded theory approach is a commonly used research method employed across a variety of different studies. It offers a unique way to gather, interpret, and analyze. With this approach, data is gathered and analyzed simultaneously.  Existing analysis frames and codes are disregarded, and data is analyzed inductively, with new codes and frames generated from the research.

Ethnographic research

Ethnography  is a descriptive form of a qualitative study of people and their cultures. Its primary goal is to study people's behavior in their natural environment. This method necessitates that the researcher adapts to their target audience's setting. 

Thereby, you will be able to understand their motivation, lifestyle, ambitions, traditions, and culture in situ. But, the researcher must be prepared to deal with geographical constraints while collecting data i.e., audiences can’t be studied in a laboratory or research facility.

This study can last from a couple of days to several years. Thus, it is time-consuming and complicated, requiring you to have both the time to gather the relevant data as well as the expertise in analyzing, observing, and interpreting data to draw meaningful conclusions.

Narrative framework

A narrative framework is a qualitative research approach that relies on people's written text or visual images. It entails people analyzing these events or narratives to determine certain topics or issues. With this approach, you can understand how people represent themselves and their experiences to a larger audience.

Phenomenological approach

The phenomenological study seeks to investigate the experiences of a particular phenomenon within a group of individuals or communities. It analyzes a certain event through interviews with persons who have witnessed it to determine the connections between their views. Even though this method relies heavily on interviews, other data sources (recorded notes), and observations could be employed to enhance the findings.

• Qualitative research methods (tools)

Some of the instruments involved in qualitative research include:

Document research: Also known as document analysis because it involves evaluating written documents. These can include personal and non-personal materials like archives, policy publications, yearly reports, diaries, or letters.

Focus groups:  This is where a researcher poses questions and generates conversation among a group of people. The major goal of focus groups is to examine participants' experiences and knowledge, including research into how and why individuals act in various ways.

Secondary study: Involves acquiring existing information from texts, images, audio, or video recordings.

Observations:   This requires thorough field notes on everything you see, hear, or experience. Compared to reported conduct or opinion, this study method can assist you in getting insights into a specific situation and observable behaviors.

Structured interviews :  In this approach, you will directly engage people one-on-one. Interviews are ideal for learning about a person's subjective beliefs, motivations, and encounters.

Surveys:  This is when you distribute questionnaires containing open-ended questions

• What are common examples of qualitative research?

Everyday examples of qualitative research include:

Conducting a demographic analysis of a business

For instance, suppose you own a business such as a grocery store (or any store) and believe it caters to a broad customer base, but after conducting a demographic analysis, you discover that most of your customers are men.

You could do 1:1 interviews with female customers to learn why they don't shop at your store.

In this case, interviewing potential female customers should clarify why they don't find your shop appealing. It could be because of the products you sell or a need for greater brand awareness, among other possible reasons.

Launching or testing a new product

Suppose you are the product manager at a SaaS company looking to introduce a new product. Focus groups can be an excellent way to determine whether your product is marketable.

In this instance, you could hold a focus group with a sample group drawn from your intended audience. The group will explore the product based on its new features while you ensure adequate data on how users react to the new features. The data you collect will be key to making sales and marketing decisions.

Conducting studies to explain buyers' behaviors

You can also use qualitative research to understand existing buyer behavior better. Marketers analyze historical information linked to their businesses and industries to see when purchasers buy more.

Qualitative research can help you determine when to target new clients and peak seasons to boost sales by investigating the reason behind these behaviors.

• Qualitative research: data collection

Data collection is gathering information on predetermined variables to gain appropriate answers, test hypotheses, and analyze results. Researchers will collect non-numerical data for qualitative data collection to obtain detailed explanations and draw conclusions.

To get valid findings and achieve a conclusion in qualitative research, researchers must collect comprehensive and multifaceted data.

Qualitative data is usually gathered through interviews or focus groups with videotapes or handwritten notes. If there are recordings, they are transcribed before the data analysis process. Researchers keep separate folders for the recordings acquired from each focus group when collecting qualitative research data to categorize the data.

• Qualitative research: data analysis

Qualitative data analysis is organizing, examining, and interpreting qualitative data. Its main objective is identifying trends and patterns, responding to research questions, and recommending actions based on the findings. Textual analysis is a popular method for analyzing qualitative data.

Textual analysis differs from other qualitative research approaches in that researchers consider the social circumstances of study participants to decode their words, behaviors, and broader meaning. 

Learn more about qualitative research data analysis software

• When to use qualitative research

Qualitative research is helpful in various situations, particularly when a researcher wants to capture accurate, in-depth insights. 

Here are some instances when qualitative research can be valuable:

Examining your product or service to improve your marketing approach

When researching market segments, demographics, and customer service teams

Identifying client language when you want to design a quantitative survey

When attempting to comprehend your or someone else's strengths and weaknesses

Assessing feelings and beliefs about societal and public policy matters

Collecting information about a business or product's perception

Analyzing your target audience's reactions to marketing efforts

When launching a new product or coming up with a new idea

When seeking to evaluate buyers' purchasing patterns

• Qualitative research methods vs. quantitative research methods

Qualitative research examines people's ideas and what influences their perception, whereas quantitative research draws conclusions based on numbers and measurements.

Qualitative research is descriptive, and its primary goal is to comprehensively understand people's attitudes, behaviors, and ideas.

In contrast, quantitative research is more restrictive because it relies on numerical data and analyzes statistical data to make decisions. This research method assists researchers in gaining an initial grasp of the subject, which deals with numbers. For instance, the number of customers likely to purchase your products or use your services.

What is the most important feature of qualitative research?

A distinguishing feature of qualitative research is that it’s conducted in a real-world setting instead of a simulated environment. The researcher is examining actual phenomena instead of experimenting with different variables to see what outcomes (data) might result.

Can I use qualitative and quantitative approaches together in a study?

Yes, combining qualitative and quantitative research approaches happens all the time and is known as mixed methods research. For example, you could study individuals’ perceived risk in a certain scenario, such as how people rate the safety or riskiness of a given neighborhood. Simultaneously, you could analyze historical data objectively, indicating how safe or dangerous that area has been in the last year. To get the most out of mixed-method research, it’s important to understand the pros and cons of each methodology, so you can create a thoughtfully designed study that will yield compelling results.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 17 February 2024

Last updated: 19 November 2023

Last updated: 5 March 2024

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

Criteria for Good Qualitative Research: A Comprehensive Review

• Regular Article
• Open access
• Published: 18 September 2021
• Volume 31 , pages 679–689, ( 2022 )

Cite this article

You have full access to this open access article

• Drishti Yadav   ORCID: orcid.org/0000-0002-2974-0323 1  

71k Accesses

23 Citations

73 Altmetric

Explore all metrics

This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then, references of relevant articles were surveyed to find noteworthy, distinct, and well-defined pointers to good qualitative research. This review presents an investigative assessment of the pivotal features in qualitative research that can permit the readers to pass judgment on its quality and to condemn it as good research when objectively and adequately utilized. Overall, this review underlines the crux of qualitative research and accentuates the necessity to evaluate such research by the very tenets of its being. It also offers some prospects and recommendations to improve the quality of qualitative research. Based on the findings of this review, it is concluded that quality criteria are the aftereffect of socio-institutional procedures and existing paradigmatic conducts. Owing to the paradigmatic diversity of qualitative research, a single and specific set of quality criteria is neither feasible nor anticipated. Since qualitative research is not a cohesive discipline, researchers need to educate and familiarize themselves with applicable norms and decisive factors to evaluate qualitative research from within its theoretical and methodological framework of origin.

Similar content being viewed by others

What is Qualitative in Qualitative Research

Patrik Aspers & Ugo Corte

Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations

Gordon W. Cheung, Helena D. Cooper-Thomas, … Linda C. Wang

Sampling Techniques for Quantitative Research

Avoid common mistakes on your manuscript.

Introduction

“… It is important to regularly dialogue about what makes for good qualitative research” (Tracy, 2010 , p. 837)

To decide what represents good qualitative research is highly debatable. There are numerous methods that are contained within qualitative research and that are established on diverse philosophical perspectives. Bryman et al., ( 2008 , p. 262) suggest that “It is widely assumed that whereas quality criteria for quantitative research are well‐known and widely agreed, this is not the case for qualitative research.” Hence, the question “how to evaluate the quality of qualitative research” has been continuously debated. There are many areas of science and technology wherein these debates on the assessment of qualitative research have taken place. Examples include various areas of psychology: general psychology (Madill et al., 2000 ); counseling psychology (Morrow, 2005 ); and clinical psychology (Barker & Pistrang, 2005 ), and other disciplines of social sciences: social policy (Bryman et al., 2008 ); health research (Sparkes, 2001 ); business and management research (Johnson et al., 2006 ); information systems (Klein & Myers, 1999 ); and environmental studies (Reid & Gough, 2000 ). In the literature, these debates are enthused by the impression that the blanket application of criteria for good qualitative research developed around the positivist paradigm is improper. Such debates are based on the wide range of philosophical backgrounds within which qualitative research is conducted (e.g., Sandberg, 2000 ; Schwandt, 1996 ). The existence of methodological diversity led to the formulation of different sets of criteria applicable to qualitative research.

Among qualitative researchers, the dilemma of governing the measures to assess the quality of research is not a new phenomenon, especially when the virtuous triad of objectivity, reliability, and validity (Spencer et al., 2004 ) are not adequate. Occasionally, the criteria of quantitative research are used to evaluate qualitative research (Cohen & Crabtree, 2008 ; Lather, 2004 ). Indeed, Howe ( 2004 ) claims that the prevailing paradigm in educational research is scientifically based experimental research. Hypotheses and conjectures about the preeminence of quantitative research can weaken the worth and usefulness of qualitative research by neglecting the prominence of harmonizing match for purpose on research paradigm, the epistemological stance of the researcher, and the choice of methodology. Researchers have been reprimanded concerning this in “paradigmatic controversies, contradictions, and emerging confluences” (Lincoln & Guba, 2000 ).

In general, qualitative research tends to come from a very different paradigmatic stance and intrinsically demands distinctive and out-of-the-ordinary criteria for evaluating good research and varieties of research contributions that can be made. This review attempts to present a series of evaluative criteria for qualitative researchers, arguing that their choice of criteria needs to be compatible with the unique nature of the research in question (its methodology, aims, and assumptions). This review aims to assist researchers in identifying some of the indispensable features or markers of high-quality qualitative research. In a nutshell, the purpose of this systematic literature review is to analyze the existing knowledge on high-quality qualitative research and to verify the existence of research studies dealing with the critical assessment of qualitative research based on the concept of diverse paradigmatic stances. Contrary to the existing reviews, this review also suggests some critical directions to follow to improve the quality of qualitative research in different epistemological and ontological perspectives. This review is also intended to provide guidelines for the acceleration of future developments and dialogues among qualitative researchers in the context of assessing the qualitative research.

The rest of this review article is structured in the following fashion: Sect.  Methods describes the method followed for performing this review. Section Criteria for Evaluating Qualitative Studies provides a comprehensive description of the criteria for evaluating qualitative studies. This section is followed by a summary of the strategies to improve the quality of qualitative research in Sect.  Improving Quality: Strategies . Section  How to Assess the Quality of the Research Findings? provides details on how to assess the quality of the research findings. After that, some of the quality checklists (as tools to evaluate quality) are discussed in Sect.  Quality Checklists: Tools for Assessing the Quality . At last, the review ends with the concluding remarks presented in Sect.  Conclusions, Future Directions and Outlook . Some prospects in qualitative research for enhancing its quality and usefulness in the social and techno-scientific research community are also presented in Sect.  Conclusions, Future Directions and Outlook .

For this review, a comprehensive literature search was performed from many databases using generic search terms such as Qualitative Research , Criteria , etc . The following databases were chosen for the literature search based on the high number of results: IEEE Explore, ScienceDirect, PubMed, Google Scholar, and Web of Science. The following keywords (and their combinations using Boolean connectives OR/AND) were adopted for the literature search: qualitative research, criteria, quality, assessment, and validity. The synonyms for these keywords were collected and arranged in a logical structure (see Table 1 ). All publications in journals and conference proceedings later than 1950 till 2021 were considered for the search. Other articles extracted from the references of the papers identified in the electronic search were also included. A large number of publications on qualitative research were retrieved during the initial screening. Hence, to include the searches with the main focus on criteria for good qualitative research, an inclusion criterion was utilized in the search string.

From the selected databases, the search retrieved a total of 765 publications. Then, the duplicate records were removed. After that, based on the title and abstract, the remaining 426 publications were screened for their relevance by using the following inclusion and exclusion criteria (see Table 2 ). Publications focusing on evaluation criteria for good qualitative research were included, whereas those works which delivered theoretical concepts on qualitative research were excluded. Based on the screening and eligibility, 45 research articles were identified that offered explicit criteria for evaluating the quality of qualitative research and were found to be relevant to this review.

Figure  1 illustrates the complete review process in the form of PRISMA flow diagram. PRISMA, i.e., “preferred reporting items for systematic reviews and meta-analyses” is employed in systematic reviews to refine the quality of reporting.

PRISMA flow diagram illustrating the search and inclusion process. N represents the number of records

Criteria for Evaluating Qualitative Studies

Fundamental criteria: general research quality.

Various researchers have put forward criteria for evaluating qualitative research, which have been summarized in Table 3 . Also, the criteria outlined in Table 4 effectively deliver the various approaches to evaluate and assess the quality of qualitative work. The entries in Table 4 are based on Tracy’s “Eight big‐tent criteria for excellent qualitative research” (Tracy, 2010 ). Tracy argues that high-quality qualitative work should formulate criteria focusing on the worthiness, relevance, timeliness, significance, morality, and practicality of the research topic, and the ethical stance of the research itself. Researchers have also suggested a series of questions as guiding principles to assess the quality of a qualitative study (Mays & Pope, 2020 ). Nassaji ( 2020 ) argues that good qualitative research should be robust, well informed, and thoroughly documented.

Qualitative Research: Interpretive Paradigms

All qualitative researchers follow highly abstract principles which bring together beliefs about ontology, epistemology, and methodology. These beliefs govern how the researcher perceives and acts. The net, which encompasses the researcher’s epistemological, ontological, and methodological premises, is referred to as a paradigm, or an interpretive structure, a “Basic set of beliefs that guides action” (Guba, 1990 ). Four major interpretive paradigms structure the qualitative research: positivist and postpositivist, constructivist interpretive, critical (Marxist, emancipatory), and feminist poststructural. The complexity of these four abstract paradigms increases at the level of concrete, specific interpretive communities. Table 5 presents these paradigms and their assumptions, including their criteria for evaluating research, and the typical form that an interpretive or theoretical statement assumes in each paradigm. Moreover, for evaluating qualitative research, quantitative conceptualizations of reliability and validity are proven to be incompatible (Horsburgh, 2003 ). In addition, a series of questions have been put forward in the literature to assist a reviewer (who is proficient in qualitative methods) for meticulous assessment and endorsement of qualitative research (Morse, 2003 ). Hammersley ( 2007 ) also suggests that guiding principles for qualitative research are advantageous, but methodological pluralism should not be simply acknowledged for all qualitative approaches. Seale ( 1999 ) also points out the significance of methodological cognizance in research studies.

Table 5 reflects that criteria for assessing the quality of qualitative research are the aftermath of socio-institutional practices and existing paradigmatic standpoints. Owing to the paradigmatic diversity of qualitative research, a single set of quality criteria is neither possible nor desirable. Hence, the researchers must be reflexive about the criteria they use in the various roles they play within their research community.

Improving Quality: Strategies

Another critical question is “How can the qualitative researchers ensure that the abovementioned quality criteria can be met?” Lincoln and Guba ( 1986 ) delineated several strategies to intensify each criteria of trustworthiness. Other researchers (Merriam & Tisdell, 2016 ; Shenton, 2004 ) also presented such strategies. A brief description of these strategies is shown in Table 6 .

It is worth mentioning that generalizability is also an integral part of qualitative research (Hays & McKibben, 2021 ). In general, the guiding principle pertaining to generalizability speaks about inducing and comprehending knowledge to synthesize interpretive components of an underlying context. Table 7 summarizes the main metasynthesis steps required to ascertain generalizability in qualitative research.

Figure  2 reflects the crucial components of a conceptual framework and their contribution to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice (Johnson et al., 2020 ). The synergy and interrelationship of these components signifies their role to different stances of a qualitative research study.

Essential elements of a conceptual framework

In a nutshell, to assess the rationale of a study, its conceptual framework and research question(s), quality criteria must take account of the following: lucid context for the problem statement in the introduction; well-articulated research problems and questions; precise conceptual framework; distinct research purpose; and clear presentation and investigation of the paradigms. These criteria would expedite the quality of qualitative research.

How to Assess the Quality of the Research Findings?

The inclusion of quotes or similar research data enhances the confirmability in the write-up of the findings. The use of expressions (for instance, “80% of all respondents agreed that” or “only one of the interviewees mentioned that”) may also quantify qualitative findings (Stenfors et al., 2020 ). On the other hand, the persuasive reason for “why this may not help in intensifying the research” has also been provided (Monrouxe & Rees, 2020 ). Further, the Discussion and Conclusion sections of an article also prove robust markers of high-quality qualitative research, as elucidated in Table 8 .

Quality Checklists: Tools for Assessing the Quality

Numerous checklists are available to speed up the assessment of the quality of qualitative research. However, if used uncritically and recklessly concerning the research context, these checklists may be counterproductive. I recommend that such lists and guiding principles may assist in pinpointing the markers of high-quality qualitative research. However, considering enormous variations in the authors’ theoretical and philosophical contexts, I would emphasize that high dependability on such checklists may say little about whether the findings can be applied in your setting. A combination of such checklists might be appropriate for novice researchers. Some of these checklists are listed below:

The most commonly used framework is Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ). This framework is recommended by some journals to be followed by the authors during article submission.

Standards for Reporting Qualitative Research (SRQR) is another checklist that has been created particularly for medical education (O’Brien et al., 2014 ).

Also, Tracy ( 2010 ) and Critical Appraisal Skills Programme (CASP, 2021 ) offer criteria for qualitative research relevant across methods and approaches.

Further, researchers have also outlined different criteria as hallmarks of high-quality qualitative research. For instance, the “Road Trip Checklist” (Epp & Otnes, 2021 ) provides a quick reference to specific questions to address different elements of high-quality qualitative research.

Conclusions, Future Directions, and Outlook

This work presents a broad review of the criteria for good qualitative research. In addition, this article presents an exploratory analysis of the essential elements in qualitative research that can enable the readers of qualitative work to judge it as good research when objectively and adequately utilized. In this review, some of the essential markers that indicate high-quality qualitative research have been highlighted. I scope them narrowly to achieve rigor in qualitative research and note that they do not completely cover the broader considerations necessary for high-quality research. This review points out that a universal and versatile one-size-fits-all guideline for evaluating the quality of qualitative research does not exist. In other words, this review also emphasizes the non-existence of a set of common guidelines among qualitative researchers. In unison, this review reinforces that each qualitative approach should be treated uniquely on account of its own distinctive features for different epistemological and disciplinary positions. Owing to the sensitivity of the worth of qualitative research towards the specific context and the type of paradigmatic stance, researchers should themselves analyze what approaches can be and must be tailored to ensemble the distinct characteristics of the phenomenon under investigation. Although this article does not assert to put forward a magic bullet and to provide a one-stop solution for dealing with dilemmas about how, why, or whether to evaluate the “goodness” of qualitative research, it offers a platform to assist the researchers in improving their qualitative studies. This work provides an assembly of concerns to reflect on, a series of questions to ask, and multiple sets of criteria to look at, when attempting to determine the quality of qualitative research. Overall, this review underlines the crux of qualitative research and accentuates the need to evaluate such research by the very tenets of its being. Bringing together the vital arguments and delineating the requirements that good qualitative research should satisfy, this review strives to equip the researchers as well as reviewers to make well-versed judgment about the worth and significance of the qualitative research under scrutiny. In a nutshell, a comprehensive portrayal of the research process (from the context of research to the research objectives, research questions and design, speculative foundations, and from approaches of collecting data to analyzing the results, to deriving inferences) frequently proliferates the quality of a qualitative research.

Prospects : A Road Ahead for Qualitative Research

Irrefutably, qualitative research is a vivacious and evolving discipline wherein different epistemological and disciplinary positions have their own characteristics and importance. In addition, not surprisingly, owing to the sprouting and varied features of qualitative research, no consensus has been pulled off till date. Researchers have reflected various concerns and proposed several recommendations for editors and reviewers on conducting reviews of critical qualitative research (Levitt et al., 2021 ; McGinley et al., 2021 ). Following are some prospects and a few recommendations put forward towards the maturation of qualitative research and its quality evaluation:

In general, most of the manuscript and grant reviewers are not qualitative experts. Hence, it is more likely that they would prefer to adopt a broad set of criteria. However, researchers and reviewers need to keep in mind that it is inappropriate to utilize the same approaches and conducts among all qualitative research. Therefore, future work needs to focus on educating researchers and reviewers about the criteria to evaluate qualitative research from within the suitable theoretical and methodological context.

There is an urgent need to refurbish and augment critical assessment of some well-known and widely accepted tools (including checklists such as COREQ, SRQR) to interrogate their applicability on different aspects (along with their epistemological ramifications).

Efforts should be made towards creating more space for creativity, experimentation, and a dialogue between the diverse traditions of qualitative research. This would potentially help to avoid the enforcement of one's own set of quality criteria on the work carried out by others.

Moreover, journal reviewers need to be aware of various methodological practices and philosophical debates.

It is pivotal to highlight the expressions and considerations of qualitative researchers and bring them into a more open and transparent dialogue about assessing qualitative research in techno-scientific, academic, sociocultural, and political rooms.

Frequent debates on the use of evaluative criteria are required to solve some potentially resolved issues (including the applicability of a single set of criteria in multi-disciplinary aspects). Such debates would not only benefit the group of qualitative researchers themselves, but primarily assist in augmenting the well-being and vivacity of the entire discipline.

To conclude, I speculate that the criteria, and my perspective, may transfer to other methods, approaches, and contexts. I hope that they spark dialog and debate – about criteria for excellent qualitative research and the underpinnings of the discipline more broadly – and, therefore, help improve the quality of a qualitative study. Further, I anticipate that this review will assist the researchers to contemplate on the quality of their own research, to substantiate research design and help the reviewers to review qualitative research for journals. On a final note, I pinpoint the need to formulate a framework (encompassing the prerequisites of a qualitative study) by the cohesive efforts of qualitative researchers of different disciplines with different theoretic-paradigmatic origins. I believe that tailoring such a framework (of guiding principles) paves the way for qualitative researchers to consolidate the status of qualitative research in the wide-ranging open science debate. Dialogue on this issue across different approaches is crucial for the impending prospects of socio-techno-educational research.

Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16 (10), 1472–1482.

Article   Google Scholar  

Barker, C., & Pistrang, N. (2005). Quality criteria under methodological pluralism: Implications for conducting and evaluating research. American Journal of Community Psychology, 35 (3–4), 201–212.

Bryman, A., Becker, S., & Sempik, J. (2008). Quality criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal of Social Research Methodology, 11 (4), 261–276.

Caelli, K., Ray, L., & Mill, J. (2003). ‘Clear as mud’: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2 (2), 1–13.

CASP (2021). CASP checklists. Retrieved May 2021 from https://casp-uk.net/casp-tools-checklists/

Cohen, D. J., & Crabtree, B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. The Annals of Family Medicine, 6 (4), 331–339.

Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (pp. 1–32). Sage Publications Ltd.

Google Scholar  

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38 (3), 215–229.

Epp, A. M., & Otnes, C. C. (2021). High-quality qualitative research: Getting into gear. Journal of Service Research . https://doi.org/10.1177/1094670520961445

Guba, E. G. (1990). The paradigm dialog. In Alternative paradigms conference, mar, 1989, Indiana u, school of education, San Francisco, ca, us . Sage Publications, Inc.

Hammersley, M. (2007). The issue of quality in qualitative research. International Journal of Research and Method in Education, 30 (3), 287–305.

Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., & Mokkink, L. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19 , 1609406920976417.

Hays, D. G., & McKibben, W. B. (2021). Promoting rigorous research: Generalizability and qualitative research. Journal of Counseling and Development, 99 (2), 178–188.

Horsburgh, D. (2003). Evaluation of qualitative research. Journal of Clinical Nursing, 12 (2), 307–312.

Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10 (1), 42–46.

Johnson, J. L., Adkins, D., & Chauvin, S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84 (1), 7120.

Johnson, P., Buehring, A., Cassell, C., & Symon, G. (2006). Evaluating qualitative management research: Towards a contingent criteriology. International Journal of Management Reviews, 8 (3), 131–156.

Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23 (1), 67–93.

Lather, P. (2004). This is your father’s paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10 (1), 15–34.

Levitt, H. M., Morrill, Z., Collins, K. M., & Rizo, J. L. (2021). The methodological integrity of critical qualitative research: Principles to support design and research review. Journal of Counseling Psychology, 68 (3), 357.

Lincoln, Y. S., & Guba, E. G. (1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation, 1986 (30), 73–84.

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163–188). Sage Publications.

Madill, A., Jordan, A., & Shirley, C. (2000). Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies. British Journal of Psychology, 91 (1), 1–20.

Mays, N., & Pope, C. (2020). Quality in qualitative research. Qualitative Research in Health Care . https://doi.org/10.1002/9781119410867.ch15

McGinley, S., Wei, W., Zhang, L., & Zheng, Y. (2021). The state of qualitative research in hospitality: A 5-year review 2014 to 2019. Cornell Hospitality Quarterly, 62 (1), 8–20.

Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and implementation. San Francisco, US.

Meyer, M., & Dykes, J. (2019). Criteria for rigor in visualization design study. IEEE Transactions on Visualization and Computer Graphics, 26 (1), 87–97.

Monrouxe, L. V., & Rees, C. E. (2020). When I say… quantification in qualitative research. Medical Education, 54 (3), 186–187.

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52 (2), 250.

Morse, J. M. (2003). A review committee’s guide for evaluating qualitative proposals. Qualitative Health Research, 13 (6), 833–851.

Nassaji, H. (2020). Good qualitative research. Language Teaching Research, 24 (4), 427–431.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89 (9), 1245–1251.

O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19 , 1609406919899220.

Reid, A., & Gough, S. (2000). Guidelines for reporting and evaluating qualitative research: What are the alternatives? Environmental Education Research, 6 (1), 59–91.

Rocco, T. S. (2010). Criteria for evaluating qualitative studies. Human Resource Development International . https://doi.org/10.1080/13678868.2010.501959

Sandberg, J. (2000). Understanding human competence at work: An interpretative approach. Academy of Management Journal, 43 (1), 9–25.

Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2 (1), 58–72.

Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5 (4), 465–478.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 (2), 63–75.

Sparkes, A. C. (2001). Myth 94: Qualitative health researchers will agree about validity. Qualitative Health Research, 11 (4), 538–552.

Spencer, L., Ritchie, J., Lewis, J., & Dillon, L. (2004). Quality in qualitative evaluation: A framework for assessing research evidence.

Stenfors, T., Kajamaa, A., & Bennett, D. (2020). How to assess the quality of qualitative research. The Clinical Teacher, 17 (6), 596–599.

Taylor, E. W., Beck, J., & Ainsworth, E. (2001). Publishing qualitative adult education research: A peer review perspective. Studies in the Education of Adults, 33 (2), 163–179.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19 (6), 349–357.

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16 (10), 837–851.

Download references

Open access funding provided by TU Wien (TUW).

Author information

Authors and affiliations.

Faculty of Informatics, Technische Universität Wien, 1040, Vienna, Austria

Drishti Yadav

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Drishti Yadav .

Ethics declarations

Conflict of interest.

The author declares no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Yadav, D. Criteria for Good Qualitative Research: A Comprehensive Review. Asia-Pacific Edu Res 31 , 679–689 (2022). https://doi.org/10.1007/s40299-021-00619-0

Download citation

Accepted : 28 August 2021

Published : 18 September 2021

Issue Date : December 2022

DOI : https://doi.org/10.1007/s40299-021-00619-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Evaluative criteria
• Find a journal
• Publish with us
• Track your research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on 4 April 2022 by Pritha Bhandari . Revised on 30 January 2023.

Qualitative research involves collecting and analysing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analysing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, and history.

• How does social media shape body image in teenagers?
• How do children and adults interpret healthy eating in the UK?
• What factors influence employee retention in a large organisation?
• How is anxiety experienced around the world?
• How can teachers integrate social issues into science curriculums?

Table of contents

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography, action research, phenomenological research, and narrative research. They share some similarities, but emphasise different aims and perspectives.

Prevent plagiarism, run a free check.

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

• Observations: recording what you have seen, heard, or encountered in detailed field notes.
• Interviews:  personally asking people questions in one-on-one conversations.
• Focus groups: asking questions and generating discussion among a group of people.
• Surveys : distributing questionnaires with open-ended questions.
• Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
• You take field notes with observations and reflect on your own experiences of the company culture.
• You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
• You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves ‘instruments’ in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analysing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

• Prepare and organise your data. This may mean transcribing interviews or typing up fieldnotes.
• Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
• Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorise your data.
• Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
• Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analysing qualitative data. Although these methods share similar processes, they emphasise different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

• Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

• Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

• Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

• Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Researchers must consider practical and theoretical limitations in analysing and interpreting their data. Qualitative research suffers from:

• Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

• Subjectivity

Due to the researcher’s primary role in analysing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

• Limited generalisability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalisable conclusions because the data may be biased and unrepresentative of the wider population .

• Labour-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organisation to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organisations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organise your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Bhandari, P. (2023, January 30). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved 20 March 2024, from https://www.scribbr.co.uk/research-methods/introduction-to-qualitative-research/

Is this article helpful?

Pritha Bhandari

• Privacy Policy

Buy Me a Coffee

Home » Research Paper – Structure, Examples and Writing Guide

Research Paper – Structure, Examples and Writing Guide

Table of Contents

Research Paper

Definition:

Research Paper is a written document that presents the author’s original research, analysis, and interpretation of a specific topic or issue.

It is typically based on Empirical Evidence, and may involve qualitative or quantitative research methods, or a combination of both. The purpose of a research paper is to contribute new knowledge or insights to a particular field of study, and to demonstrate the author’s understanding of the existing literature and theories related to the topic.

Structure of Research Paper

The structure of a research paper typically follows a standard format, consisting of several sections that convey specific information about the research study. The following is a detailed explanation of the structure of a research paper:

The title page contains the title of the paper, the name(s) of the author(s), and the affiliation(s) of the author(s). It also includes the date of submission and possibly, the name of the journal or conference where the paper is to be published.

The abstract is a brief summary of the research paper, typically ranging from 100 to 250 words. It should include the research question, the methods used, the key findings, and the implications of the results. The abstract should be written in a concise and clear manner to allow readers to quickly grasp the essence of the research.

Introduction

The introduction section of a research paper provides background information about the research problem, the research question, and the research objectives. It also outlines the significance of the research, the research gap that it aims to fill, and the approach taken to address the research question. Finally, the introduction section ends with a clear statement of the research hypothesis or research question.

Literature Review

The literature review section of a research paper provides an overview of the existing literature on the topic of study. It includes a critical analysis and synthesis of the literature, highlighting the key concepts, themes, and debates. The literature review should also demonstrate the research gap and how the current study seeks to address it.

The methods section of a research paper describes the research design, the sample selection, the data collection and analysis procedures, and the statistical methods used to analyze the data. This section should provide sufficient detail for other researchers to replicate the study.

The results section presents the findings of the research, using tables, graphs, and figures to illustrate the data. The findings should be presented in a clear and concise manner, with reference to the research question and hypothesis.

The discussion section of a research paper interprets the findings and discusses their implications for the research question, the literature review, and the field of study. It should also address the limitations of the study and suggest future research directions.

The conclusion section summarizes the main findings of the study, restates the research question and hypothesis, and provides a final reflection on the significance of the research.

The references section provides a list of all the sources cited in the paper, following a specific citation style such as APA, MLA or Chicago.

How to Write Research Paper

You can write Research Paper by the following guide:

• Choose a Topic: The first step is to select a topic that interests you and is relevant to your field of study. Brainstorm ideas and narrow down to a research question that is specific and researchable.
• Conduct a Literature Review: The literature review helps you identify the gap in the existing research and provides a basis for your research question. It also helps you to develop a theoretical framework and research hypothesis.
• Develop a Thesis Statement : The thesis statement is the main argument of your research paper. It should be clear, concise and specific to your research question.
• Plan your Research: Develop a research plan that outlines the methods, data sources, and data analysis procedures. This will help you to collect and analyze data effectively.
• Collect and Analyze Data: Collect data using various methods such as surveys, interviews, observations, or experiments. Analyze data using statistical tools or other qualitative methods.
• Organize your Paper : Organize your paper into sections such as Introduction, Literature Review, Methods, Results, Discussion, and Conclusion. Ensure that each section is coherent and follows a logical flow.
• Write your Paper : Start by writing the introduction, followed by the literature review, methods, results, discussion, and conclusion. Ensure that your writing is clear, concise, and follows the required formatting and citation styles.
• Edit and Proofread your Paper: Review your paper for grammar and spelling errors, and ensure that it is well-structured and easy to read. Ask someone else to review your paper to get feedback and suggestions for improvement.
• Cite your Sources: Ensure that you properly cite all sources used in your research paper. This is essential for giving credit to the original authors and avoiding plagiarism.

Research Paper Example

Note : The below example research paper is for illustrative purposes only and is not an actual research paper. Actual research papers may have different structures, contents, and formats depending on the field of study, research question, data collection and analysis methods, and other factors. Students should always consult with their professors or supervisors for specific guidelines and expectations for their research papers.

Research Paper Example sample for Students:

Title: The Impact of Social Media on Mental Health among Young Adults

Abstract: This study aims to investigate the impact of social media use on the mental health of young adults. A literature review was conducted to examine the existing research on the topic. A survey was then administered to 200 university students to collect data on their social media use, mental health status, and perceived impact of social media on their mental health. The results showed that social media use is positively associated with depression, anxiety, and stress. The study also found that social comparison, cyberbullying, and FOMO (Fear of Missing Out) are significant predictors of mental health problems among young adults.

Introduction: Social media has become an integral part of modern life, particularly among young adults. While social media has many benefits, including increased communication and social connectivity, it has also been associated with negative outcomes, such as addiction, cyberbullying, and mental health problems. This study aims to investigate the impact of social media use on the mental health of young adults.

Literature Review: The literature review highlights the existing research on the impact of social media use on mental health. The review shows that social media use is associated with depression, anxiety, stress, and other mental health problems. The review also identifies the factors that contribute to the negative impact of social media, including social comparison, cyberbullying, and FOMO.

Methods : A survey was administered to 200 university students to collect data on their social media use, mental health status, and perceived impact of social media on their mental health. The survey included questions on social media use, mental health status (measured using the DASS-21), and perceived impact of social media on their mental health. Data were analyzed using descriptive statistics and regression analysis.

Results : The results showed that social media use is positively associated with depression, anxiety, and stress. The study also found that social comparison, cyberbullying, and FOMO are significant predictors of mental health problems among young adults.

Discussion : The study’s findings suggest that social media use has a negative impact on the mental health of young adults. The study highlights the need for interventions that address the factors contributing to the negative impact of social media, such as social comparison, cyberbullying, and FOMO.

Conclusion : In conclusion, social media use has a significant impact on the mental health of young adults. The study’s findings underscore the need for interventions that promote healthy social media use and address the negative outcomes associated with social media use. Future research can explore the effectiveness of interventions aimed at reducing the negative impact of social media on mental health. Additionally, longitudinal studies can investigate the long-term effects of social media use on mental health.

Limitations : The study has some limitations, including the use of self-report measures and a cross-sectional design. The use of self-report measures may result in biased responses, and a cross-sectional design limits the ability to establish causality.

Implications: The study’s findings have implications for mental health professionals, educators, and policymakers. Mental health professionals can use the findings to develop interventions that address the negative impact of social media use on mental health. Educators can incorporate social media literacy into their curriculum to promote healthy social media use among young adults. Policymakers can use the findings to develop policies that protect young adults from the negative outcomes associated with social media use.

References :

• Twenge, J. M., & Campbell, W. K. (2019). Associations between screen time and lower psychological well-being among children and adolescents: Evidence from a population-based study. Preventive medicine reports, 15, 100918.
• Primack, B. A., Shensa, A., Escobar-Viera, C. G., Barrett, E. L., Sidani, J. E., Colditz, J. B., … & James, A. E. (2017). Use of multiple social media platforms and symptoms of depression and anxiety: A nationally-representative study among US young adults. Computers in Human Behavior, 69, 1-9.
• Van der Meer, T. G., & Verhoeven, J. W. (2017). Social media and its impact on academic performance of students. Journal of Information Technology Education: Research, 16, 383-398.

Appendix : The survey used in this study is provided below.

Social Media and Mental Health Survey

• How often do you use social media per day?
• Less than 30 minutes
• 30 minutes to 1 hour
• 1 to 2 hours
• 2 to 4 hours
• More than 4 hours
• Which social media platforms do you use?
• Others (Please specify)
• How often do you experience the following on social media?
• Social comparison (comparing yourself to others)
• Cyberbullying
• Fear of Missing Out (FOMO)
• Have you ever experienced any of the following mental health problems in the past month?
• Do you think social media use has a positive or negative impact on your mental health?
• Very positive
• Somewhat positive
• Somewhat negative
• Very negative
• In your opinion, which factors contribute to the negative impact of social media on mental health?
• Social comparison
• In your opinion, what interventions could be effective in reducing the negative impact of social media on mental health?
• Education on healthy social media use
• Counseling for mental health problems caused by social media
• Social media detox programs
• Regulation of social media use

Thank you for your participation!

Applications of Research Paper

Research papers have several applications in various fields, including:

• Advancing knowledge: Research papers contribute to the advancement of knowledge by generating new insights, theories, and findings that can inform future research and practice. They help to answer important questions, clarify existing knowledge, and identify areas that require further investigation.
• Informing policy: Research papers can inform policy decisions by providing evidence-based recommendations for policymakers. They can help to identify gaps in current policies, evaluate the effectiveness of interventions, and inform the development of new policies and regulations.
• Improving practice: Research papers can improve practice by providing evidence-based guidance for professionals in various fields, including medicine, education, business, and psychology. They can inform the development of best practices, guidelines, and standards of care that can improve outcomes for individuals and organizations.
• Educating students : Research papers are often used as teaching tools in universities and colleges to educate students about research methods, data analysis, and academic writing. They help students to develop critical thinking skills, research skills, and communication skills that are essential for success in many careers.
• Fostering collaboration: Research papers can foster collaboration among researchers, practitioners, and policymakers by providing a platform for sharing knowledge and ideas. They can facilitate interdisciplinary collaborations and partnerships that can lead to innovative solutions to complex problems.

When to Write Research Paper

Research papers are typically written when a person has completed a research project or when they have conducted a study and have obtained data or findings that they want to share with the academic or professional community. Research papers are usually written in academic settings, such as universities, but they can also be written in professional settings, such as research organizations, government agencies, or private companies.

Here are some common situations where a person might need to write a research paper:

• For academic purposes: Students in universities and colleges are often required to write research papers as part of their coursework, particularly in the social sciences, natural sciences, and humanities. Writing research papers helps students to develop research skills, critical thinking skills, and academic writing skills.
• For publication: Researchers often write research papers to publish their findings in academic journals or to present their work at academic conferences. Publishing research papers is an important way to disseminate research findings to the academic community and to establish oneself as an expert in a particular field.
• To inform policy or practice : Researchers may write research papers to inform policy decisions or to improve practice in various fields. Research findings can be used to inform the development of policies, guidelines, and best practices that can improve outcomes for individuals and organizations.
• To share new insights or ideas: Researchers may write research papers to share new insights or ideas with the academic or professional community. They may present new theories, propose new research methods, or challenge existing paradigms in their field.

Purpose of Research Paper

The purpose of a research paper is to present the results of a study or investigation in a clear, concise, and structured manner. Research papers are written to communicate new knowledge, ideas, or findings to a specific audience, such as researchers, scholars, practitioners, or policymakers. The primary purposes of a research paper are:

• To contribute to the body of knowledge : Research papers aim to add new knowledge or insights to a particular field or discipline. They do this by reporting the results of empirical studies, reviewing and synthesizing existing literature, proposing new theories, or providing new perspectives on a topic.
• To inform or persuade: Research papers are written to inform or persuade the reader about a particular issue, topic, or phenomenon. They present evidence and arguments to support their claims and seek to persuade the reader of the validity of their findings or recommendations.
• To advance the field: Research papers seek to advance the field or discipline by identifying gaps in knowledge, proposing new research questions or approaches, or challenging existing assumptions or paradigms. They aim to contribute to ongoing debates and discussions within a field and to stimulate further research and inquiry.
• To demonstrate research skills: Research papers demonstrate the author’s research skills, including their ability to design and conduct a study, collect and analyze data, and interpret and communicate findings. They also demonstrate the author’s ability to critically evaluate existing literature, synthesize information from multiple sources, and write in a clear and structured manner.

Characteristics of Research Paper

Research papers have several characteristics that distinguish them from other forms of academic or professional writing. Here are some common characteristics of research papers:

• Evidence-based: Research papers are based on empirical evidence, which is collected through rigorous research methods such as experiments, surveys, observations, or interviews. They rely on objective data and facts to support their claims and conclusions.
• Structured and organized: Research papers have a clear and logical structure, with sections such as introduction, literature review, methods, results, discussion, and conclusion. They are organized in a way that helps the reader to follow the argument and understand the findings.
• Formal and objective: Research papers are written in a formal and objective tone, with an emphasis on clarity, precision, and accuracy. They avoid subjective language or personal opinions and instead rely on objective data and analysis to support their arguments.
• Citations and references: Research papers include citations and references to acknowledge the sources of information and ideas used in the paper. They use a specific citation style, such as APA, MLA, or Chicago, to ensure consistency and accuracy.
• Peer-reviewed: Research papers are often peer-reviewed, which means they are evaluated by other experts in the field before they are published. Peer-review ensures that the research is of high quality, meets ethical standards, and contributes to the advancement of knowledge in the field.
• Objective and unbiased: Research papers strive to be objective and unbiased in their presentation of the findings. They avoid personal biases or preconceptions and instead rely on the data and analysis to draw conclusions.

Advantages of Research Paper

Research papers have many advantages, both for the individual researcher and for the broader academic and professional community. Here are some advantages of research papers:

• Contribution to knowledge: Research papers contribute to the body of knowledge in a particular field or discipline. They add new information, insights, and perspectives to existing literature and help advance the understanding of a particular phenomenon or issue.
• Opportunity for intellectual growth: Research papers provide an opportunity for intellectual growth for the researcher. They require critical thinking, problem-solving, and creativity, which can help develop the researcher’s skills and knowledge.
• Career advancement: Research papers can help advance the researcher’s career by demonstrating their expertise and contributions to the field. They can also lead to new research opportunities, collaborations, and funding.
• Academic recognition: Research papers can lead to academic recognition in the form of awards, grants, or invitations to speak at conferences or events. They can also contribute to the researcher’s reputation and standing in the field.
• Impact on policy and practice: Research papers can have a significant impact on policy and practice. They can inform policy decisions, guide practice, and lead to changes in laws, regulations, or procedures.
• Advancement of society: Research papers can contribute to the advancement of society by addressing important issues, identifying solutions to problems, and promoting social justice and equality.

Limitations of Research Paper

Research papers also have some limitations that should be considered when interpreting their findings or implications. Here are some common limitations of research papers:

• Limited generalizability: Research findings may not be generalizable to other populations, settings, or contexts. Studies often use specific samples or conditions that may not reflect the broader population or real-world situations.
• Potential for bias : Research papers may be biased due to factors such as sample selection, measurement errors, or researcher biases. It is important to evaluate the quality of the research design and methods used to ensure that the findings are valid and reliable.
• Ethical concerns: Research papers may raise ethical concerns, such as the use of vulnerable populations or invasive procedures. Researchers must adhere to ethical guidelines and obtain informed consent from participants to ensure that the research is conducted in a responsible and respectful manner.
• Limitations of methodology: Research papers may be limited by the methodology used to collect and analyze data. For example, certain research methods may not capture the complexity or nuance of a particular phenomenon, or may not be appropriate for certain research questions.
• Publication bias: Research papers may be subject to publication bias, where positive or significant findings are more likely to be published than negative or non-significant findings. This can skew the overall findings of a particular area of research.
• Time and resource constraints: Research papers may be limited by time and resource constraints, which can affect the quality and scope of the research. Researchers may not have access to certain data or resources, or may be unable to conduct long-term studies due to practical limitations.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Research Paper Conclusion – Writing Guide and...

Appendices – Writing Guide, Types and Examples

How to Cite Research Paper – All Formats and...

Research Report – Example, Writing Guide and...

Delimitations in Research – Types, Examples and...

Scope of the Research – Writing Guide and...

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved March 22, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Pediatr Psychol

Commentary: Writing and Evaluating Qualitative Research Reports

Yelena p. wu.

1 Division of Public Health, Department of Family and Preventive Medicine, University of Utah,

2 Cancer Control and Population Sciences, Huntsman Cancer Institute,

Deborah Thompson

3 Department of Pediatrics-Nutrition, USDA/ARS Children’s Nutrition Research Center, Baylor College of Medicine,

Karen J. Aroian

4 College of Nursing, University of Central Florida,

Elizabeth L. McQuaid

5 Department of Psychiatry and Human Behavior, Brown University, and

Janet A. Deatrick

6 School of Nursing, University of Pennsylvania

Objective  To provide an overview of qualitative methods, particularly for reviewers and authors who may be less familiar with qualitative research. Methods  A question and answer format is used to address considerations for writing and evaluating qualitative research. Results and Conclusions  When producing qualitative research, individuals are encouraged to address the qualitative research considerations raised and to explicitly identify the systematic strategies used to ensure rigor in study design and methods, analysis, and presentation of findings. Increasing capacity for review and publication of qualitative research within pediatric psychology will advance the field’s ability to gain a better understanding of the specific needs of pediatric populations, tailor interventions more effectively, and promote optimal health.

The Journal of Pediatric Psychology (JPP) has a long history of emphasizing high-quality, methodologically rigorous research in social and behavioral aspects of children’s health ( Palermo, 2013 , 2014 ). Traditionally, research published in JPP has focused on quantitative methodologies. Qualitative approaches are of interest to pediatric psychologists given the important role of qualitative research in developing new theories ( Kelly & Ganong, 2011 ), illustrating important clinical themes ( Kars, Grypdonck, de Bock, & van Delden, 2015 ), developing new instruments ( Thompson, Bhatt, & Watson, 2013 ), understanding patients’ and families’ perspectives and needs ( Bevans, Gardner, Pajer, Riley, & Forrest, 2013 ; Lyons, Goodwin, McCreanor, & Griffin, 2015 ), and documenting new or rarely examined issues ( Haukeland, Fjermestad, Mossige, & Vatne, 2015 ; Valenzuela et al., 2011 ). Further, these methods are integral to intervention development ( Minges et al., 2015 ; Thompson et al., 2007 ) and understanding intervention outcomes ( de Visser et al., 2015 ; Hess & Straub, 2011 ). For example, when designing an intervention, qualitative research can identify patient and family preferences for and perspectives on desirable intervention characteristics and perceived needs ( Cassidy et al., 2013 ; Hess & Straub, 2011 ; Thompson, 2014 ), which may lead to a more targeted, effective intervention.

Both qualitative and quantitative approaches are concerned with issues such as generalizability of study findings (e.g., to whom the study findings can be applied) and rigor. However, qualitative and quantitative methods have different approaches to these issues. The purpose of qualitative research is to contribute knowledge or understanding by describing phenomenon within certain groups or populations of interest. As such, the purpose of qualitative research is not to provide generalizable findings. Instead, qualitative research has a discovery focus and often uses an iterative approach. Thus, qualitative work is often foundational to future qualitative, quantitative, or mixed-methods studies.

At the time of this writing, three of six current calls for papers for special issues of JPP specifically note that manuscripts incorporating qualitative approaches would be welcomed. Despite apparent openness to broadening JPP’s emphasis beyond its traditional quantitative approach, few published articles have used qualitative methods. For example, of 232 research articles published in JPP from 2012 to 2014 (excluding commentaries and reviews), only five used qualitative methods (2% of articles).

The goal of the current article is to present considerations for writing and evaluating qualitative research within the context of pediatric psychology to provide a framework for writing and reviewing manuscripts reporting qualitative findings. The current article may be especially useful to reviewers and authors who are less familiar with qualitative methods. The tenets presented here are grounded in the well-established literature on reporting and evaluating qualitative research, including guidelines and checklists ( Eakin & Mykhalovskiy, 2003 ; Elo et al., 2014 ; Mays & Pope, 2000 ; Tong, Sainsbury, & Craig, 2007 ). For example, the Consolidated Criteria for Reporting Qualitative Research checklist describes essential elements for reporting qualitative findings ( Tong et al., 2007 ). Although the considerations presented in the current manuscript have broad applicability to many fields, examples were purposively selected for the field of pediatric psychology.

Our goal is that this article will stimulate publication of more qualitative research in pediatric psychology and allied fields. More specifically, the goal is to encourage high-quality qualitative research by addressing key issues involved in conducting qualitative studies, and the process of conducting, reporting, and evaluating qualitative findings. Readers interested in more in-depth information on designing and implementing qualitative studies, relevant theoretical frameworks and approaches, and analytic approaches are referred to the well-developed literature in this area ( Clark, 2003 ; Corbin & Strauss, 2008 ; Creswell, 1994 ; Eakin & Mykhalovskiy, 2003 ; Elo et al., 2014 ; Mays & Pope, 2000 ; Miles, Huberman, & Saldaña, 2013 ; Ritchie & Lewis, 2003 ; Saldaña, 2012 ; Sandelowski, 1995 , 2010 ; Tong et al., 2007 ; Yin, 2015 ). Researchers new to qualitative research are also encouraged to obtain specialized training in qualitative methods and/or to collaborate with a qualitative expert in an effort to ensure rigor (i.e., validity).

We begin the article with a definition of qualitative research and an overview of the concept of rigor. While we recognize that qualitative methods comprise multiple and distinct approaches with unique purposes, we present an overview of considerations for writing and evaluating qualitative research that cut across qualitative methods. Specifically, we present basic principles in three broad areas: (1) study design and methods, (2) analytic considerations, and (3) presentation of findings (see Table 1 for a summary of the principles addressed in each area). Each area is addressed using a “question and answer” format. We present a brief explanation of each question, options for how one could address the issue raised, and a suggested recommendation. We recognize, however, that there are no absolute “right” or “wrong” answers and that the most “right” answer for each situation depends on the specific study and its purpose. In fact, our strongest recommendation is that authors of qualitative research manuscripts be explicit about their rationale for design, analytic choices, and strategies so that readers and reviewers can evaluate the rationale and rigor of the study methods.

Summary of Overarching Principles to Address in Qualitative Research Manuscripts

What Is Qualitative Research?

Qualitative methods are used across many areas of health research, including health psychology ( Gough & Deatrick, 2015 ), to study the meaning of people’s lives in their real-world roles, represent their views and perspectives, identify important contextual conditions, discover new or additional insights about existing social and behavioral concepts, and acknowledge the contribution of multiple perspectives ( Yin, 2015 ). Qualitative research is a family of approaches rather than a single approach. There are multiple and distinct qualitative methodologies or stances (e.g., constructivism, post-positivism, critical theory), each with different underlying ontological and epistemological assumptions ( Lincoln, Lynham, & Guba, 2011 ). However, certain features are common to most qualitative approaches and distinguish qualitative research from quantitative research ( Creswell, 1994 ).

Key to all qualitative methodologies is that multiple perspectives about a phenomenon of interest are essential, and that those perspectives are best inductively derived or discovered from people with personal experience regarding that phenomenon. These perspectives or definitions may differ from “conventional wisdom.” Thus, meanings need to be discovered from the population under study to ensure optimal understanding. For instance, in a recent qualitative study about texting while driving, adolescents said that they did not approve of texting while driving. The investigators, however, discovered that the respondents did not consider themselves driving while a vehicle was stopped at a red light. In other words, the respondents did approve of texting while stopped at a red light. In addition, the adolescents said that they highly valued being constantly connected via texting. Thus, what is meant by “driving” and the value of “being connected” need to be considered when approaching the issue of texting while driving with adolescents ( McDonald & Sommers, 2015 ).

Qualitative methods are also distinct from a mixed-method approach (i.e., integration of qualitative and quantitative approaches; Creswell, 2013b ). A mixed-methods study may include a first phase of quantitative data collection that provides results that inform a second phase of the study that includes qualitative data collection, or vice versa. A mixed-methods study may also include concurrent quantitative and qualitative data collection. The timing, priority, and stage of integration of the two approaches (quantitative and qualitative) are complex and vary depending on the research question; they also dictate how to attend to differing qualitative and quantitative principles ( Creswell et al., 2011 ). Understanding the basic tenets of qualitative research is preliminary to integrating qualitative research with another approach that has different tenets. A full discussion of the integration of qualitative and quantitative research approaches is beyond the scope of this article. Readers interested in the topic are referred to one of the many excellent resources on the topic ( Creswell, 2013b ).

What Are Typical Qualitative Research Questions?

Qualitative research questions are typically open-ended and are framed in the spirit of discovery and exploration and to address existing knowledge gaps. The current manuscript provides exemplar pediatric qualitative studies that illustrate key issues that arise when reporting and evaluating qualitative studies. Example research questions that are contained in the studies cited in the current manuscript are presented in Table 2 .

Example Qualitative Research Questions From the Pediatric Literature

What Are Rigor and Transparency in Qualitative Research?

There are several overarching principles with unique application in qualitative research, including definitions of scientific rigor and the importance of transparency. Quantitative research generally uses the terms reliability and validity to describe the rigor of research, while in qualitative research, rigor refers to the goal of seeking to understand the tacit knowledge of participants’ conception of reality ( Polanyi, 1958 ). For example, Haukeland and colleagues (2015) used qualitative analysis to identify themes describing the emotional experiences of a unique and understudied population—pediatric siblings of children with rare medical conditions such as Turner syndrome and Duchenne muscular dystrophy. Within this context, the authors’ rendering of the diverse and contradictory emotions experienced by siblings of children with these rare conditions represents “rigor” within a qualitative framework.

While debate exists regarding the terminology describing and strategies for strengthening scientific rigor in qualitative studies ( Guba, 1981 ; Morse, 2015a , 2015b ; Sandelowski, 1993a ; Whittemore, Chase, & Mandle, 2001 ), little debate exists regarding the importance of explaining strategies used to strengthen rigor. Such strategies should be appropriate for the specific study; therefore, it is wise to clearly describe what is relevant for each study. For example, in terms of strengthening credibility or the plausibility of data analysis and interpretation, prolonged engagement with participants is appropriate when conducting an observational study (e.g., observations of parent–child mealtime interactions; Hughes et al., 2011 ; Power et al., 2015 ). For an interview-only study, however, it would be more practical to strengthen credibility through other strategies (e.g., keeping detailed field notes about the interviews included in the analysis).

Dependability is the stability of a data analysis protocol. For instance, stepwise development of a coding system from an “a priori” list of codes based on the underlying conceptual framework or existing literature (e.g., creating initial codes for potential barriers to medication adherence based on prior studies) may be essential for analysis of data from semi-structured interviews using multiple coders. But this may not be the ideal strategy if the purpose is to inductively derive all possible coding categories directly from data in an area where little is known. For some research questions, the strategy may be to strengthen confirmability or to verify a specific phenomenon of interest using different sources of data before generating conclusions. This process, which is commonly referred to in the research literature as triangulation, may also include collecting different types of data (e.g., interview data, observational data), using multiple coders to incorporate different ways of interpreting the data, or using multiple theories ( Krefting, 1991 ; Ritchie & Lewis, 2003 ). Alternatively, another investigator may use triangulation to provide complementarity data ( Krefting, 1991 ) to garner additional information to deepen understanding. Because the purpose of qualitative research is to discover multiple perspectives about a phenomenon, it is not necessarily appropriate to attain concordance across studies or investigators when independently analyzing data. Some qualitative experts also believe that it is inappropriate to use triangulation to confirm findings, but this debate has not been resolved within the field ( Ritchie & Lewis, 2003 ; Tobin & Begley, 2004 ). More agreement exists, however, regarding the value of triangulation to complement, deepen, or expand understanding of a particular topic or issue ( Ritchie & Lewis, 2003 ). Finally, instead of basing a study on a sample that allows for generalizing statistical results to other populations, investigators in qualitative research studies are focused on designing a study and conveying the results so that the reader understands the transferability of the results. Strategies for transferability may include explanations of how the sample was selected and descriptive characteristics of study participants, which provides a context for the results and enables readers to decide if other samples share critical attributes. A study is deemed transferable if relevant contextual features are common to both the study sample and the larger population.

Strategies to enhance rigor should be used systematically across each phase of a study. That is, rigor needs to be identified, managed, and documented throughout the research process: during the preparation phase (data collection and sampling), organization phase (analysis and interpretation), and reporting phase (manuscript or final report; Elo et al., 2014 ). From this perspective, the strategies help strengthen the trustworthiness of the overall study (i.e., to what extent the study findings are worth heeding; Eakin & Mykhalovskiy, 2003 ; Lincoln & Guba, 1985 ).

A good example of managing and documenting rigor and trustworthiness can be found in a study of family treatment decisions for children with cancer ( Kelly & Ganong, 2011 ). The researchers describe how they promoted the rigor of the study and strengthening its credibility by triangulating data sources (e.g., obtaining data from children’s custodial parents, stepparents, etc.), debriefing (e.g., holding detailed conversations with colleagues about the data and interpretations of the data), member checking (i.e., presenting preliminary findings to participants to obtain their feedback and interpretation), and reviewing study procedure decisions and analytic procedures with a second party.

Transparency is another key concept in written reports of qualitative research. In other words, enough detail should be provided for the reader to understand what was done and why ( Ritchie & Lewis, 2003 ). Examples of information that should be included are a clear rationale for selecting a particular population or people with certain characteristics, the research question being investigated, and a meaningful explanation of why this research question was selected (i.e., the gap in knowledge or understanding that is being investigated; Ritchie & Lewis, 2003 ). Clearly describing recruitment, enrollment, data collection, and data analysis or extraction methods are equally important ( Dixon-Woods, Shaw, Agarwal, & Smith, 2004 ). Coherency among methods and transparency about research decisions adds to the robustness of qualitative research ( Tobin & Begley, 2004 ) and provides a context for understanding the findings and their implications.

Study Design and Methods

Is qualitative research hypothesis driven.

In contrast to quantitative research, qualitative research is not typically hypothesis driven ( Creswell, 1994 ; Ritchie & Lewis, 2003 ). A risk associated with using hypotheses in qualitative research is that the findings could be biased by the hypotheses. Alternatively, qualitative research is exploratory and typically guided by a research question or conceptual framework rather than hypotheses ( Creswell, 1994 ; Ritchie & Lewis, 2003 ). As previously stated, the goal of qualitative research is to increase understanding in areas where little is known by developing deeper insight into complex situations or processes. According to Richards and Morse (2013) , “If you know what you are likely to find, …  you should not be working qualitatively” (p. 28). Thus, we do not recommend that a hypothesis be stated in manuscripts presenting qualitative data.

What Is the Role of Theory in Qualitative Research?

Consistent with the exploratory nature of qualitative research, one particular qualitative method, grounded theory, is used specifically for discovering substantive theory (i.e., working theories of action or processes developed for a specific area of concern; Bryant & Charmaz, 2010 ; Glaser & Strauss, 1967 ). This method uses a series of structured steps to break down qualitative data into codes, organize the codes into conceptual categories, and link the categories into a theory that explains the phenomenon under study. For example, Kelly and Ganong (2011) used grounded theory methods to produce a substantive theory about how single and re-partnered parents (e.g., households with a step-parent) made treatment decisions for children with childhood cancer. The theory of decision making developed in this study included “moving to place,” which described the ways in which parents from different family structures (e.g., single and re-partnered parents) were involved in the child’s treatment decision-making. The resulting theory also delineated the causal conditions, context, and intervening factors that contributed to the strategies used for moving to place.

Theories may be used in other types of qualitative research as well, serving as the impetus or organizing framework for the study ( Sandelowski, 1993b ). For example, Izaguirre and Keefer (2014) used Social Cognitive Theory ( Bandura, 1986 ) to investigate self-efficacy among adolescents with inflammatory bowel disease. The impetus for selecting the theory was to inform the development of a self-efficacy measure for adolescent self-management. In another study on health care transition in youth with Type 1 Diabetes ( Pierce, Wysocki, & Aroian, 2016 ), the investigators adapted a social-ecological model—the Socio-ecological Model of Adolescent and Young Adult Transition Readiness (SMART) model ( Schwartz, Tuchman, Hobbie, & Ginsberg, 2011 )—to their study population ( Pierce & Wysocki, 2015 ). Pierce et al. (2016) are currently using the adapted SMART model to focus their data collection and structure the preliminary analysis of their data about diabetes health care transition.

Regardless of whether theory is induced from data or selected in advance to guide the study, consistent with the principle of transparency , its role should be clearly identified and justified in the research publication ( Bradbury-Jones, Taylor, & Herber, 2014 ; Kelly, 2010 ). Methodological congruence is an important guiding principle in this regard ( Richards & Morse, 2013 ). If a theory frames the study at the outset, it should guide and direct all phases. The resulting publication(s) should relate the phenomenon of interest and the research question(s) to the theory and specify how the theory guided data collection and analysis. The publication(s) should also discuss how the theory fits with the finished product. For instance, authors should describe how the theory provided a framework for the presentation of the findings and discuss the findings in context with the relevant theoretical literature.

A study examining parents’ motivations to promote vegetable consumption in their children ( Hingle et al., 2012 ) provides an example of methodological congruence. The investigators adapted the Model of Goal Directed Behavior ( Bagozzi & Pieters, 1998 ) for parenting practices relevant to vegetable consumption (Model of Goal Directed Vegetable Parenting Practices; MGDVPP). Consistent with the adapted theoretical model and in keeping with the congruence principle, interviews were guided by the theoretical constructs contained within the MGDVPP, including parents’ attitudes, subjective norms, and perceived behavioral control related to promoting vegetable consumption in children ( Hingle et al., 2012 ). The study discovered that the adapted model successfully identified parents’ motivations to encourage their children to eat more vegetables.

The use of the theory should be consistent with the basic goal of qualitative research, which is discovery. Alternatively stated, theories should be used as broad orienting frameworks for exploring topical areas without imposing preconceived ideas and biases. The theory should be consistent with the study findings and not be used to force-fit the researcher’s interpretation of the data ( Sandelowski, 1993b ). Divergence from the theory when it does not fit the study findings is illustrated in a qualitative study of hypertension prevention beliefs in Hispanics ( Aroian, Peters, Rudner, & Waser, 2012 ). This study used the Theory of Planned Behavior as a guiding theoretical framework but found that coding separately for normative and control beliefs was not the best organizing schema for presenting the study findings. When divergence from the original theory occurs, the research report should explain and justify how and why the theory was modified ( Bradbury-Jones et al., 2014 ).

What Are Typical Sampling Methods in Qualitative Studies?

Qualitative sampling methods should be “purposeful” ( Coyne, 1997 ; Patton, 2015 ; Tuckett, 2004 ). Purposeful sampling is based on the study purpose and investigator judgments about which people and settings will provide the richest information for the research questions. The logic underlying this type of sampling differs from the logic underlying quantitative sampling ( Patton, 2015 ). Quantitative research strives for empirical generalization. In qualitative studies, generalizability beyond the study sample is typically not the intent; rather, the focus is on deriving depth and context-embedded meaning for the relevant study population.

Purposeful sampling is a broad term. Theoretical sampling is one particular type of purposeful sampling unique to grounded theory methods ( Coyne, 1997 ). In theoretical sampling, study participants are chosen according to theoretical categories that emerge from ongoing data collection and analyses ( Bryant & Charmaz, 2010 ). Data collection and analysis are conducted concurrently to allow generating and testing hypotheses that emerge from analyzing incoming data. The following example from the previously mentioned qualitative interview study about transition from pediatric to adult care in adolescents with type 1 diabetes ( Pierce et al., 2016 ) illustrates the process of theoretical sampling: An adolescent study participant stated that he was “turned off” by the “childish” posters in his pediatrician’s office. He elaborated that he welcomed transitioning to adult care because his diabetes was discovered when he was 18, an age when he reportedly felt more “mature” than most pediatric patients. These data were coded as “developmental misfit” and prompted a tentative hypothesis about developmental stage at entry for pediatric diabetes care and readiness for health care transition. Examining this hypothesis prompted seeking study participants who varied according to age or developmental stage at time of diagnosis to examine the theoretical relevance of an emerging theme about developmental fit.

Not all purposeful sampling, however, is “theoretical.” For example, ethnographic studies typically seek to understand a group’s cultural beliefs and practices ( Creswell, 2013a ). Consistent with this purpose, researchers conducting an ethnographic study might purposefully select study participants according to specific characteristics that reflect the social roles and positions in a given group or society (e.g., socioeconomic status, education; Johnson, 1990 ).

Random sampling is generally not used in qualitative research. Random selection requires a sufficiently large sample to maximize the potential for chance and, as will be discussed below, sample size is intentionally small in qualitative studies. However, random sampling may be used to verify or clarify findings ( Patton, 2015 ). Validating study findings with a randomly selected subsample can be used to address the possibility that a researcher is inadvertently giving greater attention to cases that reinforce his or her preconceived ideas.

Regardless of the sampling method used, qualitative researchers should clearly describe the sampling strategy and justify how it fits the study when reporting study findings (transparency). A common error is to refer to theoretical sampling when the cases were not chosen according to emerging theoretical concepts. Another common error is to apply sampling principles from quantitative research (e.g., cluster sampling) to convince skeptical reviewers about the rigor or validity of qualitative research. Rigor is best achieved by being purposeful, making sound decisions, and articulating the rationale for those decisions. As mentioned earlier in the discussion of transferability , qualitative researchers are encouraged to describe their methods of sample selection and descriptive characteristics about their sample so that readers and reviewers can judge how the current sample may differ from others. Understanding the characteristics of each qualitative study sample is essential for the iterative nature of qualitative research whereby qualitative findings inform the development of future qualitative, quantitative, or mixed-methods studies. Reviewers should evaluate sampling decisions based on how they fit the study purpose and how they influence the quality of the end product.

What Sample Size Is Needed for Qualitative Research?

No definitive rules exist about sample size in qualitative research. However, sample sizes are typically smaller than those in quantitative studies ( Patton, 2015 ). Small samples often generate a large volume of data and information-rich cases, ultimately leading to insight regarding the phenomenon under study ( Patton, 2015 ; Ritchie & Lewis, 2003 ). Sample sizes of 20–30 cases are typical, but a qualitative sample can be even smaller under some circumstances ( Mason, 2010 ).

Sample size adequacy is evaluated based on the quality of the study findings, specifically the full development of categories and inter-relationships or the adequacy of information about the phenomenon under study ( Corbin & Strauss, 2008 ; Ritchie & Lewis, 2003 ). Small sample sizes are of concern if they do not result in these outcomes. Data saturation (i.e., the point at which no new information, categories, or themes emerge) is often used to judge informational adequacy ( Morgan, 1998 ; Ritchie & Lewis, 2003 ). Although enough participants should be included to obtain saturation ( Morgan, 1998 ), informational adequacy pertains to more than sample size. It is also a function of the quality of the data, which is influenced by study participant characteristics (e.g., cognitive ability, knowledge, representativeness) and the researcher’s data-gathering skills and analytical ability to generate meaningful findings ( Morse, 2015b ; Patton, 2015 ).

Sample size is also influenced by type of qualitative research, the study purpose, the sample, the depth and complexity of the topic investigated, and the method of data collection. In general, the more heterogeneous the sample, the larger the sample size, particularly if the goal is to investigate similarities and differences by specific characteristics ( Ritchie & Lewis, 2003 ). For instance, in a study to conduct an initial exploration of factors underlying parents’ motivations to use good parenting practices, theoretical saturation (i.e., the point at which no new information, categories, or themes emerge) was obtained with a small sample ( n  = 15), most likely because the study was limited to parents of young children ( Hingle et al., 2012 ). If the goal of the study had been, for example, to identify racial/ethnic, gender, or age differences in food parenting practices, a larger sample would likely be needed to obtain saturation or informational adequacy.

Studies that seek to understand maximum variation in a phenomenon might also need a larger sample than one that is seeking to understand extreme or atypical cases. For example, a qualitative study of diet and physical activity in young Australian men conducted focus groups to identify perceived motivators and barriers to healthy eating and physical activity and examine the influence of body weight on their perceptions. Examining the influence of body weight status required 10 focus groups to allow for group assignment based on body mass index ( Ashton et al., 2015 ). More specifically, 61 men were assigned to a healthy-weight focus group ( n  = 3), an overweight/obese focus group ( n  = 3), or a mixed-weight focus group ( n  = 4). Had the researcher not been interested in whether facilitators and barriers differed by weight status, its likely theoretical saturation could have been obtained with fewer groups. Depth of inquiry also influences sample size ( Sandelowski, 1995 ). For instance, an in-depth analysis of an intervention for children with cancer and their families included 16 family members from three families. Study data comprised 52 hrs of videotaped intervention sessions and 10 interviews ( West, Bell, Woodgate, & Moules, 2015 ). Depth was obtained through multiple data points and types of data, which justified sampling only a few families.

Authors of publications describing qualitative findings should show evidence that the data were “saturated” by a sample with sufficient variation to permit detailing shared and divergent perspectives, meanings, or experiences about the topic of inquiry. Decisions related to the sample (e.g., targeted recruitment) should be detailed in publications so that peer reviewers have the context for evaluating the sample and determining how the sample influenced the study findings ( Patton, 2015 ).

Qualitative Data Analysis

When conducting qualitative research, voluminous amounts of data are gathered and must be prepared (i.e., transcribed) and managed. During the analytic process, data are systematically transformed through identifying, defining, interpreting, and describing findings that are meant to comprehensively describe the phenomenon or the abstract qualities that they have in common. The process should be systematic ( dependability ) and well-documented in the analysis section of a qualitative manuscript. For example, Kelly and Ganong (2011) , in their study of medical treatment decisions made by families of children with cancer, described their analytic procedure by outlining their approach to coding and use of memoing (e.g., keeping careful notes about emerging ideas about the data throughout the analytic process), comparative analysis (e.g., comparing data against one another and looking for similarities and differences), and diagram drawing (e.g., pictorially representing the data structure, including relationships between codes).

How Should Researchers Document Coding Reliability?

Because the intent of qualitative research is to account for multiple perspectives, the goal of qualitative analysis is to comprehensively incorporate those perspectives into discernible findings. Researchers accustomed to doing quantitative studies may expect authors to quantify interrater reliability (e.g., kappa statistic) but this is not typical in qualitative research. Rather, the emphasis in qualitative research is on (1) training those gathering data to be rigorous and produce high-quality data and on (2) using systematic processes to document key decisions (e.g., code book), clear direction, and open communication among team members during data analysis. The goal is to make the most of the collective insight of the investigative team to triangulate or complement each other’s efforts to process and interpret the data. Instead of evaluating if two independent raters came to the same numeric rating, reviewers of qualitative manuscripts should judge to what extent the overall process of coding, data management, and data interpretation were systematic and rigorous. Authors of qualitative reports should articulate their coding procedures for others to evaluate. Together, these strategies promote trustworthiness of the study findings.

An example of how these processes are described in the report of a qualitative study is as follows:

The first two authors independently applied the categories to a sample of two interviews and compared their application of the categories to identify lack of clarity and overlap in categories. The investigators created a code book that contained a definition of categories, guidelines for their application, and excerpts of data exemplifying the categories. The first two authors independently coded the data and compared how they applied the categories to the data and resolved any differences during biweekly meetings. ATLAS.ti, version 6.2, was used to document and accommodate ongoing changes and additions to the coding structure ( Palma et al., 2015 , p. 224).

Do I Need to Use a Specialized Qualitative Data Software Program for Analysis?

Multiple computer software packages for qualitative data analysis are currently available ( Silver & Lewins, 2014 ; Yin, 2015 ). These packages allow the researcher to import qualitative data (e.g., interview transcripts) into the software program and organize data segments (e.g., delineate which interview excerpts are relevant to particular themes). Qualitative analysis software can be useful for organizing and sorting through data, including during the analysis phase. Some software programs also offer sophisticated coding and visualization capabilities that facilitate and enhance interpretation and understanding. For example, if data segments are coded by specific characteristics (e.g., gender, race/ethnicity), the data can be sorted and analyzed by these characteristics, which may contribute to an understanding of whether and/or how a particular phenomenon may vary by these characteristics.

The strength of computer software packages for qualitative data analysis is their potential to contribute to methodological rigor by organizing the data for systematic analyses ( John & Johnson, 2000 ; MacMillan & Koenig, 2004 ). However, the programs do not replace the researchers’ analyses. The researcher or research team is ultimately responsible for analyzing the data, identifying the themes and patterns, and placing the findings within the context of the literature. In other words, qualitative data analysis software programs contribute to, but do not ensure scientific rigor or “objectivity” in, the analytic process. In fact, using a software program for analysis is not essential if the researcher demonstrates the use of alternative tools and procedures for rigor.

Presentation of Findings

Should there be overlap between presentation of themes in the results and discussion sections.

Qualitative papers sometimes combine results and discussion into one section to provide a cohesive presentation of the findings along with meaningful linkages to the existing literature ( Burnard, 2004 ; Burnard, Gill, Stewart, Treasure, & Chadwick, 2008 ). Although doing so is an acceptable method for reporting qualitative findings, some journals prefer the two sections to be distinct.

When the journal style is to distinguish the two sections, the results section should describe the findings, that is, the themes, while the discussion section should pull the themes together to make larger-level conclusions and place the findings within the context of the existing literature. For instance, the findings section of a study of how rural African-American adolescents, parents, and community leaders perceived obesity and topics for a proposed obesity prevention program, contained a description of themes about adolescent eating patterns, body shape, and feedback on the proposed weight gain prevention program according to each subset of participants (i.e., adolescents, parents, community leaders). The discussion section then put these themes within the context of findings from prior qualitative and intervention studies in related populations ( Cassidy et al., 2013 ). In the Discussion, when making linkages to the existing literature, it is important to avoid the temptation to extrapolate beyond the findings or to over-interpret them ( Burnard, 2004 ). Linkages between the findings and the existing literature should be supported by ample evidence to avoid spurious or misleading connections ( Burnard, 2004 ).

What Should I Include in the Results Section?

The results section of a qualitative research report is likely to contain more material than customary in quantitative research reports. Findings in a qualitative research paper typically include researcher interpretations of the data as well as data exemplars and the logic that led to researcher interpretations ( Sandelowski & Barroso, 2002 ). Interpretation pertains to the researcher breaking down and recombining the data and creating new meanings (e.g., abstract categories, themes, conceptual models). Select quotes from interviews or other types of data (e.g., participant observation, focus groups) are presented to illustrate or support researcher interpretations. Researchers trained in the quantitative tradition, where interpretation is restricted to the discussion section, may find this surprising; however, in qualitative methods, researcher interpretations represent an important component of the study results. The presentation of the findings, including researcher interpretations (e.g., themes) and data (e.g., quotes) supporting those interpretations, adds to the trustworthiness of the study ( Elo et al., 2014 ).

The Results section should contain a balance between data illustrations (i.e., quotes) and researcher interpretations ( Lofland & Lofland, 2006 ; Sandelowski, 1998 ). Because interpretation arises out of the data, description and interpretation should be combined. Description should be sufficient to support researcher interpretations, and quotes should be used judiciously ( Morrow, 2005 ; Sandelowski, 1994 ). Not every theme needs to be supported by multiple quotes. Rather, quotes should be carefully selected to provide “voice” to the participants and to help the reader understand the phenomenon from the participant’s perspective within the context of the researcher’s interpretation ( Morrow, 2005 ; Ritchie & Lewis, 2003 ). For example, researchers who developed a grounded theory of sexual risk behavior of urban American Indian adolescent girls identified desire for better opportunities as a key deterrent to neighborhood norms for early sexual activity. They illustrated this theme with the following quote: “I don’t want to live in the ‘hood and all that…My sisters are stuck there because they had babies. That isn’t going to happen to me” ( Saftner, Martyn, Momper, Loveland-Cherry, & Low, 2015 , p. 372).

There is no precise formula for the proportion of description to interpretation. Both descriptive and analytic excess should be avoided ( Lofland & Lofland, 2006 ). The former pertains to presentation of unedited field notes or interview transcripts rather than selecting and connecting data to analytic concepts that explain or summarize the data. The latter pertains to focusing on the mechanics of analysis and interpretation without substantiating researcher interpretations with quotes. Reviewer requests for methodological rigor can result in researchers writing qualitative research papers that suffer from analytic excess ( Sandelowski & Barroso, 2002 ). Page limitations of most journals provide a safeguard against descriptive excess, but page limitations should not circumvent researchers from providing the basis for their interpretations.

Additional potential problems with qualitative results sections include under-elaboration, where themes are too few and not clearly defined. The opposite problem, over-elaboration, pertains to too many analytic distinctions that could be collapsed under a higher level of abstraction. Quotes can also be under- or over-interpreted. Care should be taken to ensure the quote(s) selected clearly support the theme to which they are attached. And finally, findings from a qualitative study should be interesting and make clear contributions to the literature ( Lofland & Lofland, 2006 ; Morse, 2015b ).

Should I Quantify My Results? (e.g., Frequency With Which Themes Were Endorsed)

There is controversy over whether to quantify qualitative findings, such as providing counts for the frequency with which particular themes are endorsed by study participants ( Morgan, 1993 ; Sandelowski, 2001 ). Qualitative papers usually report themes and patterns that emerge from the data without quantification ( Dey, 1993 ). However, it is possible to quantify qualitative findings, such as in qualitative content analysis. Qualitative content analysis is a method through which a researcher identifies the frequency with which a phenomenon, such as specific words, phrases, or concepts, is mentioned ( Elo et al., 2014 ; Morgan, 1993 ). Although this method may appeal to quantitative reviewers, it is important to note that this method only fits specific study purposes, such as studies that investigate the language used by a particular group when communicating about a specific topic. In addition, results may be quantified to provide information on whether themes appeared to be common or atypical. Authors should avoid using imprecise language, such as “some participants” or “many participants.” A good example of quantification of results to illustrate more or less typical themes comes from a manuscript describing a qualitative study of school nurses’ perceived barriers to addressing obesity with students and their families. The authors described that all but one nurse reported not having the resources they needed to discuss weight with students and families whereas one-quarter of nurses reported not feeling competent to discuss weight issues ( Steele et al., 2011 ). If quantification of findings is used, authors should provide justification that explains how quantification is consistent with the aims or goals of the study ( Sandelowski, 2001 ).

Conclusions

This article highlighted key theoretical and logistical considerations that arise in designing, conducting, and reporting qualitative research studies (see Table 1 for a summary). This type of research is vital for obtaining patient, family, community, and other stakeholder perspectives about their needs and interests, and will become increasingly critical as our models of health care delivery evolve. For example, qualitative research could contribute to the study of health care providers and systems with the goal of optimizing our health care delivery models. Given the increasing diversity of the populations we serve, qualitative research will also be critical in providing guidance in how to tailor health interventions to key characteristics and increase the likelihood of acceptable, effective treatment approaches. For example, applying qualitative research methods could enhance our understanding of refugee experiences in our health care system, clarify treatment preferences for emerging adults in the midst of health care transitions, examine satisfaction with health care delivery, and evaluate the applicability of our theoretical models of health behavior changes across racial and ethnic groups. Incorporating patient perspectives into treatment is essential to meeting this nation’s priority on patient-centered health care ( Institute of Medicine Committee on Quality of Health Care in America, 2001 ). Authors of qualitative studies who address the methodological choices addressed in this review will make important contributions to the field of pediatric psychology. Qualitative findings will lead to a more informed field that addresses the needs of a wide range of patient populations and produces effective and acceptable population-specific interventions to promote health.

Acknowledgments

The authors thank Bridget Grahmann for her assistance with manuscript preparation.

This work was supported by National Cancer Institute of the National Institutes of Health (K07CA196985 to Y.W.). This work is a publication of the United States Department of Agriculture/Agricultural Research Center (USDA/ARS), Children’s Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, Texas. It is also a publication of the USDA/ARS, Children’s Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, Texas, and funded in part with federal funds from the USDA/ARS under Cooperative Agreement No. 58‐6250‐0‐008 (to D.T.). The contents of this publication do not necessarily reflect the views or policies of the USDA, nor does mention of trade names, commercial products, or organizations imply endorsement from the U.S. government. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of interest : None declared.

• Aroian K. J., Peters R. M., Rudner N., Waser L. (2012). Hypertension prevention beliefs of hispanics . Journal of Transcultural Nursing , 23 , 134–142. doi:10.1177/1043659611433871. [ PubMed ] [ Google Scholar ]
• Ashton L. M., Hutchesson M. J., Rollo M. E., Morgan P. J., Thompson D. I., Collins C. E. (2015). Young adult males’ motivators and perceived barriers towards eating healthily and being active: A qualitative study . The International Journal of Behavioral Nutrition and Physical Activity , 12 , 93 doi:10.1186/s12966‐015‐0257‐6. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bagozzi R., Pieters R. (1998). Goal-directed emotions . Cognition & Emotion , 12 ( 1 ), 1–26. [ Google Scholar ]
• Bandura A. (1986). Social foundations of thought and action: A social cognitive theory . Englewood Cliffs, NJ: Prentice-Hall Inc. [ Google Scholar ]
• Bevans K. B., Gardner W., Pajer K., Riley A. W., Forrest C. B. (2013). Qualitative development of the PROMIS ® pediatric stress response item banks . Journal of Pediatric Psychology , 38 , 173–191. doi:10.1093/jpepsy/jss107. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bradbury-Jones C., Taylor J., Herber O. (2014). How theory is used and articulated in qualitative research: Development of a new typology . Social Science and Medicine , 120 , 135–141. doi:10.1016/j.socscimed.2014.09.014. [ PubMed ] [ Google Scholar ]
• Bryant A., Charmaz K. (2010). The Sage handbook of grounded theory . Thousand Oaks, CA: Sage. [ Google Scholar ]
• Burnard P. (2004). Writing a qualitative research report . Nurse Education Today , 24 , 174–179. doi:10.1016/j.nedt.2003.11.005. [ PubMed ] [ Google Scholar ]
• Burnard P., Gill P., Stewart K., Treasure E., Chadwick B. (2008). Analysing and presenting qualitative data . British Dental Journal , 204 , 429–432. doi:10.1038/sj.bdj.2008.292. [ PubMed ] [ Google Scholar ]
• Cassidy O., Sbrocco T., Vannucci A., Nelson B., Jackson-Bowen D., Heimdal J., Heimdal J., Mirza N., Wilfley D. E., Osborn R., Shomaker L. B., Young J. F., Waldron H., Carter M., Tanofsky-Kraff M., (2013). Adapting interpersonal psychotherapy for the prevention of excessive weight gain in rural African American girls . Journal of Pediatric Psychology , 38 , 965–977. doi:10.1093/jpepsy/jst029. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Clark J. (2003). How to peer review a qualitative manuscript . Peer Review in Health Sciences , 2 , 219–235. [ Google Scholar ]
• Corbin S., Strauss A. (2008). Basics of qualitative research (3rd ed.). Los Angeles, CA: Sage Publications. [ Google Scholar ]
• Coyne I. T. (1997). Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? Journal of Advanced Nursing , 26 , 623–630. doi:10.1046/j.1365‐2648.1997.t01‐25‐00999.x. [ PubMed ] [ Google Scholar ]
• Creswell J. W. (1994). Research design: Qualitative & quantitative approaches . Journal of Marketing Research , 33 , 252 doi:10.2307/3152153. [ Google Scholar ]
• Creswell J. W. (2013a). Qualitative inquiry and research design: Choosing among five approaches . Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Creswell J. W. (2013b). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Creswell J. W., Klassen A. C., Plano Clark V. L., Smith K. C.;for the Office of Behavioral and Social Sciences Research. (2011). Best practices for mixed methods research in the health sciences . Retrieved from National Institutes of Health: http://obssr.od.nih.gov/mixed_methods_research .
• de Visser R. O., Graber R., Hart A., Abraham C., Scanlon T., Watten P., Memon A. (2015). Using qualitative methods within a mixed-methods approach to developing and evaluating interventions to address harmful alcohol use among young people . Health Psychology , 34 , 349–360. doi:10.1037/hea0000163. [ PubMed ] [ Google Scholar ]
• Dey I. (1993). Qualitative data analysis: A user-friendly guide for social scientists . New York, NY: Routledge. [ Google Scholar ]
• Dixon-Woods M., Shaw R. L., Agarwal S., Smith J. A. (2004). The problem of appraising qualitative research . Quality and Safety in Health Care , 13 , 223–225. doi:10.1136/qhc.13.3.223. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Eakin J. M., Mykhalovskiy E. (2003). Reframing the evaluation of qualitative health research: Reflections on a review of appraisal guidelines in the health sciences . Journal of Evaluation in Clinical Practice , 9 , 187–194. doi:10.1046/j.1365‐2753.2003.00392.x. [ PubMed ] [ Google Scholar ]
• Elo S., Kääriäinen M., Kanste O., Pölkki T., Utriainen K., Kyngäs H. (2014). Qualitative content analysis: A focus on trustworthiness . SAGE Open , 4 ( 1 ), 1–10. doi:10.1177/2158244014522633. [ Google Scholar ]
• Glaser B., Strauss A. (1967). The discovery grounded theory: Strategies for qualitative inquiry . Nursing Research , 17 , 364 doi:10.1097/00006199‐196807000‐00014. [ Google Scholar ]
• Gough B., Deatrick J. A. (2015). Qualitative health psychology research: Diversity, power, and impact . Health Psychology , 34 , 289–292. doi:10.1037/hea0000206. [ PubMed ] [ Google Scholar ]
• Guba E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries . Educational Communication and Technology , 29 , 75–91. doi:10.1007/BF02766777. [ Google Scholar ]
• Haukeland Y. B., Fjermestad K. W., Mossige S., Vatne T. M. (2015). Emotional experiences among siblings of children with rare disorders . Journal of Pediatric Psychology , 40 , 12–20. doi:10.1093/jpepsy/jsv022. [ PubMed ] [ Google Scholar ]
• Hess J. S., Straub D. M. (2011). Brief report: Preliminary findings from a pilot health care transition education intervention for adolescents and young adults with special health care needs . Journal of Pediatric Psychology , 36 , 172–178. doi:10.1093/jpepsy/jsq091. [ PubMed ] [ Google Scholar ]
• Hingle M., Beltran A., O’Connor T., Thompson D., Baranowski J., Baranowski T. (2012). A model of goal directed vegetable parenting practices . Appetite , 58 , 444–449. doi:10.1016/j.appet.2011.12.011. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Hughes S. O., Power T. G., Papaioannou M. A., Cross M. B., Nicklas T. A., Hall S. K., Shewchuk R. M. (2011). Emotional climate, feeding practices, and feeding styles: An observational analysis of the dinner meal in Head Start families . The International Journal of Behavavioral Nutrition and Physical Activity , 8 , 60 doi:10.1186/1479‐5868‐8‐60. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Institute of Medicine Committee on Quality of Health Care in America. (2001). Crossing the quality chasm: A new health system for the 21st century . National Academies Press. Washington, DC. [ Google Scholar ]
• Izaguirre M. R., Keefer L. (2014). Development of a self-efficacy scale for adolescents and young adults with inflammatory bowel disease . Journal of Pediatric Gastroenterology and Nutrition , 59 , 29–32. doi:10.1097/mpg.0000000000000357. [ PubMed ] [ Google Scholar ]
• John W. S., Johnson P. (2000). The pros and cons of data analysis software for qualitative research . Journal of Nursing Scholarship , 32 , 393–397. [ PubMed ] [ Google Scholar ]
• Johnson J. C. (1990). Selecting ethnographic informants . Sage Publications. Thousand Oaks, CA. [ Google Scholar ]
• Kars M. C., Grypdonck M. H., de Bock L. C., van Delden J. J. (2015). The parents’ ability to attend to the “voice of their child” with incurable cancer during the palliative phase . Health Psychology , 34 , 446–452. doi:10.1037/hea0000166. [ PubMed ] [ Google Scholar ]
• Kelly K., Ganong L. (2011). Moving to place: Childhood cancer treatment decision making in single-parent and repartnered family structures . Qualitative Health Research , 21 , 349–364. doi:10.1177/1049732310385823. [ PubMed ] [ Google Scholar ]
• Kelly M. (2010). The role of theory in qualitative health research . Family Practice , 27 , 285–290. doi:10.1093/fampra/cmp077. [ PubMed ] [ Google Scholar ]
• Krefting L. (1991). Rigor in qualitative research: The assessment of trustworthiness . The American Journal of Occupational Therapy , 45 , 214–222. doi:10.5014/ajot.45.3.214. [ PubMed ] [ Google Scholar ]
• Lincoln Y. S., Guba E. G. (1985). Naturalistic inquiry . Newbury Park, CA: Sage Publications. [ Google Scholar ]
• Lincoln Y. S., Lynham S. A., Guba E. G. (2011). Paradigmatic controversies, contradictions, and emerging confluences, revisited . In Denzin N. K., Lincoln Y. S. (Eds.), The Sage handbook of qualitative research (4th ed., pp. 97–128). Thousand Oaks, CA: Sage. [ Google Scholar ]
• Lofland J., Lofland L. H. (2006). Analyzing social settings: A guide to qualitative observation and analysis . Belmont, CA: Wadsworth Publishing Company. [ Google Scholar ]
• Lyons A. C., Goodwin I., McCreanor T., Griffin C. (2015). Social networking and young adults’ drinking practices: Innovative qualitative methods for health behavior research . Health Psychology , 34 , 293–302. doi:10.1037/hea0000168. [ PubMed ] [ Google Scholar ]
• MacMillan K., Koenig T. (2004). The wow factor: Preconceptions and expectations for data analysis software in qualitative research . Social Science Computer Review , 22 , 179–186. doi:10.1177/0894439303262625. [ Google Scholar ]
• Mason M. (Producer). (2010). Sample size and saturation in PhD studies using qualitative interviews . Forum: Qualitative Social Research . Retrieved from http://nbn-resolving.de/urn:nbn:de:0114-fqs100387 .
• Mays N., Pope C. (2000). Qualitative research in health care: Assessing quality in qualitative research . British Medical Journal , 320 , 50 doi:10.1136/bmj.320.7226.50. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McDonald C. C., Sommers M. S. (2015). Teen drivers’ perceptions of inattention and cell phone use while eriving . Traffic Injury Prevention , 16 ( Suppl 2 ), S52–S58. doi:10.1080/15389588.2015.1062886. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Miles M. B., Huberman A. M., Saldaña J. (2013). Qualitative data analysis: A methods sourcebook (3rd ed.). Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Minges K. E., Owen N., Salmon J., Chao A., Dunstan D. W., Whittemore R. (2015). Reducing youth screen time: Qualitative metasynthesis of findings on barriers and facilitators . Health Psychology , 34 , 381–397. doi:10.1037/hea0000172. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Morgan D. L. (1993). Qualitative content analysis: A guide to paths not taken . Qualitative Health Research , 3 , 112–121. doi:10.1177/104973239300300107. [ PubMed ] [ Google Scholar ]
• Morgan D. L. (1998). Planning Focus Groups: Focus Group Kit #2 . Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Morrow S. (2005). Quality and trustworthiness in qualitative research in counseling psychology . Journal of Counseling Psychology , 52 , 250–260. doi:10.1037/0022‐0167.52.2.250. [ Google Scholar ]
• Morse J. M. (2015a). Critical analysis of strategies for determining rigor in qualitative inquiry . Qualitative Health Research , 25 , 1212–1222. doi:10.1177/1049732315588501. [ PubMed ] [ Google Scholar ]
• Morse J. M. (2015b). Data were saturated . Qualitative Health Research , 25 , 587–588. doi:10.1177/1049732315576699. [ PubMed ] [ Google Scholar ]
• Palermo T. M. (2013). New guidelines for publishing review articles in JPP: Systematic reviews and topical reviews . Journal of Pediatric Psychology , 38 , 5–9. doi:10.1093/jpepsy/jss124. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Palermo T. M. (2014). Evidence-based interventions in pediatric psychology: Progress over the decades . Journal of Pediatric Psychology , 39 , 753–762. doi:10.1093/jpepsy/jsu048. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Palma E., Deatrick J., Hobbie W., Ogle S., Kobayashi K., Maldonado L. (2015). Maternal caregiving demands for adolescent and young adult survivors of pediatric brain tumors . Oncology Nursing Forum , 42 , 222–229. doi:10.1188/15.ONF.. [ PubMed ] [ Google Scholar ]
• Patton M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Pierce J. S., Wysocki T. (2015). Topical Review: Advancing research on the transition to adult care for type 1 diabetes . Journal of Pediatric Psychology , 40 , 1041–1047. doi:10.1093/jpepsy/jsv064. [ PubMed ] [ Google Scholar ]
• Pierce J. S., Wysocki T., Aroian K. (2016). Multiple stakeholder perspectives on health care transition outcomes in Type 1 Diabetes . Unpublished data. [ Google Scholar ]
• Polanyi M. (1958). Personal knowledge . New York, NY: Harper & Row. [ Google Scholar ]
• Power T. G., Hughes S. O., Goodell L. S., Johnson S. L., Duran J. A., Williams K., Beck A. D., Frankel L. A. (2015). Feeding practices of low-income mothers: How do they compare to current recommendations? The International Journal of Behavioral Nutrition and Physical Activity , 12 , 34 doi:10.1186/s12966‐015‐0179‐3. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Richards L., Morse J. M. (2013). Readdme first for a user’s guide to qualitative methods (3rd ed.). Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Ritchie J., Lewis J. (Eds.). (2003). Qualitative research practice: A guide for social science students and researchers . Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Saftner M. A., Martyn K. K., Momper S. L., Loveland-Cherry C. J., Low L. K. (2015). Urban American Indian adolescent girls framing sexual risk behavior . Journal of Transcultural Nursing , 26 , 365–375. doi:10.1177/1043659614524789. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Saldaña J. (2012). The coding manual for qualitative researchers . Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Sandelowski M. (1993a). Rigor or rigor mortis: The problem of rigor in qualitative research revisited . Advances in Nursing Science , 16 , 1–8. [ PubMed ] [ Google Scholar ]
• Sandelowski M. (1993b). Theory unmasked: The uses and guises of theory in qualitative research . Research in Nursing & Health , 16 , 213–218. doi:10.1002/nur.4770160308. [ PubMed ] [ Google Scholar ]
• Sandelowski M. (1994). The use of quotes in qualitative research . Research in Nursing and Health , 17 , 479–482. [ PubMed ] [ Google Scholar ]
• Sandelowski M. (1995). Sample size in qualitative research . Research in Nursing and Health , 18 , 179–183. [ PubMed ] [ Google Scholar ]
• Sandelowski M. (1998). Writing a good read: Strategies for re-presenting qualitative data . Research in Nursing and Health , 21 , 375–382. doi:10.1016/s1361‐3111(98)80052‐6. [ PubMed ] [ Google Scholar ]
• Sandelowski M. (2001). Real qualitative researchers do not count: The use of numbers in qualitative research . Research in Nursing and Health , 24 , 230–240. [ PubMed ] [ Google Scholar ]
• Sandelowski M. (2010). What’s in a name? Qualitative description revisited . Research in Nursing and Health , 33 , 77–84. doi:10.1002/nur.20362.. [ PubMed ] [ Google Scholar ]
• Sandelowski M., Barroso J. (2002). Finding the findings in qualitative studies . Journal of Nursing Scholarship , 34 , 213–219. [ PubMed ] [ Google Scholar ]
• Schwartz L. A., Tuchman L. K., Hobbie W. L., Ginsberg J. P. (2011). A social-ecological model of readiness for transition to adult-oriented care for adolescents and young adults with chronic health conditions . Child: Care, Health, and Development , 37 , 883–895. doi:10.1111/j.1365‐2214.2011.01282.x. [ PubMed ] [ Google Scholar ]
• Silver C., Lewins A. (2014). Using software in qualitative research: A step-by-step guide (2nd ed.). London: Sage Publications. [ Google Scholar ]
• Steele R. G., Wu Y. P., Jensen C. D., Pankey S., Davis A. M., Aylward B. S. (2011). School nurses’ perceived barriers to discussing weight with children and their families: A qualitative approach . Journal of School Health , 81 , 128–137. doi:10.1111/j.1746‐1561.2010.00571.x. [ PubMed ] [ Google Scholar ]
• Thompson D. (2014). Talk to me, please!: The importance of qualitative research to games for health . Games for Health: Research, Development, and Clinical Applications , 3 , 117–118. doi:10.1089/g4h.2014.0023. [ PubMed ] [ Google Scholar ]
• Thompson D., Baranowski T., Buday R., Baranowski J., Juliano M., Frazior M., Wilsdon J., Jago R. (2007). In pursuit of change: Youth response to intensive goal setting embedded in a serious video game . Journal of Diabetes Science and Technology , 1 , 907–917. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Thompson D., Bhatt R., Watson K. (2013). Physical activity problem-solving inventory for adolescents: Development and initial validation . Pediatric Exercise Science , 25 , 448–467. [ PubMed ] [ Google Scholar ]
• Tobin G. A., Begley C. M. (2004). Methodological rigour within a qualitative framework . Journal of Advanced Nursing , 48 , 388–396. doi:10.1111/j.1365‐2648.2004.03207.x. [ PubMed ] [ Google Scholar ]
• Tong A., Sainsbury P., Craig J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups . International Journal for Quality in Health Care , 19 , 349–357. doi:10.1093/intqhc/mzm042. [ PubMed ] [ Google Scholar ]
• Tuckett A. G. (2004). Qualitative research sampling: The very real complexities . Nurse Researcher , 12 , 47–61. doi:10.7748/nr2004.07.12.1.47.c5930. [ PubMed ] [ Google Scholar ]
• Valenzuela J. M., Buchanan C. L., Radcliffe J., Ambrose C., Hawkins L. A., Tanney M., Rudy B. J. (2011). Transition to adult services among behaviorally infected adolescents with HIV—a qualitative study . Journal of Pediatric Psychology , 36 , 134–140. doi:10.1093/jpepsy/jsp051. [ PubMed ] [ Google Scholar ]
• West C. H., Bell J. M., Woodgate R. L., Moules N. J. (2015). Waiting to return to normal: An exploration of family systems intervention in childhood cancer . Journal of Family Nursing , 21 , 261–294. doi:10.1177/1074840715576795. [ PubMed ] [ Google Scholar ]
• Whittemore R., Chase S. K., Mandle C. L. (2001). Validity in qualitative research . Qualitative Health Research , 11 , 522–537. doi:10.1177/104973201129119299. [ PubMed ] [ Google Scholar ]
• Yin R. K. (2015). Qualitative research from start to finish (2nd ed.). New York, NY: Guilford Press. [ Google Scholar ]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 19 March 2024

TacticAI: an AI assistant for football tactics

• Zhe Wang   ORCID: orcid.org/0000-0002-0748-5376 1   na1 ,
• Petar Veličković   ORCID: orcid.org/0000-0002-2820-4692 1   na1 ,
• Daniel Hennes   ORCID: orcid.org/0000-0002-3646-5286 1   na1 ,
• Nenad Tomašev   ORCID: orcid.org/0000-0003-1624-0220 1 ,
• Laurel Prince 1 ,
• Michael Kaisers 1 ,
• Yoram Bachrach 1 ,
• Romuald Elie 1 ,
• Li Kevin Wenliang 1 ,
• Federico Piccinini 1 ,
• William Spearman 2 ,
• Ian Graham 3 ,
• Jerome Connor 1 ,
• Yi Yang 1 ,
• Adrià Recasens 1 ,
• Mina Khan 1 ,
• Nathalie Beauguerlange 1 ,
• Pablo Sprechmann 1 ,
• Pol Moreno 1 ,
• Nicolas Heess   ORCID: orcid.org/0000-0001-7876-9256 1 ,
• Michael Bowling   ORCID: orcid.org/0000-0003-2960-8418 4 ,
• Demis Hassabis 1 &
• Karl Tuyls   ORCID: orcid.org/0000-0001-7929-1944 5  

Nature Communications volume  15 , Article number:  1906 ( 2024 ) Cite this article

20k Accesses

241 Altmetric

Metrics details

• Computational science
• Information technology

Identifying key patterns of tactics implemented by rival teams, and developing effective responses, lies at the heart of modern football. However, doing so algorithmically remains an open research challenge. To address this unmet need, we propose TacticAI, an AI football tactics assistant developed and evaluated in close collaboration with domain experts from Liverpool FC. We focus on analysing corner kicks, as they offer coaches the most direct opportunities for interventions and improvements. TacticAI incorporates both a predictive and a generative component, allowing the coaches to effectively sample and explore alternative player setups for each corner kick routine and to select those with the highest predicted likelihood of success. We validate TacticAI on a number of relevant benchmark tasks: predicting receivers and shot attempts and recommending player position adjustments. The utility of TacticAI is validated by a qualitative study conducted with football domain experts at Liverpool FC. We show that TacticAI’s model suggestions are not only indistinguishable from real tactics, but also favoured over existing tactics 90% of the time, and that TacticAI offers an effective corner kick retrieval system. TacticAI achieves these results despite the limited availability of gold-standard data, achieving data efficiency through geometric deep learning.

Similar content being viewed by others

Artificial intelligence and illusions of understanding in scientific research

Lisa Messeri & M. J. Crockett

Highly accurate protein structure prediction with AlphaFold

John Jumper, Richard Evans, … Demis Hassabis

Solving olympiad geometry without human demonstrations

Trieu H. Trinh, Yuhuai Wu, … Thang Luong

Introduction

Association football, or simply football or soccer, is a widely popular and highly professionalised sport, in which two teams compete to score goals against each other. As each football team comprises up to 11 active players at all times and takes place on a very large pitch (also known as a soccer field), scoring goals tends to require a significant degree of strategic team-play. Under the rules codified in the Laws of the Game 1 , this competition has nurtured an evolution of nuanced strategies and tactics, culminating in modern professional football leagues. In today’s play, data-driven insights are a key driver in determining the optimal player setups for each game and developing counter-tactics to maximise the chances of success 2 .

When competing at the highest level the margins are incredibly tight, and it is increasingly important to be able to capitalise on any opportunity for creating an advantage on the pitch. To that end, top-tier clubs employ diverse teams of coaches, analysts and experts, tasked with studying and devising (counter-)tactics before each game. Several recent methods attempt to improve tactical coaching and player decision-making through artificial intelligence (AI) tools, using a wide variety of data types from videos to tracking sensors and applying diverse algorithms ranging from simple logistic regression to elaborate neural network architectures. Such methods have been employed to help predict shot events from videos 3 , forecast off-screen movement from spatio-temporal data 4 , determine whether a match is in-play or interrupted 5 , or identify player actions 6 .

The execution of agreed-upon plans by players on the pitch is highly dynamic and imperfect, depending on numerous factors including player fitness and fatigue, variations in player movement and positioning, weather, the state of the pitch, and the reaction of the opposing team. In contrast, set pieces provide an opportunity to exert more control on the outcome, as the brief interruption in play allows the players to reposition according to one of the practiced and pre-agreed patterns, and make a deliberate attempt towards the goal. Examples of such set pieces include free kicks, corner kicks, goal kicks, throw-ins, and penalties 2 .

Among set pieces, corner kicks are of particular importance, as an improvement in corner kick execution may substantially modify game outcomes, and they lend themselves to principled, tactical and detailed analysis. This is because corner kicks tend to occur frequently in football matches (with ~10 corners on average taking place in each match 7 ), they are taken from a fixed, rigid position, and they offer an immediate opportunity for scoring a goal—no other set piece simultaneously satisfies all of the above. In practice, corner kick routines are determined well ahead of each match, taking into account the strengths and weaknesses of the opposing team and their typical tactical deployment. It is for this reason that we focus on corner kick analysis in particular, and propose TacticAI, an AI football assistant for supporting the human expert with set piece analysis, and the development and improvement of corner kick routines.

TacticAI is rooted in learning efficient representations of corner kick tactics from raw, spatio-temporal player tracking data. It makes efficient use of this data by representing each corner kick situation as a graph—a natural representation for modelling relationships between players (Fig.  1 A, Table  2 ), and these player relationships may be of higher importance than the absolute distances between them on the pitch 8 . Such a graph input is a natural candidate for graph machine learning models 9 , which we employ within TacticAI to obtain high-dimensional latent player representations. In the Supplementary Discussion section, we carefully contrast TacticAI against prior art in the area.

A How corner kick situations are converted to a graph representation. Each player is treated as a node in a graph, with node, edge and graph features extracted as detailed in the main text. Then, a graph neural network operates over this graph by performing message passing; each node’s representation is updated using the messages sent to it from its neighbouring nodes. B How TacticAI processes a given corner kick. To ensure that TacticAI’s answers are robust in the face of horizontal or vertical reflections, all possible combinations of reflections are applied to the input corner, and these four views are then fed to the core TacticAI model, where they are able to interact with each other to compute the final player representations—each internal blue arrow corresponds to a single message passing layer from ( A ). Once player representations are computed, they can be used to predict the corner’s receiver, whether a shot has been taken, as well as assistive adjustments to player positions and velocities, which increase or decrease the probability of a shot being taken.

Uniquely, TacticAI takes advantage of geometric deep learning 10 to explicitly produce player representations that respect several symmetries of the football pitch (Fig.  1 B). As an illustrative example, we can usually safely assume that under a horizontal or vertical reflection of the pitch state, the game situation is equivalent. Geometric deep learning ensures that TacticAI’s player representations will be identically computed under such reflections, such that this symmetry does not have to be learnt from data. This proves to be a valuable addition, as high-quality tracking data is often limited—with only a few hundred matches played each year in every league. We provide an in-depth overview of how we employ geometric deep learning in TacticAI in the “Methods” section.

From these representations, TacticAI is then able to answer various predictive questions about the outcomes of a corner—for example, which player is most likely to make first contact with the ball, or whether a shot will take place. TacticAI can also be used as a retrieval system—for mining similar corner kick situations based on the similarity of player representations—and a generative recommendation system, suggesting adjustments to player positions and velocities to maximise or minimise the estimated shot probability. Through several experiments within a case study with domain expert coaches and analysts from Liverpool FC, the results of which we present in the next section, we obtain clear statistical evidence that TacticAI readily provides useful, realistic and accurate tactical suggestions.

To demonstrate the diverse qualities of our approach, we design TacticAI with three distinct predictive and generative components: receiver prediction, shot prediction, and tactic recommendation through guided generation, which also correspond to the benchmark tasks for quantitatively evaluating TacticAI. In addition to providing accurate quantitative insights for corner kick analysis with its predictive components, the interplay between TacticAI’s predictive and generative components allows coaches to sample alternative player setups for each routine of interest, and directly evaluate the possible outcomes of such alternatives.

We will first describe our quantitative analysis, which demonstrates that TacticAI’s predictive components are accurate at predicting corner kick receivers and shot situations on held-out test corners and that the proposed player adjustments do not strongly deviate from ground-truth situations. However, such an analysis only gives an indirect insight into how useful TacticAI would be once deployed. We tackle this question of utility head-on and conduct a comprehensive case study in collaboration with our partners at Liverpool FC—where we directly ask human expert raters to judge the utility of TacticAI’s predictions and player adjustments. The following sections expand on the specific results and analysis we have performed.

In what follows, we will describe TacticAI’s components at a minimal level necessary to understand our evaluation. We defer detailed descriptions of TacticAI’s components to the “Methods” section. Note that, all our error bars reported in this research are standard deviations.

Benchmarking TacticAI

We evaluate the three components of TacticAI on a relevant benchmark dataset of corner kicks. Our dataset consists of 7176 corner kicks from the 2020 to 2021 Premier League seasons, which we randomly shuffle and split into a training (80%) and a test set (20%). As previously mentioned, TacticAI operates on graphs. Accordingly, we represent each corner kick situation as a graph, where each node corresponds to a player. The features associated with each node encode the movements (velocities and positions) and simple profiles (heights and weights) of on-pitch players at the timestamp when the corresponding corner kick was being taken by the attacking kicker (see the “Methods” section), and no information of ball movement was encoded. The graphs are fully connected; that is, for every pair of players, we will include the edge connecting them in the graph. Each of these edges encodes a binary feature, indicating whether the two players are on opposing teams or not. For each task, we generated the relevant dataset of node/edge/graph features and corresponding labels (Tables  1 and 2 , see the “Methods” section). The components were then trained separately with their corresponding corner kick graphs. In particular, we only employ a minimal set of features to construct the corner kick graphs, without encoding the movements of the ball nor explicitly encoding the distances between players into the graphs. We used a consistent training-test split for all benchmark tasks, as this made it possible to benchmark not only the individual components but also their interactions.

Accurate receiver and shot prediction through geometric deep learning

One of TacticAI’s key predictive models forecasts the receiver out of the 22 on-pitch players. The receiver is defined as the first player touching the ball after the corner is taken. In our evaluation, all methods used the same set of features (see the “Receiver prediction” entry in Table  1 and the “Methods” section). We leveraged the receiver prediction task to benchmark several different TacticAI base models. Our best-performing model—achieving 0.782 ± 0.039 in top-3 test accuracy after 50,000 training steps—was a deep graph attention network 11 , 12 , leveraging geometric deep learning 10 through the use of D 2 group convolutions 13 . We supplement this result with a detailed ablation study, verifying that both our choice of base architecture and group convolution yielded significant improvements in the receiver prediction task (Supplementary Table  2 , see the subsection “Ablation study” in the “Methods” section). Considering that corner kick receiver prediction is a highly challenging task with many factors that are unseen by our model—including fatigue and fitness levels, and actual ball trajectory—we consider TacticAI’s top-3 accuracy to reflect a high level of predictive power, and keep the base TacticAI architecture fixed for subsequent studies. In addition to this quantitative evaluation with the evaluation dataset, we also evaluate the performance of TacticAI’s receiver prediction component in a case study with human raters. Please see the “Case study with expert raters” section for more details.

For shot prediction, we observe that reusing the base TacticAI architecture to directly predict shot events—i.e., directly modelling the probability \({\mathbb{P}}(\,{{\mbox{shot}}}| {{\mbox{corner}}}\,)\) —proved challenging, only yielding a test F 1 score of 0.52 ± 0.03, for a GATv2 base model. Note that here we use the F 1 score—the harmonic mean of precision and recall—as it is commonly used in binary classification problems over imbalanced datasets, such as shot prediction. However, given that we already have a potent receiver predictor, we decided to use its output to give us additional insight into whether or not a shot had been taken. Hence, we opted to decompose the probability of taking a shot as

where \({\mathbb{P}}(\,{{\mbox{receiver}}}| {{\mbox{corner}}}\,)\) are the probabilities computed by TacticAI’s receiver prediction system, and \({\mathbb{P}}(\,{{\mbox{shot}}}| {{\mbox{receiver}}},{{\mbox{corner}}}\,)\) models the conditional shot probability after a specific player makes first contact with the ball. This was implemented through providing an additional global feature to indicate the receiver in the corresponding corner kick (Table  1 ) while the architecture otherwise remained the same as that of receiver prediction (Supplementary Fig.  2 , see the “Methods” section). At training time, we feed the ground-truth receiver as input to the model—at inference time, we attempt every possible receiver, weighing their contributions using the probabilities given by TacticAI’s receiver predictor, as per Eq. ( 1 ). This two-phased approach yielded a final test F 1 score of 0.68 ± 0.04 for shot prediction, which encodes significantly more signal than the unconditional shot predictor, especially considering the many unobservables associated with predicting shot events. Just as for receiver prediction, this performance can be further improved using geometric deep learning; a conditional GATv2 shot predictor with D 2 group convolutions achieves an F 1 score of 0.71 ± 0.01.

Moreover, we also observe that, even just through predicting the receivers, without explicitly classifying any other salient features of corners, TacticAI learned generalisable representations of the data. Specifically, team setups with similar tactical patterns tend to cluster together in TacticAI’s latent space (Fig.  2 ). However, no clear clusters are observed in the raw input space (Supplementary Fig.  1 ). This indicates that TacticAI can be leveraged as a useful corner kick retrieval system, and we will present our evaluation of this hypothesis in the “Case study with expert raters” section.

We visualise the latent representations of attacking and defending teams in 1024 corner kicks using t -SNE. A latent team embedding in one corner kick sample is the mean of the latent player representations on the same attacking ( A – C ) or defending ( D ) team. Given the reference corner kick sample ( A ), we retrieve another corner kick sample ( B ) with respect to the closest distance of their representations in the latent space. We observe that ( A ) and ( B ) are both out-swing corner kicks and share similar patterns of their attacking tactics, which are highlighted with rectangles having the same colours, although they bear differences with respect to the absolute positions and velocities of the players. All the while, the latent representation of an in-swing attack ( C ) is distant from both ( A ) and ( B ) in the latent space. The red arrows are only used to demonstrate the difference between in- and out-swing corner kicks, not the actual ball trajectories.

Lastly, it is worth emphasising that the utility of the shot predictor likely does not come from forecasting whether a shot event will occur—a challenging problem with many imponderables—but from analysing the difference in predicted shot probability across multiple corners. Indeed, in the following section, we will show how TacticAI’s generative tactic refinements can directly influence the predicted shot probabilities, which will then corresponds to highly favourable evaluation by our expert raters in the “Case study with expert raters” section.

Controlled tactic refinement using class-conditional generative models

Equipped with components that are able to potently relate corner kicks with their various outcomes (e.g. receivers and shot events), we can explore the use of TacticAI to suggest adjustments of tactics, in order to amplify or reduce the likelihood of certain outcomes.

Specifically, we aim to produce adjustments to the movements of players on one of the two teams, including their positions and velocities, which would maximise or minimise the probability of a shot event, conditioned on the initial corner setup, consisting of the movements of players on both teams and their heights and weights. In particular, although in real-world scenarios both teams may react simultaneously to the movements of each other, in our study, we focus on moderate adjustments to player movements, which help to detect players that are not responding to a tactic properly. Due to this reason, we simplify the process of tactic refinement through generating the adjustments for only one team while keeping the other fixed. The way we train a model for this task is through an auto-encoding objective: we feed the ground-truth shot outcome (a binary indicator) as an additional graph-level feature to TacticAI’s model (Table  1 ), and then have it learn to reconstruct a probability distribution of the input player coordinates (Fig.  1 B, also see the “Methods” section). As a consequence, our tactic adjustment system does not depend on the previously discussed shot predictor—although we can use the shot predictor to evaluate whether the adjustments make a measurable difference in shot probability.

This autoencoder-based generative model is an individual component that separates from TacticAI’s predictive systems. All three systems share the encoder architecture (without sharing parameters), but use different decoders (see the “Methods” section). At inference time, we can instead feed in a desired shot outcome for the given corner setup, and then sample new positions and velocities for players on one team using this probability distribution. This setup, in principle, allows for flexible downstream use, as human coaches can optimise corner kick setups through generating adjustments conditioned on the specific outcomes of their interest—e.g., increasing shot probability for the attacking team, decreasing it for the defending team (Fig.  3 ) or amplifying the chance that a particular striker receives the ball.

TacticAI makes it possible for human coaches to redesign corner kick tactics in ways that help maximise the probability of a positive outcome for either the attacking or the defending team by identifying key players, as well as by providing temporally coordinated tactic recommendations that take all players into consideration. As demonstrated in the present example ( A ), for a corner kick in which there was a shot attempt in reality ( B ), TacticAI can generate a tactically-adjusted setting in which the shot probability has been reduced, by adjusting the positioning of the defenders ( D ). The suggested defender positions result in reduced receiver probability for attacking players 2–5 (see bottom row), while the receiver probability of Attacker 1, who is distant from the goalpost, has been increased ( C ). The model is capable of generating multiple such scenarios. Coaches can inspect the different options visually and additionally consult TacticAI’s quantitative analysis of the presented tactics.

We first evaluate the generated adjustments quantitatively, by verifying that they are indistinguishable from the original corner kick distribution using a classifier. To do this, we synthesised a dataset consisting of 200 corner kick samples and their corresponding conditionally generated adjustments. Specifically, for corners without a shot event, we generated adjustments for the attacking team by setting the shot event feature to 1, and vice-versa for the defending team when a shot event did happen. We found that the real and generated samples were not distinguishable by an MLP classifier, with an F 1 score of 0.53 ± 0.05, indicating random chance level accuracy. This result indicates that the adjustments produced by TacticAI are likely similar enough to real corner kicks that the MLP is unable to tell them apart. Note that, in spite of this similarity, TacticAI recommends player-level adjustments that are not negligible—in the following section we will illustrate several salient examples of this. To more realistically validate the practical indistinguishability of TacticAI’s adjustments from realistic corners, we also evaluated the realism of the adjustments in a case study with human experts, which we will present in the following section.

In addition, we leveraged our TacticAI shot predictor to estimate whether the proposed adjustments were effective. We did this by analysing 100 corner kick samples in which threatening shots occurred, and then, for each sample, generated one defensive refinement through setting the shot event feature to 0. We observed that the average shot probability significantly decreased, from 0.75 ± 0.14 for ground-truth corners to 0.69 ± 0.16 for adjustments ( z  = 2.62,  p  < 0.001). This observation was consistent when testing for attacking team refinements (shot probability increased from 0.18 ± 0.16 to 0.31 ± 0.26 ( z  = −4.46,  p  < 0.001)). Moving beyond this result, we also asked human raters to assess the utility of TacticAI’s proposed adjustments within our case study, which we detail next.

Case study with expert raters

Although quantitative evaluation with well-defined benchmark datasets was critical for the technical development of TacticAI, the ultimate test of TacticAI as a football tactic assistant is its practical downstream utility being recognised by professionals in the industry. To this end, we evaluated TacticAI through a case study with our partners at Liverpool FC (LFC). Specifically, we invited a group of five football experts: three data scientists, one video analyst, and one coaching assistant. Each of them completed four tasks in the case study, which evaluated the utility of TacticAI’s components from several perspectives; these include (1) the realism of TacticAI’s generated adjustments, (2) the plausibility of TacticAI’s receiver predictions, (3) effectiveness of TacticAI’s embeddings for retrieving similar corners, and (4) usefulness of TacticAI’s recommended adjustments. We provide an overview of our study’s results here and refer the interested reader to Supplementary Figs.  3 – 5 and the  Supplementary Methods for additional details.

We first simultaneously evaluated the realism of the adjusted corner kicks generated by TacticAI, and the plausibility of its receiver predictions. Going through a collection of 50 corner kick samples, we first asked the raters to classify whether a given sample was real or generated by TacticAI, and then they were asked to identify the most likely receivers in the corner kick sample (Supplementary Fig.  3 ).

On the task of classifying real and generated samples, first, we found that the raters’ average F 1 score of classifying the real vs. generated samples was only 0.60 ± 0.04, with individual F 1 scores ( \({F}_{1}^{A}=0.54,{F}_{1}^{B}=0.64,{F}_{1}^{C}=0.65,{F}_{1}^{D}=0.62,{F}_{1}^{E}=0.56\) ), indicating that the raters were, in many situations, unable to distinguish TacticAI’s adjustments from real corners.

The previous evaluation focused on analysing realism detection performance across raters. We also conduct a study that analyses realism detection across samples. Specifically, we assigned ratings for each sample—assigning +1 to a sample if it was identified as real by a human rater, and 0 otherwise—and computed the average rating for each sample across the five raters. Importantly, by studying the distribution of ratings, we found that there was no significant difference between the average ratings assigned to real and generated corners ( z  = −0.34,  p  > 0.05) (Fig.  4 A). Hence, the real and generated samples were assigned statistically indistinguishable average ratings by human raters.

In task 1, we tested the statistical difference between the real corner kick samples and the synthetic ones generated by TacticAI from two aspects: ( A.1 ) the distributions of their assigned ratings, and ( A.2 ) the corresponding histograms of the rating values. Analogously, in task 2 (receiver prediction), ( B.1 ) we track the distributions of the top-3 accuracy of receiver prediction using those samples, and ( B.2 ) the corresponding histogram of the mean rating per sample. No statistical difference in the mean was observed in either cases (( A.1 ) ( z  = −0.34,  p  > 0.05), and ( B.1 ) ( z  = 0.97,  p  > 0.05)). Additionally, we observed a statistically significant difference between the ratings of different raters on receiver prediction, with three clear clusters emerging ( C ). Specifically, Raters A and E had similar ratings ( z  = 0.66,  p  > 0.05), and Raters B and D also rated in similar ways ( z  = −1.84,  p  > 0.05), while Rater C responded differently from all other raters. This suggests a good level of variety of the human raters with respect to their perceptions of corner kicks. In task 3—identifying similar corners retrieved in terms of salient strategic setups—there were no significant differences among the distributions of the ratings by different raters ( D ), suggesting a high level of agreement on the usefulness of TacticAI’s capability of retrieving similar corners ( F 1,4  = 1.01,  p  > 0.1). Finally, in task 4, we compared the ratings of TacticAI’s strategic refinements across the human raters ( E ) and found that the raters also agreed on the general effectiveness of the refinements recommended by TacticAI ( F 1,4  = 0.45,  p  > 0.05). Note that the violin plots used in B.1 and C – E model a continuous probability distribution and hence assign nonzero probabilities to values outside of the allowed ranges. We only label y -axis ticks for the possible set of ratings.

For the task of identifying receivers, we rated TacticAI’s predictions with respect to a rater as +1 if at least one of the receivers identified by the rater appeared in TacticAI’s top-3 predictions, and 0 otherwise. The average top-3 accuracy among the human raters was 0.79 ± 0.18; specifically, 0.81 ± 0.17 for the real samples, and 0.77 ± 0.21 for the generated ones. These scores closely line up with the accuracy of TacticAI in predicting receivers for held-out test corners, validating our quantitative study. Further, after averaging the ratings for receiver prediction sample-wise, we found no statistically significant difference between the average ratings of predicting receivers over the real and generated samples ( z  = 0.97,  p  > 0.05) (Fig.  4 B). This indicates that TacticAI was equally performant in predicting the receivers of real corners and TacticAI-generated adjustments, and hence may be leveraged for this purpose even in simulated scenarios.

There is a notably high variance in the average receiver prediction rating of TacticAI. We hypothesise that this is due to the fact that different raters may choose to focus on different salient features when evaluating the likely receivers (or even the amount of likely receivers). We set out to validate this hypothesis by testing the pair-wise similarity of the predictions by the human raters through running a one-away analysis of variance (ANOVA), followed by a Tukey test. We found that the distributions of the five raters’ predictions were significantly different ( F 1,4  = 14.46,  p  < 0.001) forming three clusters (Fig.  4 C). This result indicates that different human raters—as suggested by their various titles at LFC—may often use very different leads when suggesting plausible receivers. The fact that TacticAI manages to retain a high top-3 accuracy in such a setting suggests that it was able to capture the salient patterns of corner kick strategies, which broadly align with human raters’ preferences. We will further test this hypothesis in the third task—identifying similar corners.

For the third task, we asked the human raters to judge 50 pairs of corners for their similarity. Each pair consisted of a reference corner and a retrieved corner, where the retrieved corner was chosen either as the nearest-neighbour of the reference in terms of their TacticAI latent space representations, or—as a feature-level heuristic—the cosine similarities of their raw features (Supplementary Fig.  4 ) in our corner kick dataset. We score the raters’ judgement of a pair as +1 if they considered the corners presented in the case to be usefully similar, otherwise, the pair is scored with 0. We first computed, for each rater, the recall with which they have judged a baseline- or TacticAI-retrieved pair as usefully similar—see description of Task 3 in the  Supplementary Methods . For TacticAI retrievals, the average recall across all raters was 0.59 ± 0.09, and for the baseline system, the recall was 0.36 ± 0.10. Secondly, we assess the statistical difference between the results of the two methods by averaging the ratings for each reference–retrieval pair, finding that the average rating of TacticAI retrievals is significantly higher than the average rating of baseline method retrievals ( z  = 2.34,  p  < 0.05). These two results suggest that TacticAI significantly outperforms the feature-space baseline as a method for mining similar corners. This indicates that TacticAI is able to extract salient features from corners that are not trivial to extract from the input data alone, reinforcing it as a potent tool for discovering opposing team tactics from available data. Finally, we observed that this task exhibited a high level of inter-rater agreement for TacticAI-retrieved pairs ( F 1,4  = 1.01,  p  > 0.1) (Fig.  4 D), suggesting that human raters were largely in agreement with respect to their assessment of TacticAI’s performance.

Finally, we evaluated TacticAI’s player adjustment recommendations for their practical utility. Specifically, each rater was given 50 tactical refinements together with the corresponding real corner kick setups—see Supplementary Fig.  5 , and the “Case study design” section in the  Supplementary Methods . The raters were then asked to rate each refinement as saliently improving the tactics (+1), saliently making them worse (−1), or offering no salient differences (0). We calculated the average rating assigned by each of the raters (giving us a value in the range [− 1, 1] for each rater). The average of these values across all five raters was 0.7 ± 0.1. Further, for 45 of the 50 situations (90%), the human raters found TacticAI’s suggestion to be favourable on average (by majority voting). Both of these results indicate that TacticAI’s recommendations are salient and useful to a downstream football club practitioner, and we set out to validate this with statistical tests.

We performed statistical significance testing of the observed positive ratings. First, for each of the 50 situations, we averaged its ratings across all five raters and then ran a t -test to assess whether the mean rating was significantly larger than zero. Indeed, the statistical test indicated that the tactical adjustments recommended by TacticAI were constructive overall ( \({t}_{49}^{{{{{{{{\rm{avg}}}}}}}}}=9.20,\, p \, < \, 0.001\) ). Secondly, we verified that each of the five raters individually found TacticAI’s recommendations to be constructive, running a t -test on each of their ratings individually. For all of the five raters, their average ratings were found to be above zero with statistical significance ( \({t}_{49}^{A}=5.84,\, {p}^{A} \, < \, 0.001;{t}_{49}^{B}=7.88,\; {p}^{B} \, < \, 0.001;{t}_{49}^{C}=7.00,\; {p}^{C} \, < \, 0.001;{t}_{49}^{D}=6.04,\; {p}^{D} \, < \, 0.001;{t}_{49}^{E}=7.30,\, {p}^{E} \, < \, 0.001\) ). In addition, their ratings also shared a high level of inter-agreement ( F 1,4  = 0.45,  p  > 0.05) (Fig.  4 E), suggesting a level of practical usefulness that is generally recognised by human experts, even though they represent different backgrounds.

Taking all of these results together, we find TacticAI to possess strong components for prediction, retrieval, and tactical adjustments on corner kicks. To illustrate the kinds of salient recommendations by TacticAI, in Fig.  5 we present four examples with a high degree of inter-rater agreement.

These examples are selected from our case study with human experts, to illustrate the breadth of tactical adjustments that TacticAI suggests to teams defending a corner. The density of the yellow circles coincides with the number of times that the corresponding change is recognised as constructive by human experts. Instead of optimising the movement of one specific player, TacticAI can recommend improvements for multiple players in one generation step through suggesting better positions to block the opposing players, or better orientations to track them more efficiently. Some specific comments from expert raters follow. In A , according to raters, TacticAI suggests more favourable positions for several defenders, and improved tracking runs for several others—further, the goalkeeper is positioned more deeply, which is also beneficial. In B , TacticAI suggests that the defenders furthest away from the corner make improved covering runs, which was unanimously deemed useful, with several other defenders also positioned more favourably. In C , TacticAI recommends improved covering runs for a central group of defenders in the penalty box, which was unanimously considered salient by our raters. And in D , TacticAI suggests substantially better tracking runs for two central defenders, along with a better positioning for two other defenders in the goal area.

We have demonstrated an AI assistant for football tactics and provided statistical evidence of its efficacy through a comprehensive case study with expert human raters from Liverpool FC. First, TacticAI is able to accurately predict the first receiver after a corner kick is taken as well as the probability of a shot as the direct result of the corner. Second, TacticAI has been shown to produce plausible tactical variations that improve outcomes in a salient way, while being indistinguishable from real scenarios by domain experts. And finally, the system’s latent player representations are a powerful means to retrieve similar set-piece tactics, allowing coaches to analyse relevant tactics and counter-tactics that have been successful in the past.

The broader scope of strategy modelling in football has previously been addressed from various individual angles, such as pass prediction 14 , 15 , 16 , shot prediction 3 or corner kick tactical classification 7 . However, to the best of our knowledge, our work stands out by combining and evaluating predictive and generative modelling of corner kicks for tactic development. It also stands out in its method of applying geometric deep learning, allowing for efficiently incorporating various symmetries of the football pitch for improved data efficiency. Our method incorporates minimal domain knowledge and does not rely on intricate feature engineering—though its factorised design naturally allows for more intricate feature engineering approaches when such features are available.

Our methodology requires the position and velocity estimates of all players at the time of execution of the corner and subsequent events. Here, we derive these from high-quality tracking and event data, with data availability from tracking providers limited to top leagues. Player tracking based on broadcast video would increase the reach and training data substantially, but would also likely result in noisier model inputs. While the attention mechanism of GATs would allow us to perform introspection of the most salient factors contributing to the model outcome, our method does not explicitly model exogenous (aleatoric) uncertainty, which would be valuable context for the football analyst.

While the empirical study of our method’s efficacy has been focused on corner kicks in association football, it readily generalises to other set pieces (such as throw-ins, which similarly benefit from similarity retrieval, pass and/or shot prediction) and other team sports with suspended play situations. The learned representations and overall framing of TacticAI also lay the ground for future research to integrate a natural language interface that enables domain-grounded conversations with the assistant, with the aim to retrieve particular situations of interest, make predictions for a given tactical variant, compare and contrast, and guide through an interactive process to derive tactical suggestions. It is thus our belief that TacticAI lays the groundwork for the next-generation AI assistant for football.

We devised TacticAI as a geometric deep learning pipeline, further expanded in this section. We process labelled spatio-temporal football data into graph representations, and train and evaluate on benchmarking tasks cast as classification or regression. These steps are presented in sequence, followed by details on the employed computational architecture.

Raw corner kick data

The raw dataset consisted of 9693 corner kicks collected from the 2020–21, 2021–22, and 2022–23 (up to January 2023) Premier League seasons. The dataset was provided by Liverpool FC and comprises four separate data sources, described below.

Our primary data source is spatio-temporal trajectory frames (tracking data), which tracked all on-pitch players and the ball, for each match, at 25 frames per second. In addition to player positions, their velocities are derived from position data through filtering. For each corner kick, we only used the frame in which the kick is being taken as input information.

Secondly, we also leverage event stream data, which annotated the events or actions (e.g., passes, shots and goals) that have occurred in the corresponding tracking frames.

Thirdly, the line-up data for the corresponding games, which recorded the players’ profiles, including their heights, weights and roles, is also used.

Lastly, we have access to miscellaneous game data, which contains the game days, stadium information, and pitch length and width in meters.

Graph representation and construction

We assumed that we were provided with an input graph \({{{{{{{\mathcal{G}}}}}}}}=({{{{{{{\mathcal{V}}}}}}}},\,{{{{{{{\mathcal{E}}}}}}}})\) with a set of nodes \({{{{{{{\mathcal{V}}}}}}}}\) and edges \({{{{{{{\mathcal{E}}}}}}}}\subseteq {{{{{{{\mathcal{V}}}}}}}}\times {{{{{{{\mathcal{V}}}}}}}}\) . Within the context of football games, we took \({{{{{{{\mathcal{V}}}}}}}}\) to be the set of 22 players currently on the pitch for both teams, and we set \({{{{{{{\mathcal{E}}}}}}}}={{{{{{{\mathcal{V}}}}}}}}\times {{{{{{{\mathcal{V}}}}}}}}\) ; that is, we assumed all pairs of players have the potential to interact. Further analyses, leveraging more specific choices of \({{{{{{{\mathcal{E}}}}}}}}\) , would be an interesting avenue for future work.

Additionally, we assume that the graph is appropriately featurised. Specifically, we provide a node feature matrix, \({{{{{{{\bf{X}}}}}}}}\in {{\mathbb{R}}}^{| {{{{{{{\mathcal{V}}}}}}}}| \times k}\) , an edge feature tensor, \({{{{{{{\bf{E}}}}}}}}\in {{\mathbb{R}}}^{| {{{{{{{\mathcal{V}}}}}}}}| \times | {{{{{{{\mathcal{V}}}}}}}}| \times l}\) , and a graph feature vector, \({{{{{{{\bf{g}}}}}}}}\in {{\mathbb{R}}}^{m}\) . The appropriate entries of these objects provide us with the input features for each node, edge, and graph. For example, \({{{{{{{{\bf{x}}}}}}}}}_{u}\in {{\mathbb{R}}}^{k}\) would provide attributes of an individual player \(u\in {{{{{{{\mathcal{V}}}}}}}}\) , such as position, height and weight, and \({{{{{{{{\bf{e}}}}}}}}}_{uv}\in {{\mathbb{R}}}^{l}\) would provide the attributes of a particular pair of players \((u,\, v)\in {{{{{{{\mathcal{E}}}}}}}}\) , such as their distance, and whether they belong to the same team. The graph feature vector, g , can be used to store global attributes of interest to the corner kick, such as the game time, current score, or ball position. For a simplified visualisation of how a graph neural network would process such an input, refer to Fig.  1 A.

To construct the input graphs, we first aligned the four data sources with respect to their game IDs and timestamps and filtered out 2517 invalid corner kicks, for which the alignment failed due to missing data, e.g., missing tracking frames or event labels. This filtering yielded 7176 valid corner kicks for training and evaluation. We summarised the exact information that was used to construct the input graphs in Table  2 . In particular, other than player heights (measured in centimeters (cm)) and weights (measured in kilograms (kg)), the players were anonymous in the model. For the cases in which the player profiles were missing, we set their heights and weights to 180 cm and 75 kg, respectively, as defaults. In total, we had 385 such occurrences out of a total of 213,246( = 22 × 9693) during data preprocessing. We downscaled the heights and weights by a factor of 100. Moreover, for each corner kick, we zero-centred the positions of on-pitch players and normalised them onto a 10 m × 10 m pitch, and their velocities were re-scaled accordingly. For the cases in which the pitch dimensions were missing, we used a standard pitch dimension of 110 m × 63 m as default.

We summarised the grouping of the features in Table  1 . The actual features used in different benchmark tasks may differ, and we will describe this in more detail in the next section. To focus on modelling the high-level tactics played by the attacking and defending teams, other than a binary indicator for ball possession—which is 1 for the corner kick taker and 0 for all other players—no information of ball movement, neither positions nor velocities, was used to construct the input graphs. Additionally, we do not have access to the player’s vertical movement, therefore only information on the two-dimensional movements of each player is provided in the data. We do however acknowledge that such information, when available, would be interesting to consider in a corner kick outcome predictor, considering the prevalence of aerial battles in corners.

Benchmark tasks construction

TacticAI consists of three predictive and generative models, which also correspond to three benchmark tasks implemented in this study. Specifically, (1) Receiver prediction, (2) Threatening shot prediction, and (3) Guided generation of team positions and velocities (Table  1 ). The graphs of all the benchmark tasks used the same feature space of nodes and edges, differing only in the global features.

For all three tasks, our models first transform the node features to a latent node feature matrix, \({{{{{{{\bf{H}}}}}}}}={f}_{{{{{{{{\mathcal{G}}}}}}}}}({{{{{{{\bf{X}}}}}}}},\, {{{{{{{\bf{E}}}}}}}},\, {{{{{{{\bf{g}}}}}}}})\) , from which we could answer queries: either about individual players—in which case we learned a relevant classifier or regressor over the h u vectors (the rows of H )—or about the occurrence of a global event (e.g. shot taken)—in which case we classified or regressed over the aggregated player vectors, ∑ u h u . In both cases, the classifiers were trained using stochastic gradient descent over an appropriately chosen loss function, such as categorical cross-entropy for classifiers, and mean squared error for regressors.

For different tasks, we extracted the corresponding ground-truth labels from either the event stream data or the tracking data. Specifically, (1) We modelled receiver prediction as a node classification task and labelled the first player to touch the ball after the corner was taken as the target node. This player could be either an attacking or defensive player. (2) Shot prediction was modelled as graph classification. In particular, we considered a next-ball-touch action by the attacking team as a shot if it was a direct corner, a goal, an aerial, hit on the goalposts, a shot attempt saved by the goalkeeper, or missing target. This yielded 1736 corners labelled as a shot being taken, and 5440 corners labelled as a shot not being taken. (3) For guided generation of player position and velocities, no additional label was needed, as this model relied on a self-supervised reconstruction objective.

The entire dataset was split into training and evaluation sets with an 80:20 ratio through random sampling, and the same splits were used for all tasks.

Graph neural networks

The central model of TacticAI is the graph neural network (GNN) 9 , which computes latent representations on a graph by repeatedly combining them within each node’s neighbourhood. Here we define a node’s neighbourhood, \({{{{{{{{\mathcal{N}}}}}}}}}_{u}\) , as the set of all first-order neighbours of node u , that is, \({{{{{{{{\mathcal{N}}}}}}}}}_{u}=\{v\,| \,(v,\, u)\in {{{{{{{\mathcal{E}}}}}}}}\}\) . A single GNN layer then transforms the node features by passing messages between neighbouring nodes 17 , following the notation of related work 10 , and the implementation of the CLRS-30 benchmark baselines 18 :

where \(\psi :{{\mathbb{R}}}^{k}\times {{\mathbb{R}}}^{k}\times {{\mathbb{R}}}^{l}\times {{\mathbb{R}}}^{m}\to {{\mathbb{R}}}^{{k}^{{\prime} }}\) and \(\phi :{{\mathbb{R}}}^{k}\times {{\mathbb{R}}}^{{k}^{{\prime} }}\to {{\mathbb{R}}}^{{k}^{{\prime} }}\) are two learnable functions (e.g. multilayer perceptrons), \({{{{{{{{\bf{h}}}}}}}}}_{u}^{(t)}\) are the features of node u after t GNN layers, and ⨁ is any permutation-invariant aggregator, such as sum, max, or average. By definition, we set \({{{{{{{{\bf{h}}}}}}}}}_{u}^{(0)}={{{{{{{{\bf{x}}}}}}}}}_{u}\) , and iterate Eq. ( 2 ) for T steps, where T is a hyperparameter. Then, we let \({{{{{{{\bf{H}}}}}}}}={f}_{{{{{{{{\mathcal{G}}}}}}}}}({{{{{{{\bf{X}}}}}}}},\, {{{{{{{\bf{E}}}}}}}},\, {{{{{{{\bf{g}}}}}}}})={{{{{{{{\bf{H}}}}}}}}}^{(T)}\) be the final node embeddings coming out of the GNN.

It is well known that Eq. ( 2 ) is remarkably general; it can be used to express popular models such as Transformers 19 as a special case, and it has been argued that all discrete deep learning models can be expressed in this form 20 , 21 . This makes GNNs a perfect framework for benchmarking various approaches to modelling player–player interactions in the context of football.

Different choices of ψ , ϕ and ⨁ yield different architectures. In our case, we utilise a message function that factorises into an attentional mechanism, \(a:{{\mathbb{R}}}^{k}\times {{\mathbb{R}}}^{k}\times {{\mathbb{R}}}^{l}\times {{\mathbb{R}}}^{m}\to {\mathbb{R}}\) :

yielding the graph attention network (GAT) architecture 12 . In our work, specifically, we use a two-layer multilayer perceptron for the attentional mechanism, as proposed by GATv2 11 :

where \({{{{{{{{\bf{W}}}}}}}}}_{1},\, {{{{{{{{\bf{W}}}}}}}}}_{2}\in {{\mathbb{R}}}^{k\times h}\) , \({{{{{{{{\bf{W}}}}}}}}}_{e}\in {{\mathbb{R}}}^{l\times h}\) , \({{{{{{{{\bf{W}}}}}}}}}_{g}\in {{\mathbb{R}}}^{m\times h}\) and \({{{{{{{\bf{a}}}}}}}}\in {{\mathbb{R}}}^{h}\) are the learnable parameters of the attentional mechanism, and LeakyReLU is the leaky rectified linear activation function. This mechanism computes coefficients of interaction (a single scalar value) for each pair of connected nodes ( u ,  v ), which are then normalised across all neighbours of u using the \({{{{{{{\rm{softmax}}}}}}}}\) function.

Through early-stage experimentation, we have ascertained that GATs are capable of matching the performance of more generic choices of ψ (such as the MPNN 17 ) while being more scalable. Hence, we focus our study on the GAT model in this work. More details can be found in the subsection “Ablation study” section.

Geometric deep learning

In spite of the power of Eq. ( 2 ), using it in its full generality is often prone to overfitting, given the large number of parameters contained in ψ and ϕ . This problem is exacerbated in the football analytics domain, where gold-standard data is generally very scarce—for example, in the English Premier League, only a few hundred games are played every season.

In order to tackle this issue, we can exploit the immense regularity of data arising from football games. Strategically equivalent game states are also called transpositions, and symmetries such as arriving at the same chess position through different move sequences have been exploited computationally since the 1960s 22 . Similarly, game rotations and reflections may yield equivalent strategic situations 23 . Using the blueprint of geometric deep learning (GDL) 10 , we can design specialised GNN architectures that exploit this regularity.

That is, geometric deep learning is a generic methodology for deriving mathematical constraints on neural networks, such that they will behave predictably when inputs are transformed in certain ways. In several important cases, these constraints can be directly resolved, directly informing neural network architecture design. For a comprehensive example of point clouds under 3D rotational symmetry, see Fuchs et al. 24 .

To elucidate several aspects of the GDL framework on a high level, let us assume that there exists a group of input data transformations (symmetries), \({\mathfrak{G}}\) under which the ground-truth label remains unchanged. Specifically, if we let y ( X ,  E ,  g ) be the label given to the graph featurised with X ,  E ,  g , then for every transformation \({\mathfrak{g}}\in {\mathfrak{G}}\) , the following property holds:

This condition is also referred to as \({\mathfrak{G}}\) -invariance. Here, by \({\mathfrak{g}}({{{{{{{\bf{X}}}}}}}})\) we denote the result of transforming X by \({\mathfrak{g}}\) —a concept also known as a group action. More generally, it is a function of the form \({\mathfrak{G}}\times {{{{{{{\mathcal{S}}}}}}}}\to {{{{{{{\mathcal{S}}}}}}}}\) for some state set \({{{{{{{\mathcal{S}}}}}}}}\) . Note that a single group element, \({\mathfrak{g}}\in {\mathfrak{G}}\) can easily produce different actions on different \({{{{{{{\mathcal{S}}}}}}}}\) —in this case, \({{{{{{{\mathcal{S}}}}}}}}\) could be \({{\mathbb{R}}}^{| {{{{{{{\mathcal{V}}}}}}}}| \times k}\) ( X ), \({{\mathbb{R}}}^{| {{{{{{{\mathcal{V}}}}}}}}| \times | {{{{{{{\mathcal{V}}}}}}}}| \times l}\) ( E ) and \({{\mathbb{R}}}^{m}\) ( g ).

It is worth noting that GNNs may also be derived using a GDL perspective if we set the symmetry group \({\mathfrak{G}}\) to \({S}_{| {{{{{{{\mathcal{V}}}}}}}}}|\) , the permutation group of \(| {{{{{{{\mathcal{V}}}}}}}}|\) objects. Owing to the design of Eq. ( 2 ), its outputs will not be dependent on the exact permutation of nodes in the input graph.

Frame averaging

A simple mechanism to enforce \({\mathfrak{G}}\) -invariance, given any predictor \({f}_{{{{{{{{\mathcal{G}}}}}}}}}({{{{{{{\bf{X}}}}}}}},\, {{{{{{{\bf{E}}}}}}}},\, {{{{{{{\bf{g}}}}}}}})\) , performs frame averaging across all \({\mathfrak{G}}\) -transformed inputs:

This ensures that all \({\mathfrak{G}}\) -transformed versions of a particular input (also known as that input’s orbit) will have exactly the same output, satisfying Eq. ( 5 ). A variant of this approach has also been applied in the AlphaGo architecture 25 to encode symmetries of a Go board.

In our specific implementation, we set \({\mathfrak{G}}={D}_{2}=\{{{{{{{{\rm{id}}}}}}}},\leftrightarrow,\updownarrow,\leftrightarrow \updownarrow \}\) , the dihedral group. Exploiting D 2 -invariance allows us to encode quadrant symmetries. Each element of the D 2 group encodes the presence of vertical or horizontal reflections of the input football pitch. Under these transformations, the pitch is assumed completely symmetric, and hence many predictions, such as which player receives the corner kick, or takes a shot from it, can be safely assumed unchanged. As an example of how to compute transformed features in Eq. ( 6 ), ↔( X ) horizontally reflects all positional features of players in X (e.g. the coordinates of the player), and negates the x -axis component of their velocity.

Group convolutions

While the frame averaging approach of Eq. ( 6 ) is a powerful way to restrict GNNs to respect input symmetries, it arguably misses an opportunity for the different \({\mathfrak{G}}\) -transformed views to interact while their computations are being performed. For small groups such as D 2 , a more fine-grained approach can be assumed, operating over a single GNN layer in Eq. ( 2 ), which we will write shortly as \({{{{{{{{\bf{H}}}}}}}}}^{(t)}={g}_{{{{{{{{\mathcal{G}}}}}}}}}({{{{{{{{\bf{H}}}}}}}}}^{(t-1)},\, {{{{{{{\bf{E}}}}}}}},\, {{{{{{{\bf{g}}}}}}}})\) . The condition that we need a symmetry-respecting GNN layer to satisfy is as follows, for all transformations \({\mathfrak{g}}\in {\mathfrak{G}}\) :

that is, it does not matter if we apply \({\mathfrak{g}}\) it to the input or the output of the function \({g}_{{{{{{{{\mathcal{G}}}}}}}}}\) —the final answer is the same. This condition is also referred to as \({\mathfrak{G}}\) -equivariance, and it has recently proved to be a potent paradigm for developing powerful GNNs over biochemical data 24 , 26 .

To satisfy D 2 -equivariance, we apply the group convolution approach 13 . Therein, views of the input are allowed to directly interact with their \({\mathfrak{G}}\) -transformed variants, in a manner very similar to grid convolutions (which is, indeed, a special case of group convolutions, setting \({\mathfrak{G}}\) to be the translation group). We use \({{{{{{{{\bf{H}}}}}}}}}_{{\mathfrak{g}}}^{(t)}\) to denote the \({\mathfrak{g}}\) -transformed view of the latent node features at layer t . Omitting E and g inputs for brevity, and using our previously designed layer \({g}_{{{{{{{{\mathcal{G}}}}}}}}}\) as a building block, we can perform a group convolution as follows:

Here, ∥ is the concatenation operation, joining the two node feature matrices column-wise; \({{\mathfrak{g}}}^{-1}\) is the inverse transformation to \({\mathfrak{g}}\) (which must exist as \({\mathfrak{G}}\) is a group); and \({{\mathfrak{g}}}^{-1}{\mathfrak{h}}\) is the composition of the two transformations.

Effectively, Eq. ( 8 ) implies our D 2 -equivariant GNN needs to maintain a node feature matrix \({{{{{{{{\bf{H}}}}}}}}}_{{\mathfrak{g}}}^{(t)}\) for every \({\mathfrak{G}}\) -transformation of the current input, and these views are recombined by invoking \({g}_{{{{{{{{\mathcal{G}}}}}}}}}\) on all pairs related together by applying a transformation \({\mathfrak{h}}\) . Note that all reflections are self-inverses, hence, in D 2 , \({\mathfrak{g}}={{\mathfrak{g}}}^{-1}\) .

It is worth noting that both the frame averaging in Eq. ( 6 ) and group convolution in Eq. ( 8 ) are similar in spirit to data augmentation. However, whereas standard data augmentation would only show one view at a time to the model, a frame averaging/group convolution architecture exhaustively generates all views and feeds them to the model all at once. Further, group convolutions allow these views to explicitly interact in a way that does not break symmetries. Here lies the key difference between the two approaches: frame averaging and group convolutions rigorously enforce the symmetries in \({\mathfrak{G}}\) , whereas data augmentation only provides implicit hints to the model about satisfying them. As a consequence of the exhaustive generation, Eqs. ( 6 ) and ( 8 ) are only feasible for small groups like D 2 . For larger groups, approaches like Steerable CNNs 27 may be employed.

Network architectures

While the three benchmark tasks we are performing have minor differences in the global features available to the model, the neural network models designed for them all have the same encoder–decoder architecture. The encoder has the same structure in all tasks, while the decoder model is tailored to produce appropriately shaped outputs for each benchmark task.

Given an input graph, TacticAI’s model first generates all relevant D 2 -transformed versions of it, by appropriately reflecting the player coordinates and velocities. We refer to the original input graph as the identity view, and the remaining three D 2 -transformed graphs as reflected views.

Once the views are prepared, we apply four group convolutional layers (Eq. ( 8 )) with a GATv2 base model (Eqs. ( 3 ) and ( 4 )) as the \({g}_{{{{{{{{\mathcal{G}}}}}}}}}\) function. Specifically, this means that, in Eqs. ( 3 ) and ( 4 ), every instance of \({{{{{{{{\bf{h}}}}}}}}}_{u}^{(t-1)}\) is replaced by the concatenation of \({({{{{{{{{\bf{h}}}}}}}}}_{{\mathfrak{h}}}^{(t-1)})}_{u}\parallel {({{{{{{{{\bf{h}}}}}}}}}_{{{\mathfrak{g}}}^{-1}{\mathfrak{h}}}^{(t-1)})}_{u}\) . Each GATv2 layer has eight attention heads and computes four latent features overall per player. Accordingly, once the four group convolutions are performed, we have a representation of \({{{{{{{\bf{H}}}}}}}}\in {{\mathbb{R}}}^{4\times 22\times 4}\) , where the first dimension corresponds to the four views ( \({{{{{{{{\bf{H}}}}}}}}}_{{{{{{{{\rm{id}}}}}}}}},\, {{{{{{{{\bf{H}}}}}}}}}_{\leftrightarrow },\, {{{{{{{{\bf{H}}}}}}}}}_{\updownarrow },\, {{{{{{{{\bf{H}}}}}}}}}_{\leftrightarrow \updownarrow }\in {{\mathbb{R}}}^{22\times 4}\) ), the second dimension corresponds to the players (eleven on each team), and the third corresponds to the 4-dimensional latent vector for each player node in this particular view. How this representation is used by the decoder depends on the specific downstream task, as we detail below.

For receiver prediction, which is a fully invariant function (i.e. reflections do not change the receiver), we perform simple frame averaging across all views, arriving at

and then learn a node-wise classifier over the rows of \({{{{{{{{\bf{H}}}}}}}}}^{{{{{{{{\rm{node}}}}}}}}}\in {{\mathbb{R}}}^{22\times 4}\) . We further decode H node into a logit vector \({{{{{{{\bf{O}}}}}}}}\in {{\mathbb{R}}}^{22}\) with a linear layer before computing the corresponding softmax cross entropy loss.

For shot prediction, which is once again fully invariant (i.e. reflections do not change the probability of a shot), we can further average the frame-averaged features across all players to get a global graph representation:

and then learn a binary classifier over \({{{{{{{{\bf{h}}}}}}}}}^{{{{{{{{\rm{graph}}}}}}}}}\in {{\mathbb{R}}}^{4}\) . Specifically, we decode the hidden vector into a single logit with a linear layer and compute the sigmoid binary cross-entropy loss with the corresponding label.

For guided generation (position/velocity adjustments), we generate the player positions and velocities with respect to a particular outcome of interest for the human coaches, predicted over the rows of the hidden feature matrix. For example, the model may adjust the defensive setup to decrease the shot probability by the attacking team. The model output is now equivariant rather than invariant—reflecting the pitch appropriately reflects the predicted positions and velocity vectors. As such, we cannot perform frame averaging, and take only the identity view’s features, \({{{{{{{{\bf{H}}}}}}}}}_{{{{{{{{\rm{id}}}}}}}}}\in {{\mathbb{R}}}^{22\times 4}\) . From this latent feature matrix, we can then learn a conditional distribution from each row, which models the positions or velocities of the corresponding player. To do this, we extend the backbone encoder with conditional variational autoencoder (CVAE 28 , 29 ). Specifically, for the u -th row of H id , h u , we first map its latent embedding to the parameters of a two-dimensional Gaussian distribution \({{{{{{{\mathcal{N}}}}}}}}({\mu }_{u}| {\sigma }_{u})\) , and then sample the coordinates and velocities from this distribution. At training time, we can efficiently propagate gradients through this sampling operation using the reparameterisation trick 28 : sample a random value \({\epsilon }_{u} \sim {{{{{{{\mathcal{N}}}}}}}}(0,1)\) for each player from the unit Gaussian distribution, and then treat μ u  +  σ u ϵ u as the sample for this player. In what follows, we omit edge features for brevity. For each corner kick sample X with the corresponding outcome o (e.g. a binary value indicating a shot event), we extend the standard VAE loss 28 , 29 to our case of outcome-conditional guided generation as

where h u is the player embedding corresponding to the u th row of H id , and \({\mathbb{KL}}\) is Kullback–Leibler (KL) divergence. Specifically, the first term is the generation loss between the real player input x u and the reconstructed sample decoded from h u with the decoder p ϕ . Using the KL term, the distribution of the latent embedding h u is regularised towards p ( h u ∣ o ), which is a multivariate Gaussian in our case.

A complete high-level summary of the generic encoder–decoder equivariant architecture employed by TacticAI can be summarised in Supplementary Fig.  2 . In the following section, we will provide empirical evidence for justifying these architectural decisions. This will be done through targeted ablation studies on our predictive benchmarks (receiver prediction and shot prediction).

Ablation study

We leveraged the receiver prediction task as a way to evaluate various base model architectures, and directly quantitatively assess the contributions of geometric deep learning in this context. We already see that the raw corner kick data can be better represented through geometric deep learning, yielding separable clusters in the latent space that could correspond to different attacking or defending tactics (Fig.  2 ). In addition, we hypothesise that these representations can also yield better performance on the task of receiver prediction. Accordingly, we ablate several design choices using deep learning on this task, as illustrated by the following four questions:

Does a factorised graph representation help? To assess this, we compare it against a convolutional neural network (CNN 30 ) baseline, which does not leverage a graph representation.

Does a graph structure help? To assess this, we compare against a Deep Sets 31 baseline, which only models each node in isolation without considering adjacency information—equivalently, setting each neighbourhood \({{{{{{{{\mathcal{N}}}}}}}}}_{u}\) to a singleton set { u }.

Are attentional GNNs a good strategy? To assess this, we compare against a message passing neural network 32 , MPNN baseline, which uses the fully potent GNN layer from Eq. ( 2 ) instead of the GATv2.

Does accounting for symmetries help? To assess this, we compare our geometric GATv2 baseline against one which does not utilise D 2 group convolutions but utilises D 2 frame averaging, and one which does not explicitly utilise any aspect of D 2 symmetries at all.

Each of these models has been trained for a fixed budget of 50,000 training steps. The test top- k receiver prediction accuracies of the trained models are provided in Supplementary Table  2 . As already discussed in the section “Results”, there is a clear advantage to using a full graph structure, as well as directly accounting for reflection symmetry. Further, the usage of the MPNN layer leads to slight overfitting compared to the GATv2, illustrating how attentional GNNs strike a good balance of expressivity and data efficiency for this task. Our analysis highlights the quantitative benefits of both graph representation learning and geometric deep learning for football analytics from tracking data. We also provide a brief ablation study for the shot prediction task in Supplementary Table  3 .

Training details

We train each of TacticAI’s models in isolation, using NVIDIA Tesla P100 GPUs. To minimise overfitting, each model’s learning objective is regularised with an L 2 norm penalty with respect to the network parameters. During training, we use the Adam stochastic gradient descent optimiser 33 over the regularised loss.

All models, including baselines, have been given an equal hyperparameter tuning budget, spanning the number of message passing steps ({1, 2, 4}), initial learning rate ({0.0001, 0.00005}), batch size ({128, 256}) and L 2 regularisation coefficient ({0.01, 0.005, 0.001, 0.0001, 0}). We summarise the chosen hyperparameters of each TacticAI model in Supplementary Table  1 .

Data availability

The data collected in the human experiments in this study have been deposited in the Zenodo database under accession code https://zenodo.org/records/10557063 , and the processed data which is used in the statistical analysis and to generate the relevant figures in the main text are available under the same accession code. The input and output data generated and/or analysed during the current study are protected and are not available due to data privacy laws and licensing restrictions. However, contact details of the input data providers are available from the corresponding authors on reasonable request.

Code availability

All the core models described in this research were built with the Graph Neural Network processors provided by the CLRS Algorithmic Reasoning Benchmark 18 , and their source code is available at https://github.com/google-deepmind/clrs . We are unable to release our code for this work as it was developed in a proprietary context; however, the corresponding authors are open to answer specific questions concerning re-implementations on request. For general data analysis, we used the following freely available packages: numpy v1.25.2 , pandas v1.5.3 , matplotlib v3.6.1 , seaborn v0.12.2 and scipy v1.9.3 . Specifically, the code of the statistical analysis conducted in this study is available at https://zenodo.org/records/10557063 .

The International Football Association Board (IFAB). Laws of the Game (The International Football Association Board, 2023).

Tuyls, K. et al. Game plan: what AI can do for football, and what football can do for AI. J. Artif. Intell. Res. 71 , 41–88 (2021).

Article   Google Scholar  

Goka, R., Moroto, Y., Maeda, K., Ogawa, T. & Haseyama, M. Prediction of shooting events in soccer videos using complete bipartite graphs and players’ spatial–temporal relations. Sensors 23 , 4506 (2023).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Omidshafiei, S. et al. Multiagent off-screen behavior prediction in football. Sci. Rep. 12 , 8638 (2022).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Lang, S., Wild, R., Isenko, A. & Link, D. Predicting the in-game status in soccer with machine learning using spatiotemporal player tracking data. Sci. Rep. 12 , 16291 (2022).

Baccouche, M., Mamalet, F., Wolf, C., Garcia, C. & Baskurt, A. Action classification in soccer videos with long short-term memory recurrent neural networks. In International Conference on Artificial Neural Networks (eds Diamantaras, K., Duch, W. & Iliadis, L. S.) pages 154–159 (Springer, 2010).

Shaw, L. & Gopaladesikan, S. Routine inspection: a playbook for corner kicks. In Machine Learning and Data Mining for Sports Analytics: 7th International Workshop, MLSA 2020, Co-located with ECML/PKDD 2020 , Proceedings, Ghent, Belgium, September 14–18, 2020, Vol . 7 , 3–16 (Springer, 2020).

Araújo, D. & Davids, K. Team synergies in sport: theory and measures. Front. Psychol. 7 , 1449 (2016).

Article   PubMed   PubMed Central   Google Scholar  

Veličković, P. Everything is connected: graph neural networks. Curr. Opin. Struct. Biol. 79 , 102538 (2023).

Article   PubMed   Google Scholar  

Bronstein, M. M., Bruna, J., Cohen, T. & Veličković, P. Geometric deep learning: grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478 (2021).

Brody, S., Alon, U. & Yahav, E. How attentive are graph attention networks? In International Conference on Learning Representations (ICLR, 2022). https://openreview.net/forum?id=F72ximsx7C1 .

Veličković, P. et al. Graph attention networks. In International Conference on Learning Representations (ICLR, 2018). https://openreview.net/forum?id=rJXMpikCZ .

Cohen, T. & Welling, M. Group equivariant convolutional networks. In International Conference on Machine Learning (eds Balcan, M. F. & Weinberger, K. Q.) 2990–2999 (PMLR, 2016).

Honda, Y., Kawakami, R., Yoshihashi, R., Kato, K. & Naemura, T. Pass receiver prediction in soccer using video and players’ trajectories. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 3502–3511 (2022). https://ieeexplore.ieee.org/document/9857310 .

Hubáček, O., Sourek, G. & Železný, F. Deep learning from spatial relations for soccer pass prediction. In MLSA@PKDD/ECML (eds Brefeld, U., Davis, J., Van Haaren, J. & Zimmermann, A.) Vol. 11330, (Lecture Notes in Computer Science, Springer, Cham, 2018).

Sanyal, S. Who will receive the ball? Predicting pass recipient in soccer videos. J Visual Commun. Image Represent. 78 , 103190 (2021).

Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In International Conference on Machine Learning (Precup, D. & Teh, Y. W.) 1263–1272 (PMLR, 2017).

Veličković, P. et al. The CLRS algorithmic reasoning benchmark. In International Conference on Machine Learning (eds Chaudhuri, K. et al.) 22084–22102 (PMLR, 2022).

Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems (eds Guyon, I. et al.) Vol. 30 (Curran Associates, Inc., 2017).

Veličković, P. Message passing all the way up. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning (GTRL, 2022). https://openreview.net/forum?id=Bc8GiEZkTe5 .

Baranwal, A., Kimon, F. & Aukosh, J. Optimality of message-passing architectures for sparse graphs. In Thirty-seventh Conference on Neural Information Processing Systems (2023). https://papers.nips.cc/paper_files/paper/2023/hash/7e991aa4cd2fdf0014fba2f000f542d0-Abstract-Conference.html .

Greenblatt, R. D., Eastlake III, D. E. & Crocker, S. D. The Greenblatt chess program. In Proc. Fall Joint Computer Conference , 14–16 , 801–810 (Association for Computing Machinery, 1967). https://dl.acm.org/doi/10.1145/1465611.1465715 .

Schijf, M., Allis, L. V. & Uiterwijk, J. W. Proof-number search and transpositions. ICGA J. 17 , 63–74 (1994).

Fuchs, F., Worrall, D., Fischer, V. & Welling, M. SE(3)-transformers: 3D roto-translation equivariant attention networks. Adv. Neural Inf. Process. Syst. 33 , 1970–1981 (2020).

Google Scholar  

Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529 , 484–489 (2016).

Article   ADS   CAS   PubMed   Google Scholar  

Satorras, V. G., Hoogeboom, E. & Welling, M. E ( n ) equivariant graph neural networks. In International Conference on Machine Learning (eds Meila, M. & Zhang, T.) 9323–9332 (PMLR, 2021).

Cohen, T. S. & Welling, M. Steerable CNNs. In International Conference on Learning Representations (ICLR, 2017). https://openreview.net/forum?id=rJQKYt5ll .

Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14–16, 2014, Conference Track Proceedings (ICLR, 2014). https://openreview.net/forum?id=33X9fd2-9FyZd .

Sohn, K., Lee, H. & Yan, X. Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems (eds Cortes, C, Lawrence, N., Lee, D., Sugiyama, M. & Garnett, R.) Vol. 28 (Curran Associates, Inc., 2015).

Fernández, J. & Bornn, L. Soccermap: a deep learning architecture for visually-interpretable analysis in soccer. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part V (eds Dong, Y., Ifrim, G., Mladenić, D., Saunders, C. & Van Hoecke, S.) 491–506 (Springer, 2021).

Zaheer, M. et al. Deep sets. In Advances in Neural Information Processing Systems Vol. 30 (eds Guyon, I., et al.) (Curran Associates, Inc., 2017).

Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In Proc. 34th International Conference on Machine Learning , Vol. 70 of Proceedings of Machine Learning Research, 6–11 Aug 2017 (eds Precup, D. & Whye Teh, Y) 1263–1272 (PMLR, 2017).

Kingma, G. E. & Ba, J. Adam: a method for stochastic optimization. In ICLR (Poster) , (eds Bengio, Y. & LeCun, Y.) (International Conference of Learning Representations (ICLR), 2015). https://openreview.net/forum?id=8gmWwjFyLj .

Download references

Acknowledgements

We gratefully acknowledge the support of James French, Timothy Waskett, Hans Leitert and Benjamin Hervey for their extensive efforts in analysing TacticAI’s outputs. Further, we are thankful to Kevin McKee, Sherjil Ozair and Beatrice Bevilacqua for useful technical discussions, and Marc Lanctôt and Satinder Singh for reviewing the paper prior to submission.

Author information

These authors contributed equally: Zhe Wang, Petar Veličković, Daniel Hennes.

Authors and Affiliations

Google DeepMind, 6-8 Handyside Street, London, N1C 4UZ, UK

Zhe Wang, Petar Veličković, Daniel Hennes, Nenad Tomašev, Laurel Prince, Michael Kaisers, Yoram Bachrach, Romuald Elie, Li Kevin Wenliang, Federico Piccinini, Jerome Connor, Yi Yang, Adrià Recasens, Mina Khan, Nathalie Beauguerlange, Pablo Sprechmann, Pol Moreno, Nicolas Heess & Demis Hassabis

Liverpool FC, AXA Training Centre, Simonswood Lane, Kirkby, Liverpool, L33 5XB, UK

William Spearman

Liverpool FC, Kirkby, UK

University of Alberta, Amii, Edmonton, AB, T6G 2E8, Canada

Michael Bowling

Google DeepMind, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

Z.W., D. Hennes, L.P. and K.T. coordinated and organised the research effort leading to this paper. P.V. and Z.W. developed the core TacticAI models. Z.W., W.S. and I.G. prepared the Premier League corner kick dataset used for training and evaluating these models. P.V., Z.W., D. Hennes and N.T. designed the case study with human experts and Z.W. and P.V. performed the qualitative evaluation and statistical analysis of its outcomes. Z.W., P.V., D. Hennes, N.T., L.P., M. Kaisers, Y.B., R.E., L.K.W., F.P., W.S., I.G., N.H., M.B., D. Hassabis and K.T. contributed to writing the paper and providing feedback on the final manuscript. J.C., Y.Y., A.R., M. Khan, N.B., P.S. and P.M. contributed valuable technical and implementation discussions throughout the work’s development.

Corresponding authors

Correspondence to Zhe Wang , Petar Veličković or Karl Tuyls .

Ethics declarations

Competing interests.

The authors declare no competing interests but the following competing interests: TacticAI was developed during the course of the Authors’ employment at Google DeepMind and Liverpool Football Club, as applicable to each Author.

Peer review

Peer review information.

Nature Communications thanks Rui Luo and the other anonymous reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, Z., Veličković, P., Hennes, D. et al. TacticAI: an AI assistant for football tactics. Nat Commun 15 , 1906 (2024). https://doi.org/10.1038/s41467-024-45965-x

Download citation

Received : 13 October 2023

Accepted : 08 February 2024

Published : 19 March 2024

DOI : https://doi.org/10.1038/s41467-024-45965-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.