observation protocol qualitative research template

From John W. Creswell \(2016\). 30 Essential Skills for the Qualitative Researcher \ . Thousand Oaks, CA: Sage.

Qualitative Research: Observation

• Getting Started
• Focus Groups
• Observation
• Case Studies
• Data Collection
• Cleaning Text
• Analysis Tools
• Institutional Review

Participant Observation

Photo: https://slideplayer.com/slide/4599875/

Field Guide

• Participant Observation Field Guide

What is an observation?

A way to gather data by watching people, events, or noting physical characteristics in their natural setting. Observations can be overt (subjects know they are being observed) or covert (do not know they are being watched).

• Researcher becomes a participant in the culture or context being observed.
• Requires researcher to be accepted as part of culture being observed in order for success

Direct Observation

• Researcher strives to be as unobtrusive as possible so as not to bias the observations; more detached.
• Technology can be useful (i.e video, audiorecording).

Indirect Observation

• Results of an interaction, process or behavior are observed (for example, measuring the amount of plate waste left by students in a school cafeteria to determine whether a new food is acceptable to them).

Suggested Readings and Film

• Born into Brothels . (2004) Oscar winning documentary, an example of participatory observation, portrays the life of children born to prostitutes in Calcutta. New York-based photographer Zana Briski gave cameras to the children of prostitutes and taught them photography
• Davies, J. P., & Spencer, D. (2010).  Emotions in the field: The psychology and anthropology of fieldwork experience . Stanford, CA: Stanford University Press.
• DeWalt, K. M., & DeWalt, B. R. (2011).  Participant observation : A guide for fieldworkers .   Lanham, Md: Rowman & Littlefield.
• Reinharz, S. (2011).  Observing the observer: Understanding our selves in field research . NY: Oxford University Press.
• Schensul, J. J., & LeCompte, M. D. (2013).  Essential ethnographic methods: A mixed methods approach . Lanham, MD: AltaMira Press.
• Skinner, J. (2012).  The interview: An ethnographic approach . NY: Berg.
• << Previous: Focus Groups
• Next: Case Studies >>
• Last Updated: Mar 1, 2024 10:13 AM
• URL: https://guides.library.duke.edu/qualitative-research

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

Prompts, Not Questions: Four Techniques for Crafting Better Interview Protocols

• Published: 05 June 2021
• Volume 44 , pages 507–528, ( 2021 )

Cite this article

• Tomás R. Jiménez 1 &
• Marlene Orozco 2  

5821 Accesses

21 Citations

14 Altmetric

Explore all metrics

A Correction to this article was published on 05 August 2023

This article has been updated

We offer effective ways to write interview protocol “prompts” that are generative of the most critical types of information researchers wish to learn from interview respondents: salience of events, attributes, and experiences; the structure of what is normal; perceptions of cause and effect; and views about sensitive topics. We offer tips for writing and putting into practice protocol prompts that we have found to be effective at obtaining each of these kinds of information. In doing so, we encourage researchers to think of an interview protocol as a series of prompts, rather than a list of questions, for respondents to talk about certain topics related to the main research question(s). We provide illustrative examples from our own research and that of our students and professional colleagues to show how generally minor tweaks to typical interview prompts result in richer interview data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Survey Interviewing: Departures from the Script

I Got an Interview, How Do I Prepare for It?

Qualitative Interviewing

Change history, 05 august 2023.

A Correction to this paper has been published: https://doi.org/10.1007/s11133-023-09543-9

Barry, Robert A. 1957. The social desirability variable in personality assessment and research . New York: The Dryden Press.

Google Scholar  

Bates, Timothy, and Alicia Robb. 2006. Small business viability in America’s urban minority communities. Urban Studies 51 (13): 2844–2862.

Article   Google Scholar  

Becker, Howard. 1998. Tricks of the trade: How to think about your research while you’re doing it . Chicago: University of Chicago Press.

Book   Google Scholar  

Carian, Emily K. 2019. Constructing manhood: Men's rights activists and feminist men's shared meanings of gender . PhD Dissertation. Department of Sociology, Stanford University.

Carian, Emily, and Jasmine Hill. 2021. Teaching interviews: Illuminating frameworks of social desirability in the classroom . San Bernardino. Unpublished Manuscript. Department of Sociology, California State University.

Carr, Deborah, Elizabeth Heger Boyle, Benjamin Cornwell, Shelley Correll, Robert Crosnoe, Jeremy Freese, and Mary C. Waters. 2017. The art and science of social research . New York: W.W. Norton and Company, Inc..

Glynn, Carroll J., Andrew F. Hayes, and James Shanahan. 1997. Perceived support for one's opinions and willingness to speak out: A meta-analysis of survey studies on the "spiral of silence". The Public Opinion Quarterly 61 (3): 452–463.

Goffman, Erving. 1959. The presentation of self in everyday life . New York: Anchor Books.

Grazian, David. 2015. American zoo: A sociological safari . Princeton: Princeton University Press.

Hart, Chloe Grace. 2021. Trajectory guarding: Managing unwanted, ambiguously sexual interactions at work. American Sociological Review 86 (2): 256–278.

Holland, Paul W. 1986. Statistics and causal inference. Journal of the American Statistical Association 81 (396): 945–960.

Jiménez, Tomás R. 2010. Replenished ethnicity: Mexican Americans, immigration, and identity . Berkeley: University of California Press.

Jiménez, Tomás R. 2017. The other side of assimilation: How immigrants are changing American life . Oakland: University of California Press.

Jiménez, Tomás R., and Adam L. Horowitz. 2013. When White is just alright: How immigrants redefine achievement and reconfigure the ethnoracial hierarchy. American Sociological Review 78 (5): 849–871.

Jiménez, Tomás R., Deborah J. Schildkraut, Yuen J. Huo, and John F. Dovidio. 2021. States of belonging: Immigration policies, attitudes, and inclusion . New York: Russell Sage Foundation Press.

Lamont, Michele. 2000. The dignity of working men: Morality and the boundaries of race, class, and immigration . Cambridge: Harvard University Press.

Lofland, John, David Snow, Leon Anderson, and Lyn H. Lofland. 2006. Analyzing social settings: A guide to qualitative research and analysis . Fourth edition. Belmont: Wadsworth Publishing.

Lamont, Michèle, and Ann Swidler. 2014. Methodological pluralism and the possibilities and limits of interviewing. Qualitative Sociology 37 (2): 153–171.

Marrow, Helen B. 2011. New destination dreaming: Immigration, race, and legal status in the rural American south . Stanford: Stanford University Press.

McCracken, Grant. 1998. The long interview . Qualitative Research Methods Series. Newbury Park: Sage Publications.

Morgan, Stephen L., and Christopher Winship. 2014. Counterfactuals and causal inference: Methods and principles for social research . 2nd edition. New York: Cambridge University Press.

Nederhof, Anton J. 1985. Methods of coping with social desirability bias: A review. European Journal of Social Psychology 15 (3): 263–280.

Noelle-Neumann, Elisabeth. 1974. The spiral of silence a theory of public opinion. Journal of Communication 24 (2): 43–51.

Orozco, Marlene. 2021. The salience of ethnic identity in entrepreneurship: An ethnic strategies of business action framework. Unpublished Manuscript.

Rivera, Lauren A. 2016. Pedigree: How elite students get elite jobs . Princeton: Princeton University Press.

Roth, Wendy. 2012. Race migrations: Latinos and the cultural transformation of race . Stanford: Stanford University Press.

Roth, Wendy D., and Biorn Ivemark. 2018. Genetic options: The impact of genetic ancestry testing on consumers’ racial and ethnic identities. American Journal of Sociology 124 (1): 150–184.

Seidman, Irving. 2019. Interviewing as qualitative research: A guide for researchers in education and the social sciences . New York: Teachers College Press.

Sobotka, Tagart. 2021. Bad doctors, enablers, and the powerless: The opioid crisis and the construction of blame. PhD Dissertation. Department of Sociology, Stanford University.

Smith, Tom W., Davern, Michael, Freese, Jeremy, and Stephen L. Morgan. 2019. General social surveys, 1972–2018: Cumulative codebook. GSS NORC.  https://gss.norc.org/documents/codebook/gss_codebook.pdf .

Stuart, Forrest. 2016. Down, out, and under arrest: Policing and everyday life in skid row . Chicago: University of Chicago Press.

Vasquez, Jessica M. 2011. Mexican Americans across generations : Immigrant families, racial realities . New York: New York University Press.

Waldinger, Roger David. 1996. Still the promised city?: African-Americans and new immigrants in postindustrial New York . Cambridge: Harvard University Press.

Warikoo, Natasha Kumar. 2011. Balancing acts: Youth culture in the global city . Berkeley: University of California Press.

Waters, Mary C. 1990. Ethnic options: Choosing identities in America . Berkeley: University of California Press.

Weiss, Robert Stuart. 1995. Learning from strangers: The art and method of qualitative interview studies . New York: Free Press.

Widen, Sherri, Marlene Orozco, Eileen Lai Horng, and Susanna Loeb. 2019. Reaching unconnected caregivers: Using a text-message education program to better understand how to support informal caregivers role in child development. Journal of Early Childhood Research 18 (1): 29–43.

Download references

Acknowledgements

We would like to thank our colleagues who supported this work and provided examples from their research: Emily Carian, Molly King, Tagart Sobotka, and Chloe Hart. Special thanks to Forrest Stuart for his input on several drafts. We would also like to thank the participants of the Migration, Ethnicity, Race and Nation workshop at Stanford for their comments on the manuscript.

Author information

Authors and affiliations.

Department of Sociology, Stanford University, 450 Jane Stanford Way, Building 120, Room 160, Stanford, CA, 94305-2047, USA

Tomás R. Jiménez

Department of Sociology, Stanford University, 450 Jane Stanford Way, Building 120, Room 030B, Stanford, CA, 94305-2047, USA

Marlene Orozco

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tomás R. Jiménez .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Jiménez, T.R., Orozco, M. Prompts, Not Questions: Four Techniques for Crafting Better Interview Protocols. Qual Sociol 44 , 507–528 (2021). https://doi.org/10.1007/s11133-021-09483-2

Download citation

Accepted : 04 May 2021

Published : 05 June 2021

Issue Date : December 2021

DOI : https://doi.org/10.1007/s11133-021-09483-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Ethnography
• Counterfactual
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 7, Issue 9
• Protocol for a qualitative study exploring perspectives on the INternational CLassification of Diseases (11th revision); Using lived experience to improve mental health Diagnosis in NHS England: INCLUDE study
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Corinna Hackmann 1 ,
• Amanda Green 1 ,
• Caitlin Notley 2 ,
• Amorette Perkins 1 ,
• Geoffrey M Reed 3 ,
• Joseph Ridler 1 ,
• Jon Wilson 1 , 2 ,
• Tom Shakespeare 2
• 1 Department of Research and Development , Norfolk and Suffolk NHS Foundation Trust, Hellesdon Hospital , Norwich , UK
• 2 Department of Clinical Psychology , Norwich Medical School, University of East Anglia , Norwich , UK
• 3 Department of Psychiatry , Global Mental Health Program, Columbia University Medical Centre , New York , New York , USA
• Correspondence to Dr Corinna Hackmann; Corinna.hackmann{at}nsft.nhs.uk

Introduction Developed in dialogue with WHO, this research aims to incorporate lived experience and views in the refinement of the International Classification of Diseases Mental and Behavioural Disorders 11th Revision (ICD-11). The validity and clinical utility of psychiatric diagnostic systems has been questioned by both service users and clinicians, as not all aspects reflect their lived experience or are user friendly. This is critical as evidence suggests that diagnosis can impact service user experience, identity, service use and outcomes. Feedback and recommendations from service users and clinicians should help minimise the potential for unintended negative consequences and improve the accuracy, validity and clinical utility of the ICD-11.

Methods and analysis The name INCLUDE reflects the value of expertise by experience as all aspects of the proposed study are co-produced. Feedback on the planned criteria for the ICD-11 will be sought through focus groups with service users and clinicians. The data from these groups will be coded and inductively analysed using a thematic analysis approach. Findings from this will be used to form the basis of co-produced recommendations for the ICD-11. Two service user focus groups will be conducted for each of these diagnoses: Personality Disorder, Bipolar I Disorder, Schizophrenia, Depressive Disorder and Generalised Anxiety Disorder. There will be four focus groups with clinicians (psychiatrists, general practitioners and clinical psychologists).

Ethics and dissemination This study has received ethical approval from the Coventry and Warwickshire HRA Research Ethics Committee (16/WM/0479). The output for the project will be recommendations that reflect the views and experiences of experts by experience (service users and clinicians). The findings will be disseminated via conferences and peer-reviewed publications. As the ICD is an international tool, the aim is for the methodology to be internationally disseminated for replication by other groups.

Trial registration number ClinicalTrials.gov: NCT03131505 .

• International Classification of Diseases
• Personality Disorders
• Anxiety Disorders

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-018399

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

This study is the first to gather expert by experience views on the proposed criteria to be fed into the revision process of the International Classification of Diseases.

All aspects of the proposed study have been co-produced with experts by experience and agreed with a representative from WHO.

Qualitative focus group data will be thematically analysed to form the basis of co-produced recommendations to be fed back to WHO.

The themes and resulting recommendations will be limited to five diagnostic categories and will only reflect views from the UK.

Introduction

Diagnostic systems have a number of functions both from the perspective of the clinician and service user. 1–3 Diagnosis offers indications for treatment, may guide expectation regarding prognosis and can help people to make sense of their experiences of living with mental health (MH) difficulties. 1 2 In order for a diagnostic system to be useful, it is critical that it reflects the day-to-day experiences of people living with the symptoms. Service users have reported relief derived from diagnostic definitions that resonate with and explain their experiences. 1 4 On the other hand, some feel their diagnosis does not ‘fit’ with or describe their experiences, and thus has limited utility other than being a ‘tick box’ exercise of labelling and categorising. 5–7 To date, it appears that no revision of the major systems for psychiatric diagnosis (International Classification of Diseases (ICD) or Diagnostic and Statistical Manual of Mental Disorders) has sought feedback from service users prior to publication.

Diagnostic systems are designed for clinicians; despite this, service users can easily access the diagnostic criteria. Research shows that the labels, language and descriptions used in these systems can impact people’s self-perception, their interpretations of how other people view them and their understanding of the implications of having a diagnosis, including the prognosis and potential for recovery. 5 8 9 These interpretations can have a direct impact on factors such as self-worth and self-stigmatisation, social and occupational functioning, recovery and service use. 5 6 10 For example, service users have reported that terms like ‘disorder’ and ‘enduring’ suggest permanency, impeding their hope for recovery. 5 Similarly, others have reported that the descriptions and terms used in diagnostic systems (eg, language like ‘deviant’, ‘incompetent’, ‘disregard for social obligations’ and ‘limited capacity’) can be stigmatising and unhelpful, leading to feelings of rejection, anger and possible avoidance of services. 5 6 8 Clarity on the perceptions of individuals receiving a diagnosis, in terms of the language, meaning and implications of what is included in the system, may help to minimise possible negative consequences.

Evidence suggests that clinicians also have concerns regarding the validity and clinical utility of the current diagnostic systems. 3 11–13 For instance, health professionals have reported that some diagnostic definitions feel arbitrary, artificial or unreflective of the typical presentations they observe in practice. 11 12 Other evidence suggests that clinicians find the categories difficult to use, particularly for distinguishing between disorders. 9 12 14 Clinicians have also expressed reservations regarding the terminology and associated stigma, particularly for conditions such as Schizophrenia and Personality Disorder. 13 15 These findings are from studies that have been conducted after the criteria have been released. Prospective input from clinicians on the proposed criteria as part of the process of revision may therefore improve the validity and clinical utility of diagnostic systems.

The value of expertise by experience is increasingly recognised by policymakers, 16–18 service providers and researchers. 19 20 Many have argued that processes of diagnosis could be improved by including perspectives of those with lived experience. 10 21 It has been suggested that within the diagnostic categories, "the traditional language is useful for listing and sorting but not for living and experiencing. ‘Naming' a thing is not the same as 'knowing'a thing" (p90) 22 and therefore categories could be improved by viewing service users as ‘authors of knowledge from whom others have something to learn’ (p291). 21 Likewise, it has been argued that diagnostic systems could be improved by addressing problems identified by practising clinicians. 3

Input regarding the proposed content for the ICD-11 from service users and clinicians should be used to support the process of revision and improvement. Feedback and clarity from service users on (1) whether the content of the system is in line with their experience of symptoms and (2) their interpretations of the content and language should facilitate the development of a system that is more accurate and valid, with minimised unintended negative impact.

Aims and objectives

This research project will use a focus group methodology to ask service users and clinicians who use the ICD diagnostic tool (psychiatrists and general practitioners) their views on the proposed content for the ICD-11. Data collected through collaborative discussion in the groups will be inductively analysed, and resulting themes will be triangulated with an advisory group (involving additional service users and clinicians). The output will be recommendations for improvement to ICD-11 content that have been co-produced with a feedback group (of different service users and clinicians).

Research questions

What are the views and perspectives of service users and clinicians on the content of the ICD-11?

How could the system be improved for the benefit of service users and clinicians?

Methods and analysis

Study design.

This is a qualitative study. Data will be collected through focus groups. Focus groups are an appropriate method of data collection to answer the study research questions seeking to explore views and perspectives of service users and clinicians, where our analysis will aim to define key themes and points of consensus or divergence gathered through interaction, 23 24 drawing on participants own perspectives and choice of language. 25 Participants will be given a copy of the proposed diagnostic criteria relevant to their diagnosis to discuss in the group. This will include both the technical version (as it is proposed for the ICD-11) and a lay translation of the criteria. Thematic analysis 26–28 will be used to identify emergent recurring and/or salient themes in the focus group data. The themes will form the basis for co-produced recommendations to support the development of the ICD-11. Data collection for this study commenced in February 2017 and analyses are planned to be completed and fed back to WHO by the end of December 2017.

Co-production

The research team that developed this project includes a service user expert by experience (AG), two academics (TS, CN), two research clinicians (a consultant psychiatrist (JW) and a clinical psychologist (CH)) and two research assistant psychologists (AP, JR). A service user expert by experience research team member will be involved in all aspects of the research, including design, facilitating focus groups, analysis, write-up and dissemination.

In developing the project, team members consulted a local service user governor, service users and the service user involvement leads at the hosting National Health Service organisation. This input helped shape the design (changing and broadening the process of recruitment of service users and supporting the use of focus groups) and the initial selection of the diagnoses that were included.

Co-production with service users, clinicians and researchers will continue throughout the project. Data analysis will be co-produced through involvement of the service user expert by experience on the research team and the advisory and feedback groups.

Diagnoses under investigation

With agreement from WHO, five diagnoses have been selected for exploration: Personality Disorder, Bipolar I Disorder, Schizophrenia, Depressive Disorder and Generalised Anxiety Disorder. These diagnoses include a wide range of symptom phenomena. Personality Disorder, Bipolar I Disorder and Schizophrenia are found to be more stigmatised, rejected and negatively viewed than other diagnoses, meaning they may have a particularly negative impact and be more consistently associated with harm. 29 30 Depressive Disorder and Generalised Anxiety Disorder are highly prevalent, making the largest contribution to the burden of disease in middle-income and high-income countries, including the UK. 31

Lay translation

The lay translations of the criteria have been produced by members of the research team including psychiatrists and other clinicians, and approved by a representative from WHO to ensure they reflect the proposed ICD-11. Documents have been created presenting lay translations alongside the technical version as it is written in the ICD-11, so that participants are easily able to refer to either source. Copies of these are available in English for researchers wishing to replicate this study.

Recruitment

Sampling will be purposive and include a number of pathways to ensure maximum inclusivity. Recruitment of service users will be both via clinicians in a MH trust and self-referral via a number of routes. Promotion of the study will be via clinicians, service user involvement leads in a MH trust and non-governmental organisations (NGOs). Clinicians in the MH trust will be asked to identify potential participants and seek consent to be contacted by the research team. Service user involvement leads will disseminate information about the study to service users and the membership of a MH trust (which includes many previous service users), providing a telephone number and email address to self-refer if interested. NGOs will promote the study using the same materials. The study will be promoted through recruitment posters, service user involvement forums and on social media. Clinicians will be recruited via team leaders, word of mouth and email communications promoting the project.

Once self-referral or consent to contact has been established, a member of the research team will make contact, provide potential participants with a brief overview of the study, and answer any questions. If the individual wishes to be involved in the study, they will be sent a copy of the participant information sheet via post or email. This information sheet outlines the purpose and nature of the study, and the ethical safeguards regarding data protection and privacy. Potential participants will have at least 72 hours to consider whether they would like to be involved in the study. If the individual would like to take part in the study, researchers will arrange to meet them at least 1 week before the focus group to complete the consent process and give them the relevant proposed diagnostic criteria to read and consider.

Sample size

There will be two service user focus groups for each of the five diagnoses. Additionally, there will be four clinician focus groups. The ICD system is primarily used by medical doctors in the UK, although clinical psychologists have been included in this study as they also apply the system in their work. 32 In this study, the diagnostic criteria presented to participants are divided into distinct discussion points. During the focus groups, these discussion points will be addressed one by one and participants will be asked for their feedback through predefined questions and prompts. This includes asking people their views of the proposed features, the language used, the positives and negatives of what is included and how the classification might be improved for the benefit of service users. In light of this, the number of groups was agreed based on research stating that using more standardised interviews decreases variability and thus requires fewer focus groups. 33 In total, there will be 14 groups, containing three to six participants each. This will give a total sample of 42–84 participants (30–60 service users and 12–24 clinicians). The advisory group will comprise three to five additional service users and three clinicians. Lastly, the feedback group will comprise five service users and three clinicians. The focus group size was chosen to allow participants opportunity to discuss their views and experiences in detail, while increasing recruitment feasibility. 34 The sample size should be sufficient in providing data to meet the aims and to cover a range of views. Evidence suggests that the majority of themes are discovered in the first two to three focus groups. 35

Inclusion and exclusion criteria

Adult service users (18 years and older) may be included in the focus groups if they have formally received at least one of the five diagnoses under investigation and have accessed services within the last 5 years (including those currently in receipt of services). People with multiple diagnoses may only take part in one focus group, but will be given the option of which group. Clinicians will have had experience working in MH, including the use of the psychiatric diagnoses under investigation. Individuals may only participate in either one focus group, the advisory group or the feedback group.

Individuals will be excluded if they are under the age of 18 years, lack the capacity to consent, or have an inability to speak fluent English (as fluent English is required to participate in the focus groups). Individuals will also be excluded if their participation is deemed unsafe to themselves or others by their lead clinician or clinicians on the research team.

Data collection

Focus groups are the most applicable method for data collection to meet our research aims, as attitudes, opinions and beliefs are more likely to be revealed in the reflective process facilitated by the social interaction that a focus group entails than by other methods. 23–25 Additionally, focus groups have proved to be a useful way of exploring stigma issues in MH, 36 and service users are often familiar with group settings for discussing MH issues.

The summary of the new diagnostic guidelines and lay translation will enable participants to reflect on both the content and the language of the proposed criteria. During the groups, topic guides will encourage participants to discuss and share views of the relevant diagnostic category. This includes their overarching views, thoughts and feelings; as well as, specific reflections on areas such as the language used, aspects that may be helpful or unhelpful, and suggestions for improvement.

Each focus group will be led by an experienced and trained member of the research team and have an assistant facilitator. Service user focus groups will last 60–90 min, and clinician focus groups will last 2–2.5 hours to account for the discussion of multiple diagnoses.

The focus groups will be audio-recorded and transcribed verbatim. The transcripts will first be read and descriptively openly coded (using the same language as participants where possible) by the lead researcher. Approximately 25% of the transcripts will be independently open coded by another member of the research team, as a validity check. Codes will be compared and discussed until consensus is reached. The five diagnoses will initially be analysed separately to produce themes that are relevant to each diagnosis. Following this, these themes will be compared with identify common themes relevant to all the diagnostic categories. Analysis of data will mainly be descriptive. We will take a critical realist epistemological stance to analysis, recognising that there are multiple individual realities, but taking a pragmatic approach to analysing data at face value, drawing on the perspectives of individuals as they choose to represent themselves through discussion. 37 Thematic analysis will be used to inductively code themes that reoccur or appear important. 26–28 The concept of salience will be referred to here, to guide coding that is conceptually and inherently significant, not just frequently occurring. A qualitative data management software system (NVIVO-11) will be used to facilitate data analysis.

In addition to descriptive data for thematic coding, focus groups generate data that is conversational. Analysis of this requires an inductive approach that focuses on instances in the data where there is marked agreement (consensus), disagreement or divergence. These instances will be identified as ‘critical moments’. The sample size is small and purposive. Consequently, summary quantified coding matrices will not be produced. Instead there will be a focus on the 'critical moments' to direct the analysis and eventual findings, reporting on the issues that are of central importance to the participants.

Following analysis of each focus group, a second stage analysis will be conducted to compare and contrast findings across groups. The analysis will seek out consensus, disagreement and inconsistency within service user and clinician focus groups, and between diagnoses. This second stage analysis will involve discussions within the research team to refine the themes and to develop higher level themes, that is, grouping the open codes into meaningful conceptual categories. This will allow tentative conclusions to be drawn about aspects of the diagnostic criteria which may be particularly pertinent for some groups and less important for others. It will also enable conclusions to be drawn regarding generic language or overall responses to the diagnostic criteria, in comparison to more nuanced reactions to diagnostically specific categories.

The output from the analysis will be higher level themes and categories that form the basis of recommendations for the ICD-11. These themes will be triangulated with the advisory group. The resulting themes will be discussed with the feedback group in order to co-produce the recommendations. These recommendations will be contextualised with a description of the themes and identified areas of agreement and disagreement for feedback to WHO.

Data protection

All confidential data will be kept for 5 years on password-protected computers and/or locked filing cabinets only accessible to members of the research team. During transcription, audio-recordings will be anonymised, with all identifiable information removed prior to using the software analysis tool. All audio-recordings will be destroyed immediately after transcription.

Ethics and dissemination

Ethical considerations.

Written informed consent to participate and be audio-recorded will be obtained from all participants. Data management and storage will be subject to the UK Data Protection Act 1998. Ethical approval for the current study was obtained from the Coventry and Warwickshire Research Ethics Committee (Rec Ref: 16/WM/0479).

Declaration of Helsinki

This study complies with the Declaration of Helsinki, adopted by the 18th World Medical Association (WMA) General Assembly, Helsinki, Finland, June 1964 and last revised by the 64th WMA General Assembly, Fortaleza, Brazil, October (2013).

Output and dissemination

This research has been designed to obtain feedback with recommendations for the ICD-11, and to develop a methodology that can be replicated in other countries that use the ICD system. Additionally, the findings, and learning in terms of the process of co-producing and conducting research with experts by experience, will be disseminated via peer-reviewed publications, conferences, media and lay reports.

Service user involvement in MH is a priority. 19 Studies have found that both clinicians and service users have questioned the accuracy, validity and clinical utility of the ICD and other psychiatric diagnostic tools. 3 8 9 11 12 38 Despite this, to date, service user and clinician feedback has not been obtained prior to revision of the ICD manual. In light of this, is not clear whether the content resonates with the experiences of people giving and receiving the diagnoses, could lack clinical utility, or even, cause harm (eg, in terms of the language used).

Limitations

This study is designed to input feedback from service users and clinicians in the forthcoming revision of the ICD. The usefulness of the data and resulting recommendations is dependent on input, that is, reflective of the views of service users and clinicians that the new system will impact. The current study will include two focus groups for each disorder in an attempt to minimise bias 35 and to account for group-think processes that may occur within individual groups. Taking a critical realist epistemological stance is a pragmatic approach to work with discursive data created through the interactional context of a focus group. It is acknowledged that there are multiple competing realities and perspectives that may differ across time and context, and the analysis findings will be limited to the time and context of this study. Transferability of findings is nonetheless maximised by triangulation to ensure the inclusion of multiple stakeholder perspectives, enabled by the advisory and feedback groups of experts by experience that will co-produce the recommendations reported to WHO. Interpretation of the feedback will take into account potential limitations regarding the generalisability of the findings. The current project is exploring only five of the diagnoses that are included in the ICD-11. The ICD is internationally used, and the current project will reflect the experiences and views of service users and clinicians in the UK only. Future research may include both additional diagnostic categories and encapsulate expertise by experience and relevant clinicians in different countries.

The current study will use feedback from experts by experience to co-produce recommendations for the revised diagnostic system proposed for the ICD-11. This feedback aims to improve the accuracy, validity and clinical utility of the manual, and minimise the potential for unintended negative consequences. This qualitative approach has not been previously employed by any countries that use the ICD system. Our vision is that this process will become a routine feature in future revisions of all diagnostic systems.

Acknowledgments

We would like to thank the library services at Norfolk and Suffolk Foundation Trust for aiding the searching and retrieval of documents. We would like to thank Kevin James (service user governor), Lesley Drew and Sharon Picken (service user involvement leads) for their input during the development of the project. We would also really like to thank Dr Bonnie Teague who generously offered the benefit of her wisdom and proof reading skills.

• Kilbride M ,
• Welford M , et al
• Johnstone L ,
• Bonnington O ,
• Stalker K ,
• Ferguson I ,
• Castillo H ,
• van Rijswijk E ,
• van Hout H ,
• van de Lisdonk E , et al
• Shadbolt N ,
• Starcevic V , et al
• Milton AC ,
• Kelly B , et al
• 16. ↵ Department of Health . Putting People First: Planning together – peer support and self-directed support . London : Department of Health , 2010 .
• 17. ↵ Department of Health . No Health without Mental Health: A cross-government mental health outcomes strategy for people of all ages . London : Department of Health , 2011 .
• 18. ↵ Department of Health . Closing the Gap: Priorities for essential change in mental health . London : Social Care, Local Government and Care Partnership Directorate , 2014 .
• Simpson EL ,
• Beresford P
• Malone P , et al
• Wilkinson S
• MacQueen KM ,
• Stevens S ,
• Serfaty M , et al
• Carlyle D , et al
• 31. ↵ World Health Organization . The global burden of disease: 2004 update . Switzerland : World Health Organization , 2008 .
• 32. ↵ The British Psychological Society . Diagnosis – policy and guidance . http://www.bps.org.uk/system/files/documents/diagnosis-policyguidance.pdf ( accessed Jul 2017 ).
• Schulze B ,
• Angermeyer MC
• Denzin NK ,

Contributors CH is the chief investigator for this project and wrote the protocol. TS is supervising the project and helped to develop all aspects of the project. AG is the expert by experience on the research team, and led on developing the co-production, and the public and patient involvement. CN led the development of the methodology. AP had a specific contribution to the literature review. GMR is the WHO consultant for the project. GMR developed the original idea for the project and has had input into the development of the lay criteria. JR provided input to ethical considerations and the lay criteria. JW led on the development of the lay criteria. All authors supported the development and critical review of the protocol.

Competing interests None declared.

Ethics approval Coventry and Warwickshire HRA Research Ethics Committee (16/WM/0479).

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

This site uses session cookies and persistent cookies to improve the content and structure of the site.

By clicking “ Accept All Cookies ”, you agree to the storing of cookies on this device to enhance site navigation and content, analyse site usage, and assist in our marketing efforts.

By clicking ' See cookie policy ' you can review and change your cookie preferences and enable the ones you agree to.

By dismissing this banner , you are rejecting all cookies and therefore we will not store any cookies on this device.

Qualitative protocol guidance and template: consultation

This HRA consultation has closed and is displayed for reference only. CTIMP protocol guidance and templates can be found in the  protocol page of the research planning section of the site.

The HRA continues to work toward improving the quality and consistency of health research in the UK. As part of this, throughout 2015 the HRA developed a suite of protocol guidance and templates for different study types. Moving into the next phase we have produced protocol guidance and a template for qualitative studies.

The HRA is aware that the quality and content of protocols for Qualitative research varies widely. Strong feedback to the HRA highlighted that this considerable variability of protocols was causing delays to reviews. In response to this the HRA facilitated work to develop guidance and a template to assist organisations and individuals to improve the consistency and quality of their qualitative protocols.

A protocol which contains all the elements that review bodies consider is less likely to be delayed during the review process because the reviewers are less likely to require clarification from the applicant.

A multidisciplinary group from research active organisations provided expertise to the project which has produced this detailed guidance and template which are published for use and comment.

Is it mandatory to use this guidance and template?

No. The use of this collated consensus guidance and template is not mandatory. The guidance and template are published as standards to encourage and enable responsible research. The documents will:

• Support researchers developing protocols where the sponsor does not already use a template
• Support sponsors wishing to develop template protocols in line with validated guidance
• Support sponsors to review their existing protocol template to assess whether it is in line with national guidance.

Can sponsors continue to use their own protocol templates?

Yes. The HRA acknowledge that institutions have specific needs, including specialised additional material, and may have their own templates. The HRA asks that sponsors advise those preparing protocols how their template has regard for the HRA guidance and template. In addition, the HRA recommends that each protocol states clearly how it meets HRA guidance:

• The protocol has regard for the HRA guidance and order of content
• The protocol has regard for the HRA guidance
• The protocol does not have regard to the HRA guidance and order of content

What are the benefits of using the guidance and template?

By clearly defining the expected components of a protocol, the guidance and template help researchers to be sure that they have covered all the elements required by sponsors, Research Ethics Committees and NHS sites. In the future this will also apply to applications for HRA Approval. Protocols which have regard for the guidance and template are less likely to raise queries that can cause delays.

Who can use the template? The template can be used by all individuals and sponsoring organisations involved in authoring Qualitative research projects.

How do I provide feedback?

The feedback period has now closed. 

• Privacy notice
• Terms & conditions
• Accessibility statement
• Feedback or concerns

Search form

• About Faculty Development and Support
• Programs and Funding Opportunities

Consultations, Observations, and Services

• Strategic Resources & Digital Publications
• Canvas @ Yale Support
• Learning Environments @ Yale
• Teaching Workshops
• Teaching Consultations and Classroom Observations
• Teaching Programs
• Spring Teaching Forum
• Written and Oral Communication Workshops and Panels
• Writing Resources & Tutorials
• About the Graduate Writing Laboratory
• Writing and Public Speaking Consultations
• Writing Workshops and Panels
• Writing Peer-Review Groups
• Writing Retreats and All Writes
• Online Writing Resources for Graduate Students
• About Teaching Development for Graduate and Professional School Students
• Teaching Programs and Grants
• Teaching Forums
• Resources for Graduate Student Teachers
• About Undergraduate Writing and Tutoring
• Academic Strategies Program
• The Writing Center
• STEM Tutoring & Programs
• Humanities & Social Sciences
• Center for Language Study
• Online Course Catalog
• Antiracist Pedagogy
• NECQL 2019: NorthEast Consortium for Quantitative Literacy XXII Meeting
• STEMinar Series
• Teaching in Context: Troubling Times
• Helmsley Postdoctoral Teaching Scholars
• Pedagogical Partners
• Instructional Materials
• Evaluation & Research
• STEM Education Job Opportunities
• Yale Connect
• Online Education Legal Statements

You are here

Classroom observation protocols & teaching inventories.

A variety of published tools can assist instructors when assessing their teaching practices. Many such tools, including classroom observation protocols and teaching inventories, have been utilized in science, technology, engineering and math (STEM) courses, but are easily adaptable to other disciplines.

In observation protocols, an observer witnesses classroom teaching or views a videotape of instruction. While doing so, the observer fills out the protocol, typically consisting of questions that (1) ask whether particular teaching and learning behaviors were observed, (2) use a Likert-scale to capture the extent to which the behavior was seen in the classroom, and/or (3) allow for open-ended general feedback. Because observation protocols are typically designed to measure particular approaches, instructors should be careful to choose one for its specific assessment purpose. In contrast, teaching inventories can often be completed quickly by the instructor to obtain an overall assessment of practices. Teaching inventories are often used in more low-key, self-assessing reflective teaching approaches.

Instructors may keep in mind several benefits and challenges with classroom observation protocols and inventories. Training is often required if an observation protocol is used for research or other purposes where reliability between observers is essential. Also, while using a protocol just once provides a snapshot view of the classroom,  multiple observations can enhance the reliability of the assessments. Some protocols may also pose judgment on instruction, which can be awkward to share with the instructor being assessed. Teaching inventories, while often quick to complete, involve self-report of teaching practices, and can lack total objectivity. They also tend to focus on evidence of quantity (e.g. how often a particular behavior is observed) over quality. Where possible, coupling teaching inventories with observation protocols may be desirable.

A variety of published observation protocols and teaching inventories have been implemented and researched extensively in higher education.

Classroom Observation Protocols

• Assesses multiple dimensions of teaching and is customizable. 
• http://tdop.wceruw.org/
• Measures student- versus teacher-centered practices. 
• http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/index.htm
• COPUS (Classroom Observation Protocol for Undergraduate STEM) (Smith, et. al, 2013)

Teaching Inventories

• Measures whether the instructors’ teaching practices are focused on information transmission or conceptual change. 
• Characterizes general teaching practices in math and science. 
• Describes practices to develop anti-racist teaching approaches​

Poorvu Center staff are available to discuss these and a variety of other teaching inventories that might be of use to instructors.

Recommendations

• Choose Carefully - Instructors should choose a teaching inventory and/or observation protocol that measures the behaviors for which feedback is desired. Additionally, instructor and observer should meet beforehand to align goals, ensuring that the observer knows to pay particular attention to specific practices. 
• Debrief -  Instructor and observer should schedule a time to debrief soon after the observation. This often happens over coffee, in a no-judgment, evaluation-free climate.
• Compare Data Points - Instructors may consider using both a teaching inventory for self-assessment purposes, and have an observer use a teaching protocol in class. When these instruments assess similar items, the outcomes/feedback can be useful to compare.
• Assess Again - After receiving feedback from the observer and reflecting upon practices, instructors might consider asking the observer to re-assess practices during a subsequent class in which changes have been made.
• Consider Total Alignment - Instructors can assess the syllabus and the flow of course design in tandem. The “Downloads” section at the bottom of this page includes an assessment for considering, as an example, the degree of inclusivity in the syllabus and course design.

References and Additional Resources

Blonder, B., Bowles, T., De Master, K., Fanshel, R. Z., Girotto, M., Kahn, A., Keenan, T., Mascarenhas, M., Mgbara, W., Pickett, S., Potts, M., & Rodriguez, M. (2022). Advancing Inclusion and Anti-Racism in the College Classroom: A rubric and resource guide for instructors (1.0.0). Zenodo. https://doi.org/10.5281/zenodo.5874656

Osthoff, E., Clune, W., Ferrare, J., Kretchmar, K., & White, P. (2009). Implementing immersion: Design, professional development, classroom enactment and learning effects of an extended science inquiry unit in an urban district. Madison: University of Wisconsin–Madison, Wisconsin Center for Educational Research.

Piburn, M., Sawada, D., Falconer, K., Turley, J. Benford, R., Bloom, I. (2000). Reformed Teaching Observation Protocol (RTOP). ACEPT IN-003.

Trigwell, K., Prosser, M. Development and Use of the Approaches to Teaching Inventory. (2004). Educational Psychology Review, 16(4): 409-424.

Smith, M., Jones, F., Gilbert, S., and Wieman, C. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices. CBE-Life Sciences Education, Vol. 12.4.

Wieman C., Gilbert S. (2014). The Teaching Practices Inventory: A New Tool for Characterizing College and University Teaching in Mathematics and Science. CBE-Life Sciences Education, 13(3):552–569.

Downloads 

YOU MAY BE INTERESTED IN

Reserve a Room

The Poorvu Center for Teaching and Learning partners with departments and groups on-campus throughout the year to share its space. Please review the reservation form and submit a request.

Instructional Enhancement Fund

The Instructional Enhancement Fund (IEF) awards grants of up to $500 to support the timely integration of new learning activities into an existing undergraduate or graduate course. All Yale instructors of record, including tenured and tenure-track faculty, clinical instructional faculty, lecturers, lectors, and part-time acting instructors (PTAIs), are eligible to apply. Award decisions are typically provided within two weeks to help instructors implement ideas for the current semester.

The Poorvu Center for Teaching and Learning routinely supports members of the Yale community with individual instructional consultations and classroom observations.