research report quality

Criteria for Good Qualitative Research: A Comprehensive Review

• Regular Article
• Open access
• Published: 18 September 2021
• Volume 31 , pages 679–689, ( 2022 )

Cite this article

You have full access to this open access article

• Drishti Yadav   ORCID: orcid.org/0000-0002-2974-0323 1  

75k Accesses

27 Citations

72 Altmetric

Explore all metrics

This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then, references of relevant articles were surveyed to find noteworthy, distinct, and well-defined pointers to good qualitative research. This review presents an investigative assessment of the pivotal features in qualitative research that can permit the readers to pass judgment on its quality and to condemn it as good research when objectively and adequately utilized. Overall, this review underlines the crux of qualitative research and accentuates the necessity to evaluate such research by the very tenets of its being. It also offers some prospects and recommendations to improve the quality of qualitative research. Based on the findings of this review, it is concluded that quality criteria are the aftereffect of socio-institutional procedures and existing paradigmatic conducts. Owing to the paradigmatic diversity of qualitative research, a single and specific set of quality criteria is neither feasible nor anticipated. Since qualitative research is not a cohesive discipline, researchers need to educate and familiarize themselves with applicable norms and decisive factors to evaluate qualitative research from within its theoretical and methodological framework of origin.

Similar content being viewed by others

Mixed methods research: what it is and what it could be

Rob Timans, Paul Wouters & Johan Heilbron

Saturation in qualitative research: exploring its conceptualization and operationalization

Benjamin Saunders, Julius Sim, … Clare Jinks

What Is Sustainability? A Review of the Concept and Its Applications

Avoid common mistakes on your manuscript.

Introduction

“… It is important to regularly dialogue about what makes for good qualitative research” (Tracy, 2010 , p. 837)

To decide what represents good qualitative research is highly debatable. There are numerous methods that are contained within qualitative research and that are established on diverse philosophical perspectives. Bryman et al., ( 2008 , p. 262) suggest that “It is widely assumed that whereas quality criteria for quantitative research are well‐known and widely agreed, this is not the case for qualitative research.” Hence, the question “how to evaluate the quality of qualitative research” has been continuously debated. There are many areas of science and technology wherein these debates on the assessment of qualitative research have taken place. Examples include various areas of psychology: general psychology (Madill et al., 2000 ); counseling psychology (Morrow, 2005 ); and clinical psychology (Barker & Pistrang, 2005 ), and other disciplines of social sciences: social policy (Bryman et al., 2008 ); health research (Sparkes, 2001 ); business and management research (Johnson et al., 2006 ); information systems (Klein & Myers, 1999 ); and environmental studies (Reid & Gough, 2000 ). In the literature, these debates are enthused by the impression that the blanket application of criteria for good qualitative research developed around the positivist paradigm is improper. Such debates are based on the wide range of philosophical backgrounds within which qualitative research is conducted (e.g., Sandberg, 2000 ; Schwandt, 1996 ). The existence of methodological diversity led to the formulation of different sets of criteria applicable to qualitative research.

Among qualitative researchers, the dilemma of governing the measures to assess the quality of research is not a new phenomenon, especially when the virtuous triad of objectivity, reliability, and validity (Spencer et al., 2004 ) are not adequate. Occasionally, the criteria of quantitative research are used to evaluate qualitative research (Cohen & Crabtree, 2008 ; Lather, 2004 ). Indeed, Howe ( 2004 ) claims that the prevailing paradigm in educational research is scientifically based experimental research. Hypotheses and conjectures about the preeminence of quantitative research can weaken the worth and usefulness of qualitative research by neglecting the prominence of harmonizing match for purpose on research paradigm, the epistemological stance of the researcher, and the choice of methodology. Researchers have been reprimanded concerning this in “paradigmatic controversies, contradictions, and emerging confluences” (Lincoln & Guba, 2000 ).

In general, qualitative research tends to come from a very different paradigmatic stance and intrinsically demands distinctive and out-of-the-ordinary criteria for evaluating good research and varieties of research contributions that can be made. This review attempts to present a series of evaluative criteria for qualitative researchers, arguing that their choice of criteria needs to be compatible with the unique nature of the research in question (its methodology, aims, and assumptions). This review aims to assist researchers in identifying some of the indispensable features or markers of high-quality qualitative research. In a nutshell, the purpose of this systematic literature review is to analyze the existing knowledge on high-quality qualitative research and to verify the existence of research studies dealing with the critical assessment of qualitative research based on the concept of diverse paradigmatic stances. Contrary to the existing reviews, this review also suggests some critical directions to follow to improve the quality of qualitative research in different epistemological and ontological perspectives. This review is also intended to provide guidelines for the acceleration of future developments and dialogues among qualitative researchers in the context of assessing the qualitative research.

The rest of this review article is structured in the following fashion: Sect.  Methods describes the method followed for performing this review. Section Criteria for Evaluating Qualitative Studies provides a comprehensive description of the criteria for evaluating qualitative studies. This section is followed by a summary of the strategies to improve the quality of qualitative research in Sect.  Improving Quality: Strategies . Section  How to Assess the Quality of the Research Findings? provides details on how to assess the quality of the research findings. After that, some of the quality checklists (as tools to evaluate quality) are discussed in Sect.  Quality Checklists: Tools for Assessing the Quality . At last, the review ends with the concluding remarks presented in Sect.  Conclusions, Future Directions and Outlook . Some prospects in qualitative research for enhancing its quality and usefulness in the social and techno-scientific research community are also presented in Sect.  Conclusions, Future Directions and Outlook .

For this review, a comprehensive literature search was performed from many databases using generic search terms such as Qualitative Research , Criteria , etc . The following databases were chosen for the literature search based on the high number of results: IEEE Explore, ScienceDirect, PubMed, Google Scholar, and Web of Science. The following keywords (and their combinations using Boolean connectives OR/AND) were adopted for the literature search: qualitative research, criteria, quality, assessment, and validity. The synonyms for these keywords were collected and arranged in a logical structure (see Table 1 ). All publications in journals and conference proceedings later than 1950 till 2021 were considered for the search. Other articles extracted from the references of the papers identified in the electronic search were also included. A large number of publications on qualitative research were retrieved during the initial screening. Hence, to include the searches with the main focus on criteria for good qualitative research, an inclusion criterion was utilized in the search string.

From the selected databases, the search retrieved a total of 765 publications. Then, the duplicate records were removed. After that, based on the title and abstract, the remaining 426 publications were screened for their relevance by using the following inclusion and exclusion criteria (see Table 2 ). Publications focusing on evaluation criteria for good qualitative research were included, whereas those works which delivered theoretical concepts on qualitative research were excluded. Based on the screening and eligibility, 45 research articles were identified that offered explicit criteria for evaluating the quality of qualitative research and were found to be relevant to this review.

Figure  1 illustrates the complete review process in the form of PRISMA flow diagram. PRISMA, i.e., “preferred reporting items for systematic reviews and meta-analyses” is employed in systematic reviews to refine the quality of reporting.

PRISMA flow diagram illustrating the search and inclusion process. N represents the number of records

Criteria for Evaluating Qualitative Studies

Fundamental criteria: general research quality.

Various researchers have put forward criteria for evaluating qualitative research, which have been summarized in Table 3 . Also, the criteria outlined in Table 4 effectively deliver the various approaches to evaluate and assess the quality of qualitative work. The entries in Table 4 are based on Tracy’s “Eight big‐tent criteria for excellent qualitative research” (Tracy, 2010 ). Tracy argues that high-quality qualitative work should formulate criteria focusing on the worthiness, relevance, timeliness, significance, morality, and practicality of the research topic, and the ethical stance of the research itself. Researchers have also suggested a series of questions as guiding principles to assess the quality of a qualitative study (Mays & Pope, 2020 ). Nassaji ( 2020 ) argues that good qualitative research should be robust, well informed, and thoroughly documented.

Qualitative Research: Interpretive Paradigms

All qualitative researchers follow highly abstract principles which bring together beliefs about ontology, epistemology, and methodology. These beliefs govern how the researcher perceives and acts. The net, which encompasses the researcher’s epistemological, ontological, and methodological premises, is referred to as a paradigm, or an interpretive structure, a “Basic set of beliefs that guides action” (Guba, 1990 ). Four major interpretive paradigms structure the qualitative research: positivist and postpositivist, constructivist interpretive, critical (Marxist, emancipatory), and feminist poststructural. The complexity of these four abstract paradigms increases at the level of concrete, specific interpretive communities. Table 5 presents these paradigms and their assumptions, including their criteria for evaluating research, and the typical form that an interpretive or theoretical statement assumes in each paradigm. Moreover, for evaluating qualitative research, quantitative conceptualizations of reliability and validity are proven to be incompatible (Horsburgh, 2003 ). In addition, a series of questions have been put forward in the literature to assist a reviewer (who is proficient in qualitative methods) for meticulous assessment and endorsement of qualitative research (Morse, 2003 ). Hammersley ( 2007 ) also suggests that guiding principles for qualitative research are advantageous, but methodological pluralism should not be simply acknowledged for all qualitative approaches. Seale ( 1999 ) also points out the significance of methodological cognizance in research studies.

Table 5 reflects that criteria for assessing the quality of qualitative research are the aftermath of socio-institutional practices and existing paradigmatic standpoints. Owing to the paradigmatic diversity of qualitative research, a single set of quality criteria is neither possible nor desirable. Hence, the researchers must be reflexive about the criteria they use in the various roles they play within their research community.

Improving Quality: Strategies

Another critical question is “How can the qualitative researchers ensure that the abovementioned quality criteria can be met?” Lincoln and Guba ( 1986 ) delineated several strategies to intensify each criteria of trustworthiness. Other researchers (Merriam & Tisdell, 2016 ; Shenton, 2004 ) also presented such strategies. A brief description of these strategies is shown in Table 6 .

It is worth mentioning that generalizability is also an integral part of qualitative research (Hays & McKibben, 2021 ). In general, the guiding principle pertaining to generalizability speaks about inducing and comprehending knowledge to synthesize interpretive components of an underlying context. Table 7 summarizes the main metasynthesis steps required to ascertain generalizability in qualitative research.

Figure  2 reflects the crucial components of a conceptual framework and their contribution to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice (Johnson et al., 2020 ). The synergy and interrelationship of these components signifies their role to different stances of a qualitative research study.

Essential elements of a conceptual framework

In a nutshell, to assess the rationale of a study, its conceptual framework and research question(s), quality criteria must take account of the following: lucid context for the problem statement in the introduction; well-articulated research problems and questions; precise conceptual framework; distinct research purpose; and clear presentation and investigation of the paradigms. These criteria would expedite the quality of qualitative research.

How to Assess the Quality of the Research Findings?

The inclusion of quotes or similar research data enhances the confirmability in the write-up of the findings. The use of expressions (for instance, “80% of all respondents agreed that” or “only one of the interviewees mentioned that”) may also quantify qualitative findings (Stenfors et al., 2020 ). On the other hand, the persuasive reason for “why this may not help in intensifying the research” has also been provided (Monrouxe & Rees, 2020 ). Further, the Discussion and Conclusion sections of an article also prove robust markers of high-quality qualitative research, as elucidated in Table 8 .

Quality Checklists: Tools for Assessing the Quality

Numerous checklists are available to speed up the assessment of the quality of qualitative research. However, if used uncritically and recklessly concerning the research context, these checklists may be counterproductive. I recommend that such lists and guiding principles may assist in pinpointing the markers of high-quality qualitative research. However, considering enormous variations in the authors’ theoretical and philosophical contexts, I would emphasize that high dependability on such checklists may say little about whether the findings can be applied in your setting. A combination of such checklists might be appropriate for novice researchers. Some of these checklists are listed below:

The most commonly used framework is Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ). This framework is recommended by some journals to be followed by the authors during article submission.

Standards for Reporting Qualitative Research (SRQR) is another checklist that has been created particularly for medical education (O’Brien et al., 2014 ).

Also, Tracy ( 2010 ) and Critical Appraisal Skills Programme (CASP, 2021 ) offer criteria for qualitative research relevant across methods and approaches.

Further, researchers have also outlined different criteria as hallmarks of high-quality qualitative research. For instance, the “Road Trip Checklist” (Epp & Otnes, 2021 ) provides a quick reference to specific questions to address different elements of high-quality qualitative research.

Conclusions, Future Directions, and Outlook

This work presents a broad review of the criteria for good qualitative research. In addition, this article presents an exploratory analysis of the essential elements in qualitative research that can enable the readers of qualitative work to judge it as good research when objectively and adequately utilized. In this review, some of the essential markers that indicate high-quality qualitative research have been highlighted. I scope them narrowly to achieve rigor in qualitative research and note that they do not completely cover the broader considerations necessary for high-quality research. This review points out that a universal and versatile one-size-fits-all guideline for evaluating the quality of qualitative research does not exist. In other words, this review also emphasizes the non-existence of a set of common guidelines among qualitative researchers. In unison, this review reinforces that each qualitative approach should be treated uniquely on account of its own distinctive features for different epistemological and disciplinary positions. Owing to the sensitivity of the worth of qualitative research towards the specific context and the type of paradigmatic stance, researchers should themselves analyze what approaches can be and must be tailored to ensemble the distinct characteristics of the phenomenon under investigation. Although this article does not assert to put forward a magic bullet and to provide a one-stop solution for dealing with dilemmas about how, why, or whether to evaluate the “goodness” of qualitative research, it offers a platform to assist the researchers in improving their qualitative studies. This work provides an assembly of concerns to reflect on, a series of questions to ask, and multiple sets of criteria to look at, when attempting to determine the quality of qualitative research. Overall, this review underlines the crux of qualitative research and accentuates the need to evaluate such research by the very tenets of its being. Bringing together the vital arguments and delineating the requirements that good qualitative research should satisfy, this review strives to equip the researchers as well as reviewers to make well-versed judgment about the worth and significance of the qualitative research under scrutiny. In a nutshell, a comprehensive portrayal of the research process (from the context of research to the research objectives, research questions and design, speculative foundations, and from approaches of collecting data to analyzing the results, to deriving inferences) frequently proliferates the quality of a qualitative research.

Prospects : A Road Ahead for Qualitative Research

Irrefutably, qualitative research is a vivacious and evolving discipline wherein different epistemological and disciplinary positions have their own characteristics and importance. In addition, not surprisingly, owing to the sprouting and varied features of qualitative research, no consensus has been pulled off till date. Researchers have reflected various concerns and proposed several recommendations for editors and reviewers on conducting reviews of critical qualitative research (Levitt et al., 2021 ; McGinley et al., 2021 ). Following are some prospects and a few recommendations put forward towards the maturation of qualitative research and its quality evaluation:

In general, most of the manuscript and grant reviewers are not qualitative experts. Hence, it is more likely that they would prefer to adopt a broad set of criteria. However, researchers and reviewers need to keep in mind that it is inappropriate to utilize the same approaches and conducts among all qualitative research. Therefore, future work needs to focus on educating researchers and reviewers about the criteria to evaluate qualitative research from within the suitable theoretical and methodological context.

There is an urgent need to refurbish and augment critical assessment of some well-known and widely accepted tools (including checklists such as COREQ, SRQR) to interrogate their applicability on different aspects (along with their epistemological ramifications).

Efforts should be made towards creating more space for creativity, experimentation, and a dialogue between the diverse traditions of qualitative research. This would potentially help to avoid the enforcement of one's own set of quality criteria on the work carried out by others.

Moreover, journal reviewers need to be aware of various methodological practices and philosophical debates.

It is pivotal to highlight the expressions and considerations of qualitative researchers and bring them into a more open and transparent dialogue about assessing qualitative research in techno-scientific, academic, sociocultural, and political rooms.

Frequent debates on the use of evaluative criteria are required to solve some potentially resolved issues (including the applicability of a single set of criteria in multi-disciplinary aspects). Such debates would not only benefit the group of qualitative researchers themselves, but primarily assist in augmenting the well-being and vivacity of the entire discipline.

To conclude, I speculate that the criteria, and my perspective, may transfer to other methods, approaches, and contexts. I hope that they spark dialog and debate – about criteria for excellent qualitative research and the underpinnings of the discipline more broadly – and, therefore, help improve the quality of a qualitative study. Further, I anticipate that this review will assist the researchers to contemplate on the quality of their own research, to substantiate research design and help the reviewers to review qualitative research for journals. On a final note, I pinpoint the need to formulate a framework (encompassing the prerequisites of a qualitative study) by the cohesive efforts of qualitative researchers of different disciplines with different theoretic-paradigmatic origins. I believe that tailoring such a framework (of guiding principles) paves the way for qualitative researchers to consolidate the status of qualitative research in the wide-ranging open science debate. Dialogue on this issue across different approaches is crucial for the impending prospects of socio-techno-educational research.

Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16 (10), 1472–1482.

Article   Google Scholar  

Barker, C., & Pistrang, N. (2005). Quality criteria under methodological pluralism: Implications for conducting and evaluating research. American Journal of Community Psychology, 35 (3–4), 201–212.

Bryman, A., Becker, S., & Sempik, J. (2008). Quality criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal of Social Research Methodology, 11 (4), 261–276.

Caelli, K., Ray, L., & Mill, J. (2003). ‘Clear as mud’: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2 (2), 1–13.

CASP (2021). CASP checklists. Retrieved May 2021 from https://casp-uk.net/casp-tools-checklists/

Cohen, D. J., & Crabtree, B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. The Annals of Family Medicine, 6 (4), 331–339.

Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (pp. 1–32). Sage Publications Ltd.

Google Scholar  

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38 (3), 215–229.

Epp, A. M., & Otnes, C. C. (2021). High-quality qualitative research: Getting into gear. Journal of Service Research . https://doi.org/10.1177/1094670520961445

Guba, E. G. (1990). The paradigm dialog. In Alternative paradigms conference, mar, 1989, Indiana u, school of education, San Francisco, ca, us . Sage Publications, Inc.

Hammersley, M. (2007). The issue of quality in qualitative research. International Journal of Research and Method in Education, 30 (3), 287–305.

Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., & Mokkink, L. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19 , 1609406920976417.

Hays, D. G., & McKibben, W. B. (2021). Promoting rigorous research: Generalizability and qualitative research. Journal of Counseling and Development, 99 (2), 178–188.

Horsburgh, D. (2003). Evaluation of qualitative research. Journal of Clinical Nursing, 12 (2), 307–312.

Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10 (1), 42–46.

Johnson, J. L., Adkins, D., & Chauvin, S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84 (1), 7120.

Johnson, P., Buehring, A., Cassell, C., & Symon, G. (2006). Evaluating qualitative management research: Towards a contingent criteriology. International Journal of Management Reviews, 8 (3), 131–156.

Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23 (1), 67–93.

Lather, P. (2004). This is your father’s paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10 (1), 15–34.

Levitt, H. M., Morrill, Z., Collins, K. M., & Rizo, J. L. (2021). The methodological integrity of critical qualitative research: Principles to support design and research review. Journal of Counseling Psychology, 68 (3), 357.

Lincoln, Y. S., & Guba, E. G. (1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation, 1986 (30), 73–84.

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163–188). Sage Publications.

Madill, A., Jordan, A., & Shirley, C. (2000). Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies. British Journal of Psychology, 91 (1), 1–20.

Mays, N., & Pope, C. (2020). Quality in qualitative research. Qualitative Research in Health Care . https://doi.org/10.1002/9781119410867.ch15

McGinley, S., Wei, W., Zhang, L., & Zheng, Y. (2021). The state of qualitative research in hospitality: A 5-year review 2014 to 2019. Cornell Hospitality Quarterly, 62 (1), 8–20.

Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and implementation. San Francisco, US.

Meyer, M., & Dykes, J. (2019). Criteria for rigor in visualization design study. IEEE Transactions on Visualization and Computer Graphics, 26 (1), 87–97.

Monrouxe, L. V., & Rees, C. E. (2020). When I say… quantification in qualitative research. Medical Education, 54 (3), 186–187.

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52 (2), 250.

Morse, J. M. (2003). A review committee’s guide for evaluating qualitative proposals. Qualitative Health Research, 13 (6), 833–851.

Nassaji, H. (2020). Good qualitative research. Language Teaching Research, 24 (4), 427–431.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89 (9), 1245–1251.

O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19 , 1609406919899220.

Reid, A., & Gough, S. (2000). Guidelines for reporting and evaluating qualitative research: What are the alternatives? Environmental Education Research, 6 (1), 59–91.

Rocco, T. S. (2010). Criteria for evaluating qualitative studies. Human Resource Development International . https://doi.org/10.1080/13678868.2010.501959

Sandberg, J. (2000). Understanding human competence at work: An interpretative approach. Academy of Management Journal, 43 (1), 9–25.

Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2 (1), 58–72.

Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5 (4), 465–478.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 (2), 63–75.

Sparkes, A. C. (2001). Myth 94: Qualitative health researchers will agree about validity. Qualitative Health Research, 11 (4), 538–552.

Spencer, L., Ritchie, J., Lewis, J., & Dillon, L. (2004). Quality in qualitative evaluation: A framework for assessing research evidence.

Stenfors, T., Kajamaa, A., & Bennett, D. (2020). How to assess the quality of qualitative research. The Clinical Teacher, 17 (6), 596–599.

Taylor, E. W., Beck, J., & Ainsworth, E. (2001). Publishing qualitative adult education research: A peer review perspective. Studies in the Education of Adults, 33 (2), 163–179.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19 (6), 349–357.

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16 (10), 837–851.

Download references

Open access funding provided by TU Wien (TUW).

Author information

Authors and affiliations.

Faculty of Informatics, Technische Universität Wien, 1040, Vienna, Austria

Drishti Yadav

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Drishti Yadav .

Ethics declarations

Conflict of interest.

The author declares no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Yadav, D. Criteria for Good Qualitative Research: A Comprehensive Review. Asia-Pacific Edu Res 31 , 679–689 (2022). https://doi.org/10.1007/s40299-021-00619-0

Download citation

Accepted : 28 August 2021

Published : 18 September 2021

Issue Date : December 2022

DOI : https://doi.org/10.1007/s40299-021-00619-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Evaluative criteria
• Find a journal
• Publish with us
• Track your research

Research report guide: Definition, types, and tips

Last updated

5 March 2024

Reviewed by

From successful product launches or software releases to planning major business decisions, research reports serve many vital functions. They can summarize evidence and deliver insights and recommendations to save companies time and resources. They can reveal the most value-adding actions a company should take.

However, poorly constructed reports can have the opposite effect! Taking the time to learn established research-reporting rules and approaches will equip you with in-demand skills. You’ll be able to capture and communicate information applicable to numerous situations and industries, adding another string to your resume bow.

• What are research reports?

A research report is a collection of contextual data, gathered through organized research, that provides new insights into a particular challenge (which, for this article, is business-related). Research reports are a time-tested method for distilling large amounts of data into a narrow band of focus.

Their effectiveness often hinges on whether the report provides:

Strong, well-researched evidence

Comprehensive analysis

Well-considered conclusions and recommendations

Though the topic possibilities are endless, an effective research report keeps a laser-like focus on the specific questions or objectives the researcher believes are key to achieving success. Many research reports begin as research proposals, which usually include the need for a report to capture the findings of the study and recommend a course of action.

A description of the research method used, e.g., qualitative, quantitative, or other

Statistical analysis

Causal (or explanatory) research (i.e., research identifying relationships between two variables)

Inductive research, also known as ‘theory-building’

Deductive research, such as that used to test theories

Action research, where the research is actively used to drive change

• Importance of a research report

Research reports can unify and direct a company's focus toward the most appropriate strategic action. Of course, spending resources on a report takes up some of the company's human and financial resources. Choosing when a report is called for is a matter of judgment and experience.

Some development models used heavily in the engineering world, such as Waterfall development, are notorious for over-relying on research reports. With Waterfall development, there is a linear progression through each step of a project, and each stage is precisely documented and reported on before moving to the next.

The pace of the business world is faster than the speed at which your authors can produce and disseminate reports. So how do companies strike the right balance between creating and acting on research reports?

The answer lies, again, in the report's defined objectives. By paring down your most pressing interests and those of your stakeholders, your research and reporting skills will be the lenses that keep your company's priorities in constant focus.

Honing your company's primary objectives can save significant amounts of time and align research and reporting efforts with ever-greater precision.

Some examples of well-designed research objectives are:

Proving whether or not a product or service meets customer expectations

Demonstrating the value of a service, product, or business process to your stakeholders and investors

Improving business decision-making when faced with a lack of time or other constraints

Clarifying the relationship between a critical cause and effect for problematic business processes

Prioritizing the development of a backlog of products or product features

Comparing business or production strategies

Evaluating past decisions and predicting future outcomes

• Features of a research report

Research reports generally require a research design phase, where the report author(s) determine the most important elements the report must contain.

Just as there are various kinds of research, there are many types of reports.

Here are the standard elements of almost any research-reporting format:

Report summary. A broad but comprehensive overview of what readers will learn in the full report. Summaries are usually no more than one or two paragraphs and address all key elements of the report. Think of the key takeaways your primary stakeholders will want to know if they don’t have time to read the full document.

Introduction. Include a brief background of the topic, the type of research, and the research sample. Consider the primary goal of the report, who is most affected, and how far along the company is in meeting its objectives.

Methods. A description of how the researcher carried out data collection, analysis, and final interpretations of the data. Include the reasons for choosing a particular method. The methods section should strike a balance between clearly presenting the approach taken to gather data and discussing how it is designed to achieve the report's objectives.

Data analysis. This section contains interpretations that lead readers through the results relevant to the report's thesis. If there were unexpected results, include here a discussion on why that might be. Charts, calculations, statistics, and other supporting information also belong here (or, if lengthy, as an appendix). This should be the most detailed section of the research report, with references for further study. Present the information in a logical order, whether chronologically or in order of importance to the report's objectives.

Conclusion. This should be written with sound reasoning, often containing useful recommendations. The conclusion must be backed by a continuous thread of logic throughout the report.

• How to write a research paper

With a clear outline and robust pool of research, a research paper can start to write itself, but what's a good way to start a research report?

Research report examples are often the quickest way to gain inspiration for your report. Look for the types of research reports most relevant to your industry and consider which makes the most sense for your data and goals.

The research report outline will help you organize the elements of your report. One of the most time-tested report outlines is the IMRaD structure:

Introduction

...and Discussion

Pay close attention to the most well-established research reporting format in your industry, and consider your tone and language from your audience's perspective. Learn the key terms inside and out; incorrect jargon could easily harm the perceived authority of your research paper.

Along with a foundation in high-quality research and razor-sharp analysis, the most effective research reports will also demonstrate well-developed:

Internal logic

Narrative flow

Conclusions and recommendations

Readability, striking a balance between simple phrasing and technical insight

How to gather research data for your report

The validity of research data is critical. Because the research phase usually occurs well before the writing phase, you normally have plenty of time to vet your data.

However, research reports could involve ongoing research, where report authors (sometimes the researchers themselves) write portions of the report alongside ongoing research.

One such research-report example would be an R&D department that knows its primary stakeholders are eager to learn about a lengthy work in progress and any potentially important outcomes.

However you choose to manage the research and reporting, your data must meet robust quality standards before you can rely on it. Vet any research with the following questions in mind:

Does it use statistically valid analysis methods?

Do the researchers clearly explain their research, analysis, and sampling methods?

Did the researchers provide any caveats or advice on how to interpret their data?

Have you gathered the data yourself or were you in close contact with those who did?

Is the source biased?

Usually, flawed research methods become more apparent the further you get through a research report.

It's perfectly natural for good research to raise new questions, but the reader should have no uncertainty about what the data represents. There should be no doubt about matters such as:

Whether the sampling or analysis methods were based on sound and consistent logic

What the research samples are and where they came from

The accuracy of any statistical functions or equations

Validation of testing and measuring processes

When does a report require design validation?

A robust design validation process is often a gold standard in highly technical research reports. Design validation ensures the objects of a study are measured accurately, which lends more weight to your report and makes it valuable to more specialized industries.

Product development and engineering projects are the most common research-report examples that typically involve a design validation process. Depending on the scope and complexity of your research, you might face additional steps to validate your data and research procedures.

If you’re including design validation in the report (or report proposal), explain and justify your data-collection processes. Good design validation builds greater trust in a research report and lends more weight to its conclusions.

Choosing the right analysis method

Just as the quality of your report depends on properly validated research, a useful conclusion requires the most contextually relevant analysis method. This means comparing different statistical methods and choosing the one that makes the most sense for your research.

Most broadly, research analysis comes down to quantitative or qualitative methods (respectively: measurable by a number vs subjectively qualified values). There are also mixed research methods, which bridge the need for merging hard data with qualified assessments and still reach a cohesive set of conclusions.

Some of the most common analysis methods in research reports include:

Significance testing (aka hypothesis analysis), which compares test and control groups to determine how likely the data was the result of random chance.

Regression analysis , to establish relationships between variables, control for extraneous variables , and support correlation analysis.

Correlation analysis (aka bivariate testing), a method to identify and determine the strength of linear relationships between variables. It’s effective for detecting patterns from complex data, but care must be exercised to not confuse correlation with causation.

With any analysis method, it's important to justify which method you chose in the report. You should also provide estimates of the statistical accuracy (e.g., the p-value or confidence level of quantifiable data) of any data analysis.

This requires a commitment to the report's primary aim. For instance, this may be achieving a certain level of customer satisfaction by analyzing the cause and effect of changes to how service is delivered. Even better, use statistical analysis to calculate which change is most positively correlated with improved levels of customer satisfaction.

• Tips for writing research reports

There's endless good advice for writing effective research reports, and it almost all depends on the subjective aims of the people behind the report. Due to the wide variety of research reports, the best tips will be unique to each author's purpose.

Consider the following research report tips in any order, and take note of the ones most relevant to you:

No matter how in depth or detailed your report might be, provide a well-considered, succinct summary. At the very least, give your readers a quick and effective way to get up to speed.

Pare down your target audience (e.g., other researchers, employees, laypersons, etc.), and adjust your voice for their background knowledge and interest levels

For all but the most open-ended research, clarify your objectives, both for yourself and within the report.

Leverage your team members’ talents to fill in any knowledge gaps you might have. Your team is only as good as the sum of its parts.

Justify why your research proposal’s topic will endure long enough to derive value from the finished report.

Consolidate all research and analysis functions onto a single user-friendly platform. There's no reason to settle for less than developer-grade tools suitable for non-developers.

What's the format of a research report?

The research-reporting format is how the report is structured—a framework the authors use to organize their data, conclusions, arguments, and recommendations. The format heavily determines how the report's outline develops, because the format dictates the overall structure and order of information (based on the report's goals and research objectives).

What's the purpose of a research-report outline?

A good report outline gives form and substance to the report's objectives, presenting the results in a readable, engaging way. For any research-report format, the outline should create momentum along a chain of logic that builds up to a conclusion or interpretation.

What's the difference between a research essay and a research report?

There are several key differences between research reports and essays:

Research report:

Ordered into separate sections

More commercial in nature

Often includes infographics

Heavily descriptive

More self-referential

Usually provides recommendations

Research essay

Does not rely on research report formatting

More academically minded

Normally text-only

Less detailed

Omits discussion of methods

Usually non-prescriptive 

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 6 October 2023

Last updated: 5 March 2024

Last updated: 25 November 2023

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

Child Care and Early Education Research Connections

Assessing research quality.

This page presents information and tools to help evaluate the quality of a research study, as well as information on the ethics of research.

The quality of social science and policy research can vary considerably. It is important that consumers of research keep this in mind when reading the findings from a research study or when considering whether or not to use data from a research study for secondary analysis.

Announcements

Find announcements, including conferences and meetings, Research Connections newsletters, opportunities, and more.

Search Resources

Search all resources in the Research Connections Library.

Explore Our Topics

Research Connections' resources are organized into topical categories and subcategories.

Key Questions to Ask

This section outlines key questions to ask in assessing the quality of research.

Research Assessment Tools

This section provides resources related to quantitative and qualitative assessment tools.

Ethics of Research

This section provides an overview of three basic ethical principles.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 24 June 2021

Initial evidence of research quality of registered reports compared with the standard publishing model

• Courtney K. Soderberg   ORCID: orcid.org/0000-0003-1227-7042 1   na1 ,
• Timothy M. Errington   ORCID: orcid.org/0000-0002-4959-5143 1   na1 ,
• Sarah R. Schiavone   ORCID: orcid.org/0000-0002-8829-137X 2 ,
• Julia Bottesini 2 ,
• Felix Singleton Thorn   ORCID: orcid.org/0000-0002-0237-6146 3 ,
• Simine Vazire   ORCID: orcid.org/0000-0002-3933-9752 2 , 3 ,
• Kevin M. Esterling   ORCID: orcid.org/0000-0002-5529-6422 4 &
• Brian A. Nosek   ORCID: orcid.org/0000-0001-6797-5476 1 , 5  

Nature Human Behaviour volume  5 ,  pages 990–997 ( 2021 ) Cite this article

4044 Accesses

90 Citations

254 Altmetric

Metrics details

In registered reports (RRs), initial peer review and in-principle acceptance occur before knowing the research outcomes. This combats publication bias and distinguishes planned from unplanned research. How RRs could improve the credibility of research findings is straightforward, but there is little empirical evidence. Also, there could be unintended costs such as reducing novelty. Here, 353 researchers peer reviewed a pair of papers from 29 published RRs from psychology and neuroscience and 57 non-RR comparison papers. RRs numerically outperformed comparison papers on all 19 criteria (mean difference 0.46, scale range −4 to +4) with effects ranging from RRs being statistically indistinguishable from comparison papers in novelty (0.13, 95% credible interval [−0.24, 0.49]) and creativity (0.22, [−0.14, 0.58]) to sizeable improvements in rigour of methodology (0.99, [0.62, 1.35]) and analysis (0.97, [0.60, 1.34]) and overall paper quality (0.66, [0.30, 1.02]). RRs could improve research quality while reducing publication bias and ultimately improve the credibility of the published literature.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

A systematic review and multivariate meta-analysis of the physical and mental health benefits of touch interventions

Julian Packheiser, Helena Hartmann, … Frédéric Michon

Worldwide divergence of values

Joshua Conrad Jackson & Danila Medvedev

Microdosing with psilocybin mushrooms: a double-blind placebo-controlled study

Federico Cavanna, Stephanie Muller, … Enzo Tagliazucchi

Data availability

All data files are available on OSF: https://osf.io/aj4zr/ .

Code availability

All files and scripts are available on OSF: https://osf.io/aj4zr/ .

Chambers, C. What’s next for registered reports? Nature 573 , 187–189 (2019).

Article   CAS   Google Scholar  

Chambers, C. The registered reports revolution. Lessons in cultural reform. Significance 16 , 23–27 (2019).

Article   Google Scholar  

Nosek, B. A. & Lakens, D. Registered reports: a method to increase the credibility of published results. Soc. Psychol. 45 , 137–141 (2014).

Nosek, B. A., Spies, J. R. & Motyl, M. Scientific utopia: II. restructuring incentives and practices to promote truth over publishability. Perspect. Psychol. Sci. 7 , 615–631 (2012).

Smith, R. Peer review: a flawed process at the heart of science and journals. J. R. Soc. Med. 99 , 178–182 (2006).

Fanelli, D. Negative results are disappearing from most disciplines and countries. Scientometrics 90 , 891–904 (2012).

Fanelli, D. ‘Positive’ results increase down the hierarchy of the sciences. PLoS ONE 5 , e10068 (2010).

Franco, A., Malhotra, N. & Simonovits, G. Publication bias in the social sciences: unlocking the file drawer. Science 345 , 1502–1505 (2014).

Dickersin, K. The existence of publication bias and risk factors for its occurrence. JAMA 263 , 1385–1389 (1990).

Mahoney, M. J. Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cogn. Ther. Res. 1 , 161–175 (1977).

Greenwald, A. G. Consequences of prejudice against the null hypothesis. Psychol. Bull. 82 , 1–20 (1975).

Sterling, T. D. Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. J. Am. Stat. Assoc. 54 , 30–34 (1959).

Google Scholar  

Makel, M. C., Plucker, J. A. & Hegarty, B. Replications in psychology research: How often do they really occur? Perspect. Psychol. Sci. 7 , 537–542 (2012).

Schmidt, S. Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev. Gen. Psychol. 13 , 90–100 (2009).

Makel, M. C. & Plucker, J. A. Facts are more important than novelty. Educ. Res. 43 , 304–316 (2014).

Schimmack, U. The ironic effect of significant results on the credibility of multiple-study articles. Psychol. Methods 17 , 551–566 (2012).

Giner-Sorolla, R. Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspect. Psychol. Sci. 7 , 562–571 (2012).

Begley, C. G. & Ellis, L. M. Raise standards for preclinical cancer research. Nature 483 , 531–533 (2012).

Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10 , 712–712 (2011).

Camerer, C. F. et al. Evaluating replicability of laboratory experiments in economics. Science 351 , 1433–1436 (2016).

Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2 , 637–644 (2018).

Klein, R. A. et al. Many Labs 2: investigating variation in replicability across samples and settings. Adv. Methods Pract. Psychol. Sci. 1 , 443–490 (2018).

Klein, R. A. et al. Investigating variation in replicability: a ‘many labs’ replication project. Soc. Psychol. 45 , 142–152 (2014).

Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349 , aac4716 (2015).

Ebersole, C. R. et al. Many Labs 3: evaluating participant pool quality across the academic semester via replication. J. Exp. Soc. Psychol. 67 , 68–82 (2016).

Allen, C. & Mehler, D. M. A. Open science challenges, benefits and tips in early career and beyond. PLoS Biol. 17 , e3000246 (2019).

Scheel, A. M., Schijen, M. & Lakens, D. An excess of positive results: comparing the standard psychology literature with registered reports . Preprint at PsyArXiv https://osf.io/p6e9c (2020).

Hummer, L. T., Singleton Thorn, F., Nosek, B. A. & Errington, T. M. Evaluating registered reports: a naturalistic comparative study of article impact. Preprint at OSF https://osf.io/5y8w7 (2017).

Cropley, A. Research as artisanship versus research as generation of novelty: the march to nowhere. Creat. Res. J 30 , 323–328 (2018).

Baumeister, R. F. Charting the future of social psychology on stormy seas: winners, losers, and recommendations. J. Exp. Soc. Psychol. 66 , 153–158 (2016).

Nosek, B. A. & Errington, T. M. The best time to argue about what a replication means? Before you do it. Nature 583 , 518–520 (2020).

Gelman, A., Hill, J. & Yajima, M. Why we (usually) don’t have to worry about multiple comparisons. J. Res. Educ. Eff. 5 , 189–211 (2012).

Epskamp, S. & Nuijten, M. B. statcheck: extract statistics from articles and recompute P values. R package version 1.3.1 (2018).

Hardwicke, T. E. & Ioannidis, J. P. A. Mapping the universe of registered reports. Nat. Hum. Behav. 2 , 793–796 (2018).

Chambers, C. D. & Mellor, D. T. Protocol transparency is vital for registered reports. Nat. Hum. Behav. 2 , 791–792 (2018).

John, L. K., Loewenstein, G. & Prelec, D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23 , 524–532 (2012).

Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22 , 1359–1366 (2011).

Nuijten, M. B., van Assen, M. A. L. M., Hartgerink, C. H. J., Epskamp, S. & Wicherts, J. M. The validity of the tool ‘statcheck’ in discovering statistical reporting inconsistencies. Preprint at PsyArXiv https://osf.io/tcxaj (2017).

Stan Development Team. Stan Modeling Language Users Guide and Reference Manual (2020).

Download references

Acknowledgements

The authors thank L. Hummer for help with study planning, A. Denis and Z. Loomas for help preparing survey materials, B. Bouza and N. Buttrick for help with implementing the survey in Qualtrics and A. Allard for help coding the articles. This research was funded by grants from Arnold Ventures and James S. McDonnell Foundation (grant # 220020498) to B.A.N. and supported by the National Science Foundation Graduate Research Fellowship Program (grant # 1247392 awarded to S.R.S). The funders had no role in study design, analysis, decision to publish, or preparation of the manuscript.

Author information

These authors contributed equally: Courtney K. Soderberg, Timothy M. Errington.

Authors and Affiliations

Center for Open Science, Charlottesville, VA, USA

Courtney K. Soderberg, Timothy M. Errington & Brian A. Nosek

Department of Psychology, University of California, Davis, Davis, CA, USA

Sarah R. Schiavone, Julia Bottesini & Simine Vazire

School of Psychological Sciences, University of Melbourne, Melbourne, Victoria, Australia

Felix Singleton Thorn & Simine Vazire

Department of Political Science, University of California, Riverside, Riverside, CA, USA

Kevin M. Esterling

Department of Psychology, University of Virginia, Charlottesville, VA, USA

Brian A. Nosek

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Survey: C.K.S., T.M.E. and B.A.N. Article coding: S.R.S., J.B. and S.V. Data curation: Survey: C.K.S. Article coding: S.R.S. and J.B. Formal analysis: Survey: C.K.S. and K.M.E. Article coding: S.V., S.R.S. and J.B. Investigation: Survey: C.K.S. and T.M.E. Article coding: S.R.S., J.B. and S.V. Methodology: Survey: C.K.S., T.M.E., K.M.E. and B.A.N. Article coding: S.R.S., J.B. and S.V. Software: Article coding: S.R.S. and J.B. Visualization: Survey: C.K.S. Article coding: S.R.S. and J.B. Validation: Survey: K.M.E. Article coding: S.V. Project administration: T.M.E. Resources: T.M.E. and F.S.T. Supervision: T.M.E. and B.A.N. Funding acquisition: T.M.E. and B.A.N. Writing, original draft: C.K.S., T.M.E., K.M.E. and B.A.N. Writing, review and editing: C.K.S., T.M.E., S.R.S., J.B., F.S.T., S.V., K.M.E. and B.A.N.

Corresponding author

Correspondence to Brian A. Nosek .

Ethics declarations

Competing interests.

T.M.E., C.K.S. and B.A.N. are employees of the nonprofit Center for Open Science (COS), which has a mission to increase openness, integrity and reproducibility of research. COS offers support to journals, editors and researchers in adopting and conducting RRs. The remaining authors declare no conflicts of interest.

Additional information

Peer review information Nature Human Behaviour thanks Balazs Aczel, Marcel van Assen and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1 plot of correlations between all difference score outcome variables..

Correlation matrix of the 19 outcome variables with larger darker blue circles indicating stronger positive correlations than smaller lighter blue circles.

Extended Data Fig. 2 Posterior probability distributions for parameter estimates for each DV and each level of Familiar comparing the difference of RRs and comparison articles.

Horizontal lines indicate 80% (thick) and 95% (thin) credible intervals and dots show the mean of the posteriors. Positive values indicate a performance advantage for RRs, negative values indicate a performance advantage for comparison articles.

Extended Data Fig. 3 Posterior probability distributions for parameter estimates for each DV and each level of Improve comparing the difference of RRs and comparison articles.

Extended data fig. 4 posterior probability distributions for parameter estimates for each dv and each level of ‘guessed right’ comparing the difference of rrs and comparison articles., supplementary information.

Supplementary Methods, Supplementary Discussion, Supplementary Figs. 1–6, Supplementary Tables 1–10 and Supplementary References.

Reporting summary

Peer review information, rights and permissions.

Reprints and permissions

About this article

Cite this article.

Soderberg, C.K., Errington, T.M., Schiavone, S.R. et al. Initial evidence of research quality of registered reports compared with the standard publishing model. Nat Hum Behav 5 , 990–997 (2021). https://doi.org/10.1038/s41562-021-01142-4

Download citation

Received : 07 January 2021

Accepted : 19 May 2021

Published : 24 June 2021

Issue Date : August 2021

DOI : https://doi.org/10.1038/s41562-021-01142-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Registered report adoption in academic journals: assessing rates in different research domains.

• Ting-Yu Lin
• Hao-Chien Cheng
• Tsung-Min Hung

Scientometrics (2024)

Open Science

• Alexander Maedche
• Edona Elshan
• Oliver Werth

Business & Information Systems Engineering (2024)

A guide for social science journal editors on easing into open science

• Priya Silverstein
• Colin Elman

Research Integrity and Peer Review (2024)

Nature welcomes Registered Reports

Nature (2023)

Empirical evidence of widespread exaggeration bias and selective reporting in ecology

• Kaitlin Kimmel
• Meghan L. Avolio
• Paul J. Ferraro

Nature Ecology & Evolution (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Research Reports: Definition and How to Write Them

Reports are usually spread across a vast horizon of topics but are focused on communicating information about a particular topic and a niche target market. The primary motive of research reports is to convey integral details about a study for marketers to consider while designing new strategies.

Certain events, facts, and other information based on incidents need to be relayed to the people in charge, and creating research reports is the most effective communication tool. Ideal research reports are extremely accurate in the offered information with a clear objective and conclusion. These reports should have a clean and structured format to relay information effectively.

What are Research Reports?

Research reports are recorded data prepared by researchers or statisticians after analyzing the information gathered by conducting organized research, typically in the form of surveys or qualitative methods .

A research report is a reliable source to recount details about a conducted research. It is most often considered to be a true testimony of all the work done to garner specificities of research.

The various sections of a research report are:

• Background/Introduction
• Implemented Methods
• Results based on Analysis
• Deliberation

Learn more: Quantitative Research

Components of Research Reports

Research is imperative for launching a new product/service or a new feature. The markets today are extremely volatile and competitive due to new entrants every day who may or may not provide effective products. An organization needs to make the right decisions at the right time to be relevant in such a market with updated products that suffice customer demands.

The details of a research report may change with the purpose of research but the main components of a report will remain constant. The research approach of the market researcher also influences the style of writing reports. Here are seven main components of a productive research report:

• Research Report Summary: The entire objective along with the overview of research are to be included in a summary which is a couple of paragraphs in length. All the multiple components of the research are explained in brief under the report summary.  It should be interesting enough to capture all the key elements of the report.
• Research Introduction: There always is a primary goal that the researcher is trying to achieve through a report. In the introduction section, he/she can cover answers related to this goal and establish a thesis which will be included to strive and answer it in detail.  This section should answer an integral question: “What is the current situation of the goal?”.  After the research design was conducted, did the organization conclude the goal successfully or they are still a work in progress –  provide such details in the introduction part of the research report.
• Research Methodology: This is the most important section of the report where all the important information lies. The readers can gain data for the topic along with analyzing the quality of provided content and the research can also be approved by other market researchers . Thus, this section needs to be highly informative with each aspect of research discussed in detail.  Information needs to be expressed in chronological order according to its priority and importance. Researchers should include references in case they gained information from existing techniques.
• Research Results: A short description of the results along with calculations conducted to achieve the goal will form this section of results. Usually, the exposition after data analysis is carried out in the discussion part of the report.

Learn more: Quantitative Data

• Research Discussion: The results are discussed in extreme detail in this section along with a comparative analysis of reports that could probably exist in the same domain. Any abnormality uncovered during research will be deliberated in the discussion section.  While writing research reports, the researcher will have to connect the dots on how the results will be applicable in the real world.
• Research References and Conclusion: Conclude all the research findings along with mentioning each and every author, article or any content piece from where references were taken.

Learn more: Qualitative Observation

15 Tips for Writing Research Reports

Writing research reports in the manner can lead to all the efforts going down the drain. Here are 15 tips for writing impactful research reports:

• Prepare the context before starting to write and start from the basics:  This was always taught to us in school – be well-prepared before taking a plunge into new topics. The order of survey questions might not be the ideal or most effective order for writing research reports. The idea is to start with a broader topic and work towards a more specific one and focus on a conclusion or support, which a research should support with the facts.  The most difficult thing to do in reporting, without a doubt is to start. Start with the title, the introduction, then document the first discoveries and continue from that. Once the marketers have the information well documented, they can write a general conclusion.
• Keep the target audience in mind while selecting a format that is clear, logical and obvious to them:  Will the research reports be presented to decision makers or other researchers? What are the general perceptions around that topic? This requires more care and diligence. A researcher will need a significant amount of information to start writing the research report. Be consistent with the wording, the numbering of the annexes and so on. Follow the approved format of the company for the delivery of research reports and demonstrate the integrity of the project with the objectives of the company.
• Have a clear research objective: A researcher should read the entire proposal again, and make sure that the data they provide contributes to the objectives that were raised from the beginning. Remember that speculations are for conversations, not for research reports, if a researcher speculates, they directly question their own research.
• Establish a working model:  Each study must have an internal logic, which will have to be established in the report and in the evidence. The researchers’ worst nightmare is to be required to write research reports and realize that key questions were not included.

Learn more: Quantitative Observation

• Gather all the information about the research topic. Who are the competitors of our customers? Talk to other researchers who have studied the subject of research, know the language of the industry. Misuse of the terms can discourage the readers of research reports from reading further.
• Read aloud while writing. While reading the report, if the researcher hears something inappropriate, for example, if they stumble over the words when reading them, surely the reader will too. If the researcher can’t put an idea in a single sentence, then it is very long and they must change it so that the idea is clear to everyone.
• Check grammar and spelling. Without a doubt, good practices help to understand the report. Use verbs in the present tense. Consider using the present tense, which makes the results sound more immediate. Find new words and other ways of saying things. Have fun with the language whenever possible.
• Discuss only the discoveries that are significant. If some data are not really significant, do not mention them. Remember that not everything is truly important or essential within research reports.

Learn more: Qualitative Data

• Try and stick to the survey questions. For example, do not say that the people surveyed “were worried” about an research issue , when there are different degrees of concern.
• The graphs must be clear enough so that they understand themselves. Do not let graphs lead the reader to make mistakes: give them a title, include the indications, the size of the sample, and the correct wording of the question.
• Be clear with messages. A researcher should always write every section of the report with an accuracy of details and language.
• Be creative with titles – Particularly in segmentation studies choose names “that give life to research”. Such names can survive for a long time after the initial investigation.
• Create an effective conclusion: The conclusion in the research reports is the most difficult to write, but it is an incredible opportunity to excel. Make a precise summary. Sometimes it helps to start the conclusion with something specific, then it describes the most important part of the study, and finally, it provides the implications of the conclusions.
• Get a couple more pair of eyes to read the report. Writers have trouble detecting their own mistakes. But they are responsible for what is presented. Ensure it has been approved by colleagues or friends before sending the find draft out.

Learn more: Market Research and Analysis

MORE LIKE THIS

The Power of AI in Customer Experience — Tuesday CX Thoughts

Apr 16, 2024

Employee Lifecycle Management Software: Top of 2024

Apr 15, 2024

Top 15 Sentiment Analysis Software That Should Be on Your List

Top 13 A/B Testing Software for Optimizing Your Website

Apr 12, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

Understanding science: How to evaluate the research quality of scientific studies?

• All Management Learning Resources
• Evaluating research quality

• Executive summary

As you continue to develop your evidence-based management skills, learning to evaluate research for its quality and applicability to your own management practice is paramount. While peer-reviewed, empirical research is generally of a much higher quality than other published information found from other sources, not all journal articles are created equally. Some research findings are incredibly fascinating and provocative, yet hard to reproduce; other trends may sound exciting, but not be relevant to your own industry or management practice (Creswell, 2002).

How can you, as an evidence-based manager, sift through published work to find articles of value and relevance? The key is learning some basics about evaluating the quality of research. This CQ Dossier will explain two of the core attributes that all valuable, useful research must exhibit: reliability and validity (Kmet et al, 2004).

Reliability: Consistency and reproducibility of results

Measure reliability: consistent and similar test scores over time, validity: good study design and accurate result presentation, internal validity: cause and effect relationship, design validity: research design and implementation, external validity: applicability and transferability of research results, key take-aways, references and further reading.

Put broadly, reliability is all about the consistency and reproducibility of results. A single empirical study is never sufficient to prove that an intervention works, or that a relationship between factors is robust and genuine (Aguinis et al, 2017). This is because findings sometimes result from random human error, statistical abnormalities, the selection of a very specific and unique sample, and any number of other issues. In order to truly have faith that a research trend is real, it must be found multiple times, in a variety of contexts. This is at the core of evaluating a study’s reliability (Creswell, 2002).

As a manager, you can use reliability to guide your work in several ways. First, only consider implementing an intervention if it has been shown to work in a variety of studies, conducted in a variety of settings (Aguinis et al, 2017; Kmet et al, 2004). A single test is never sufficient proof that a strategy works.

 In addition to pertaining to how reproducible a study’s findings are, reliability can also be used to examine a measure or instrument (Santos, 1999). A reliable measure of worker satisfaction, for example, will yield consistent, similar scores over time, when administered to the same individuals. An unreliable measure will not show such consistency – a single employee may seem highly satisfied, according to the measure, on one day, and highly dissatisfied the next. This can taint statistical analysis and conclusions about results in an irreparable way.

When seeking methods of measuring factors, such as employee performance, personality, or motivation, make sure to select a measure that has been tested for reliability in empirical research papers. Chronbach’s alpha is the statistic commonly used to test a measure’s reliability (Santos, 1999); look for values of 0.80 or higher to indicate a robust, consistent test.

Validity refers to a variety of crucial research attributes, which any quality study should exhibit. In a general sense, a study can be said to be “valid” if the results are accurately represented, the study is well-designed, and the results can be used to draw useful, broad conclusions. The obverse is also true: if a study is misreported, not well controlled and designed, or does not provide practical and applicable conclusions, it is likely to be invalid. Each of these attributes is discussed below.

A study’s cause-effect validity, often referred to as internal validity, is the degree to which the study has identified a genuine cause and effect relationship. In management research, it can be very difficult to prove conclusively that an intervention or strategy had a direct effect on outcomes the manager cares about (Landers & Behrend, 2015). This is because the average workplace is dynamic and complicated, with many other factors impacting employee behavior.

In order to truly prove that a factor has a causal impact on workplace outcomes, a study must

• manipulate the variable that is believed to be the cause, under controlled conditions;
• demonstrate that a change in outcomes followed the manipulation of the causal variable; and
• must rule out any alternate, plausible explanations of why a change in outcomes was observed (Landers & Behrend, 2015).

If a study does not provide such information, and does not test an intervention under such rigorous conditions, you do not have conclusive proof that it is effective. All results, then, should be viewed with skepticism.

When reading an empirical study, pay close attention to the details about the method. Make sure you can answer the following questions:

• Was the study conducted in an actual workplace, or in a laboratory?
• Were the participants actual employees, or random volunteers?
• Were external factors measured and controlled for? Did the authors consider, and address, criticisms of their conclusions?

These questions will help to give you a sense of the research quality (Becker et al, 2016).

Many psychological studies are conducted in laboratories, with volunteer samples that may not resemble the employees you, as a manager, will be working with (Becker et al, 2016). When studies are conducted in organizational settings, they tend to be a bit less well-controlled, and any observed effects may be a statistical fluke. Some researchers, in addition, are not adept at acknowledging that their conclusions may be only one of interpretation of the results, among many possible alternatives. Make sure to base your own management decisions on research that is carefully conducted, with results that are reported fairly, with limitations acknowledged.

 A study can be said to be “externally valid” if the researcher’s findings apply easily to the outside world. This type of validity is not absolute: findings that may be valid for one industry or group of people may not be valid for another (Green & Glasgow, 2006). For example, if you are seeking to introduce an employee wellness program, it is probably best to select one that has been tested in the industry you occupy, in an organization with a similar size and comparable demographics to yours (Kessler & Vesterlund, 2015).

Organizational research, as mentioned above, is conducted in a wide variety of settings, with a wide variety of types of people. As a result, some conclusions may not be applicable to your own management practice or your organization (Green & Glasgow, 2006). Make sure to pay close attention to a study’s demographics, setting, organization size, and country of origin; the more factors that make your organization distinct from the one being tested, the less likely it is that the researcher’s findings will apply to you. Conversely, if a strategy or intervention has worked in many places, industries, and cultures, it is likelier to be relevant and useful in your workplace (Kessler & Vesterlund, 2015).

• While all peer-reviewed research is quite rigorous, it’s important to learn how to determine an individual study’s quality
• A reliable research finding is one that has been reproduced, typically in a variety of settings
• A reliable measure is one which shows great consistency, when administered to the same people multiple times over a prolonged period
• A valid study should provide strong evidence that an intervention or strategy directly causes an improvement in outcomes
• Valid studies must also be well-controlled, and ought to be conducted in true organizational settings
• A researcher’s conclusions may not be valid for your own management practice if the research was conducted in an industry, country, or culture very different from your own

Management skills newsletter

Join our monthly newsletter to receive management tips, tricks and insights directly into your inbox!

Aguinis, H., Cascio, W. F., & Ramani, R. S. (2017). Science’s reproducibility and replicability crisis: International business is not immune.

Becker, T. E., Atinc, G., Breaugh, J. A., Carlson, K. D., Edwards, J. R., & Spector, P. E. (2016). Statistical control in correlational studies: 10 essential recommendations for organizational researchers.  Journal of Organizational Behavior ,  37 (2), 157-167.

Creswell, J. W. (2002).  Educational research: Planning, conducting, and evaluating quantitative  (pp. 146-166). Upper Saddle River, NJ: Prentice Hall.

Green, L. W., & Glasgow, R. E. (2006). Evaluating the relevance, generalization, and applicability of research: issues in external validation and translation methodology.  Evaluation & the health professions ,  29 (1), 126-153.

Kessler, J., & Vesterlund, L. (2015). The external validity of laboratory experiments: The misleading emphasis on quantitative effects.  Handbook of Experimental Economic Methodology, Oxford University Press, Oxford, UK.

Kmet, L. M., Lee, R. C., & Cook, L. S. (2004). Standard quality assessment criteria for evaluating primary research papers from a variety of fields.

Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples.  Industrial and Organizational Psychology ,  8 (2), 142-164.

Santos, J. R. A. (1999). Cronbach’s alpha: A tool for assessing the reliability of scales.  Journal of extension ,  37 (2), 1-5.

About the Author

Related Dossiers

Understanding science: Why social science matters in management and how to read scientific papers

Understanding science: Introduction to statistics for management practitioners

Learn what really matters based on evidence: An Introduction to Evidence-based Management

Related Services

Leadership guide for professionals

• Payment processed via PayPal
• No PayPal account required
• Pay with PayPal, credit card, bank transfer
• Right of return within 14 days
• 14 day satisfaction guarantee

Career counseling for personality development: learn about your personality traits, strenghts and weaknesses to kick-start professional growth

Funding source and research report quality in nutrition practice-related research

Affiliation.

• 1 Research and Strategic Business Development, American Dietetic Association, Chicago, Illinois, United States of America. [email protected]
• PMID: 22163017
• PMCID: PMC3232225
• DOI: 10.1371/journal.pone.0028437

Background: The source of funding is one of many possible causes of bias in scientific research. One method of detecting potential for bias is to evaluate the quality of research reports. Research exploring the relationship between funding source and nutrition-related research report quality is limited and in other disciplines the findings are mixed.

Objective: The purpose of this study is to determine whether types of funding sources of nutrition research are associated with differences in research report quality.

Design: A retrospective study of research reporting quality, research design and funding source was conducted on 2539 peer reviewed research articles from the American Dietetic Association's Evidence Analysis Library® database.

Results: Quality rating frequency distributions indicate 43.3% of research reports were rated as positive, 50.1% neutral, and 6.6% as negative. Multinomial logistic regression results showed that while both funding source and type of research design are significant predictors of quality ratings (χ2 = 118.99, p≤0.001), the model's usefulness in predicting overall research report quality is little better than chance. Compared to research reports with government funding, those not acknowledging any funding sources, followed by studies with University/hospital funding were more likely to receive neutral vs positive quality ratings, OR = 1.85, P <0.001 and OR = 1.54, P<0.001, respectively and those that did not report funding were more likely to receive negative quality ratings (OR = 4.97, P<0.001). After controlling for research design, industry funded research reports were no more likely to receive a neutral or negative quality rating than those funded by government sources.

Conclusion: Research report quality cannot be accurately predicted from the funding source after controlling for research design. Continued vigilance to evaluate the quality of all research regardless of the funding source and to further understand other factors that affect quality ratings are warranted.

Publication types

• Research Support, Non-U.S. Gov't
• Dietetics / methods
• Drug Industry / economics
• Foundations / economics
• Industry / economics
• Nutritional Sciences / standards*
• Publications
• Quality Control
• Regression Analysis
• Research Design
• Research Report
• Research Support as Topic
• Retrospective Studies
• Societies, Medical

A New Vision for High-Quality Preschool Curriculum

A high-quality preschool education can foster critical development and learning that promotes joyful, affirming, and enriching learning opportunities that prepare children for success in school and life. While preschool programs generally provide emotionally supportive environments, their curricula often fall short in advancing learning in math, early literacy, and science, and lack the necessary support for multilingual learners emerging bilingualism. Additionally, access to high-quality, effective early learning experiences may be limited and inadequate based on factors such as a childs race, location, gender, language, identified disability, and socioeconomic status.

A New Vision for High-Quality Preschool Curriculum examines preschool curriculum quality for children from ages three to five, with special attention to the needs of Black and Latine children, multilingual learners, children with disabilities and children experiencing poverty in the United States. The report articulates a vision for high-quality preschool curricula for all children, grounded in an equity and justice-oriented principles from inception to implementation and evaluation.

Read Full Description

• Issue Brief: Early Childhood Organizations and Educators
• Issue Brief: Funders
• Issue Brief: Policymakers
• Issue Brief: Curriculum Developers and Publishers

Recent News

Review Examines COVID-19 Vaccination and Adverse Events

Science Academies Issue Statements to Inform G7 Talks

Supporting Family Caregivers in STEMM

Pregnant and Lactating Women in Clinical Research

• Load More...

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Korean J Women Health Nurs
• v.28(2); 2022 Jun 30

Strategies to improve the quality of reporting nursing research

Associated data.

Research that demonstrates high quality reporting is important to improve clinical nursing practice and facilitate follow-up studies in various disciplines. Poor reporting is unethical [ 1 , 2 ], and such papers are often confusing, resulting in impractical, futile information that can even be detrimental to patient care [ 1 , 3 ]. Therefore, quality journals are invested in publishing good papers with good reporting quality, often maintained effectively via a peer review system [ 3 , 4 ]. However, the effective use of reporting guidelines is also considered a useful strategy to enhance the reporting quality of research published in academic journals [ 3 , 5 ].

The most frequently recommended and well-known reporting guidelines were developed by the EQUATOR Network ( https://www.equator-network.org/ ). EQUATOR is an acronym for Enhancing the QUAlity and Transparency Of health Research. As the name suggests, the EQUATOR Network, an international organization, is a new initiative to persuade clear and transparent health research reporting [ 2 , 6 ]. It aims to ensure accuracy, completion, and transparency in reporting health research studies to promote functionality and replicability of research and to make health research credible and valuable by popularizing reporting guidelines for health research [ 6 , 7 ].

The Korean Journal of Women Health Nursing (KJWHN), as the official journal of the Korean Society of Women Health Nursing, is constantly striving to improve the journal quality by publishing quality research reports. Not only is this beneficial for our readership, but this would also be one of the strategies for the journal to be indexed in international journal databases, such as the Social Science Citation Index (SSCI). As one of the efforts to improve the quality of reporting of published research, while adopting a double-blind peer review system comprising a professional reviewer pool, KJWHN also recommends using the reporting guidelines of the EQUATOR Network in its author guidelines [ 8 ]. In addition, the editorial board of KJWHN has published three articles analyzing adherence to reporting guidelines of research published in KJWHN and highlighted areas for reporting improvement [ 9 - 11 ]. KJWHN’s effort to adopt and inform the reporting guidelines of the EQUATOR Network is advanced level compared to other journals in Korea.

Despite the concerns regarding the strict use of reporting guidelines inhibiting the creativity of researchers [ 12 ], the perceived value of reporting guidelines has increased, and adopting reporting guidelines has become an international trend and principle in manuscript writing. Nevertheless, difficulties in correctly selecting and using guidelines according to research design have been reported [ 13 , 14 ]. Therefore, in this editorial, to improve the use of reporting guidelines, I want to (1) introduce the EQUATOR Network and reporting guideline development; (2) analyze the status of recommendations for reporting guidelines in nursing journals; (3) explain how to find appropriate reporting guidelines considering the research design; and (4) suggest strategies to efficiently use reporting guidelines.

The EQUATOR Network and reporting guidelines development

The EQUATOR Network was officially launched in 2008 as a result of the EQUATOR project, which was funded by the United Kingdom in 2006 and intended to make a map for preparing and disseminating health research reporting guidelines and establish a global collaborative relationship among key persons [ 6 ]. Twenty-seven key individuals including representatives of reporting guideline development groups, journal editors, peer reviewers, and funders from 10 countries collaborated on the project. The EQUATOR Network also established four national centers between 2014 and 2016, to focus on activities that raise awareness and support good research reporting practices [ 6 ].

As a simple structured tool, reporting guidelines are used by researchers while writing manuscripts and they comprise a basic list of information required to enable reviewers and readers to make accurate appraisal of the research quality [ 6 ]. As such, they are tools that help achieve good quality reporting in health studies. Adhering to reporting guidelines properly can also facilitate accurate replicating by other researchers and effective use by nurses and healthcare professionals to make a better clinical decision [ 4 - 6 ]. Currently, there are hundreds of general guidelines according to research types and special guidelines, which are expanded forms of general reporting guidelines.

The status of recommendations for reporting guidelines in nursing journals

In addition to many biomedical journals, nursing journals also emphasize the use of reporting guidelines while preparing manuscripts, but there are no previous studies on how much the reporting guidelines are specifically recommended in nursing journal guidelines. Therefore, for this editorial, 122 journals listed in the nursing category of SSCI as of June 1, 2022 ( Supplementary material 1 ), were analyzed in terms of reference to the reporting guidelines of the EQUATOR Network.

Among the 122 SSCI nursing journals, 94 journals (77.0%) specified adopting the EQUATOR Network’s guidelines in their author guidelines, whereas 28 journals (23.0%) did not mention reporting guidelines at all. This rate is quite high compared to a 2012 study of journals in other disciplines, in which 46% of journals mentioned reporting guidelines in their journal instruction [ 3 ]. However, a sizable proportion of nursing journals still require improvement.

Among the 94 journals that mentioned reporting guidelines use, 85 (90.4%) provided information about representative reporting guidelines according to the research design, such as CONSORT (CONsolidated Standards Of Reporting Trials) guidelines along with the EQUATOR Network link URL, whereas 9 journals (9.6%) simply mentioned the use of the reporting guidelines of the EQUATOR Network without listing reporting guidelines specifically.

The reporting guidelines that were frequently mentioned more than 10 times in nursing journals are PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) for systematic review (n=71), CONSORT for randomized trials (n=64), SQUIRE (Standards for QUality Improvement Reporting Excellence) for quality improvement studies (n=42), STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) for observational studies (n=41), COREQ (COnsolidated criteria for REporting Qualitative research) for qualitative research (n=37), TREND (Transparent Reporting of Evaluations with Nonrandomized Designs) for nonrandomized trials (n=27), STARD (STAndards for the Reporting of Diagnostic accuracy) for diagnostic/prognostic studies (n=19), CARE (CAse REport) for case report (n=14), SRQR (Standards for Reporting Qualitative Research) for qualitative research (n=14), and MOOSE (Meta-analysis Of Observational Studies in Epidemiology) for systematic review of observational studies (n=12), as arranged based on their frequency. Details, including the meaning of acronyms for reporting guidelines, version information, and the direct link URL of either the EQUATOR website or individual guideline website; are presented in Table 1 .

Summaries of the reporting guidelines commonly mentioned in the nursing journals (N=94)

EQUATOR: Enhancing the QUAlity and Transparency Of health Research.

If readers click the suggested link URL in Table 1 , they can easily find information about an updated or previous version of the reporting guideline, related forms (e.g., checklists or flow diagrams), history of guideline development, and related or elaborated publications to state the development process and provide a detailed explanation for correct use. In addition, reporting guidelines can be downloaded in either PDF (portable document format) or word file. Although following a reporting guideline does not guarantee acceptance for publication, it is the initial step for the successful publication of a manuscript [ 15 ].

In addition, the level of recommendation regarding reporting guidelines also varied. Some nursing journals required authors to complete and attach reporting guideline checklists while submitting the manuscript, whereas other journals only encouraged authors or reviewers to refer to reporting guidelines when writing or reviewing a manuscript, and as stated above, some journals did not mention reporting guidelines at all. Given that this is a phenomenon also frequently seen in other disciplines [ 4 ], although reporting guideline use in nursing journals is not low compared to that in other disciplines, the recommendations for using reporting guidelines in author instructions should be extended further.

How to find reporting guidelines to fit the research design

Numerous guidelines have been presented on the homepage of the EQUATOR Network [ 6 ], but it is not easy to find guidelines suitable for the study design. It has been reported that many authors struggle to follow reporting guidelines, especially in choosing the right guidelines for their study and in using them correctly [ 7 ], because while many journals mention reporting guidelines as general statements rather than suggesting clear instructions about how to select or use them [ 8 ]. Thus, this is one of the main challenges in improving the use of reporting guidelines, which EQUATOR is trying to solve. To promote the correct use of reporting guidelines, improving the author’s understanding of report guidelines according to the research design and increasing the motivation for using those guidelines are important.

Recently, algorithms and websites have been developed to help authors find reporting guidelines that fit their research design effectively. The EQUATOR Network developed the EQUATOR Reporting Guidelines Decision Tree ( https://www.equator-network.org/wp-content/uploads/2013/11/20160301-RG-Decision-Tree-used-for-EQUATOR-wizard-vn-1.pdf ), which is an algorithm that helps in the selection of reporting guidelines [ 12 ], and the EQUATOR Wizard ( https://www.penelope.ai/equator-wizard ), a new tool to help authors find the right reporting guideline or different checklists for different types of study design [ 13 ]. The UK EQUATOR Center also launched GoodReports.org, a website that aids authors in finding and using reporting guidelines [ 16 ]. Various electronic algorithms are currently being developed to facilitate the choice of correct reporting guideline(s), and other tools are being integrated into journal editorial management processes [ 1 , 2 ]. I suggest that all readers, including authors and reviewers, visit the suggested sites to determine the appropriate guidelines for their research design.

Strategies for better use of reporting guidelines for all users

As the positive influence of adhering to reporting guidelines enhancing the quality of published research is evident, I suggest strategies for the effective use of reporting guidelines by authors, reviewers, and journal editors, based on the EQUATOR Network’s suggestions and previous studies.

Firstly, for authors, try to find out the reporting guidelines when planning the study and drafting the manuscript, rather than at the stage of submitting the manuscript [ 16 ]. Authors should be encouraged to also check any new relevant guidelines that are more compatible with their research topic because there are many extended versions of the general guidelines according to the study design, which continue to be finessed and developed. In addition, authors should ensure adherence of all items in the reporting guidelines before submitting their manuscript and if not, explain why some items were not reported in their manuscript. The authors should note that research must always be reproducible.

Secondly, reviewers should understand the reporting guidelines correctly and verify the proper reporting of each item in the manuscript. To improve their detailed understanding of the frequently used reporting guidelines in nursing, and to increase general awareness about reporting guidelines, reviewers should acquire updated information. Opportunities in academic communities to actively share the experience of using reporting guidelines would be a practical measure to this aim, through events such as conferences and workshops.

Finally, the editorial board of journals should include the link to the EQUATOR Network website as well as frequently used reporting guidelines in the ‘Instructions for Authors’ to find the available reporting guidelines easily. In addition, the editorial board should attempt to provide clear instructions on the appropriate use of the guidelines so that authors and reviewers can easily assess the quality of the manuscript based on reporting guidelines. Journal editors and reviewers should also review the manuscript carefully, verifying the adherence to recognized reporting guidelines pertinent to the research design in the manuscript. Similar to the previous research conducted by the editorial board of KJWHN to evaluate reporting guideline use, further evaluation studies must be conducted regularly to identify reporting areas for improvement and weakness. These efforts should be actively shared with authors and reviewers to promote their understanding and motivation for using reporting guidelines. As noted above, continuous education for guideline use in the academic conference or workshop should be provided to authors and reviewers to improve research quality in the journals.

Acknowledgments

I would like to thank Min Kyong Lee, undergraduate student of Ajou University, for her support on this work.

Authors’ contributions

All work was done by Song JE.

Conflict of interest

Ju-Eun Song has been editorial board of the Korean Journal of Women Health Nursing since January 2012. She was not involved in the review process of this editorial. Otherwise, there was no conflict of interest.

Supplementary materials

Further details on supplementary materials are presented online (available at https://doi.org/10.4069/kjwhn.2022.06.08.1 ).

Supplementary material 1.

Journals indexed in the nursing category of the Social Science Citation Index as of Jun 1, 2022 (N=122).

NTRS - NASA Technical Reports Server

Available downloads, related records.

• International edition
• Australia edition
• Europe edition

Gender medicine ‘built on shaky foundations’, Cass review finds

Analysis finds most research underpinning clinical guidelines, hormone treatments and puberty blockers to be low quality

• Review of gender services has major implications for mental health services

The head of the world’s largest review into children’s care has said that gender medicine is “built on shaky foundations”.

Dr Hilary Cass, the paediatrician commissioned to conduct a review of the services provided by the NHS to children and young people questioning their gender identity, said that while doctors tended to be cautious in implementing new findings in emerging areas of medicine, “quite the reverse happened in the field of gender care for children”.

Cass commissioned the University of York to conduct a series of analyses as part of her review.

Two papers examined the quality and development of current guidelines and recommendations for managing gender dysphoria in children and young people. Most of the 23 clinical guidelines reviewed were not independent or evidence based, the researchers found.

A third paper on puberty blockers found that of 50 studies, only one was of high quality.

Similarly, of 53 studies included in a fourth paper on the use of hormone treatment, only one was of sufficiently high quality, with little or only inconsistent evidence on key outcomes.

Here are the main findings of the reviews:

Clinical guidelines

Increasing numbers of children and young people experiencing gender dysphoria are being referred to specialist gender services. There are various guidelines outlining approaches to the clinical care of these children and adolescents.

In the first two papers, the York researchers examined the quality and development of published guidelines or clinical guidance containing recommendations for managing gender dysphoria in children and young people up to the age of 18.

They studied a total of 23 guidelines published in different countries between 1998 and 2022. All but two were published after 2010.

Most of them lacked “an independent and evidence-based approach and information about how recommendations were developed”, the researchers said.

Few guidelines were informed by a systematic review of empirical evidence and they lack transparency about how their recommendations were developed. Only two reported consulting directly with children and young people during their development, the York academics found.

“Healthcare services and professionals should take into account the poor quality and interrelated nature of published guidance to support the management of children and adolescents experiencing gender dysphoria/incongruence,” the researchers wrote.

Writing in the British Medical Journal (BMJ) , Cass said that while medicine was usually based on the pillars of integrating the best available research evidence with clinical expertise, and patient values and preferences, she “found that in gender medicine those pillars are built on shaky foundations”.

She said the World Professional Association of Transgender Healthcare (WPATH) had been “highly influential in directing international practice, although its guidelines were found by the University of York’s appraisal to lack developmental rigour and transparency”.

In the foreword to her report, Cass said while doctors tended to be cautious in implementing new findings “quite the reverse happened in the field of gender care for children”.

In one example, she said a single Dutch medical study, “suggesting puberty blockers may improve psychological wellbeing for a narrowly defined group of children with gender incongruence”, had formed the basis for their use to “spread at pace to other countries”. Subsequently, there was a “greater readiness to start masculinising/feminising hormones in mid-teens”.

She added: “Some practitioners abandoned normal clinical approaches to holistic assessment, which has meant that this group of young people have been exceptionalised compared to other young people with similarly complex presentations. They deserve very much better.”

Both papers repeatedly pointed to a key problem in this area of medicine: a dearth of good data.

She said: “Filling this knowledge gap would be of great help to the young people wanting to make informed choices about their treatment.”

Cass said the NHS should put in place a “full programme of research” looking at the characteristics, interventions and outcomes of every young person presenting to gender services, with consent routinely sought for enrolment in a research study that followed them into adulthood.

Gender medicine was “an area of remarkably weak evidence”, her review found, with study results also “exaggerated or misrepresented by people on all sides of the debate to support their viewpoint”.

Alongside a puberty blocker trial, which could be in place by December, there should be research into psychosocial interventions and the use of the masculinising and feminising hormones testosterone and oestrogen, the review found.

Hormone treatment

Many trans people who seek medical intervention in their transition opt to take hormones to masculinise or feminise their body, an approach that has been used in transgender adults for decades.

“It is a well-established practice that has transformed the lives of many transgender people,” the Cass review notes, adding that while these drugs are not without long-term problems and side-effects, for many they are dramatically outweighed by the benefits.

For birth-registered females, the approach means taking testosterone, which brings about changes including the growth of facial hair and a deepening of the voice, while for birth-registered males, it involves taking hormones including oestrogen to promote changes including the growth of breasts and an increase in body fat. Some of these changes may be irreversible.

However, in recent years a growing proportion of adolescents have begun taking these cross-sex, or gender-affirming, hormones, with the vast majority who are prescribed puberty blockers subsequently moving on to such medication.

This growing take-up among young people has led to questions over the impact of these hormones in areas ranging from mental health to sexual functioning and fertility.

Now researchers at the University of York have carried out a review of the evidence, comprising an analysis of 53 previously published studies, in an attempt to set out what is known – and what is not – about the risks, benefits and possible side-effects of such hormones on young people.

All but one study, which looked at side-effects, were rated of moderate or low quality, with the researchers finding limited evidence for the impact of such hormones on trans adolescents with respect to outcomes, including gender dysphoria and body satisfaction.

The researchers noted inconsistent findings around the impact of such hormones on growth, height, bone health and cardiometabolic effects, such as BMI and cholesterol markers. In addition, they found no study assessed fertility in birth-registered females, and only one looked at fertility in birth-registered males.

“These findings add to other systematic reviews in concluding there is insufficient and/or inconsistent evidence about the risks and benefits of hormone interventions in this population,” the authors write.

However, the review did find some evidence that masculinising or feminising hormones might help with psychological health in young trans people. An analysis of five studies in the area suggested hormone treatment may improve depression, anxiety and other aspects of mental health in adolescents after 12 months of treatment, with three of four studies reporting an improvement around suicidality and/or self-harm (one reported no change).

But unpicking the precise role of such hormones is difficult. “Most studies included adolescents who received puberty suppression, making it difficult to determine the effects of hormones alone,” the authors write, adding that robust research on psychological health with long-term follow-up was needed.

The Cass review has recommended NHS England should review the current policy on masculinising or feminising hormones, advising that while there should be the option to provide such drugs from age 16, extreme caution was recommended, and there should be a clear clinical rationale for not waiting until an individual reached 18.

Puberty blockers

Treatments to suppress puberty in adolescents became available through routine clinical practice in the UK a decade ago.

While the drugs have long been used to treat precocious puberty – when children start puberty at an extremely young age – they have only been used off-label in children with gender dysphoria or incongruence since the late 1990s. The rationale for giving puberty blockers, which originated in the Netherlands, was to buy thinking time for young people and improve their ability to smooth their transition in later life.

Data from gender clinics reported in the Cass review showed the vast majority of people who started puberty suppression went on to have masculinising or feminising hormones, suggesting that puberty blockers did not buy people time to think.

To understand the broader effects of puberty blockers, researchers at the University of York identified 50 papers that reported on the effects of the drugs in adolescents with gender dysphoria or incongruence. According to their systematic review, only one of these studies was high quality, with a further 25 papers regarded as moderate quality. The remaining 24 were deemed too weak to be included in the analysis.

Many of the reports looked at how well puberty was suppressed and the treatment’s side-effects, but fewer looked at whether the drugs had their intended benefits.

Of two studies that investigated gender dysphoria and body satisfaction, neither found a change after receiving puberty blockers. The York team found “very limited” evidence that puberty blockers improved mental health.

Overall, the researchers said “no conclusions” could be drawn about the impact on gender dysphoria, mental and psychosocial health or cognitive development, though there was some evidence bone health and height may be compromised during treatment.

Based on the York work, the Cass review finds that puberty blockers offer no obvious benefit in helping transgender males to help their transition in later life, particularly if the drugs do not lead to an increase in height in adult life. For transgender females, the benefits of stopping irreversible changes such as a deeper voice and facial hair have to be weighed up against the need for penile growth should the person opt for vaginoplasty, the creation of a vagina and vulva.

In March, NHS England announced that children with gender dysphoria would no longer receive puberty blockers as routine practice. Instead, their use will be confined to a trial that the Cass review says should form part of a broader research programme into the effects of masculinising and feminising hormones.

• Transgender
• Young people

More on this story

No case for closing Scotland’s only NHS gender services clinic, says first minister

Veteran trans campaigner: ‘Cass review has potential for positive change’

Cass review must be used as ‘watershed moment’ for NHS gender services, says Streeting

‘This isn’t how good scientific debate happens’: academics on culture of fear in gender medicine research

Five thousand children with gender-related distress awaiting NHS care in England

Ban on children’s puberty blockers to be enforced in private sector in England

What Cass review says about surge in children seeking gender services

Adult transgender clinics in England face inquiry into patient care

‘Children are being used as a football’: Hilary Cass on her review of gender identity services

Thousands of children unsure of gender identity ‘let down by NHS’, report finds

Most viewed.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

During Second Chance Month, HRSA Takes Policy Action, Releases First-Ever Funding Opportunity for Health Centers to Support Transitions in Care for People Leaving Incarceration

Latest policy makes clear HRSA-funded Health Centers can provide health care services – including chronic disease, mental health, and substance use disorder treatment – to individuals in the 90 days prior to release from incarceration as they get ready and return to the community

New funding opportunity makes $51 million available to support transitions in care prior to release from incarceration for the first time in the program’s history

Today, the U.S. Department of Health and Human Services (HHS), through the Health Resources and Services Administration (HRSA), took new policy action and announced the availability of $51 million for the first-ever funding opportunity for HRSA-funded health centers to implement innovative approaches to support transitions in care for people leaving incarceration. Aligned with the White House Second Chance Initiative , today’s action, for the first time, explicitly supports the provision of health services to individuals during the 90 days prior to their release to help them return to the community by expanding access to primary health care, including mental health and substance use disorder treatment, furthering public health and strengthening public safety.

HRSA-funded health centers provide primary care regardless of ability to pay and are a cornerstone of our country’s health care system, especially for individuals and families who are uninsured; enrolled in Medicaid; living in rural, remote, or underserved areas; struggling to afford their health insurance co-pays; experiencing homelessness; residing in public housing; or otherwise having difficulty finding a doctor or paying for the cost of care. The 1,400 HRSA-funded health centers operate more than 15,000 service sites, providing care for more than 30 million patients.

“As President Biden has often said, America is a nation of second chances. People reentering the community after incarceration deserve a fair shot at living long, healthy lives and contributing to their communities,” said HHS Secretary Xavier Becerra. “HRSA-funded health centers are well positioned to facilitate these transitions from incarceration to high quality, community-based primary care.”

“At HRSA, we know that poor care transitions as people leave prisons and jails can have devasting consequences,” said HRSA Administrator Carole Johnson. “People returning from incarceration – particularly those with substance use or mental health conditions – often have significant health care needs and are particularly vulnerable to overdose deaths and other bad health outcomes.  Yet, they struggle to get connected to the health services.  That is why we are launching this innovative HRSA effort during Second Chance Month and aim to tackle these needs head-on and ensure better care and better health outcomes for people reentering their communities after incarceration.”

HRSA’s updated policy makes clear that health centers can provide health services to incarcerated individuals who are expected to be or are scheduled for release from a carceral setting within 90 days to help ensure continuity of care as people move home to the community.

As many as 80% of individuals returning to the community after incarceration have chronic medical, psychiatric, and/or substance use conditions. Studies have shown a dramatic and concerning increase in risk for opioid overdose after release from incarceration, with one study showing the risk of death at least ten times higher than for the general public. Nearly half of individuals entering incarceration meet the criteria for having a substance use disorder, and many of these individuals struggle to access and afford medication-assisted and other substance use disorder treatments following release.

Individuals returning to the community also become disconnected from critical medications and treatment for diabetes and hypertension – among the leading causes of death in the United States. They also face disproportionate risks of bad health outcomes overall because they lack connections to services and supports to navigate the process of applying for or reinstating health insurance eligibility and other benefits in the immediate period after their release.

For this competitive funding opportunity, approximately 51 health centers will implement approaches that focus on:

• Reducing drug overdose risk
• Addressing mental health and substance use disorder treatment needs
• Managing chronic conditions
• Preventing, screening, diagnosing, and treating hepatitis C, HIV, syphilis, and other infectious diseases

Health centers can also use funds made available through the funding opportunity to provide case management services that address key social drivers of health, such as housing and food insecurity, financial strain, access to transportation, and intimate partner violence. Facilitating the successful reentry of individuals returning to the community will enhance public health and safety.

This work builds on previous action taken across the Department to support justice-involved individuals and complements efforts from SAMHSA and other federal entities that provide grant funds to expand substance use disorder treatment and related recovery and reentry services to adults. A new Medicaid 1115 waiver opportunity, for example, empowers states to provide justice-involved individuals with Medicaid coverage in the period immediately prior to their release to address various health concerns, including substance use disorders and other chronic health issues. Additionally, the HHS Roadmap for Behavioral Health Integration emphasizes engaging populations at highest risk, including individuals who are justice-involved.

Applications are due in Grants.gov on June 10, 2024, and in  HRSA Electronic Handbooks  on July 2, 2024.

Visit the Health Center Program webpage for more information about this funding opportunity.

Find a health center:  https://findahealthcenter.hrsa.gov .

Sign Up for Email Updates

Receive the latest updates from the Secretary, Blogs, and News Releases

Subscribe to RSS

Receive latest updates

Related News Releases

Fact sheet: the biden-harris administration urges congress to lower drug costs for americans with commercial insurance, hhs, doj, and ftc issue request for public input as part of inquiry into impacts of corporate ownership trend in health care, hhs secretary xavier becerra tours new england, announces $100 million investment in women’s healthcare alongside the first lady, related blog posts.

Upholding Research Integrity at HHS

Media inquiries.

For general media inquiries, please contact  [email protected] .

Presale Report

Symphony CLO 43, Ltd. (U.S. Structured Credit)

Tue 16 Apr, 2024 - 5:04 PM ET

Key Rating Drivers Asset Credit Quality (Negative): The average credit quality of the indicative portfolio is 'B/B-', which is in line with that of recent CLOs. The weighted average rating factor (WARF) of the indicative portfolio is 24.74, versus a maximum covenant, in accordance with the initial expected matrix point of 26. Issuers rated in the 'B' rating category denote a highly speculative credit quality; however, the notes benefit from appropriate credit enhancement and standard U.S. CLO structural features. Asset Security (Positive): The indicative portfolio consists of 99% first-lien senior secured loans. The weighted average recovery rate (WARR) of the indicative portfolio is 74.51% versus a minimum covenant, in accordance with the initial expected matrix point of 74.1%. Portfolio Composition (Positive): The largest three industries may comprise up to 47.5% of the portfolio balance in aggregate while the top five obligors can represent up to 12.5% of the portfolio balance in aggregate. The level of diversity resulting from the industry, obligor and geographic concentrations is in line with other recent CLOs. Portfolio Management (Neutral): The transaction has a five-year reinvestment period and reinvestment criteria similar to other CLOs. Fitch's analysis was based on a stressed portfolio created by adjusting to the indicative portfolio to reflect permissible concentration limits and collateral quality test levels.

An official website of the United States government Here's how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Air Travel Consumer Report: January 2024 Numbers

WASHINGTON – The U.S. Department of Transportation (DOT) today released its Air Travel Consumer Report (ATCR) on airline operational data compiled for the month of January 2024 for on-time performance, mishandled baggage, and mishandled wheelchairs and scooters. The ATCR is designed to assist consumers with information on the quality of services provided by airlines. 

DOT expects that airlines will operate flights as scheduled and that when they do not, airlines will provide consumers the services consumers have been promised when a flight is canceled or delayed because of an airline issue. After a two-year DOT push to improve the passenger experience, the 10 largest airlines now guarantee meals and free rebooking on the same airline and nine guarantee hotel accommodations. Consumer-friendly information regarding airline commitments to their customers is available on the Department’s Airline Customer Service Dashboard at FlightRights.Gov. DOT also pushed airlines to provide fee-free family seating and rolled out a new family seating dashboard that highlights the airlines that guarantee fee-free family seating, and those of the 10 largest that do not, making it easier for parents to avoid paying junk fees to sit with their children when they fly.

In addition, DOT is improving transportation for individuals with disabilities. In July 2023, DOT finalized a rule which requires airlines to make lavatories on new, single-aisle aircraft more accessible. Then, in February 2024, DOT issued a proposal to address other barriers that Americans who use a wheelchair encounter when it comes to air travel by, among other things, mandating enhanced training for airline employees and contractors who physically assist passenger with disabilities and handle passengers’ wheelchairs.

Further, when necessary, DOT takes enforcement action against airlines and ticket agents that fail to comply with the Department’s aviation consumer protection requirements. In 2023, DOT issued the largest fines in the history of the consumer protection office. This includes a $140 million penalty against Southwest Airlines for failing passengers during the 2022 holiday meltdown. That penalty, which was in addition to over $600 million DOT already ensured was refunded by Southwest to passengers, requires Southwest to establish a $90 million compensation system for future passengers affected by significant delays and cancellations. DOT has helped return more than $3 billion in refunds to travelers since the pandemic began.

Flight Operations

The 560,352 flights operated in January 2024 were 99.56% of the 562,845 flights operated in January 2023. Operated flights in January 2024 were down 0.44% year-over-year from the 562,845 flights operated in January 2023 and down 7.18% month-over-month from 603,756 flights operated in December 2023. 

In January 2024, the 10 marketing network carriers reported 582,425 scheduled domestic flights, 22,073 (3.8%) of which were canceled. In December 2023, airlines scheduled 606,218 domestic flights, 2,462 (1.3%) of which were canceled. In January 2023, airlines scheduled 573,877 domestic flights, 11,032 (1.9%) of which were canceled.

On January 6, 2024, the Federal Aviation Administration (FAA) ordered the grounding of Boeing 737 MAX aircraft with a mid-cabin door plug installed operated by U.S. airlines or in U.S. territory. On January 24, 2024, FAA cleared all such aircraft to return to service after each aircraft operator successfully completed a new inspection process approved by the FAA. Alaska Airlines and United Airlines have informed the DOT that the grounding of the 737 MAX9 aircraft with the mid-cabin door plug installed has impacted their on-time statistics during this reporting period.

January 2024 On-Time Arrival

In January 2024, reporting marketing carriers posted an on-time arrival rate of 72.8%, down from both 83.9% in December 2023 and from 76.2% in January 2023.

Highest Marketing Carrier On-Time Arrival Rates January 2024 (ATCR Table 1)

• Delta Airlines Network – 77.8%
• Allegiant Air – 75.6%
• Southwest Airlines – 73.9% 

Lowest Marketing Carrier On-Time Arrival Rates January 2024 (ATCR Table 1)

• Alaska Airlines Network – 64.7%
• JetBlue Airways – 69.5%
• American Airlines Network – 70.5%

January 2024 Flight Cancellations

In January 2024, reporting marketing carriers canceled 3.8% of their scheduled domestic flights, higher than both the rate of 0.4% in December 2023 and the rate of 1.9% in January 2023. 

Lowest Marketing Carrier Rates of Canceled Flights January 2024 (ATCR Table 6)

• Hawaiian Airlines – 1.5%  
• Spirit Airlines – 1.5%   
• JetBlue Airways – 1.7%    

Highest Marketing Carrier Rates of Canceled Flights January 2024 (ATCR Table 6)

• Alaska Airlines Network – 11.9%    
• United Airlines Network – 6.9%    
• Southwest Airlines – 3.1%    

Complaints About Airline Service

The release of air travel service complaint data in the Air Travel Consumer Report (ATCR) has been delayed primarily because of the continued high volume of complaints against airlines and ticket agents received by the Office of Aviation Consumer Protection (OACP) and the time needed to review and process these consumer complaints. The Department is investing in modernizing its system for handling consumer complaints with the support of a Technology Modernization Fund (TMF) investment to improve the customer experience for the tens of thousands of consumers who use the system each year and enable OACP to more effectively engage in oversight of the airline industry. 

As DOT modernizes its system, given the continued high volume of air travel service complaints concerning airlines and ticket agents, DOT has revised how it processes consumer complaints received after June 1, 2023. From June 2023 until the date its system is modernized, DOT intends to revise the ATCR to display consumer submissions (complaints, inquiries, and opinions) as opposed to complaints for this period. The Department will continue to display civil rights complaints in the ATCR in a similar manner as before and anticipates publishing submission and civil rights complaint numbers in spring.

Tarmac Delays

In January 2024, airlines reported 71 tarmac delays of more than three hours on domestic flights, compared to five tarmac delays of more than three hours on domestic flights reported in December 2023. In January 2024, airlines reported six tarmac delays of more than four hours on international flights, compared to zero tarmac delays of more than four hours on international flights reported in December 2023. 

Airlines are required to have and adhere to assurances that they will not allow aircraft to remain on the tarmac for more than three hours for domestic flights and four hours for international flights without providing passengers the option to deplane, subject to exceptions related to safety, security, and Air Traffic Control related reasons. An exception also exists for departure delays if the airline begins to return the aircraft to a suitable disembarkation point to deplane passengers by those times.

The Department investigates extended tarmac delays.

Mishandled Baggage

In January 2024, reporting marketing carriers handled 37.4 million bags and posted a mishandled baggage rate of 0.75%, higher than both the rate of 0.50% in December 2023 and the rate of 0.73% in January 2023.

The Department began displaying the mishandled baggage data as a percentage (i.e., per 100 bags enplaned) in January 2022. This is consistent with the manner that the mishandled wheelchairs and scooters rate is calculated and displayed.     In the prior three calendar year reports (2019 to 2021), the Department calculated the mishandled baggage rate based on the number of mishandled bags per 1,000 checked bags. 

Mishandled Wheelchairs and Scooters

In January 2024, reporting marketing carriers reported checking 56,659 wheelchairs and scooters and mishandling 836 for a rate of 1.48% mishandled wheelchairs and scooters, higher than the rate of 1.39% mishandled in December 2023 and lower than the rate of 1.47% mishandled in January 2023.

As described earlier, in February 2024, the Department announced its proposal to strengthen its rule implementing the Air Carrier Access Act (ACAA) to address the serious problems that individuals with disabilities using wheelchairs and scooters face when traveling by air that impact their safety and dignity, including mishandled wheelchairs and scooters and improper transfers to and from aircraft seats, aisle chairs, and personal wheelchairs. The proposed rule would require that airlines meet strict standards in accommodating passengers with disabilities by setting new standards for prompt, safe, and dignified assistance, mandating enhanced training for airline employees and contractors who physically assist passengers with disabilities and handle passengers’ wheelchairs, and outlining actions that airlines must take to protect passengers when a wheelchair is damaged during transport. The proposed rule also clarifies that damaging or delaying the return of a wheelchair is an automatic violation of the ACAA.

Bumping/Oversales

Bumping/oversales data, unlike other air carrier data, are reported quarterly rather than monthly. For the fourth quarter of 2023, the 10 U.S. reporting marketing carriers posted an involuntary denied boarding, or bumping, rate of 0.20 per 10,000 passengers, lower than both the rate of 0.35 in the third quarter of 2023 and the rate of 0.30 in the fourth quarter of 2022.

Incidents Involving Animals

As part of its IT modernization, DOT’s Office of Aviation Consumer Protection (OACP) is improving the options for covered carriers to submit their monthly and annual Reports on Incidents Involving Animals During Air Transport. While the new system is being developed, OACP is permitting covered carriers to delay submission of reports on incidents involving animals during air transport. Annual data on such incidents will be published when DOT receives carriers’ complete submissions of the 2023 data. 

In January 2024, carriers reported zero incidents involving the death, injury, or loss of an animal while traveling by air, equal to the zero reports filed in both December 2023 and in January 2023.

Consumers may file air travel consumer or civil rights complaints online at   https://secure.dot.gov/air-travel-complaint , or they may mail a complaint to the Office of Aviation Consumer Protection, U.S. Department of Transportation, C-70, W96-432, 1200 New Jersey Avenue, SE, Washington, DC 20590.

The ATCR and other aviation consumer matters of interest to the public can be found at https://www.transportation.gov/airconsumer .