types of documentation in research methodology

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

17.7 Documents and other artifacts

Learning objectives.

Learners will be able to…

• Identify key considerations when planning to analyze documents and other artifacts as a strategy for qualitative data gathering, including preparations, tools, and skills to support it
• Assess whether analyzing documents and other artifacts is an effective approach to gather data for your qualitative research proposal

Qualitative researchers may also elect to utilize existing documents (e.g. reports, newspapers, blogs, minutes) or other artifacts (e.g. photos, videos, performances, works of art) as sources of data. Artifact analysis can provide important information on a specific topic, for instance, how same-sex couples are portrayed in the media. They also may provide contextual information regarding the values and popular sentiments of a given time and/or place. When choosing to utilize documents and other artifacts as a source of data for your project, remember that you are approaching these as a researcher, not just as a consumer of media. You need to thoughtfully plan what artifacts you will include, with a clear justification for their selection that is solidly linked to your research question, as well as a plan for systematically approaching these artifacts to identify and obtain relevant information from them.

Obtaining your artifacts

As you begin considering what artifacts you will be using for your research study, there are two points to consider: what will help you to answer your research question and what can you gain access to. In addressing the first of these considerations, you may already have a good idea about what artifacts are needed because you have done a substantial amount of preliminary work and you know this area well. However, if you are unsure, or you need to supplement your existing knowledge, some general sources can include: librarians, historians, community experts, topical experts, organizations or agencies that address the issue or serve the population you will be studying, and other researchers who study this area. In considering access, if the artifacts are public the answer may be a straightforward yes, but if the documents are privately held, you may need to be granted permission – and remember, this is permission to use them for research purposes, not just to view them. When obtaining permission, get something in writing, so that you have this handy to submit with your IRB application. While the types of artifacts you might include are almost endless (given they are relevant to your research question), Table 18.4 offers a list of some ideas for different sources you might consider.

Artifact analysis skills

Consistent with other areas of research, but perhaps especially salient to the use of artifacts, you will require organizational skills. Depending on what sources you choose to include, you may literally have volumes of data. Furthermore, you might not just be dealing with a large amount of data, but also a variety of types of data. Regardless of whether you are using physical or virtual data, you need to have a way to label and catalog (or file) each artifact so that you can easily track it down. As you collect specific information from each piece, make sure it is tagged with the appropriate label so that you can track it back down, as you very well may need to reference it later. This is also very important for honest and transparency in your work as a qualitative researcher – documenting a way to trace your findings back to the raw data .

In addition to staying organized, you also need to think specifically about what you are looking for in the artifacts. This might seem silly, but depending on the amount of data you are dealing with and how broad your research topic is, it might be hard to ‘separate the wheat from the chaff’ and figure out what is important or relevant information. Sometimes this is more clearly defined and we have a prescribed list of things we are looking for. This prescribed list may come from existing literature on the topic. This prescribed list may be based on peer-reviewed literature that is more conceptual, meaning that it focuses on defining concepts, putting together propositions, formulating early stage theories, and laying out professional wisdom, rather than reporting research findings. Drawing on this literature, we can then examine our data to see if there is evidence of these ideas and what this evidence tells us about these concepts. If this is the case, make sure you document this list somewhere, and on this list define each item and provide a code that you can attach when you see it in each document. This document then becomes your codebook .

However, if you aren’t clear ahead of time what this list might be, you may take an emergent approach, meaning that you have some general ideas of what you are seeking. In this event, you will actively create a codebook as you go, like the one described above, as you encounter these ideas in your artifacts. This helps you to gain a better understanding of what items should be included in your list, rather than coming in with preconceived notions about what they should be. There will be more about tracking this in our next chapter on qualitative analysis. Whether you have a prescribed list or use a more emergent design to develop your codebook, you will likely make modifications or corrections to it along the way as your knowledge evolves. When you make these changes, it is very important to have a way to document what changes you made, when, and why. Again, this helps to keep you honest, organized, and transparent. Just as another reminder, if you are using predetermined codes that you are looking for, this is reflective of a more deductive approach, whereas seeking emergent codes is more inductive .

Finally, when using artifacts, you may also need to bring in some creative, out-of-the-box thinking. You may be bringing together many different pieces of data that look and sound nothing alike, yet you are seeking information from them that will allow you tell a cohesive story. You may need to be fluid or flexible in how you are looking at things, and potentially challenge your preconceived notions.

Capturing the data

As alluded to above, you may have physical artifacts that you are dealing with, digital artifacts or representations of these artifacts (e.g. videos, photos, recordings), or even field notes about artifacts (for instance, if you take notes of a dramatic performance that can’t be recorded). A large part of what may drive your decisions about how to capture your data may be related to your level of access to those artifacts: can you look at it? Can you touch it, can you take it home with you, can you take a picture of it? Depending on what artifacts we are talking about, some of these may be important questions. Regardless of the answers to these questions, you will need to have a clearly articulated and well-documented plan for how you are obtaining the data and how you will reference it in the future.

What types of artifacts might you have access to that might help to answer your research question(s)?

• These could be artifacts available at your field placement, publicly available media, through school, or through public institutions
• These can be documents or they can be audiovisual materials
• Think outside the box, how can you gather direct or indirect indications of the thing you are studying

Generate a list of at least 3

Again, drawing on Creswell’s (2013) suggestion of capturing ‘descriptive’ and ‘reflective’ aspects in your field notes, Table 18.5 offers some more detailed description of what to include as your capture your data and corresponding examples when focusing on an artifact.

Resources to learn more about qualitative research with artifacts.

Bowen, G. A. (2009). Document analysis as a qualitative research method .

Rowsell, J. (2011). Carrying my family with me: Artifacts as emic perspectives .

Hammond, J., & McDermott, I. (n.d.). Policy document analysis .

Wang et al. (2017). Arts-based methods in socially engaged research practice: A classification framework .

A few exemplars of studies utilizing documents and other artifacts.

Casey, R. C. (2018). Hard time: A content analysis of incarcerated women’s personal accounts .

Green, K. R. (2018). Exploring the implications of shifting HIV prevention practice Ideologies on the Work of Community-Based Organizations: A Resource dependence perspective . 

Sousa, P., & Almeida, J. L. (2016). Culturally sensitive social work: promoting cultural competence .

Secondary data analysis

I wanted to briefly provide some special attention to secondary data analysis at the end of this chapter. In the past two chapters we have focused our sights most often on what we would call raw data sources . However, you can of course conduct qualitative research with secondary data , which is data that was collected previously for another research project or other purpose; data is not originating from your research process. If you are fortunate enough to have access and permission to use qualitative data that had already been collected, you can pose a new research question that may be answered by analyzing this data. This saves you the time and energy from having to collect the data yourself!

You might procure this data because you know the researcher that collected the original data. For instance, as a student, perhaps there is a faculty member that allows you access to data they had previously collected for another project. Alternatively, maybe you locate a source of qualitative data that is publicly available. Examples of this might include interviews previously conducted with Holocaust survivors. Finally, you might register and join a research data repository . These are sites where contributing researchers can house data that other researchers can view and request permission to use. Syracuse University hosts a repository that is explicitly dedicated to qualitative data . While there are more of these emerging, it may be a challenge to find the specific data you are looking for in a repository. You should also anticipate that data from repositories will have all identifiable information removed. Sharing data you have collected with a repository is a good way to extend the potential usefulness and impact of data, but it also should be anticipated before you collect your data so that you can build it into any informed consent so participants are made aware of the possibility.

Computer Assisted Qualitative Data Analysis Software (CAQDAS)

Some qualitative researchers use software packages known as Computer Assisted Qualitative Data Analysis Software (CAQDAS) in their work. These are tools that can aid researchers in managing, organizing and manipulating/analyzing their data. Some of the more common tools include NVivo, Atlas.ti, and MAXQDA, which have licensing fees attached to them (although many have discounted student rates). However, there are also some free options available if you do some hunting. Taguette Project is the only free and open source CAQDAS project that is currently receiving updates, as previous projects like RQDA which built from the R library are not in active development. Taguette is a young project, and unlike the free alternatives for quantitative data analysis, it lacks the sophisticated analytical tools of commercial CAQDAS programs.

It is unlikely that you will be using a CAQDAS for a student project, mostly because of the additional time investment it will take to become familiar with the software and associated costs (if applicable). In fact the best way to avoid spending money on qualitative data analysis software is to do your analysis by hand or using word processing or spreadsheet software. If you continue on with other qualitative research projects, it may be worth some additional study to learn more about CAQDAS tools. If you do choose to use one of these products, it won’t magically do the analysis for you. You need to be clear about what you are using the software for and how it supports your analysis plan, which will be the focus of our next chapter.

Resources to learn more about CAQDAS.

Maher et al. (2018). Ensuring rigor in qualitative data analysis: A design research approach to coding combining NVivo with traditional material methods .

Woods et al. (2016). Advancing qualitative research using qualitative data analysis software (QDAS)? Reviewing potential versus practice in published studies using ATLAS. ti and NVivo, 1994–2013 .

Zamawe, F. C. (2015). The implication of using NVivo software in qualitative data analysis: Evidence-based reflections .

As you continue to plan your research proposal, make sure to give practical thought to how you will go about collecting your qualitative data. Hopefully this chapter helped you to consider which methods are appropriate and what skills might be required to apply that particular method well. Revisit the table in section 18.3 that summarizes each of these approaches and some of the strengths and challenges associated with each of them. Collecting qualitative data can be a labor-intensive process, to be sure. However, I personally find it very rewarding. In its very forms, we are bearing witness to people’s stories and experiences.

Key Takeaways

• Artifact analysis can be particularly useful for qualitative research as a means of studying existing data; meaning we aren’t having to collect the data ourselves, but we do have to gather it. As a limitation, we don’t have any control over how the data was created, since we weren’t involved in it.
• There are many sources of existing data that we can consider for artifact analysis. Think of all the things around us that can help to tell some story! Artifact analysis may be especially appealing as a potential time saver for student researchers if you can gain permission to use existing artifacts or use artifacts that are publicly available.
• Artifact analysis still requires a systematic and premeditated approach to how you will go about extract information from your artifacts.

Reflexive Journal Entry Prompt

Here are a few questions to get you thinking about the role that you play as you gather qualitative data.

• What are your initial thoughts about qualitative data collection?
• Why might that be?
• What excites you about this process?
• What worries you about this process?
• What aspects of yourself will strengthen or enhance this process?
• What aspects of yourself may hinder or challenge this process?

Decision Point: How will you go about qualitative data collection?

• Justify your choice(s) here in relation to your research question and availability of resources at your disposal
• who will be collecting data
• what will be involved
• how will it be safely stored and organized
• how are you protecting human participants
• if you have a team, how is communication being established so everyone is “on the same page”
• how will you know you are done
• What additional information do you need to know to use this approach?
• Harris, M. and Fallot, R. (2001). Using trauma theory to design service systems. New Directions for Mental Health Service s. Jossey Bass; Farragher, B. and Yanosy, S. (2005). Creating a trauma-sensitive culture in residential treatment. Therapeutic Communities, 26 (1), 93-109. ↵

The analysis of documents (or other existing artifacts) as a source of data.

unprocessed data that researchers can analyze using quantitative and qualitative methods (e.g., responses to a survey or interview transcripts)

A code is a label that we place on segment of data that seems to represent the main idea of that segment.

A document that we use to keep track of and define the codes that we have identified (or are using) in our qualitative data analysis.

starts by reading existing theories, then testing hypotheses and revising or confirming the theory

when a researcher starts with a set of observations and then moves from particular experiences to a more general set of propositions about those experiences

analyzing data that has been collected by another person or research group

in a literature review, a source that describes primary data collected and analyzed by the author, rather than only reviewing what other researchers have found

Data someone else has collected that you have permission to use in your research.

These are sites where contributing researchers can house data that other researchers can view and request permission to use

These are software tools that can aid qualitative researchers in managing, organizing and manipulating/analyzing their data.

Doctoral Research Methods in Social Work Copyright © by Mavs Open Press. All Rights Reserved.

Share This Book

Principles of Documenting Data

In this lesson you will learn, initial questions, documentation overview.

When to Document Data

Levels of Documentation

Project level documentation, file level documentation, documenting your data, sample solution.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Med Res Methodol

A tutorial on methodological studies: the what, when, how and why

Lawrence mbuagbaw.

1 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada

2 Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph’s Healthcare—Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario L8N 4A6 Canada

3 Centre for the Development of Best Practices in Health, Yaoundé, Cameroon

Daeria O. Lawson

Livia puljak.

4 Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000 Zagreb, Croatia

David B. Allison

5 Department of Epidemiology and Biostatistics, School of Public Health – Bloomington, Indiana University, Bloomington, IN 47405 USA

Lehana Thabane

6 Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON Canada

7 Centre for Evaluation of Medicine, St. Joseph’s Healthcare-Hamilton, Hamilton, ON Canada

8 Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON Canada

Associated Data

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

The field of meta-research (or research-on-research) has proliferated in recent years in response to issues with research quality and conduct [ 1 – 3 ]. As the name suggests, this field targets issues with research design, conduct, analysis and reporting. Various types of research reports are often examined as the unit of analysis in these studies (e.g. abstracts, full manuscripts, trial registry entries). Like many other novel fields of research, meta-research has seen a proliferation of use before the development of reporting guidance. For example, this was the case with randomized trials for which risk of bias tools and reporting guidelines were only developed much later – after many trials had been published and noted to have limitations [ 4 , 5 ]; and for systematic reviews as well [ 6 – 8 ]. However, in the absence of formal guidance, studies that report on research differ substantially in how they are named, conducted and reported [ 9 , 10 ]. This creates challenges in identifying, summarizing and comparing them. In this tutorial paper, we will use the term methodological study to refer to any study that reports on the design, conduct, analysis or reporting of primary or secondary research-related reports (such as trial registry entries and conference abstracts).

In the past 10 years, there has been an increase in the use of terms related to methodological studies (based on records retrieved with a keyword search [in the title and abstract] for “methodological review” and “meta-epidemiological study” in PubMed up to December 2019), suggesting that these studies may be appearing more frequently in the literature. See Fig.  1 .

Trends in the number studies that mention “methodological review” or “meta-

epidemiological study” in PubMed.

The methods used in many methodological studies have been borrowed from systematic and scoping reviews. This practice has influenced the direction of the field, with many methodological studies including searches of electronic databases, screening of records, duplicate data extraction and assessments of risk of bias in the included studies. However, the research questions posed in methodological studies do not always require the approaches listed above, and guidance is needed on when and how to apply these methods to a methodological study. Even though methodological studies can be conducted on qualitative or mixed methods research, this paper focuses on and draws examples exclusively from quantitative research.

The objectives of this paper are to provide some insights on how to conduct methodological studies so that there is greater consistency between the research questions posed, and the design, analysis and reporting of findings. We provide multiple examples to illustrate concepts and a proposed framework for categorizing methodological studies in quantitative research.

What is a methodological study?

Any study that describes or analyzes methods (design, conduct, analysis or reporting) in published (or unpublished) literature is a methodological study. Consequently, the scope of methodological studies is quite extensive and includes, but is not limited to, topics as diverse as: research question formulation [ 11 ]; adherence to reporting guidelines [ 12 – 14 ] and consistency in reporting [ 15 ]; approaches to study analysis [ 16 ]; investigating the credibility of analyses [ 17 ]; and studies that synthesize these methodological studies [ 18 ]. While the nomenclature of methodological studies is not uniform, the intents and purposes of these studies remain fairly consistent – to describe or analyze methods in primary or secondary studies. As such, methodological studies may also be classified as a subtype of observational studies.

Parallel to this are experimental studies that compare different methods. Even though they play an important role in informing optimal research methods, experimental methodological studies are beyond the scope of this paper. Examples of such studies include the randomized trials by Buscemi et al., comparing single data extraction to double data extraction [ 19 ], and Carrasco-Labra et al., comparing approaches to presenting findings in Grading of Recommendations, Assessment, Development and Evaluations (GRADE) summary of findings tables [ 20 ]. In these studies, the unit of analysis is the person or groups of individuals applying the methods. We also direct readers to the Studies Within a Trial (SWAT) and Studies Within a Review (SWAR) programme operated through the Hub for Trials Methodology Research, for further reading as a potential useful resource for these types of experimental studies [ 21 ]. Lastly, this paper is not meant to inform the conduct of research using computational simulation and mathematical modeling for which some guidance already exists [ 22 ], or studies on the development of methods using consensus-based approaches.

When should we conduct a methodological study?

Methodological studies occupy a unique niche in health research that allows them to inform methodological advances. Methodological studies should also be conducted as pre-cursors to reporting guideline development, as they provide an opportunity to understand current practices, and help to identify the need for guidance and gaps in methodological or reporting quality. For example, the development of the popular Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) guidelines were preceded by methodological studies identifying poor reporting practices [ 23 , 24 ]. In these instances, after the reporting guidelines are published, methodological studies can also be used to monitor uptake of the guidelines.

These studies can also be conducted to inform the state of the art for design, analysis and reporting practices across different types of health research fields, with the aim of improving research practices, and preventing or reducing research waste. For example, Samaan et al. conducted a scoping review of adherence to different reporting guidelines in health care literature [ 18 ]. Methodological studies can also be used to determine the factors associated with reporting practices. For example, Abbade et al. investigated journal characteristics associated with the use of the Participants, Intervention, Comparison, Outcome, Timeframe (PICOT) format in framing research questions in trials of venous ulcer disease [ 11 ].

How often are methodological studies conducted?

There is no clear answer to this question. Based on a search of PubMed, the use of related terms (“methodological review” and “meta-epidemiological study”) – and therefore, the number of methodological studies – is on the rise. However, many other terms are used to describe methodological studies. There are also many studies that explore design, conduct, analysis or reporting of research reports, but that do not use any specific terms to describe or label their study design in terms of “methodology”. This diversity in nomenclature makes a census of methodological studies elusive. Appropriate terminology and key words for methodological studies are needed to facilitate improved accessibility for end-users.

Why do we conduct methodological studies?

Methodological studies provide information on the design, conduct, analysis or reporting of primary and secondary research and can be used to appraise quality, quantity, completeness, accuracy and consistency of health research. These issues can be explored in specific fields, journals, databases, geographical regions and time periods. For example, Areia et al. explored the quality of reporting of endoscopic diagnostic studies in gastroenterology [ 25 ]; Knol et al. investigated the reporting of p -values in baseline tables in randomized trial published in high impact journals [ 26 ]; Chen et al. describe adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement in Chinese Journals [ 27 ]; and Hopewell et al. describe the effect of editors’ implementation of CONSORT guidelines on reporting of abstracts over time [ 28 ]. Methodological studies provide useful information to researchers, clinicians, editors, publishers and users of health literature. As a result, these studies have been at the cornerstone of important methodological developments in the past two decades and have informed the development of many health research guidelines including the highly cited CONSORT statement [ 5 ].

Where can we find methodological studies?

Methodological studies can be found in most common biomedical bibliographic databases (e.g. Embase, MEDLINE, PubMed, Web of Science). However, the biggest caveat is that methodological studies are hard to identify in the literature due to the wide variety of names used and the lack of comprehensive databases dedicated to them. A handful can be found in the Cochrane Library as “Cochrane Methodology Reviews”, but these studies only cover methodological issues related to systematic reviews. Previous attempts to catalogue all empirical studies of methods used in reviews were abandoned 10 years ago [ 29 ]. In other databases, a variety of search terms may be applied with different levels of sensitivity and specificity.

Some frequently asked questions about methodological studies

In this section, we have outlined responses to questions that might help inform the conduct of methodological studies.

Q: How should I select research reports for my methodological study?

A: Selection of research reports for a methodological study depends on the research question and eligibility criteria. Once a clear research question is set and the nature of literature one desires to review is known, one can then begin the selection process. Selection may begin with a broad search, especially if the eligibility criteria are not apparent. For example, a methodological study of Cochrane Reviews of HIV would not require a complex search as all eligible studies can easily be retrieved from the Cochrane Library after checking a few boxes [ 30 ]. On the other hand, a methodological study of subgroup analyses in trials of gastrointestinal oncology would require a search to find such trials, and further screening to identify trials that conducted a subgroup analysis [ 31 ].

The strategies used for identifying participants in observational studies can apply here. One may use a systematic search to identify all eligible studies. If the number of eligible studies is unmanageable, a random sample of articles can be expected to provide comparable results if it is sufficiently large [ 32 ]. For example, Wilson et al. used a random sample of trials from the Cochrane Stroke Group’s Trial Register to investigate completeness of reporting [ 33 ]. It is possible that a simple random sample would lead to underrepresentation of units (i.e. research reports) that are smaller in number. This is relevant if the investigators wish to compare multiple groups but have too few units in one group. In this case a stratified sample would help to create equal groups. For example, in a methodological study comparing Cochrane and non-Cochrane reviews, Kahale et al. drew random samples from both groups [ 34 ]. Alternatively, systematic or purposeful sampling strategies can be used and we encourage researchers to justify their selected approaches based on the study objective.

Q: How many databases should I search?

A: The number of databases one should search would depend on the approach to sampling, which can include targeting the entire “population” of interest or a sample of that population. If you are interested in including the entire target population for your research question, or drawing a random or systematic sample from it, then a comprehensive and exhaustive search for relevant articles is required. In this case, we recommend using systematic approaches for searching electronic databases (i.e. at least 2 databases with a replicable and time stamped search strategy). The results of your search will constitute a sampling frame from which eligible studies can be drawn.

Alternatively, if your approach to sampling is purposeful, then we recommend targeting the database(s) or data sources (e.g. journals, registries) that include the information you need. For example, if you are conducting a methodological study of high impact journals in plastic surgery and they are all indexed in PubMed, you likely do not need to search any other databases. You may also have a comprehensive list of all journals of interest and can approach your search using the journal names in your database search (or by accessing the journal archives directly from the journal’s website). Even though one could also search journals’ web pages directly, using a database such as PubMed has multiple advantages, such as the use of filters, so the search can be narrowed down to a certain period, or study types of interest. Furthermore, individual journals’ web sites may have different search functionalities, which do not necessarily yield a consistent output.

Q: Should I publish a protocol for my methodological study?

A: A protocol is a description of intended research methods. Currently, only protocols for clinical trials require registration [ 35 ]. Protocols for systematic reviews are encouraged but no formal recommendation exists. The scientific community welcomes the publication of protocols because they help protect against selective outcome reporting, the use of post hoc methodologies to embellish results, and to help avoid duplication of efforts [ 36 ]. While the latter two risks exist in methodological research, the negative consequences may be substantially less than for clinical outcomes. In a sample of 31 methodological studies, 7 (22.6%) referenced a published protocol [ 9 ]. In the Cochrane Library, there are 15 protocols for methodological reviews (21 July 2020). This suggests that publishing protocols for methodological studies is not uncommon.

Authors can consider publishing their study protocol in a scholarly journal as a manuscript. Advantages of such publication include obtaining peer-review feedback about the planned study, and easy retrieval by searching databases such as PubMed. The disadvantages in trying to publish protocols includes delays associated with manuscript handling and peer review, as well as costs, as few journals publish study protocols, and those journals mostly charge article-processing fees [ 37 ]. Authors who would like to make their protocol publicly available without publishing it in scholarly journals, could deposit their study protocols in publicly available repositories, such as the Open Science Framework ( https://osf.io/ ).

Q: How to appraise the quality of a methodological study?

A: To date, there is no published tool for appraising the risk of bias in a methodological study, but in principle, a methodological study could be considered as a type of observational study. Therefore, during conduct or appraisal, care should be taken to avoid the biases common in observational studies [ 38 ]. These biases include selection bias, comparability of groups, and ascertainment of exposure or outcome. In other words, to generate a representative sample, a comprehensive reproducible search may be necessary to build a sampling frame. Additionally, random sampling may be necessary to ensure that all the included research reports have the same probability of being selected, and the screening and selection processes should be transparent and reproducible. To ensure that the groups compared are similar in all characteristics, matching, random sampling or stratified sampling can be used. Statistical adjustments for between-group differences can also be applied at the analysis stage. Finally, duplicate data extraction can reduce errors in assessment of exposures or outcomes.

Q: Should I justify a sample size?

A: In all instances where one is not using the target population (i.e. the group to which inferences from the research report are directed) [ 39 ], a sample size justification is good practice. The sample size justification may take the form of a description of what is expected to be achieved with the number of articles selected, or a formal sample size estimation that outlines the number of articles required to answer the research question with a certain precision and power. Sample size justifications in methodological studies are reasonable in the following instances:

• Comparing two groups
• Determining a proportion, mean or another quantifier
• Determining factors associated with an outcome using regression-based analyses

For example, El Dib et al. computed a sample size requirement for a methodological study of diagnostic strategies in randomized trials, based on a confidence interval approach [ 40 ].

Q: What should I call my study?

A: Other terms which have been used to describe/label methodological studies include “ methodological review ”, “methodological survey” , “meta-epidemiological study” , “systematic review” , “systematic survey”, “meta-research”, “research-on-research” and many others. We recommend that the study nomenclature be clear, unambiguous, informative and allow for appropriate indexing. Methodological study nomenclature that should be avoided includes “ systematic review” – as this will likely be confused with a systematic review of a clinical question. “ Systematic survey” may also lead to confusion about whether the survey was systematic (i.e. using a preplanned methodology) or a survey using “ systematic” sampling (i.e. a sampling approach using specific intervals to determine who is selected) [ 32 ]. Any of the above meanings of the words “ systematic” may be true for methodological studies and could be potentially misleading. “ Meta-epidemiological study” is ideal for indexing, but not very informative as it describes an entire field. The term “ review ” may point towards an appraisal or “review” of the design, conduct, analysis or reporting (or methodological components) of the targeted research reports, yet it has also been used to describe narrative reviews [ 41 , 42 ]. The term “ survey ” is also in line with the approaches used in many methodological studies [ 9 ], and would be indicative of the sampling procedures of this study design. However, in the absence of guidelines on nomenclature, the term “ methodological study ” is broad enough to capture most of the scenarios of such studies.

Q: Should I account for clustering in my methodological study?

A: Data from methodological studies are often clustered. For example, articles coming from a specific source may have different reporting standards (e.g. the Cochrane Library). Articles within the same journal may be similar due to editorial practices and policies, reporting requirements and endorsement of guidelines. There is emerging evidence that these are real concerns that should be accounted for in analyses [ 43 ]. Some cluster variables are described in the section: “ What variables are relevant to methodological studies?”

A variety of modelling approaches can be used to account for correlated data, including the use of marginal, fixed or mixed effects regression models with appropriate computation of standard errors [ 44 ]. For example, Kosa et al. used generalized estimation equations to account for correlation of articles within journals [ 15 ]. Not accounting for clustering could lead to incorrect p -values, unduly narrow confidence intervals, and biased estimates [ 45 ].

Q: Should I extract data in duplicate?

A: Yes. Duplicate data extraction takes more time but results in less errors [ 19 ]. Data extraction errors in turn affect the effect estimate [ 46 ], and therefore should be mitigated. Duplicate data extraction should be considered in the absence of other approaches to minimize extraction errors. However, much like systematic reviews, this area will likely see rapid new advances with machine learning and natural language processing technologies to support researchers with screening and data extraction [ 47 , 48 ]. However, experience plays an important role in the quality of extracted data and inexperienced extractors should be paired with experienced extractors [ 46 , 49 ].

Q: Should I assess the risk of bias of research reports included in my methodological study?

A : Risk of bias is most useful in determining the certainty that can be placed in the effect measure from a study. In methodological studies, risk of bias may not serve the purpose of determining the trustworthiness of results, as effect measures are often not the primary goal of methodological studies. Determining risk of bias in methodological studies is likely a practice borrowed from systematic review methodology, but whose intrinsic value is not obvious in methodological studies. When it is part of the research question, investigators often focus on one aspect of risk of bias. For example, Speich investigated how blinding was reported in surgical trials [ 50 ], and Abraha et al., investigated the application of intention-to-treat analyses in systematic reviews and trials [ 51 ].

Q: What variables are relevant to methodological studies?

A: There is empirical evidence that certain variables may inform the findings in a methodological study. We outline some of these and provide a brief overview below:

• Country: Countries and regions differ in their research cultures, and the resources available to conduct research. Therefore, it is reasonable to believe that there may be differences in methodological features across countries. Methodological studies have reported loco-regional differences in reporting quality [ 52 , 53 ]. This may also be related to challenges non-English speakers face in publishing papers in English.
• Authors’ expertise: The inclusion of authors with expertise in research methodology, biostatistics, and scientific writing is likely to influence the end-product. Oltean et al. found that among randomized trials in orthopaedic surgery, the use of analyses that accounted for clustering was more likely when specialists (e.g. statistician, epidemiologist or clinical trials methodologist) were included on the study team [ 54 ]. Fleming et al. found that including methodologists in the review team was associated with appropriate use of reporting guidelines [ 55 ].
• Source of funding and conflicts of interest: Some studies have found that funded studies report better [ 56 , 57 ], while others do not [ 53 , 58 ]. The presence of funding would indicate the availability of resources deployed to ensure optimal design, conduct, analysis and reporting. However, the source of funding may introduce conflicts of interest and warrant assessment. For example, Kaiser et al. investigated the effect of industry funding on obesity or nutrition randomized trials and found that reporting quality was similar [ 59 ]. Thomas et al. looked at reporting quality of long-term weight loss trials and found that industry funded studies were better [ 60 ]. Kan et al. examined the association between industry funding and “positive trials” (trials reporting a significant intervention effect) and found that industry funding was highly predictive of a positive trial [ 61 ]. This finding is similar to that of a recent Cochrane Methodology Review by Hansen et al. [ 62 ]
• Journal characteristics: Certain journals’ characteristics may influence the study design, analysis or reporting. Characteristics such as journal endorsement of guidelines [ 63 , 64 ], and Journal Impact Factor (JIF) have been shown to be associated with reporting [ 63 , 65 – 67 ].
• Study size (sample size/number of sites): Some studies have shown that reporting is better in larger studies [ 53 , 56 , 58 ].
• Year of publication: It is reasonable to assume that design, conduct, analysis and reporting of research will change over time. Many studies have demonstrated improvements in reporting over time or after the publication of reporting guidelines [ 68 , 69 ].
• Type of intervention: In a methodological study of reporting quality of weight loss intervention studies, Thabane et al. found that trials of pharmacologic interventions were reported better than trials of non-pharmacologic interventions [ 70 ].
• Interactions between variables: Complex interactions between the previously listed variables are possible. High income countries with more resources may be more likely to conduct larger studies and incorporate a variety of experts. Authors in certain countries may prefer certain journals, and journal endorsement of guidelines and editorial policies may change over time.

Q: Should I focus only on high impact journals?

A: Investigators may choose to investigate only high impact journals because they are more likely to influence practice and policy, or because they assume that methodological standards would be higher. However, the JIF may severely limit the scope of articles included and may skew the sample towards articles with positive findings. The generalizability and applicability of findings from a handful of journals must be examined carefully, especially since the JIF varies over time. Even among journals that are all “high impact”, variations exist in methodological standards.

Q: Can I conduct a methodological study of qualitative research?

A: Yes. Even though a lot of methodological research has been conducted in the quantitative research field, methodological studies of qualitative studies are feasible. Certain databases that catalogue qualitative research including the Cumulative Index to Nursing & Allied Health Literature (CINAHL) have defined subject headings that are specific to methodological research (e.g. “research methodology”). Alternatively, one could also conduct a qualitative methodological review; that is, use qualitative approaches to synthesize methodological issues in qualitative studies.

Q: What reporting guidelines should I use for my methodological study?

A: There is no guideline that covers the entire scope of methodological studies. One adaptation of the PRISMA guidelines has been published, which works well for studies that aim to use the entire target population of research reports [ 71 ]. However, it is not widely used (40 citations in 2 years as of 09 December 2019), and methodological studies that are designed as cross-sectional or before-after studies require a more fit-for purpose guideline. A more encompassing reporting guideline for a broad range of methodological studies is currently under development [ 72 ]. However, in the absence of formal guidance, the requirements for scientific reporting should be respected, and authors of methodological studies should focus on transparency and reproducibility.

Q: What are the potential threats to validity and how can I avoid them?

A: Methodological studies may be compromised by a lack of internal or external validity. The main threats to internal validity in methodological studies are selection and confounding bias. Investigators must ensure that the methods used to select articles does not make them differ systematically from the set of articles to which they would like to make inferences. For example, attempting to make extrapolations to all journals after analyzing high-impact journals would be misleading.

Many factors (confounders) may distort the association between the exposure and outcome if the included research reports differ with respect to these factors [ 73 ]. For example, when examining the association between source of funding and completeness of reporting, it may be necessary to account for journals that endorse the guidelines. Confounding bias can be addressed by restriction, matching and statistical adjustment [ 73 ]. Restriction appears to be the method of choice for many investigators who choose to include only high impact journals or articles in a specific field. For example, Knol et al. examined the reporting of p -values in baseline tables of high impact journals [ 26 ]. Matching is also sometimes used. In the methodological study of non-randomized interventional studies of elective ventral hernia repair, Parker et al. matched prospective studies with retrospective studies and compared reporting standards [ 74 ]. Some other methodological studies use statistical adjustments. For example, Zhang et al. used regression techniques to determine the factors associated with missing participant data in trials [ 16 ].

With regard to external validity, researchers interested in conducting methodological studies must consider how generalizable or applicable their findings are. This should tie in closely with the research question and should be explicit. For example. Findings from methodological studies on trials published in high impact cardiology journals cannot be assumed to be applicable to trials in other fields. However, investigators must ensure that their sample truly represents the target sample either by a) conducting a comprehensive and exhaustive search, or b) using an appropriate and justified, randomly selected sample of research reports.

Even applicability to high impact journals may vary based on the investigators’ definition, and over time. For example, for high impact journals in the field of general medicine, Bouwmeester et al. included the Annals of Internal Medicine (AIM), BMJ, the Journal of the American Medical Association (JAMA), Lancet, the New England Journal of Medicine (NEJM), and PLoS Medicine ( n  = 6) [ 75 ]. In contrast, the high impact journals selected in the methodological study by Schiller et al. were BMJ, JAMA, Lancet, and NEJM ( n  = 4) [ 76 ]. Another methodological study by Kosa et al. included AIM, BMJ, JAMA, Lancet and NEJM ( n  = 5). In the methodological study by Thabut et al., journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al. used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine “high impact” [ 77 ]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [ 78 ]. We acknowledge that the term “generalizability” may apply differently for methodological studies, especially when in many instances it is possible to include the entire target population in the sample studied.

Finally, methodological studies are not exempt from information bias which may stem from discrepancies in the included research reports [ 79 ], errors in data extraction, or inappropriate interpretation of the information extracted. Likewise, publication bias may also be a concern in methodological studies, but such concepts have not yet been explored.

A proposed framework

In order to inform discussions about methodological studies, the development of guidance for what should be reported, we have outlined some key features of methodological studies that can be used to classify them. For each of the categories outlined below, we provide an example. In our experience, the choice of approach to completing a methodological study can be informed by asking the following four questions:

• What is the aim?

A methodological study may be focused on exploring sources of bias in primary or secondary studies (meta-bias), or how bias is analyzed. We have taken care to distinguish bias (i.e. systematic deviations from the truth irrespective of the source) from reporting quality or completeness (i.e. not adhering to a specific reporting guideline or norm). An example of where this distinction would be important is in the case of a randomized trial with no blinding. This study (depending on the nature of the intervention) would be at risk of performance bias. However, if the authors report that their study was not blinded, they would have reported adequately. In fact, some methodological studies attempt to capture both “quality of conduct” and “quality of reporting”, such as Richie et al., who reported on the risk of bias in randomized trials of pharmacy practice interventions [ 80 ]. Babic et al. investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [ 81 ]. Further, biases related to choice of outcomes can also be explored. For example, Tan et al investigated differences in treatment effect size based on the outcome reported [ 82 ].

Methodological studies may report quality of reporting against a reporting checklist (i.e. adherence to guidelines) or against expected norms. For example, Croituro et al. report on the quality of reporting in systematic reviews published in dermatology journals based on their adherence to the PRISMA statement [ 83 ], and Khan et al. described the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals based on the CONSORT extension for harms [ 84 ]. Other methodological studies investigate reporting of certain features of interest that may not be part of formally published checklists or guidelines. For example, Mbuagbaw et al. described how often the implications for research are elaborated using the Evidence, Participants, Intervention, Comparison, Outcome, Timeframe (EPICOT) format [ 30 ].

Sometimes investigators may be interested in how consistent reports of the same research are, as it is expected that there should be consistency between: conference abstracts and published manuscripts; manuscript abstracts and manuscript main text; and trial registration and published manuscript. For example, Rosmarakis et al. investigated consistency between conference abstracts and full text manuscripts [ 85 ].

In addition to identifying issues with reporting in primary and secondary studies, authors of methodological studies may be interested in determining the factors that are associated with certain reporting practices. Many methodological studies incorporate this, albeit as a secondary outcome. For example, Farrokhyar et al. investigated the factors associated with reporting quality in randomized trials of coronary artery bypass grafting surgery [ 53 ].

Methodological studies may also be used to describe methods or compare methods, and the factors associated with methods. Muller et al. described the methods used for systematic reviews and meta-analyses of observational studies [ 86 ].

Some methodological studies synthesize results from other methodological studies. For example, Li et al. conducted a scoping review of methodological reviews that investigated consistency between full text and abstracts in primary biomedical research [ 87 ].

Some methodological studies may investigate the use of names and terms in health research. For example, Martinic et al. investigated the definitions of systematic reviews used in overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks [ 88 ].

In addition to the previously mentioned experimental methodological studies, there may exist other types of methodological studies not captured here.

• 2. What is the design?

Most methodological studies are purely descriptive and report their findings as counts (percent) and means (standard deviation) or medians (interquartile range). For example, Mbuagbaw et al. described the reporting of research recommendations in Cochrane HIV systematic reviews [ 30 ]. Gohari et al. described the quality of reporting of randomized trials in diabetes in Iran [ 12 ].

Some methodological studies are analytical wherein “analytical studies identify and quantify associations, test hypotheses, identify causes and determine whether an association exists between variables, such as between an exposure and a disease.” [ 89 ] In the case of methodological studies all these investigations are possible. For example, Kosa et al. investigated the association between agreement in primary outcome from trial registry to published manuscript and study covariates. They found that larger and more recent studies were more likely to have agreement [ 15 ]. Tricco et al. compared the conclusion statements from Cochrane and non-Cochrane systematic reviews with a meta-analysis of the primary outcome and found that non-Cochrane reviews were more likely to report positive findings. These results are a test of the null hypothesis that the proportions of Cochrane and non-Cochrane reviews that report positive results are equal [ 90 ].

• 3. What is the sampling strategy?

Methodological reviews with narrow research questions may be able to include the entire target population. For example, in the methodological study of Cochrane HIV systematic reviews, Mbuagbaw et al. included all of the available studies ( n  = 103) [ 30 ].

Many methodological studies use random samples of the target population [ 33 , 91 , 92 ]. Alternatively, purposeful sampling may be used, limiting the sample to a subset of research-related reports published within a certain time period, or in journals with a certain ranking or on a topic. Systematic sampling can also be used when random sampling may be challenging to implement.

• 4. What is the unit of analysis?

Many methodological studies use a research report (e.g. full manuscript of study, abstract portion of the study) as the unit of analysis, and inferences can be made at the study-level. However, both published and unpublished research-related reports can be studied. These may include articles, conference abstracts, registry entries etc.

Some methodological studies report on items which may occur more than once per article. For example, Paquette et al. report on subgroup analyses in Cochrane reviews of atrial fibrillation in which 17 systematic reviews planned 56 subgroup analyses [ 93 ].

This framework is outlined in Fig.  2 .

A proposed framework for methodological studies

Conclusions

Methodological studies have examined different aspects of reporting such as quality, completeness, consistency and adherence to reporting guidelines. As such, many of the methodological study examples cited in this tutorial are related to reporting. However, as an evolving field, the scope of research questions that can be addressed by methodological studies is expected to increase.

In this paper we have outlined the scope and purpose of methodological studies, along with examples of instances in which various approaches have been used. In the absence of formal guidance on the design, conduct, analysis and reporting of methodological studies, we have provided some advice to help make methodological studies consistent. This advice is grounded in good contemporary scientific practice. Generally, the research question should tie in with the sampling approach and planned analysis. We have also highlighted the variables that may inform findings from methodological studies. Lastly, we have provided suggestions for ways in which authors can categorize their methodological studies to inform their design and analysis.

Acknowledgements

Abbreviations, authors’ contributions.

LM conceived the idea and drafted the outline and paper. DOL and LT commented on the idea and draft outline. LM, LP and DOL performed literature searches and data extraction. All authors (LM, DOL, LT, LP, DBA) reviewed several draft versions of the manuscript and approved the final manuscript.

This work did not receive any dedicated funding.

Availability of data and materials

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Documentation in Reports and Research Papers

• An Introduction to Punctuation
• Ph.D., Rhetoric and English, University of Georgia
• M.A., Modern English and American Literature, University of Leicester
• B.A., English, State University of New York

In a report or  research paper , documentation is the evidence  provided for information and ideas borrowed from others. That evidence includes both primary sources  and secondary sources .

There are numerous documentation styles and formats, including MLA style (used for research in the humanities), APA style (psychology, sociology, education), Chicago style (history), and ACS style (chemistry).

Examples and Observations

• Adrienne Escoe "Documentation has many meanings, from the broad—anything written in any medium—to the narrow—policies and procedures manuals or perhaps records." ( T he Practical Guide to People-Friendly Documentation , 2nd. ed. ASQ Quality Press, 2001)
• Kristin R. Woolever "An issue more important than documentation form is knowing when to document. In brief, anything that is copied needs to be documented... "Perhaps the best tip for knowing when to document is to use common sense. If writers are careful to give credit where it is due and to provide the reader with easy access to all the source material, the text is probably documented appropriately." ( About Writing: A Rhetoric for Advanced Writers . Wadsworth, 1991)

Note-Taking and Documentation During the Research Process

• Linda Smoak Schwartz "The most important thing to remember when you take notes from your sources is that you must clearly distinguish between quoted, paraphrased , and summarized material that must be documented in your paper and ideas that do not require documentation because they are considered general knowledge about that subject." ( The Wadsworth Guide to MLA Documentation , 2nd ed. Wadsworth, 2011)

Library Resources Versus Internet Resources

• Susan K. Miller-Cochran and Rochelle L. Rodrigo "When you are reviewing and analyzing your resources, keep in mind that the library/Internet distinction is not quite as simple as it might seem at first. The Internet is where students often turn when they are having difficulty getting started. Many instructors warn students against using Internet resources because they are easily alterable and because anyone can construct and publish a Web site. These points are important to remember, but it is essential to use clear evaluative criteria when you are looking at any resource. Print resources can be self-published as well. Analyzing how easily a resource is changed, how often it is changed, who changed it, who reviews it, and who is responsible for the content will help you choose resources that are reliable and credible, wherever you might find them." ( The Wadsworth Guide to Research, Documentation , rev. ed. Wadsworth, 2011)

Parenthetical Documentation

• Joseph F. Trimmer "You may decide to vary the pattern of documentation by presenting the information from a source and placing the author's name and page number in parentheses at the end of the sentence. This method is particularly useful if you have already established the identity of your source in a previous sentence and now want to develop the author's idea in some detail without having to clutter your sentences with constant references to his or her name.​" ( A Guide to MLA Documentation , 9th ed. Wadsworth, 2012)
• Meaning of Tense Shift in Verbs
• What Is a Research Paper?
• What Is a Primary Source?
• Secondary Sources in Research
• An Introduction to Academic Writing
• Primary and Secondary Sources in History
• How to Develop a Research Paper Timeline
• What Is a Citation?
• How to Write a Research Paper That Earns an A
• Bibliography: Definition and Examples
• What Is a Style Guide and Which One Do You Need?
• How to Use Libraries and Archives for Research
• Finding Sources for Death Penalty Research
• Writing a Paper about an Environmental Issue
• Top 10 Genealogy Mistakes to Avoid
• How to Cite Genealogy Sources

Document Analysis

• First Online: 02 January 2023

Cite this chapter

• Benjamin Kutsyuruba 4  

Part of the book series: Springer Texts in Education ((SPTE))

4584 Accesses

1 Citations

This chapter describes the document analysis approach. As a qualitative method, document analysis entails a systematic procedure for reviewing and evaluating documents through finding, selecting, appraising (making sense of), and synthesizing data contained within them. This chapter outlines the brief history, method and use of document analysis, provides an outline of its process, strengths and limitations, and application, and offers further readings, resources, and suggestions for student engagement activities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Altheide, D. L. (1987). Ethnographic content analysis. Qualitative Sociology, 10 (1), 65–77.

Article   Google Scholar  

Altheide, D. L. (1996). Qualitative media analysis . SAGE.

Google Scholar  

Altheide, D. L. (2000). Tracking discourse and qualitative document analysis. Poetics, 27 , 287–299.

Atkinson, P. A., & Coffey, A. (1997). Analysing documentary realities. In D. Silverman (Ed.), Qualitative research: Theory, method and practice (pp. 45–62). SAGE.

Berg, B. L. (2001). Qualitative research methods for social sciences . Allyn and Bacon.

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9 (2), 27–40. https://doi.org/10.3316/qrj0902027

Bryman, A. (2003). Research methods and organization studies . Routledge.

Book   Google Scholar  

Cardno, C. (2018). Policy document analysis: A practical educational leadership tool and a qualitative research method. Educational Administration: Theory and Practice , 24 (4), 623–640. https://doi.org/10.14527/kuey.2018.016

Caulley, D. N. (1983). Document analysis in program evaluation. Evaluation and Program Planning, 6 , 19–29.

Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). SAGE.

Derrida, J. (1978). Writing and difference . Routledge & Kegan Paul.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research . Aldine De Gruyter.

Glesne, C., & Peshkin, A. (1992). Becoming qualitative researchers (2nd ed.). Longman.

Goode, W. J., & Hatt, P. K. (1952). Methods in social research . McGraw-Hill.

Hodder, I. (2000). The interpretation of documents and material culture. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 703–715). SAGE.

Krippendorff, K. (1980). Content analysis: An introduction to its methodology. SAGE.

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research, 28 , 587–604.

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2010). Practical resources for assessing and reporting intercoder reliability in content analysis research projects . Retrieved March 20, 2011, from http://matthewlombard.com/reliability/index_print.html

Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative social research (Vol. 1(2)). Retrieved March 22, 2011, from http://www.qualitative-research.net/index.php/fqs/article/viewArticle/1089/2385

McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry (7th ed.). Pearson.

Merriam, S. B. (1988a). Case study research in education: A qualitative approach . Jossey-Bass.

Merriam, S. B. (1998b). Case study research in education . Jossey-Bass.

Miller, F. A., & Alvarado, K. (2005). Incorporating documents into qualitative nursing research. Journal of Nursing Scholarship, 37 (4), 348–353.

Neuendorf, K. A. (2002). The content analysis guidebook . SAGE.

O’Leary, Z. (2014). The essential guide to doing your research project (2nd ed.). SAGE.

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). SAGE.

Prior, L. (2003). Using documents in social research . SAGE.

Prior, L. (2008a). Document analysis. In L. Given (Ed.), The SAGE encyclopaedia of qualitative research methods (pp. 231–232). SAGE. https://doi.org/10.4135/9781412963909

Prior, L. (2008b). Repositioning documents in social research. Sociology, 42 (5), 821–836. https://doi.org/10.1177/0038038508094564

Prior, L. (2012). The role of documents in social research. In S. Delamont (Ed.), Handbook of qualitative research in education (pp. 426–438). Edward Elgar.

Salminen, A., Kauppinen, K., & Lehtovaara, M. (1997). Towards a methodology for document analysis. Journal of the American Society for Information Science, 48 (7), 644–655.

Stake, R. E. (1995). The art of case study research . SAGE.

Wharton, C. (2006). Document analysis. In V. Jupp (Ed.), The SAGE dictionary of social research methods (pp. 80–81). SAGE. https://doi.org/10.4135/9780857020116

Yin, R. K. (2009). Case study research, design and methods (4th ed.). SAGE.

Additional Reading

Kutsyuruba, B. (2017). Examining education reforms through document analysis methodology. In I. Silova, A. Korzh, S. Kovalchuk, & N. Sobe (Eds.), Reimagining Utopias: Theory and method for educational research in post-socialist contexts (pp. 199–214). Sense.

Kutsyuruba, B., Christou, T., Heggie, L., Murray, J., & Deluca, C. (2015). Teacher collaborative inquiry in Ontario: An analysis of provincial and school board policies and support documents. Canadian Journal of Educational Administration and Policy, 172 , 1–38.

Kutsyuruba, B., Godden, L., & Tregunna, L. (2014). Curbing the early-career attrition: A pan-Canadian document analysis of teacher induction and mentorship programs. Canadian Journal of Educational Administration and Policy, 161 , 1–42.

Segeren, A., & Kutsyuruba, B. (2012). Twenty years and counting: An examination of the development of equity and inclusive education policy in Ontario (1990–2010). Canadian Journal of Educational Administration and Policy, 136 , 1–38.

Online Resources

Document Analysis: A How To Guide (12:27 min) https://www.youtube.com/watch?v=vOsE9saR_ck

Document Analysis with Philip Adu (1:16:40 min) https://youtu.be/bLKBffW5JPU

Download references

Author information

Authors and affiliations.

Queen’s University, Kingston, Canada

Benjamin Kutsyuruba

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Benjamin Kutsyuruba .

Editor information

Editors and affiliations.

Department of Educational Administration, College of Education, University of Saskatchewan, Saskatoon, SK, Canada

Janet Mola Okoko

Scott Tunison

Department of Educational Administration, University of Saskatchewan, Saskatoon, SK, Canada

Keith D. Walker

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Kutsyuruba, B. (2023). Document Analysis. In: Okoko, J.M., Tunison, S., Walker, K.D. (eds) Varieties of Qualitative Research Methods. Springer Texts in Education. Springer, Cham. https://doi.org/10.1007/978-3-031-04394-9_23

Download citation

DOI : https://doi.org/10.1007/978-3-031-04394-9_23

Published : 02 January 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-04396-3

Online ISBN : 978-3-031-04394-9

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• View source

• Recent Changes
• Random Page
• What links here
• Related changes
• Special pages
• Printable version
• Permanent link
• Page information

Research Documentation

When publishing research , it is important to make documentation available so that readers can understand the details of the research design that the work reports. This includes all of the technical details and decisions that could influence how the findings are read or understood. Usually, this will involve producing a document along the lines of a methodological note or appendix. That document will describe how a given study was designed and how the design was carried out. The level of detail is in such a document should be relatively high. This page will describe some common approaches to compiling this kind of material and retaining the needed information in an organized fashion throughout the life of a research project.

• Research documentation provides the context to understanding the results of a given research output.
• There is no standard form for this documentation, and its location and format will depend on the type of research output produced.
• For academic materials, this documentation often takes the form of a structured methodological appendix.
• For policy outputs or online products, it may be appropriate to include an informative README webpage or document.
• The most important process for preparing this documentation will be retaining and organizing the needed information throughout the life of the project, so that the team will not have to search through communications or data archives for small details at publication time.

What to include in research documentation

Research documentation should include all the information that is needed to understand the underlying design for the research output. This can include descriptions of:

• Populations of interest that informed the study
• Methods of sampling or other sources of data about selecting the units of observation that were actually included in the study
• Power calculations and pre-analysis plans
• Field work, including data collection or experimental manipulation, such as study protocols and monitoring or quality assurance information
• Data collection tools such as survey instruments, search keywords, and instructions or code for API requests or database queries
• Statistical approaches such as definitions of key constructed indicators, corrections or adjustments to data, and precise definitions of estimators and estimation procedures
• Data completeness, including non-observed units or quantities that were planned or "tracking" information

All of the research documentation taken together should broadly allow a reader to understand how information was gathered, what it represents, what kind of information and data files to expect, and how to relate that information to the results of the research. Research documentation is not a complete guide to data, however; it does not need to provide the level of detail or instructions that would enable a reader to approach different research questions using the same data.

Documentation will take different forms depending on the information included. Much of it will be written narrative rather than, for example, formal datasets . Understanding research documentation should not require the user to have any special software or to undertake any analytical tasks themselves. Relevant datasets (such as tracking of units of observation over time) might be included alongside the documentation, but the documentation should summarize in narrative form all the information from that dataset that is likely to affect the interpretation of the research.

Structuring research documentation as a publication appendix

If you are preparing documentation to accompany the publication of an academic output such as a working paper or journal article, the most common form of research documentation is a structured supplemental appendix. Check the journal's publication process for details. Some publishers allow unlimited supplementary materials to be included in a format such as an author-created document. These materials may or may not be included under the peer review of the main manuscript and might only be intended to provide context for readers and reviewers. In this case you should provide complete information in that material. Other publishers expect all supplementary materials to be read and reviewed as part of the publication process. In this case you should provide the minimum additional detail required to understand the research here (since much of the appendix will likely be taken up by supplementary results rather than documentation), and consider other methods for releasing complete documentation, such as self-publication on OSF or Zenodo.

Since there is unlimited space and you may have a large amount of material to include in a documentation appendix, organization is essential. It is appropriate to have several appendices that cover different aspects of the research. For example, Appendix A may include information about the study population and data, such as the total number of units available for observation , the number selected or included for observation, the number successfully included, and descriptive statistics about subgroups, strata, clusters, or other units relevant to the research. It could be accompanied by a tracking dataset with full information about the process. Appendix B might include information about an intended experimental manipulation in one section, and information about implementation, take-up, and fidelity in a second section. It could be accompanied by a dataset with key indicators. Appendix C might include data collection protocols and definitions of constructed variables and comparisons with alternative definitions, and be accompanied by data collection instruments and illustrative figures. Each appendix should included relevant references. Supplementary exhibits should be numbered to correspond with the appendix they pertain to. More granular appendices are generally preferable so that referencing and numbering remains relatively uncomplicated.

There have been many attempts to standardized some of these elements, such as the STROBE and CONSORT reporting checklists . Journals will let you know if they expect these exact templates to be followed. Even if they are not required, such templates can still be used directly or to provide inspiration or structure for the materials you might want to include.

• Documentary Research: Definition, Types, Applications & Examples

Introduction

Over the years, social scientists have used documentary research to understand series of events that have occurred or happened in the past. Here, they explore available recovered or existing documents and material to get information and gain insight into a research question or particular topic.

In this article, we would define the concept of documentary research, the various types of documentary research, its applications, and some valid examples.

Let’s dive right in.

What is Documentary Research?

In simple terms, documentary research is a form of research that uses records to get accurate information about a particular subject. It is a systematic investigation and analysis of existing records or documents. These documents can be in written forms, visual/audio materials, photographs, videos books, etc.

Documentary research is a valuable approach used in exploring historical events, cultural phenomena, and societal trends to get deep insight into a topic, subject or research question.

Documentary research is somewhat similar to content analysis, which also entails studying existing information/documents.

One of the most vital considerations when using documentary research is the quality of the material being utilized, hence the danger of falling into the single-story phenomenon. 

To forestall this, the documents being reviewed must be assessed thoroughly before it is used. (see John Scott, A Matter of Record, 1990). The criteria for authenticity involves checking the documents thoroughly to ensure their genuineness.

List of Documentary Research Methods

• Social Research Studies: This form of documentary research is commonly used in social research studies. For instance, Karl Max used documentary research extensively for his research and the documents he used include The Royal Commission, Inland revenue reports, and Her Majesty Inspectors of Factory reports, to mention a few. Emile Durkheim one of the founders of sociology authored a book on suicide and his work was recognized as the first modern example of consistent use of documents for social research.
• Archival Inquiry: This is a field of sociology explored in documentary research. It entails using primary source documents stored in archives. This form of research is popular amongst historians and the archival documents are referred to as references in their research.
• Content Analysis: This method involves the examination and interpretation of content in documents like articles, books, and speeches in other to find a connection, verify events, and identify patterns or trends.
• Historical Analysis: This is the study and analysis of occurrences that took place in the past, but were documented in records like newspapers, government records, and diaries to understand past events accurately and use the information to understand the present.
• Textual Analysis: This form of analysis is focused on printed texts, in a bid to understand pictures, symbols, and language in other to understand events or occurrences that happened in the lives of the subject.
• Oral Tradition : Oral history involves gathering information via oral summations of people who had direct experience of the events or subject being researched. These interviews are recorded and transcribed, and then analyzed as documents.
• Ethnographic Research: This form of research involves documenting the daily experiences of people in their natural environment, in other to understand how interactions in their personal space affect or impacts their experiences.
• Comparative Analysis: Comparative analysis entails comparing documents from multiple sources to understand context, and periods and uncover any similarities or differences. The goal is to understand cultural or political variations.
• Cross-Sectional Analysis: Cross-sectional analysis involves reviewing documents from multiple perspectives to understand changes, trends, or developments over a specific period.
• Aesthetic Interpretation: This is analyzing visual documents, like paintings, photographs, and footage from videos. This is often used as a supplement to text to authenticate discoveries uncovered in text documents.

Understanding the Documentary Research Methodology

Documentary research involves several key steps, such as defining the objective or research question, identifying relevant resources, revising them, and drawing up a well-informed and accurate conclusion based on fact.

Here are some key points to help you understand the documentary research methodology:

• Purpose: The essence of documentary research is to review existing documents to have insight into a research problem or question. The documents reviewed include written texts, such as books, articles, letters, diaries, newspapers, official reports, government publications, and archival materials, and non-written materials like videos, audio recordings,  photographs, and digital documents.
• Data Collection: This phase is when researchers gather relevant documents required for the research topic. These documents are evaluated carefully based on credibility and relevance. 

Explore – Data Collection Methods: Definition + Steps to Do It

• Data Analysis: Here, the gathered documents are analyzed systematically using relevant document research methodologies. This involves reading, grouping similar resources, and extracting information based on similarities, trends, etc.
• Interpretation: After data analysis, the discoveries are interpreted and the answers are applied to the research question or objective.

Read More: What is Data Interpretation? + [Types, Methods & Tools]

• Ethical Considerations: Ethical principles should be considered when carrying out documentary research. Copyright and intellectual property rights should be respected and all necessary permissions should be obtained before using confidential materials.
• Strengths and Limitations: The documentary research methodology has several advantages. One of which is that it helps researchers study past events by providing relevant documentation that sheds light. It also offers rich and detailed insights into social, cultural, and historical contexts. However, as with every good thing, there are limitations, such as some form of biases in the selected documents, which could emanate from the author or source of the document, missing data, and validity of the findings.

Related: What are Ethical Practices in Market Research?

Applications of Documentary Research

The documentary research methodology has a broad range of applications across various disciplines. They include:

• Historical Research : Documentary research is used extensively in historical studies to explore past events, in other to predict the future. Researchers review historical documents, like letters, diaries, government records, newspapers, and photographs, to better understand historical narratives, social and cultural contexts, and see how individuals or communities conducted their activities in the past.
• Social Sciences: In social sciences, documentary research helps investigate social concepts and trends. Documents like surveys, census data, and organization records are studied and analyzed, in other to understand public opinion, social inequality, and organizational behavior.
• Legal Research: Documentary research plays a vital role in legal studies. Lawyers, legal scholars, policymakers, etc analyze legal documents, regulations, court cases, and legal antecedents all in a bid to understand the legal framework and ways in which law evolves. Documentary research can support legal arguments, influence the development of legal theories, and inform policy-making.
• Education Research: Documentary research is used to understand educational policies, curriculum development, and teaching practices. Researchers review educational documents, such as textbooks,  educational policies, and assessment materials, to access educational systems, approaches, and the effect of these on learning outcomes.

Examples of Documentary Research

• The Russian Revolution (1891 – 1924), With the aid of newspaper documents and personal diaries Orlando Figes, a British historian narrated the most important milestones of the revolution in that period and proffered a comprehensive portrait of everyday occurrences as it occurred then the book Figes. depicts how the Russian Revolution was a historical process that changed the lives of its people and had its influence globally.
• The Vietnam War . The 990 minutes audiovisual documentary by Ken Burns narrates the Vietnam War (1955-1975). Throughout 10 episodes, the military operations of the Vietnam War were addressed, as well as the opposition to the war by the US.
• Bios . Lives that marked yours: Luis Alberto Spinetta. This two-hour audiovisual documentary, produced by National Geographic, intimate and deeply details the life of  Luis Alberto Spineta, an artist referred to as one of the fathers of Argentine rock. His family was part of the production,the100 hours documentary was directed by Catarina Spinetta and she used recordings, and testimonies from family members to review her father’s childhood until his final moments.
• The Secret Decrees of the Dictatorship . This publication was released between March and May 2019, and more than 7000 secret decrees issued by the Military Juntas in Argentina between 1976-1983 were reviewed by the Data Unit of the news portal. These decrees signed by different dictators focused on deportations, the prohibition of books, and the sale of weapons. All of these materials were analyzed and presented along with eight notes, published in 2019.
• World War II in Photographs, David Boyle . This book is an example of aesthetic documentary research. 900 high-quality photographs from various sources were used to portray World War II (1939–1945). The images uncover the scenarios as the warfare took place. The images were arranged in chronological order with images of the steppes of Russia, the deserts of Africa, the jungles of the South Pacific, and the seas of the Arctic and each one of them has a detailed explanation of the course of events.
• The Silence of the Others . This documentary by the Spanish Pedro Almodóvar took 7 years to produce and over 450 hours of review of materials to uncover the crimes carried out during the Franco regime and the plight of the victims seeking justice. 
• The Berlin Wall . The border through a city, Thomas Flemming. This is another example of documentary research, with documents, photos, and illustrations, this book illustrates the history of the Berlin Wall. The daily life of the people who lived to the west and east of the city was portrayed in the book as well as the events that led to the fall of the border in 1989.

Purpose of Documentary Research

The purpose of documentary research is to gather verifiable evidence, that can help researchers understand clearly events that occurred in the past/present and also uncover new knowledge by analyzing existing documents and materials. It aids researchers in exploring topics that are difficult to decipher through other research methods and proffers a historical or contextual perspective on the subject being studied.

When to Use Documentary Research

Documentary research is best when researching events that occurred in the past, especially in instances where direct observation is not applicable. Here are some instances where documentary research is particularly useful:

• Historical Studies: Documentary research is ideal when conducting historical research. Researchers can then analyze historical records or documents left behind to better understand past events, chronologically.
• Exploratory Research: In cases where there are gaps in research studies. documentary research can serve as an exploratory method to fill gaps in knowledge by exploring different perspectives that can uncover new knowledge.
• Policy Analysis : Documentary research is useful in examining policies and similar regulations. By analyzing policy documents, over a period, researchers can measure the impact policies had or have on a particular subject. Based on their review of existing documents, they can make recommendations and supervise their implementation. This method is particularly useful in fields such as public policy, education, healthcare, and social welfare.
• Comparative Studies: Documentary research is useful for comparative analysis. Researchers can analyze documents from different sources and geographical locations to identify patterns, verify results or simply identify contradictions and uncover areas that require further investigation.

Characteristics of Documentary Research

• Uses Existing Documents: Documentary research is based on existing documents as a primary source of data. These documents can be written(letters, diaries, articles, books)or unwritten documents(videos, photographs, inscriptions). These documents are analyzed to gain insight and understanding into a specific phenomenon. 
• Non-Experimental In Nature: Documentary research does not involve manipulated variables, meaning that the researcher can not change the outcomes by directly intervening in the research. All the results derived are based on phenomena that have d occurred, which have documented records to attest to their occurrence. 
• Data Analysis: Documentary research involves rigorous data analysis, as researchers have to carefully read, extract relevant information, categorize data, and use qualitative/quantitative analysis to derive results.
• Interpretation of Findings: After data analysis. The findings of the research must be interpreted in a way that gives insight and deep understanding to anyone reading about the subject being researched. The interpretation phase involves synthesizing and relating the findings to the research questions or objectives.
• Contextual Understanding : Documentary research emphasizes the importance of understanding the social, cultural, and historical, events in the context, in which the documents were recorded, reviewed, and analyzed.By context we mean, the period, cultural norms, political climate, socio-economic factors, etc where the events being studied took place and under what circumstances. This contextual understanding helps to interpret the findings and draw accurate conclusions.
• Cross Reference and Validation: Documentary research is characterized by cross-referencing or triangulation, which involves using multiple sources or methods to corroborate findings. The combination of documentary research with other research methods strengthens the validity and reliability of their findings. This enhances the robustness of the research and helps minimize potential biases or inaccuracies.
• Ethical Considerations: Documentary research requires that researchers respect ethical guidelines and principles. Copyright and intellectual property laws must be adhered to and necessary permissions obtained when using sensitive or confidential documents, as well as the privacy and anonymity of individuals mentioned in the documents. 

Advantages of Documentary Research

• Access to Existing Data: In documentary research, existing data is readily available for review and analysis. There is no need to collect new data, via surveys and the like which can take time or require intensive resources. This makes documentary research a cost-effective and efficient method.
• Rich and Dynamic Data: Documents and materials used in documentary research offer a rich pool of information and insights. This method covers a wide range of topics, periods, and perspectives. There is access to primary sources, such as original letters or historical documents, as well as secondary sources like scholarly articles or reports. This variety of data allows for a comprehensive and clear understanding of the research topic.
• Longitudinal and Historical Perspectives: Documentary research allows researchers to study phenomena over extended periods and explore historical contexts. By examining documents spanning different periods, researchers can analyze patterns, trends, changes, and continuity across social, cultural, or organizational aspects. 
• Non-Intrusive Method : Since documentary research relies on existing documents, there is no direct involvement with research subjects or settings. Hence there is no need to disturb or manipulate the research environment or intrude on the lives of individuals. This makes it an ethical and practical method, especially for sensitive or personal topics.
• Exploration of Inaccessible or Historical Data: Documentary research allows researchers to access data that cannot be duplicated anymore due to timelapse and changing circumstances. For instance, researchers can analyze archived documents, historical records, or rare texts which provide unique insights into the past or specific contexts. 
• Large-Scale Data Analysis : Documentary research deals with or involves large volumes of data. Numerous documents, texts, or media materials to identify patterns, themes, or trends can be examined. This exposure to extensive data sets enables comprehensive analysis and enhances the reliability of research findings.

Limitations of Documentary Research

• The Danger of Biased Perspectives: The documents used in documentary research are subject to bias, as they could reflect the perspectives, agendas, or limitations of the authors or organizations that produced them. Critical evaluation is necessary to ensure the credibility of the documents.
• No Control Over Data Collection : Documentary research relies on existing data that may not have been aimed at the research question it is being applied to. As researchers have limited control over the collection process, there is the potential for missing or incomplete information.
• Subjective Interpretation: Documents analyzed require interpretation of findings, which can be subjective as different researchers can interpret the same document differently, leading to variations in findings and conclusions.

Documentary research is a valuable form of research methodology as it provides access to existing documents and materials for analysis and interpretation. There are many advantages of these methods, such as diverse sources of data, historical perspectives, and access to large volumes of data from analysis.

However, there are also limitations like biases based on the author’s perspective, no control over data collection, and challenges in interpretation. A clear understanding of the pros and cons of this research method would help users make informed decisions on how to apply documentary research to their subject of study.

Connect to Formplus, Get Started Now - It's Free!

• cross-sectional study
• data analysis
• data collection methods
• data interpretation
• documentary research
• Angela Kayode-Sanni

You may also like:

Statistical Analysis Software: A Guide For Social Researchers

Introduction Social research is a complex endeavor. It takes a lot of time, energy, and resources to gather data, analyze and present...

What is Deductive Research? Meaning, Stages & Examples

Introduction Deductive research is a scientific approach that is used to test a theory or hypothesis through observations and empirical...

Statistical Analysis Plan: What is it & How to Write One

Statistics give meaning to data collected during research and make it simple to extract actionable insights from the data. As a result,...

Desk Research: Definition, Types, Application, Pros & Cons

If you are looking for a way to conduct a research study while optimizing your resources, desk research is a great option. Desk research...

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

• Privacy Policy

Home » Documentary Analysis – Methods, Applications and Examples

Documentary Analysis – Methods, Applications and Examples

Table of Contents

Documentary Analysis

Definition:

Documentary analysis, also referred to as document analysis , is a systematic procedure for reviewing or evaluating documents. This method involves a detailed review of the documents to extract themes or patterns relevant to the research topic .

Documents used in this type of analysis can include a wide variety of materials such as text (words) and images that have been recorded without a researcher’s intervention. The domain of document analysis, therefore, includes all kinds of texts – books, newspapers, letters, study reports, diaries, and more, as well as images like maps, photographs, and films.

Documentary analysis provides valuable insight and a unique perspective on the past, contextualizing the present and providing a baseline for future studies. It is also an essential tool in case studies and when direct observation or participant observation is not possible.

The process usually involves several steps:

• Sourcing : This involves identifying the document or source, its origin, and the context in which it was created.
• Contextualizing : This involves understanding the social, economic, political, and cultural circumstances during the time the document was created.
• Interrogating : This involves asking a series of questions to help understand the document better. For example, who is the author? What is the purpose of the document? Who is the intended audience?
• Making inferences : This involves understanding what the document says (either directly or indirectly) about the topic under study.
• Checking for reliability and validity : Just like other research methods, documentary analysis also involves checking for the validity and reliability of the documents being analyzed.

Documentary Analysis Methods

Documentary analysis as a qualitative research method involves a systematic process. Here are the main steps you would generally follow:

Defining the Research Question

Before you start any research , you need a clear and focused research question . This will guide your decision on what documents you need to analyze and what you’re looking for within them.

Selecting the Documents

Once you know what you’re looking for, you can start to select the relevant documents. These can be a wide range of materials – books, newspapers, letters, official reports, diaries, transcripts of speeches, archival materials, websites, social media posts, and more. They can be primary sources (directly from the time/place/person you are studying) or secondary sources (analyses created by others).

Reading and Interpreting the Documents

You need to closely read the selected documents to identify the themes and patterns that relate to your research question. This might involve content analysis (looking at what is explicitly stated) and discourse analysis (looking at what is implicitly stated or implied). You need to understand the context in which the document was created, the author’s purpose, and the audience’s perspective.

Coding and Categorizing the Data

After the initial reading, the data (text) can be broken down into smaller parts or “codes.” These codes can then be categorized based on their similarities and differences. This process of coding helps in organizing the data and identifying patterns or themes.

Analyzing the Data

Once the data is organized, it can be analyzed to make sense of it. This can involve comparing the data with existing theories, examining relationships between categories, or explaining the data in relation to the research question.

Validating the Findings

The researcher needs to ensure that the findings are accurate and credible. This might involve triangulating the data (comparing it with other sources or types of data), considering alternative explanations, or seeking feedback from others.

Reporting the Findings

The final step is to report the findings in a clear, structured way. This should include a description of the methods used, the findings, and the researcher’s interpretations and conclusions.

Applications of Documentary Analysis

Documentary analysis is widely used across a variety of fields and disciplines due to its flexible and comprehensive nature. Here are some specific applications:

Historical Research

Documentary analysis is a fundamental method in historical research. Historians use documents to reconstruct past events, understand historical contexts, and interpret the motivations and actions of historical figures. Documents analyzed may include personal letters, diaries, official records, newspaper articles, photographs, and more.

Social Science Research

Sociologists, anthropologists, and political scientists use documentary analysis to understand social phenomena, cultural practices, political events, and more. This might involve analyzing government policies, organizational records, media reports, social media posts, and other documents.

Legal Research

In law, documentary analysis is used in case analysis and statutory interpretation. Legal practitioners and scholars analyze court decisions, statutes, regulations, and other legal documents.

Business and Market Research

Companies often analyze documents to gather business intelligence, understand market trends, and make strategic decisions. This might involve analyzing competitor reports, industry news, market research studies, and more.

Media and Communication Studies

Scholars in these fields might analyze media content (e.g., news reports, advertisements, social media posts) to understand media narratives, public opinion, and communication practices.

Literary and Film Studies

In these fields, the “documents” might be novels, poems, films, or scripts. Scholars analyze these texts to interpret their meaning, understand their cultural context, and critique their form and content.

Educational Research

Educational researchers may analyze curricula, textbooks, lesson plans, and other educational documents to understand educational practices and policies.

Health Research

Health researchers may analyze medical records, health policies, clinical guidelines, and other documents to study health behaviors, healthcare delivery, and health outcomes.

Examples of Documentary Analysis

Some Examples of Documentary Analysis might be:

• Example 1 : A historian studying the causes of World War I might analyze diplomatic correspondence, government records, newspaper articles, and personal diaries from the period leading up to the war.
• Example 2 : A policy analyst trying to understand the impact of a new public health policy might analyze the policy document itself, as well as related government reports, statements from public health officials, and news media coverage of the policy.
• Example 3 : A market researcher studying consumer trends might analyze social media posts, customer reviews, industry reports, and news articles related to the market they’re studying.
• Example 4 : An education researcher might analyze curriculum documents, textbooks, and lesson plans to understand how a particular subject is being taught in schools. They might also analyze policy documents to understand the broader educational policy context.
• Example 5 : A criminologist studying hate crimes might analyze police reports, court records, news reports, and social media posts to understand patterns in hate crimes, as well as societal and institutional responses to them.
• Example 6 : A journalist writing a feature article on homelessness might analyze government reports on homelessness, policy documents related to housing and social services, news articles on homelessness, and social media posts from people experiencing homelessness.
• Example 7 : A literary critic studying a particular author might analyze their novels, letters, interviews, and reviews of their work to gain insight into their themes, writing style, influences, and reception.

When to use Documentary Analysis

Documentary analysis can be used in a variety of research contexts, including but not limited to:

• When direct access to research subjects is limited : If you are unable to conduct interviews or observations due to geographical, logistical, or ethical constraints, documentary analysis can provide an alternative source of data.
• When studying the past : Documents can provide a valuable window into historical events, cultures, and perspectives. This is particularly useful when the people involved in these events are no longer available for interviews or when physical evidence is lacking.
• When corroborating other sources of data : If you have collected data through interviews, surveys, or observations, analyzing documents can provide additional evidence to support or challenge your findings. This process of triangulation can enhance the validity of your research.
• When seeking to understand the context : Documents can provide background information that helps situate your research within a broader social, cultural, historical, or institutional context. This can be important for interpreting your other data and for making your research relevant to a wider audience.
• When the documents are the focus of the research : In some cases, the documents themselves might be the subject of your research. For example, you might be studying how a particular topic is represented in the media, how an author’s work has evolved over time, or how a government policy was developed.
• When resources are limited : Compared to methods like experiments or large-scale surveys, documentary analysis can often be conducted with relatively limited resources. It can be a particularly useful method for students, independent researchers, and others who are working with tight budgets.
• When providing an audit trail for future researchers : Documents provide a record of events, decisions, or conditions at specific points in time. They can serve as an audit trail for future researchers who want to understand the circumstances surrounding a particular event or period.

Purpose of Documentary Analysis

The purpose of documentary analysis in research can be multifold. Here are some key reasons why a researcher might choose to use this method:

• Understanding Context : Documents can provide rich contextual information about the period, environment, or culture under investigation. This can be especially useful for historical research, where the context is often key to understanding the events or trends being studied.
• Direct Source of Data : Documents can serve as primary sources of data. For instance, a letter from a historical figure can give unique insights into their thoughts, feelings, and motivations. A company’s annual report can offer firsthand information about its performance and strategy.
• Corroboration and Verification : Documentary analysis can be used to validate and cross-verify findings derived from other research methods. For example, if interviews suggest a particular outcome, relevant documents can be reviewed to confirm the accuracy of this finding.
• Substituting for Other Methods : When access to the field or subjects is not possible due to various constraints (geographical, logistical, or ethical), documentary analysis can serve as an alternative to methods like observation or interviews.
• Unobtrusive Method : Unlike some other research methods, documentary analysis doesn’t require interaction with subjects, and therefore doesn’t risk altering the behavior of those subjects.
• Longitudinal Analysis : Documents can be used to study change over time. For example, a researcher might analyze census data from multiple decades to study demographic changes.
• Providing Rich, Qualitative Data : Documents often provide qualitative data that can help researchers understand complex issues in depth. For example, a policy document might reveal not just the details of the policy, but also the underlying beliefs and attitudes that shaped it.

Advantages of Documentary Analysis

Documentary analysis offers several advantages as a research method:

• Unobtrusive : As a non-reactive method, documentary analysis does not require direct interaction with human subjects, which means that the research doesn’t affect or influence the subjects’ behavior.
• Rich Historical and Contextual Data : Documents can provide a wealth of historical and contextual information. They allow researchers to examine events and perspectives from the past, even from periods long before modern research methods were established.
• Efficiency and Accessibility : Many documents are readily accessible, especially with the proliferation of digital archives and databases. This accessibility can often make documentary analysis a more efficient method than others that require data collection from human subjects.
• Cost-Effective : Compared to other methods, documentary analysis can be relatively inexpensive. It generally requires fewer resources than conducting experiments, surveys, or fieldwork.
• Permanent Record : Documents provide a permanent record that can be reviewed multiple times. This allows for repeated analysis and verification of the data.
• Versatility : A wide variety of documents can be analyzed, from historical texts to contemporary digital content, providing flexibility and applicability to a broad range of research questions and fields.
• Ability to Cross-Verify (Triangulate) Data : Documentary analysis can be used alongside other methods as a means of triangulating data, thus adding validity and reliability to the research.

Limitations of Documentary Analysis

While documentary analysis offers several benefits as a research method, it also has its limitations. It’s important to keep these in mind when deciding to use documentary analysis and when interpreting your findings:

• Authenticity : Not all documents are genuine, and sometimes it can be challenging to verify the authenticity of a document, particularly for historical research.
• Bias and Subjectivity : All documents are products of their time and their authors. They may reflect personal, cultural, political, or institutional biases, and these biases can affect the information they contain and how it is presented.
• Incomplete or Missing Information : Documents may not provide all the information you need for your research. There may be gaps in the record, or crucial information may have been omitted, intentionally or unintentionally.
• Access and Availability : Not all documents are readily available for analysis. Some may be restricted due to privacy, confidentiality, or security considerations. Others may be difficult to locate or access, particularly historical documents that haven’t been digitized.
• Interpretation : Interpreting documents, particularly historical ones, can be challenging. You need to understand the context in which the document was created, including the social, cultural, political, and personal factors that might have influenced its content.
• Time-Consuming : While documentary analysis can be cost-effective, it can also be time-consuming, especially if you have a large number of documents to analyze or if the documents are lengthy or complex.
• Lack of Control Over Data : Unlike methods where the researcher collects the data themselves (e.g., through experiments or surveys), with documentary analysis, you have no control over what data is available. You are reliant on what others have chosen to record and preserve.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Cluster Analysis – Types, Methods and Examples

Discriminant Analysis – Methods, Types and...

MANOVA (Multivariate Analysis of Variance) –...

ANOVA (Analysis of variance) – Formulas, Types...

Graphical Methods – Types, Examples and Guide

Substantive Framework – Types, Methods and...

• Get in touch
• Enterprise & IT
• Banking & Financial Services
• News media & Entertainment
• Healthcare & Lifesciences
• Networks and Smart Devices
• Education & EdTech
• Service Design
• UI UX Design
• Data Visualization & Design
• User & Design Research
• In the News
• Our Network
• Voice Experiences
• Golden grid
• Critical Thinking
• Enterprise UX
• 20 Product performance metrics
• Types of Dashboards
• Interconnectivity and iOT
• Healthcare and Lifesciences
• Airtel XStream
• Case studies

Data Design

• UCD vs. Design Thinking

User & Design Research

Document research.

Document research method refers to the analysis of documents that contains information about the scenario or event under consideration. It is used to investigate, categorize and analyze physical sources, most commonly written documents, in the social, public or digital world. This research method is just as good as and sometimes even more cost effective than the surveys, in-depth interviews or other observation based methods such as ethnography.

Quick details: Document Research

Structure: Structured

Preparation: Documents to be researched

Deliverables: Research notes, Documentation, Insights

More about Document Research

A document is defined as written text. Documents can be files, statistical data, records of official or unofficial nature providing an account of an event, images, other written material that can be accessed in a social, public or digital context. For example, institutional memoranda and reports, census publications, government pronouncements and proceedings, diaries and other written, visual and pictorial sources in different forms and so on are socially, publically or digitally accessible either openly or on request.

Document research is not a standalone method; it is usually used in conjunction with other design methods. Document research is employed when the researcher has questions to which they seek answers. It is not as helpful in an open inquiry to determine patterns as the number of documents that would need to be investigated can be huge, turning the researcher’s task into a never-ending exercise. Therefore, This research method is often used only as a supplement to the other research methods.

Along with surveys and ethnography, document research is one of the three major types of social research and arguably has been the most widely used of the three, to study needs, behavior and expectations of user groups. The analysis of the documents in document research is either quantitative or qualitative analysis (or both). The key issues surrounding types of documents and our ability to use them as reliable sources of evidence on the user groups must be considered by all who use documents in their research.

Advantages of Document Research

1. availability.

Document research method uses documents that are public or can be accessed on request if private.

2. Time & Cost effective

As the phenomenon being investigated is clearly defined before the method is exercised, the research is focused and closed. This saves a lot of time for the researcher and also costs that would have been incurred if an expert was consulted for the study.

3. Unbiased Collection Process

As the collection process doesn’t involve direct interaction of the researcher with the user groups or author of the documents, the chances of introducing bias stays low. Again, if the document is of the statistical record type, then the data being collected is based on facts that can be verified and cross-checked for errors.

4. Researcher Presence

The researcher is not required to be present at the time of data collection.

Disadvantages of Document Research

1. limited by available data.

As the data or documents that are available on the phenomenon being investigated as the primary resources for the study, the findings will be based on only the data that is documented on the subject.

2. Errors in written material

If there are errors in the documents being referred, these errors will render erroneous findings as well .

3. Out of context

If the documents studied are out of context, they will not contribute meaning to the study or will not lead to an insightful findings.

4. Preparation before analysis

The preparation required before document analysis is performed is a task in itself. The efforts are usually directed toward recruiting the researchers, identifying sources, shortlisting the material to be researched and analyzed, among others.

Think Design's recommendation

More often than not, any design project starts with some kind of Document Research or the other. It is predominantly a secondary research method; however, the researcher/ designer is using the documented material for his own understanding of the context. 

Do not use document research as a stand-alone method and do not proceed with your design assuming that the documents you studied have provided you with all the answers. This method compliments other methods and is usually a good starting point.

Was this Page helpful?

Related methods.

• Audio/ Video Analysis
• Heatmap Analysis
• Social Network Mapping
• Time Lapse Video
• Trend Analysis
• Usage Analytics

UI UX DESIGN

Service design.

We use cookies to ensure that we give you the best experience on our website. If you continue we'll assume that you accept this. Learn more

Recent Tweets

Sign up for our newsletter.

Subscribe to our newsletter to stay updated with the latest insights in UX, CX, Data and Research.

Get in Touch

Thank you for subscribing.

You will be receive all future issues of our newsletter.

Thank you for Downloading.

One moment….

While the report downloads, could you tell us…

•   SMU Libraries
• Scholarship & Research
• Teaching & Learning
• Bridwell Library
• Business Library
• DeGolyer Library
• Fondren Library
• Hamon Arts Library
• Underwood Law Library
• Fort Burgwin Library
• Exhibits & Digital Collections
• SMU Scholar
• Special Collections & Archives
• Connect With Us
• Research Guides by Subject
• How Do I . . . ? Guides
• Find Your Librarian
• Writing Support

Types of Research Papers: Overview

• Types of Research Questions

A research paper is simply a piece of writing that uses outside sources. There are different types of research papers with varying purposes and expectations for sourcing. While this guide explains those differences broadly, disciplines and assignments vary. Ask your professor for clarification on the purpose,  types of appropriate research questions , and expectations of sources for your assignment.

Need More Help?

Related guides.

• Literature Reviews
• Annotated Bibliographies
• Starting Your Research

Research and Writing Lab

Need last minute help but didn't book an appointment? Every week we offer online drop-in labs.

Tuesdays 3:00pm - 4:30pm via Zoom @   https://smu.zoom.us/j/92637892352  and in-person, Fondren Red 1st floor (near elevators)

• Next: Types of Research Questions >>
• Last Updated: Apr 24, 2024 12:40 PM
• URL: https://guides.smu.edu/researchpapertypes

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Study Protocol

Comfort food concepts and contexts in which they are used: A scoping review protocol

Contributed equally to this work with: Juliana Miranda Pereira, Rute Guedes Melo

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft

* E-mail: [email protected]

Affiliation Department of Psycholobiology, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil

Roles Data curation, Investigation, Methodology, Validation, Visualization, Writing – original draft

Current address: Universidade Federal do Rio Grande do Norte, Centro de Biociências, Campus Universitário, Natal, RN, Brasil

Roles Methodology, Validation, Writing – review & editing

Current address: Universidade Federal do Rio Grande do Norte, Faculdade de Ciências da Saúde do Trairi, Natal, Brazil

¶ ‡ These authors also contributed equally to this work.

Affiliation Faculdade de Ciências da Saúde, Federal University of Rio Grande do Norte, Santa Cruz, Rio Grande do Norte, Brazil

Roles Supervision, Validation, Writing – review & editing

Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

¶ Membership of the Federal University of Rio Grande do Norte is provided in the Acknowledgments.

• Juliana Miranda Pereira, 
• Rute Guedes Melo, 
• Joyanne de Souza Medeiros, 
• Anna Cecília Queiroz de Medeiros, 
• Fívia de Araújo Lopes, 
• on behalf of Federal University of Rio Grande do Norte

• Published: April 24, 2024
• https://doi.org/10.1371/journal.pone.0299991
• Reader Comments

The objectives of this study are to clarify the scientific definition of comfort food, identify which methodologies are being used in research on this topic and which factors are associated with the consumption of comfort food.

Introduction

The consumption of comfort foods is subjective and influenced by individual experiences, as they are known and appreciated by the person. However, divergences about the definition of comfort food in the scientific literature reflect the heterogeneity of the methods used in the research, and consequently identification of possible factors associated with the consumption of this type of food, which can influence the knowledge about the consumption of these foods and their potential effects on the health of those who consume them.

Inclusion criteria

Works with a qualitative and quantitative approach published in full in indexed sources or in gray literature, available online in the databases consulted, without restriction on language or year of publication will be included.

The protocol was built based on the methodological recommendations of the Joanna Briggs Institute (JBI) for scoping reviews and the recommendations of the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). The Participants, Concept and Context (PCC) mnemonic strategy was built: general population, comfort food concept and world context. Based on this, search strategies were developed for different databases. Instruments were also developed for recording documents, extracting data, justifying the exclusion of documents and not obtaining access to content. A Pilot Study was conducted to test the developed methodology and instruments. The protocol has been registered with the Open Science Framework (OSF) ( https://osf.io/gnza4/ ). The results will be presented in the review resulting from this protocol in three ways: accounting of the documents will be recorded in a PRISMA Flow Diagram, the main information of the studies and their frequencies will be presented in a table, and the union of these outcomes will be presented visually in a Graphical Abstract.

Citation: Pereira JM, Guedes Melo R, de Souza Medeiros J, Queiroz de Medeiros AC, de Araújo Lopes F, on behalf of Federal University of Rio Grande do Norte (2024) Comfort food concepts and contexts in which they are used: A scoping review protocol. PLoS ONE 19(4): e0299991. https://doi.org/10.1371/journal.pone.0299991

Editor: André Ramalho, FMUP: Universidade do Porto Faculdade de Medicina, PORTUGAL

Received: November 2, 2023; Accepted: February 19, 2024; Published: April 24, 2024

Copyright: © 2024 Pereira et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The final table containing the pilot data extraction can be accessed by a link: https://osf.io/sxm85 .

Funding: This work was carried out with support from the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brazil (CAPES) - Funding Code 001. FAL is a researcher on productivity at National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico [CNPq]) - Level 2 (Number 313018/2021-9).

Competing interests: The authors have declared that no competing interests exist.

The food history of our ancestors within the context of scarcity, climate change, need for reproduction, as well as our own and our offspring’s survival, shaped the nutritional needs of modern humans. As the environment changed due to climate change, our ancestors adapted their food consumption [ 1 , 2 ]. One adaptation resulting from the evolutionary process is the strong taste preference and appreciation we have for fat and sugar, because their dietary sources such as ripe fruits and meats were scarce in the past [ 3 ].

The behaviors of searching for food (craving) and food consumption are reinforced by reward pathways in the brain as they are determinant components of human survival, which, integrated with nutritional status and sensory information, constitute the rewarding properties and affective value of food, culminating in appetite and individual eating behavior, including the pleasurable response of appreciation (liking) [ 4 ]; in addition, personality and cognitive characteristics can influence physiological changes induced by food and influence food-related memory. The already lived sensory experiences of a food contribute to the emotional weight of food situations and are related to memories. However, it is the emotions related to the situations, such as those experienced after the meal and the emotional context of the meal situation—for example, a celebration—that trigger spontaneous memories [ 5 , 6 ].

According to Locher et al. [ 7 ], the definition of comfort food was added in 1997 to two renowned dictionaries: the Collegiate Dictionary, which defined it as: “food prepared in a traditional style with a generally nostalgic or sentimental appeal”; and the Oxford dictionary, which defined the term as: “food that comforts or provides consolation; therefore, any food (often high in sugar or carbohydrate) that is associated with childhood or home cooking” [ 7 ]. The definition currently available on the Oxford Dictionary website is “foods that make you feel better, generally because they contain too much sugar or because they remind you of home” [ 8 ]. Furthermore, the term comfort food has often been used in common sense by popular media, such as food and cooking TV shows and magazines, to represent a high calorie food set [ 9 ].

Unlike what currently appears to be in public knowledge, consensus on the definition of comfort food in academic literature is difficult [ 9 ]. Two widely referenced studies in this research area were developed in 2003. Dallman et al. [ 10 ] define these foods as “palatable foods, whose sensory qualities indicate calories”. Wansink, Cheney and Chan [ 11 ] state that comfort foods are “foods that promote physiological and psychological comfort when consumed”, diverging from the previous definition.

Despite the divergence, more research was carried out on the subject after these definitions, but without defining which concept is used to guide the study development. However, it is common to use or question the consumption of hyperpalatable foods as comfort foods in their research methods, such as sugary drinks, snacks, fast foods and sweets [ 12 – 17 ]. On the other hand, some studies present a definition, such as Locher et al. [ 7 ] who defined it as “food that makes you feel good or promotes comfort”. Still, there are studies which have asked the participants to present their own definition of comfort food [ 9 ], reflecting the heterogeneity of the concept in the methods employed, and consequently in identification of possible factors associated with the consumption of this type of food, which are still not well elucidated in the scientific literature and can be influenced by the concept and methodology employed.

Reviews on the subject were developed [ 18 , 19 ] and one of them conceptualized comfort food as “those foods whose consumption provides comfort or a feeling of well-being” [ 19 ]. However, these studies did not follow search methodologies, selection or systematic analysis of scientific evidence that guarantee the rigor and coherence of the information.

It is known that comfort foods are foods already known and appreciated by the individual [ 11 ], characterizing the consumption of these foods as something subjective and influenced by individual experiences. Therefore, any food that is appreciated by the individual and that has a related affective memory can be a comfort food [ 7 ]. Likewise, “palatable calorie-reporting” foods may not be considered comfort foods for some people. In addition, it is important to consider that palatable foods are not only consumed in the context of seeking “physiological and psychological comfort”.

Therefore, categorizing comfort foods as palatable foods that suggest calories, especially ultra-processed ones, can limit knowledge about the consumption of these foods and their potential effects on the health of those who consume them. Considering the complexity of the subject, carrying out a scoping review to clarify this concept, as well as mapping the methodologies used and the associated factors is important to inform researchers about the applicability of the term, support decision-making for subsequent studies, and consequently promote progress in this knowledge area in an improved way.

A previous search was performed in JBI Evidence Synthesis, Figshare, Open Science Framework (OSF), PubMed and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases and no published scoping or systematic review on the topic or currently ongoing was identified. Therefore, objectives of this study are to clarify the scientific definition of comfort food considering how it is being understood, in what context and how it is being used; and identify which methodologies are being used in research on this topic and the factors associated with the consumption of comfort food based on available scientific documents.

The methodology proposed by the Joanna Briggs Institute (JBI) [ 20 ] for scoping reviews and the recommendations of the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA- ScR) [ 21 ] were followed in elaborating this scoping review protocol. The protocol has been registered with the Open Science Framework (OSF) ( https://osf.io/gnza4/ ). Also, this article was written based on the Prisma-P Checklist (2015) guidelines and is available on the OSF page for this protocol ( https://osf.io/jy876 ). Given the dynamism inherent in the research process, it is possible that the methodology will undergo adjustments. Therefore, if they occur, such changes will be reported and detailed in the final scoping review in order to ensure that the decisions taken were based on solid scientific criteria, transparent and consistent with the research objectives.

Review questions

The study questions were formulated using the Participants, Concept and Context (PCC) mnemonic strategy. In this protocol, participants consist of the general population, including all age groups and social characteristics; the concept to be explored is that of comfort food; and the context is global. Thus, the guiding question of this scoping review is: How is comfort food defined and how is its concept used in research in the areas of food, nutrition and eating behavior? And the complementary questions: What methodologies are used in studies on comfort food? What are the factors associated with the consumption of comfort food?

Inclusion and exclusion criteria

Works with a qualitative and quantitative approach published in full in indexed sources or in gray literature, available online in the consulted databases, without restriction on language or year of publication will be included. Research reports, editorials, letters to the editor and abstracts published in conference proceedings, reports or case series, communications and duplicate documents will be excluded. The databases to be consulted will be: Medline/PubMed; Embase; Cochrane Library Databanks; Lilacs; Cumulative Index to Nursing and Allied Health Literature (CINAHL); Scopus; Scielo; Web of science; Science direct; Directory of Open Access Journals (DOAJ); Directory of Open Access Books (DOAB). The following sites will be accessed to search the gray literature: CAPES Catalog of Theses and Dissertations; DART-Europe E-theses Portal; Electronic Theses Online Service (EThOS); Academic Archive Online (DIVA); Brazilian Digital Library of Theses and Dissertations (BDTD); Cyberthesis; Google Scholar. The complete list of databases is available in Table 1 . Access to the databases was through the Journal Portal of the Coordination for the Improvement of Higher Education Personnel (Capes) in the period of August 2023, but for each database there will be a specific search strategy and these will be available as complementary information to the final review.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0299991.t001

Search strategy

An initial search was performed in January 2023 in the PUBMED database using the term “comfort food”. Articles that presented the term were selected, and then the title, abstract and keywords were analyzed to identify used index terms which could contribute to the construction of the search strategy for the review.

Search strategies were developed by a professional librarian and were peer-reviewed. Controlled vocabularies in health sciences were used to retrieve relevant studies in national and international multidisciplinary databases and in the health area, namely: Medical Subject Headings (MeSH), Health Sciences Descriptors (DeCS) and Emtree (Embase), with the purpose of identifying descriptors to broaden the search results. In addition, keywords were used. Some of the descriptors and keywords to be used include: “Comfort food”; “Comfort eating”; “Food preferences”; and “Emotional eating”, as detailed in Table 2 .

https://doi.org/10.1371/journal.pone.0299991.t002

It is noteworthy that as greater familiarization with the databases occurs, more keywords, sources and other descriptors can be discovered and incorporated into the search strategy taking into account the interactive nature of this study. Therefore, it is reiterated that if changes occur, these will be incorporated and reported in the review. In addition, reference lists of materials included in the review will be consulted in order to identify additional sources.

Data extraction

Searches in the databases will follow an order of each database at a time, and will be carried out by one of the reviewers in an anonymous browser. Duplicates will be excluded from the results. Then, all remaining publications from the search will be extracted and transferred to Rayyan, a free and intuitive application available via website and mobile, which aims to assist in screening eligible studies for systematic reviews [ 22 ]. The initial screening will occur independently between two reviewers by reading the title and abstract of the material, keeping those that contain information relevant to the research objectives. The selection process will follow the recommendations contained in the PRISMA-ScR checklist and will have an expected deadline of early November 2023. All extracted articles will be accounted for in Table 3 . If the articles only refer to comfort food but a concept has not been attributed by the authors or a reference is made to a concept by other authors, they will be included.

https://doi.org/10.1371/journal.pone.0299991.t003

The second screening begins with a complete reading of the selected articles from which only data related to the PCC will be extracted, such as the methodology used, the concept of comfort food, in which context it was used and which factors are associated with the consumption of comfort food. These data will be recorded in Table 4 , constructed by the work team based on the recommendations of Peters et al. [ 23 ]. Considering that there may be disagreements at some article selection stage, these will be resolved in discussion with a third reviewer who will make the decision. The agreement level between the reviewers and the facilitator will be calculated and reported. In addition, how the disagreement was resolved will also be informed in the final review report.

https://doi.org/10.1371/journal.pone.0299991.t004

It is shown that the articles not included in the review will be registered in the exclusion justification table ( Table 5 ). The authors of works not available in full will be contacted in order to gain access to the content. If this is not available, the title of the study and the reason for not obtaining the data will be recorded in a table ( Table 6 ).

https://doi.org/10.1371/journal.pone.0299991.t005

https://doi.org/10.1371/journal.pone.0299991.t006

A Pilot Study was conducted following the steps mentioned above in order to assess the adequacy of the search strategy, methodology and instruments to be used in the review. A search was performed on PubMed (using the search strategy available in Table 2 ) and the first 50 resulting articles were selected. The three reviewers separately read the titles and abstracts and selected the articles. After that, there was a meeting to share the selection results, analyze divergences and align decisions. Then, the reviewers separately filled in the data extraction table with the information contained in the selected articles. At the end, improvements were discussed and put in the table, divergences were resolved and the way of tabulating the extracted data was aligned.

From the 50 articles extracted, the title and abstract were analyzed, considering the recommended inclusion and exclusion criteria. Of these, three met the criteria. Then, the full reading documents phase began. After that, the data extraction table ( Table 4 ) was complete. During this process, some improvement needs were identified, such as: adding the "type of study" and reorganizing some columns to make data identification and filling faster: the positions of "age group" and "sample number" were reversed (this one now comes first) and "group" with biological sex (this one comes after). The other columns remained as in the original table. The pilot study results are available in Table 7 and on the OSF page for this protocol ( https://osf.io/sxm85 ).

https://doi.org/10.1371/journal.pone.0299991.t007

Analysis and presentation of the results

The article accounting will be registered in the PRISMA Flow Diagram, which presents the different review stages involving the mapping of the records found, articles which were included and excluded, as well as the reason for the exclusions [ 24 ].

The main information of the studies, such as concept, application, associated factors and methodology used and their frequencies will be presented in a table. In addition, a Graphical Abstract will be created to unite and express these outcomes visually, enabling quick and easy understanding of the facts. In this context, such creation will take place through the “Mind The Graph” website, which consists of a tool with design resources and a wide collection of scientific illustrations [ 25 ]. All demonstrations of results will be accompanied by a narrative summary that will describe how the results and/or graphs relate to the questions developed and the objectives of the review. Finally, the underlying data for the results will be presented in a table, which will serve as supplementary material.

Supporting information

S1 file. checklist prisma for systematic review protocols..

https://doi.org/10.1371/journal.pone.0299991.s001

• View Article
• Google Scholar
• 2. Lieberman D. Chapter 3: Muita coisa depende do jantar. In: Lieberman D. A história do corpo humano: evolução, saúde e doença. 1st ed. São Paulo: Editora Zahar; 2015.
• PubMed/NCBI
• 5. Gibson EL. Mood, emotions and food choice. In: Richard S & Raats M, editors. The psychology of food choice. Wallingford: CABI; 2006. pp. 113–140.
• 21. PRISMA [Internet]. www.prisma-statement.org . Available from: http://www.prisma-statement.org/Extensions/ScopingReviews
• 24. PRISMA. PRISMA [Internet]. prisma-statement.org. 2020. Available from: http://prisma-statement.org/PRISMAStatement/FlowDiagram .
• 25. Free Infographic Maker—Online Graphs and Infographics Creator for Doctors and Scientists [Internet]. Mind the Graph. [cited 2023 Aug 16]. Available from: https://mindthegraph.com/app/about-us .

• Open access
• Published: 23 April 2024

The Role of Chief Medical officers in making Public Health Overview Documents in Norwegian Municipalities. A qualitative Study

• Dag-Helge Rønnevik 1 ,
• Betty J. Pettersen 1 , 2 ,
• Anders Grimsmo 1 &
• Aslak Steinsbekk 1  

BMC Public Health volume  24 , Article number:  1132 ( 2024 ) Cite this article

23 Accesses

Metrics details

To investigate how Chief Medical Officers experience their role in the municipalities´ work with making the public health overview documents, demanded by the Norwegian Public Health Act from 2012.

A qualitative study with semi-structured focus group interviews with 21 Chief Medical Officers from 20 different municipalities in Norway. The interviews were conducted in 2017. The data were analyzed thematically.

The Chief Medical Officers were mainly positive to participating in making public health overview documents. They took on roles as leaders of the work, medical advisors, data collectors towards local GPs and listening post to other sectors. Organizational factors like too small positions and a lack of tradition to involve the CMO in public health work were experienced as barriers to their involvement. The collaboration with the public health coordinators was said to be rewarding, and the intersectoral process involved employees from other sectors in a new way in public health. Although there were some positive experiences, several CMOs considered the use and impact of the public health overview document as limited.

There was a large variation in the amount and the type of involvement the Chief Medical Officers had in making the public health overview documents in Norwegian municipalities. More research is needed to understand if this has any consequences for the quality of public health work in the municipalities and whether it is a sign of a changing role of the Chief Medical Officers.

Peer Review reports

The role of Chief Medical Officers (CMOs), also termed public health physicians, medical officer of health and district medical officer, has developed over time [ 1 , 2 , 3 ]. From 1984 it became mandatory for Norwegian municipalities to have one or more CMO as medical advisor [ 4 ].

Two thirds of the 356 Norwegian municipalities have only one CMO, and about half of all CMOs have combined positions as general practitioners (GPs), often with the CMO position as a small add on, typical 20% of a man-year [ 5 ]. The position as a CMO may cover public health, infection control, environmental health care, primary health care management, quality assurance and planning processes, often in a hybrid role as advisors on different managerial levels [ 6 ]. During the COVID-19 pandemic, CMOs in Norway as well as in other countries were heavily involved in infection control and vaccination programs and they were publicly exposed through communication with the general population, management, and politicians [ 7 , 8 ].

The Public health Act (PHA) introduced in 2012 [ 9 ], stated that Norwegian municipalities must involve all sectors in promoting public health, and have a public health overview document. The overview document shall identify the public health challenges in the municipality, including assessment of causal factors and impact on population health. It should be designed to be used as a base for the municipality’s planning and policy development, also in accordance with the Planning and Building Act [ 10 ], representing a ‘Health in All Policies’ (HiAP) as proposed by the WHO [ 11 , 12 ]. The medical expertise represented by the CMOs is stated in the PHA as a prerequisite for this work. One or more CMOs must be employed as medical advisors for the municipality to take care of community medical advice in the municipality’s public health work, including epidemiological analyses.

Previous studies have found that following the introduction of the Public health act in Norway, municipalities have introduced new working methods and professionals and implemented organizational changes on both micro and macro levels [ 13 , 14 ]. This includes appointment of public health coordinators [ 15 ] and a change towards increased understanding and adoption of the new, comprehensive public health policy [ 16 ]. Municipalities that developed health overviews reported more often to prioritising fair distribution of social and economic resources among social groups in political decision-making [ 17 ]. However, a countrywide supervision in 2015 showed that many of the municipalities had not decided how they wanted the CMOs to be involved in the work [ 18 ]. Also internationally there are tensions related to the role of CMOs [ 19 ], but no study focusing on the CMOs role in making overview documents has been identified.

The aim of this study was to investigate how Chief Medical Officers experience their role in the municipalities work with making the public health overview documents demanded by the Norwegian Public Health Act from 2012.

Materials and methods

Study design.

This was a qualitative study with semi-structured focus group interviews with Chief Medical Officers, conducted in 2017. Focus groups were chosen as they foster discussions among the participants and are suitable to explore phenomena that concern common experiences, attitudes or views in a field where people interact [ 20 ]. The Consolidated criteria for Reporting Qualitative research checklist (COREQ) [ 21 ] was consulted for reporting the study.

Participants and recruitment

The goal was to include CMOs from rural and urban areas in Norway, with variation in age, experience in public health, gender and size of the municipality they worked in. Eligible CMOs were recruited from the following three meetings. A yearly course arranged by the Chief County Medical Officer in a county in Central Norway, a compulsory course in the specialist program in community medicine held in Oslo and a workshop on new ways of organizing and managing GP practices in the western part of Norway. The identified participants were invited by e-mail which included information about the study.

Data collection

The data was collected in semi-structured focus group interviews, which lasted about 90 min and were audiotaped and transcribed verbatim. The participants were given verbal information about the study before the interviews started. The interview guide (Fig.  1 ) was developed based on relevant public documents related to public health. It was discussed with a group of 20 stakeholders interested in this topic, as well as discussions among the researchers. The main questions concerned experiences with public health work in general, and in particular the public health overview document.

Interview guide

Data were analyzed thematically, inspired by Braun and Clarke [ 22 ], in an iterative process. First, meanings and patterns were identified and discussed. Then meaning units related to the aim were coded and organized in themes. Seven initial main themes were identified (challenges related to the CMO-role in general, “yes the CMO is important”, intersectoral public health, challenges related to the CMO-role in public health, the role of GPs in public health, organizational conditions, public health vs. health care services). This process was repeated in an iterative process involving all authors and ended with three main themes (types of involvement, reasons for being involved or not, and the impact of the overview document).

A total of 21 CMOs participated in the three focus group interviews (Table  1 ). Twelve were specialists in community medicine/public health, and 11 worked fulltime as CMOs. Half of them were over 45 years old. The intended variation in geography and municipality size was reached. Exact data on years of experience from community medicine and public health were not systematically collected, but it varied from a few years to more than 30 years.

Types of involvement

Most of the informants expressed it as natural for them to participate in making the public health overview document, and that they had made a substantial contribution. A few had been leaders in the process, while others took a role as medical advisors. The main contributions were collecting, selecting and evaluating data, and making recommendations in the written report. Still almost none participated in all parts of the process, and some did not take part at all.

I have given advice regarding structure, scope and working method. I wasn’t involved in the actual writing, but I know it was an extensive struggle to get data from the various sectors.

It was said that having combined positions as GP made it easier to contribute to the collection of and ensuring the quality of the data from the health care sector. This included collecting data from local GPs including helping the GPs in extracting relevant data from the electronic health records. One senior CMO rejected using local data for this purpose because he did not find it trustworthy and would rather rely on national data. Some had interviewed GPs to get their input to the overview document.

It was very rewarding [collecting data from GPs]. We had a series of meetings and the GPs received course credits, and it was just as good to discuss this as discussing other professional matters. So GPs are not hopeless in relation to thinking about public health work, I would say!

The CMOs said that their role in the process was based on their knowledge about the municipality and the health care sector, and their relationships to other stakeholders in public health. One said that the CMO is a listening post to other sectors. Some emphasized their epidemiological training as important, giving them the ability to see relationships between data, do health impact assessments and understand the burden of disease in the local community. Also having skills in presenting knowledge, training in prioritizing and giving advice were mentioned as contributions.

It is the competency in community medicine, to prioritize among many suggested proposals. An employee from the planning department may have a completely different understanding than the CMO. In a sense, it is in a group process the community medicine competency is relevant.

Reasons for being involved or not

The variation in what degree the CMOs had been involved in the process of making the public health overview documents, was mainly said to be due to organizational factors. Too small positions, a lack of tradition to involve the CMO in public health work, municipal directors with limited interests in public health, and limited access to other expertise locally were mentioned as barriers. One said he had deliberately chosen to let the public health coordinator take care of the process, so he could prioritize other tasks. Two CMOs in combined positions said it was hard to combine this kind of planning process with clinical work as GPs due to time constraint and difficulty in finding suitable meeting times. One talked about being involved as a medical alibi to legitimize the work, while another senior doctor felt being on the outside and left behind:

From one perspective, it is good that the work involves more people, but my role has become more obscure. In some way, I have been somewhat degraded […]. The structure of public health work is no longer such that I am an explicit part of the leadership, which I think I should have been, even if it shall be distributed. And not just someone they ask when it suits them.

Those who expressed being most involved described the process as rewarding. The work with the overview document was said to breathe new life into public health work, and giving the CMOs knowledge they would have missed without this process. The collaboration with the public health coordinators was especially mentioned as a valuable experience. One emphasized the importance of having office space in the same building as the public health coordinator. The intersectoral process was said to give good discussions with employees from other sectors, and through this work they saw that they gave politicians a good basis for making important decisions for the health of the population. Also, being organized at a cross-sectoral level with access to other stakeholders involved in public health, with enough time to interact with them, was seen as an advantage.

It is important to be there and remind them of this [the public health perspective] when the decisions are made, and it is time-consuming, because then you have to take part in the planning processes and be involved early enough.

Impact of the health overview

There were different experiences as to the result of the process and how the overview document had been used after completion. One CMO said that she and her CMO colleague had created a web-based version of the overview document on the municipality’s website, with navigation features to make it more accessible. Several informants had witnessed how the process of making the overview document made employees in other sectors connect to public health in a new way. One had been included in intersectoral working groups where they used the overview document as a foundation for further municipal planning.

When the planning program is approved by the politicians, we see many signs of the idea behind. We can now see the results from the work done by the public health coordinator and the CMO, having worked to collect this data that we have collected in a health overview, and that it will be used in the future.

Still, several informants said that they did not see much use or impact of the public health overview document. Reasons given were that the politicians did not care, the plan was neglected and put in a drawer and the recommendations in the overview document were not prioritized in the municipality`s budget. The perspectives of the politicians were also commented. One CMO referred to politicians becoming very focused on measures concerning somatic health, asking “do we have more or less cancer than our neighbors?”. Others commented that they had a hard time making the politicians realize that effects of interventions in public health take time.

But as it turned out, as soon as there was a cost connected to it, it went down to zero and nothing was granted for follow up. The plan was adopted, but interventions in the plan are completely dead. I haven´t bothered to get involved further since it got so negative from the administration and the politicians.

The Chief Medical Officers said they were involved in various roles and to varying degrees in making public health overview documents. This was related to the size of their position as CMOs and different traditions in the municipalities to involve CMOs. How the municipalities used the overview document afterwards also varied.

Study strength and limitations

Although no data were recorded about those invited who did not volunteer to participate, the characteristic of the participants indicates that the sought after variation was achieved. There was good variation in experiences, e.g. positive and negative attitudes towards public health work. Furthermore, the main topics were identifiable in all the interviews, which indicates that more interviews would not be likely to change the findings. Some of the informants were known by the authors’ networks and some knew each other. However, in the transcripts it was not found that the participants tried to please others, rather there were lively discussions about different experiences and viewpoints. This also shows that the use of focus groups was a good choice. All the authors were involved in the inductive analysis which ensured that they contributed with different perspectives and increased the likelihood of identifying the variations in the data. The data were collected before the Covid-19 pandemic, and it may be that interviews conducted after the pandemic would have given some different experiences. However, we assume that this would mainly concern infectious disease control, and that the experiences related to public health work would be similar to the situation in 2017.

Health in all policy

The intention behind the public health overview documents and part of the public health act in Norway, is to formalize the implementation of health in all policies (HiAP) [ 13 , 23 ]. None of the participants expressed any objection to this approach. On the contrary, the CMOs mainly expressed positive attitudes to the process of making public health overviews regardless of their own involvement. This is in accordance with an evaluation report that found that making overview documents contributed to a strengthened anchoring of public health in the municipal planning system and an increase in the use of resources for the organization and planning of public health work [ 24 ].

Some of the CMOs expressed an appreciation of and motivation for cross-sectoral work, but overall, there were few who talked about having a leading role in working with the overview. This is in line with previous evaluation of actions that is termed ‘intersectoral action for health’ (ISA), which concerns how the health sector works with other governmental and non-state sectors to improve health and well-being [ 25 ]. A Danish document analysis showed that the ideas that ISA builds on, has proven difficult to implement, and that the guidelines for municipal ISA were vague and constructed with buzzwords [ 26 ].

Furthermore, Health in all policy in Norway is nationally decided upon, but not fully recognized locally [ 27 ]. There are reports pointing out that Norwegian municipalities are at different levels regarding building competence necessary to achieve the objectives of the Public Health Act [ 28 ]. A report from 2015 concluded that the work required more resources, time, expertise and capacity than the municipalities had at their disposal [ 29 ]. It was stated that especially small municipalities did not have enough resources to meet the expectations from the governments. This is a likely reason for some of the experiences of the CMOs about the role of both the planning and use of the overview document.

The role of the CMOs

There was a large variation of how involved the CMOs were in making of the public health overview documents, from hardly taking part, to having a central role. There were also comments about the changing role of the CMOs in the municipalities over time. One possible explanation is the increased focus on implementing inter-sectoral working groups and increasing numbers of public health coordinators. Such activities are used to coordinate and implement public health policies and measures in the municipalities, and are considered important when building capacity to reduce health inequities [ 30 ].

A major contribution from the CMOs that participated in the work, was the contact with GPs, both through helping to collect data from electronic health record and to interview GPs. This could in part be explained by the opportunities that come with CMOs working in combined positions [ 31 ], i.e. as both CMOs and GPs. Although CMOs have different attitudes towards their responsibility to go beyond clinical medicine to help even out social inequality in health [ 32 ], this activity points to a wish among the CMOs to use knowledge from the clinical work in primary care as a contribution to the population health perspective.

Chief Medical Officers have experienced large variation in their involvement in making public health overview documents in Norwegian municipalities. More research is needed to understand if this has any consequences for the quality of public health work in the municipalities and whether it is a sign of a changing role of the Chief Medical Officers.

Data availability

The datasets generated and analysed during the current study are not publicly available due to data regulations but are available from the corresponding author on reasonable request.

Larsen Ø, Olsen BO. The shaping of a profession: Physicians in Norway, past and present. Oslo: Science History; 1996.

Google Scholar  

Larsen Ø. Legerollen–verktøy som må voktes [The physician role - tools to be guarded]. Michael. 2022;19:215–24.

Pettersen BJ. Local public health physicians in Norway from 1994 to 2002: workload, work content, and interaction: a story of everyday life in primary health care. Tromsø: University of Tromsø; 2008.

Lov om kommunale. Helse- og omsorgstjenester m.m. (helse- og omsorgstjenesteloven) [Health and Care services Act]. LOV-2011-06-24-30, https://lovdata.no/dokument/NL/lov/2011-06-24-30 (2012).

NOU. 2022: 5. Myndighetenes håndtering av koronapandemien– del 2 — Rapport fra Koronakommisjonen [The governments handling of the corona pandemic - part 2 - Report from the Corona Commission], https://www.regjeringen.no/no/dokumenter/nou-2022-5/id2910055/ (2022).

Fossberg BC, Frich JC. Kommuneoverlegers opplevelse Av Egen rolle [How Chief Medical officers experience their role]. Tidsskrift Norske Legeforening. 2022;142:121–6. https://doi.org/10.4045/tidsskr.21.0589 .

Article   Google Scholar  

Hungnes T, Vik E, Veddeng O. Kommunelegens rolle under koronapandemien–en kvalitativ studie [The role of chief medical doctors during the corona pandemic– a qualitative study]. Tidsskr nor Laegeforen. 2022;142:1391–5.

MacAulay M, Macintyre AK, Yashadhana A, et al. Under the spotlight: understanding the role of the Chief Medical Officer in a pandemic. J Epidemiol Community Health. 2022;76:100–4. https://doi.org/10.1136/jech-2021-216850 .

Article   PubMed   Google Scholar  

Lov om folkehelsearbeid (Folkehelseloven) [The Public Health Act]. LOV-2011-06-24-29, https://lovdata.no/dokument/NL/lov/2011-06-24-29?q=folkehelseloven (2012).

Lov om planlegging og. Byggesaksbehandling (Plan- Og bygningsloven) [The Planning and Building Act]. LOV-2008-06-27-71, https://lovdata.no/dokument/NL/lov/2008-06-27-71?q=plan-%20og%20bygning (2008).

World Health Organization. The Helsinki Statement on Health in All Policies, https://www.who.int/publications/i/item/9789241506908 (2013).

World Health Organization. Norway: Health system review, https://apps.who.int/iris/handle/10665/331786 (2020).

Hofstad H. The ambition of Health in all policies in Norway: the role of political leadership and bureaucratic change. Health Policy. 2016;120:567–75. https://doi.org/10.1016/j.healthpol.2016.03.001 .

Helgesen M, Hofstad H. Regionalt og lokalt folkehelsearbeid. Ressurser, organisering og koordinering. En baselineundersøkelse [Regional and local public health work. Resources, organization and coordination. A baseline survey], https://oda.oslomet.no/oda-xmlui/bitstream/handle/20.500.12199/5507 /2012-13.pdf?sequence=1 (2012).

Hagen SHM, Torp S. Health in all policies: a cross-sectional study of the public health coordinators’ role in Norwegian municipalities. Scand J Public Health. 2015;43:597–605. https://doi.org/10.1177/1403494815585614 .

Fosse E, Helgesen MK, Hagen S, et al. Addressing the social determinants of health at the local level: opportunities and challenges. Scand J Public Health. 2018;46:47–52. https://doi.org/10.1177/1403494817743896 .

Article   CAS   PubMed   Google Scholar  

Hagen S, Øvergård KI, Helgesen M, et al. Health Promotion at Local Level in Norway: the Use of Public Health Coordinators and Health Overviews to promote fair distribution among Social groups. Int J Health Policy Manage. 2018;7:807–17. https://doi.org/10.15171/ijhpm.2018.22 .

Norwegian Board of Health Supervision. Med blikk for betre folkehelse [Improving public health]. Report of the Norwegian Board of Health Supervision 4/2015, https://www.helsetilsynet.no/publikasjoner/rapport-fra-helsetilsynet/2015/med-blikk-for-betre-folkehelse-oppsummering-av-landsomfattande-tilsyn-i-2014/ (2015).

Cassola A, Fafard P, Nagi R, et al. Tensions and opportunities in the roles of senior public health officials in Canada: a qualitative study. Health Policy. 2022;126:988–95. https://doi.org/10.1016/j.healthpol.2022.07.009 .

Article   PubMed   PubMed Central   Google Scholar  

Morgan DL, Krueger RA. When to use focus groups and why. In: Morgan DL, editor. Successful focus groups: advancing the state of the art. Newsbury Park: Sage; 1993. pp. 3–19.

Chapter   Google Scholar  

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57. https://doi.org/10.1093/intqhc/mzm042 .

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3. https://doi.org/10.1191/1478088706qp063oa .

Nilsen HM, Feiring E. Local public health projectification in practice: a qualitative study of facilitators and barriers to a public health plan implementation. Scand J Public Health. 2023;51:874–81. https://doi.org/10.1177/14034948221080402 .

Løset GK, Veenstra M. Forebyggende helsearbeid i kommunene. Prosessevaluering av Samhandlingsreformen. [Process evaluation of the coordination reform]. NOVA Rapport 6/15, https://oda.oslomet.no/oda-xmlui/handle/20.500.12199/5088 (2015).

Rantala R, Bortz M, Armada F. Intersectoral action: local governments promoting health. Health Promot Int. 2014;29:92–102. https://doi.org/10.1093/heapro/dau047 .

Holt DH, Waldorff SB, Tjørnhøj-Thomsen T, et al. Ambiguous expectations for intersectoral action for health: a document analysis of the Danish case. Crit Public Health. 2018;28:35–47. https://doi.org/10.1080/09581596.2017.1288286 .

Amdam R. Health in all policies in Norwegian municipalities—dilemmas and lack of legitimacy and capacity. Int J Health Plann Mgmt: 1–15. https://doi.org/10.1002/hpm.3664 .

Synnevåg E, Amdam R, Fosse E. Legitimising Inter-sectoral Public Health policies: a challenge for Professional identities? Int J Integr Care. 2019;19:1–10. https://doi.org/10.5334/ijic.4641 .

Hofstad H, Lid S, Schou A et al. Lokal oversikt over folkehelsen– muligheter og utfordringer [Local public health overview - possibilities and challenges], http://www.hioa.no/extension/hioa/design/hioa/images/nibr/files/filer/2015-13.pdf (2015).

Helgesen M, Fosse E, Hagen S. Capacity to reduce inequities in health in Norwegian municipalities. Scand J Public Health. 2017;45(18suppl):77–82. https://doi.org/10.1177/1403494817709412 .

Thorlby R. Reclaiming a population health perspective. Future challenges for primary care, https://www.nuffieldtrust.org.uk/files/2017-01/reclaiming-a-population-health-perspective-web-final.pdf (2013).

Bringedal B, Bærøe K. Bør leger bidra til å utjevne sosial ulikhet i helse? [Should doctors help even out social inequality in health?]. Tidsskr nor Legeforen. 2010;130:1024–7. https://doi.org/10.4045/tidsskr.09.0648 .

Download references

Acknowledgements

We especially would like to thank the Chief Medical Officers taking part in the interviews.

This project has been made possible by funding from the Dam Foundation in partnership with The Norwegian Health Association. Additional funding has been given by Norwegian Centre of Rural Medicine, The Liaison Committee for education, research and innovation in Central Norway and the research fund of The Medical Association of Rogaland.

Open access funding provided by Norwegian University of Science and Technology

Author information

Authors and affiliations.

ISM, NTNU, Trondheim, Norway

Dag-Helge Rønnevik, Betty J. Pettersen, Anders Grimsmo & Aslak Steinsbekk

Municipality of Trondheim, Trondheim, Norway

Betty J. Pettersen

You can also search for this author in PubMed   Google Scholar

Contributions

DHR, BJP and AG led the conceptualization of the study, conducted the key informant interviews, and wrote the first draft of the article. AS provided oversight for the investigation, methodological support, and substantive editorial feedback. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Dag-Helge Rønnevik .

Ethics declarations

Ethics approval and consent to participate.

The study was approved by The Norwegian Centre for Research Data (project 52922) who ensure that the projects´ processing of data is done in accordance with the national data protection legislation. The study was carried out in accordance with the Declaration of Helsinki. Informed consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rønnevik, DH., Pettersen, B.J., Grimsmo, A. et al. The Role of Chief Medical officers in making Public Health Overview Documents in Norwegian Municipalities. A qualitative Study. BMC Public Health 24 , 1132 (2024). https://doi.org/10.1186/s12889-024-18608-5

Download citation

Received : 20 September 2023

Accepted : 15 April 2024

Published : 23 April 2024

DOI : https://doi.org/10.1186/s12889-024-18608-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Chief Medical Officer
• Public health
• Community medicine
• Epidemiology
• Preventive medicine
• General practice

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

Academic integrity and documentation, explore more of umgc.

• Online Guide to Writing

Types of Documentation

Bibliographies and Source Lists

What is a bibliography.

A bibliography is a list of books and other source material that you have used in preparing a research paper. Sometimes these lists will include works that you consulted but did not cite specifically in your assignment. Consult the style guide required for your assignment to determine the specific title of your bibliography page as well as how to cite each source type. Bibliographies are usually placed at the end of your research paper.

What is an annotated bibliography?

A special kind of bibliography, the annotated bibliography, is often used to direct your readers to other books and resources on your topic. An instructor may ask you to prepare an annotated bibliography to help you narrow down a topic for your research assignment. Such bibliographies offer a few lines of information, typically 150-300 words, summarizing the content of the resource after the bibliographic entry.   

Example of Annotated Bibliographic Entry in MLA Style

Waddell, Marie L., Robert M. Esch, and Roberta R. Walker. The Art of Styling         Sentences: 20 Patterns for Success. 3rd ed. New York: Barron’s, 1993.         A comprehensive look at 20 sentence patterns and their variations to         teach students how to write effective sentences by imitating good style.

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

Cookies on GOV.UK

We use some essential cookies to make this website work.

We’d like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.

We also use cookies set by other sites to help us deliver content from their services.

You have accepted additional cookies. You can change your cookie settings at any time.

You have rejected additional cookies. You can change your cookie settings at any time.

• Education, training and skills
• Further and higher education, skills and vocational training

Future skills projections and analysis

A one-off assessment of skills forecasting methods.

Applies to England

Ref: ISBN 978-1-83870-542-8 , RR1419

PDF , 910 KB , 150 pages

This report is an evaluation of future skills methodologies at economy-wide and sector level.

It reviews the accuracy, versatility, ease of use and relevance of each method for key users.

• central planners, for example government
• local users, such as those involved in local skills improvement plans
• users engaged in workforce planning, such as employers and training providers
• those designing qualifications and standards, such as the Institute for Apprenticeships and Technical Education

Related content

Is this page useful.

• Yes this page is useful
• No this page is not useful

Help us improve GOV.UK

Don’t include personal or financial information like your National Insurance number or credit card details.

To help us improve GOV.UK, we’d like to know more about your visit today. We’ll send you a link to a feedback form. It will take only 2 minutes to fill in. Don’t worry we won’t send you spam or share your email address with anyone.

The Federal Register

The daily journal of the united states government, request access.

Due to aggressive automated scraping of FederalRegister.gov and eCFR.gov, programmatic access to these sites is limited to access to our extensive developer APIs.

If you are human user receiving this message, we can add your IP address to a set of IPs that can access FederalRegister.gov & eCFR.gov; complete the CAPTCHA (bot test) below and click "Request Access". This process will be necessary for each IP address you wish to access the site from, requests are valid for approximately one quarter (three months) after which the process may need to be repeated.

An official website of the United States government.

If you want to request a wider IP range, first request access for your current IP, and then use the "Site Feedback" button found in the lower left-hand side to make the request.