primary research articles pubmed

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Trending Articles

• Plasma interleukin-41 serves as a potential diagnostic biomarker for Kawasaki disease. Cai X, et al. Microvasc Res. 2023. PMID: 36682486
• A gut-derived hormone regulates cholesterol metabolism. Hu X, et al. Cell. 2024. PMID: 38503280
• Safety and efficacy of givinostat in boys with Duchenne muscular dystrophy (EPIDYS): a multicentre, randomised, double-blind, placebo-controlled, phase 3 trial. Mercuri E, et al. Lancet Neurol. 2024. PMID: 38508835 Clinical Trial.
• Global, regional, and national burden of disorders affecting the nervous system, 1990-2021: a systematic analysis for the Global Burden of Disease Study 2021. GBD 2021 Nervous System Disorders Collaborators. Lancet Neurol. 2024. PMID: 38493795
• Global fertility in 204 countries and territories, 1950-2021, with forecasts to 2100: a comprehensive demographic analysis for the Global Burden of Disease Study 2021. GBD 2021 Fertility and Forecasting Collaborators. Lancet. 2024. PMID: 38521087

Latest Literature

• Am Heart J (1)
• Am J Med (2)
• Arch Phys Med Rehabil (3)
• Cell Metab (1)
• Gastroenterology (3)
• J Am Acad Dermatol (3)
• J Biol Chem (1)
• Lancet (16)
• Nat Commun (32)

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Harvard Library
• Research Guides
• Faculty of Arts & Sciences Libraries
• Identifying Articles
• PubMed at Harvard
• Searching in PubMed
• My NCBI in PubMed
• Utilizing Search Results
• Scenarios in PubMed

Primary Research Article

Review article.

Identifying and creating an APA style citation for your bibliography: 

• Author initials are separated by a period
• Multiple authors are separated by commas and an ampersand (&)  
• Title format rules change depending on what is referenced
• Double check them for accuracy 

Identifying and creating an APA style in-text citation: 

• eg. (Smith, 2022) or (Smith & Stevens, 2022) 

The structure of this changes depending on whether a direct quote or parenthetical used:

Direct Quote: the citation must follow the quote directly and contain a page number after the date

eg. (Smith, 2022, p.21)

Parenthetical: the page number is not needed

For more information, take a look at Harvard Library's Citation Styles guide !

A primary research article typically contains the following section headings:

"Methods"/"Materials and Methods"/"Experimental Methods"(different journals title this section in different ways)

"Results"

"Discussion"

If you skim the article, you should find additional evidence that an experiment was conducted by the authors themselves.

Primary research articles provide a background on their subject by summarizing previously conducted research, this typically occurs only in the Introduction section of the article.

Review articles do not report new experiments. Rather, they attempt to provide a thorough review of a specific subject by assessing either all or the best available scholarly literature on that topic.

Ways to identify a review article: 

• Author(s) summarize and analyze previously published research 
• May focus on a specific research question, comparing and contrasting previously published research 
• Overview all of the research on a particular topic 
• Does not contain "methods" or "results" type sections
• << Previous: Scenarios in PubMed
• Last Updated: Oct 3, 2023 4:16 PM
• URL: https://guides.library.harvard.edu/PubMed

Harvard University Digital Accessibility Policy

• Books, eBooks & Articles
• Databases A-Z
• Primary Sources
• E-Audiobooks
• Videos & Images
• Online Videos
• Images & Artwork
• More resources
• Research Guides
• Library Instruction
• Request Research Guide
• Interlibrary Loan
• Books on Reserve
• Research Assistance
• Writing Lab
• Online Tutoring
• Group Study Sessions
• Turabian/Chicago
• Other Citing Styles

Service Alert

Finding Primary Articles in PubMed: Home

• APA Citations

Finding Primary Articles in PubMed

From the library homepage -- library.surry.edu (opens in new window) -- click on Find Articles .

Click on the letter P or scroll through the list until you see PubMed . To limit to full text articles, click on the PubMed Central link in the PubMed description.

Type in a search for your topic. Press Enter or click the Search button.

You will retrieve a list of articles. To limit to primary research articles, click on Clinical Trial or click More to select other type of trials and original research studies.

You may also limit your article results to Free full text either on the left or you can scan below the article results for Free Article or Free PMC Article .

If the article is available for free, you will see a link to access the article in the upper right of the screen. If you can't find the article text, email Alan Unsworth, Research Librarian , to see if the article may be obtained .

• Next: APA Citations >>
• Last Updated: Nov 9, 2023 2:07 PM
• URL: https://library.surry.edu/pubmed

Literature Searching

In this guide.

• Introduction
• Steps for searching the literature in PubMed
• Step 1 - Formulate a search question
• Step 2- Identify primary concepts and gather synonyms
• Step 3 - Locate subject headings (MeSH)
• Step 4 - Combine concepts using Boolean operators
• Step 5 - Refine search terms and search in PubMed
• Step 6 - Apply limits

Steps for Searching the Literature

Searching is an iterative process and often requires re-evaluation and testing by adding or changing keywords and the ways they relate to each other. To guide your search development, you can follow the search steps below. For more information on each step, navigate to its matching tab on the right menu. 

1. Formulate a clear, well-defined, answerable search question

Generally, the basic literature search process begins with formulating a clear, well-defined research question. Asking the right research question is essential to creating an effective search. Your research question(s) must be well-defined and answerable. If the question is too broad, your search will yield more information than you can possibly look through.

2. Identify primary concepts and gather synonyms

Your research question will also help identify the primary search concepts. This will allow you to think about how you want the concepts to relate to each other. Since different authors use different terminology to refer to the same concept, you will need to gather synonyms and all the ways authors might express them. However, it is important to balance the terms so that the synonyms do not go beyond the scope of how you've defined them.

3. Locate subject headings (MeSH)

Subject databases like PubMed use 'controlled vocabularies' made up of subject headings that are preassigned to indexed articles that share a similar topic. These subject headings are organized hierarchically within a family tree of broader and narrower concepts. In PubMed and MEDLINE, the subject headings are called Medical Subject Headings (MeSH). By including MeSH terms in your search, you will not have to think about word variations, word endings, plural or singular forms, or synonyms. Some topics or concepts may even have more than one appropriate MeSH term. There are also times when a topic or concept may not have a MeSH term. 

4. Combine concepts using Boolean operators AND/OR

Once you have identified your search concepts, synonyms, and MeSH terms, you'll need to put them together using nesting and Boolean operators (e.g. AND, OR, NOT). Nesting uses parentheses to put search terms into groups. Boolean operators are used to combine similar and different concepts into one query. 

5. Refine search terms and search in PubMed

There are various database search tactics you can use, such as field tags to limit the search to certain fields, quotation marks for phrase searching, and proximity operators to search a number of spaces between terms to refine your search terms. The constructed search string is ready to be pasted into PubMed. 

6. Apply limits (optional)

If you're getting too many results, you can further refine your search results by using limits on the left box of the results page. Limits allow you to narrow your search by a number of facets such as year, journal name, article type, language, age, etc. 

Depending on the nature of the literature review, the complexity and comprehensiveness of the search strategies and the choice of databases can be different. Please contact the Lane Librarians if you have any questions. 

The type of information you gather is influenced by the type of information source or database you select to search. Bibliographic databases contain references to published literature, such as journal articles, conference abstracts, books, reports, government and legal publications, and patents. Literature reviews typically synthesis indexed, peer-reviewed articles (i.e. works that generally represent the latest original research and have undergone rigorous expert screening before publication), and gray literature (i.e. materials not formally published by commercial publishers or peer-reviewed journals). PubMed offers a breadth of health sciences literature and is a good starting point to locate journal articles.

What is PubMed?

PubMed is a free search engine accessing primarily the MEDLINE database of references and abstracts on life sciences and biomedical topics. Available to the public online since 1996, PubMed was developed and is maintained by the  National Center for Biotechnology Information (NCBI) , at the  U.S. National Library of Medicine (NLM) , located at the  National Institutes of Health (NIH) .

MEDLINE is the National Library of Medicine’s (NLM) premier bibliographic database that contains more than 27 million references to journal articles from more than 5,200 worldwide journals in life sciences with a concentration on biomedicine. The Literature Selection Technica Review Committee (LSTRC) reviews and selects journals for MEDLINE based on the research quality and impact of the journals. A distinctive feature of MEDLINE is that the records are indexed with NLM  Medical Subject Headings  (MeSH).

PubMed also contains citations for  PubMed Central (PMC)  articles. PMC is a full-text archive that includes articles from journals reviewed and selected by NLM for archiving (current and historical), as well as individual articles collected for archiving in compliance with funder policies.  PubMed allows users to search keywords in the bibliographic data, but not the full text of the PMC articles.

How to Access PubMed?

To access PubMed, go to the Lane Library homepage and click PubMed in "Top Resources" on the left. This PubMed link is coded with Find Fulltext @ Lane Library Stanford that links you to Lane's full-text articles online. 

• << Previous: Introduction
• Next: Step 1 - Formulate a search question >>
• Last Updated: Jan 9, 2024 10:30 AM
• URL: https://laneguides.stanford.edu/LitSearch

Identifying Primary and Secondary Research Articles

• Primary and Secondary

Primary Research Articles

Primary research articles report on a single study. In the health sciences, primary research articles generally describe the following aspects of the study:

• The study's hypothesis or research question
• Some articles will include information on how participants were recruited or identified, as well as additional information about participants' sex, age, or race/ethnicity
• A "methods" or "methodology" section that describes how the study was performed and what the researchers did
• Results and conclusion section

Secondary Research Articles

Review articles are the most common type of secondary research article in the health sciences. A review article is a summary of previously published research on a topic. Authors who are writing a review article will search databases for previously completed research and summarize or synthesize those articles,  as opposed to recruiting participants and performing a new research study.

Specific types of review articles include:

• Systematic Reviews
• Meta-Analysis
• Narrative Reviews
• Integrative Reviews
• Literature Reviews

Review articles often report on the following:

• The hypothesis, research question, or review topic
• Databases searched-- authors should clearly describe where and how they searched for the research included in their reviews
• Systematic Reviews and Meta-Analysis should provide detailed information on the databases searched and the search strategy the authors used.Selection criteria-- the researchers should describe how they decided which articles to include
• A critical appraisal or evaluation of the quality of the articles included (most frequently included in systematic reviews and meta-analysis)
• Discussion, results, and conclusions

Determining Primary versus Secondary Using the Database Abstract

Information found in PubMed, CINAHL, Scopus, and other databases can help you determine whether the article you're looking at is primary or secondary.

Primary research article abstract

• Note that in the "Objectives" field, the authors describe their single, individual study.
• In the materials and methods section, they describe the number of patients included in the study and how those patients were divided into groups.
• These are all clues that help us determine this abstract is describing is a single, primary research article, as opposed to a literature review.
• Primary Article Abstract

Secondary research/review article abstract

• Note that the words "systematic review" and "meta-analysis" appear in the title of the article
• The objectives field also includes the term "meta-analysis" (a common type of literature review in the health sciences)
• The "Data Source" section includes a list of databases searched
• The "Study Selection" section describes the selection criteria
• These are all clues that help us determine that this abstract is describing a review article, as opposed to a single, primary research article.
• Secondary Research Article

• Primary vs. Secondary Worksheet

Full Text Challenge

Can you determine if the following articles are primary or secondary?

• Last Updated: Feb 17, 2024 5:25 PM
• URL: https://library.usfca.edu/primary-secondary

2130 Fulton Street San Francisco, CA 94117-1080 415-422-5555

• Facebook (link is external)
• Instagram (link is external)
• Twitter (link is external)
• YouTube (link is external)
• Consumer Information
• Privacy Statement
• Web Accessibility

Copyright © 2022 University of San Francisco

Finding Primary Research Articles in the Sciences: Home

• Advanced Search-Databases
• Primary vs. Secondary
• Analyzing a Primary Research Article
• MLA, APA, and Chicago Style

This guide goes over how to find and analyze primary research articles in the sciences (e.g. nutrition, health sciences and nursing, biology, chemistry, physics, sociology, psychology). In addition, the guide explains how to tell the difference between a primary source and a secondary source in scientific subject areas.

If you are looking for how to find primary sources in the humanities and social sciences, such as direct experience accounts in newspapers, diaries, artwork and so forth, please see   Finding Primary Sources in the Humanities and Social Sciences . 

Recommended Databases

To get started, choose one of the databases below.  Once you log in, enter your search terms to start looking for primary articles. 

• Link to all Polk State College Library databases

Login Required

You must log in to use library databases and eBooks. When prompted to log in, enter your Passport credentials. 

If you have trouble, try  resetting your Passport pin , sending an email to  [email protected] ,  or calling the Help Desk at 863.292.3652 . 

You can also get help from Ask a Librarian . 

Search Tips

Keep your search terms simple.

• No need to type full sentences into the database search box.  Limit your search to 2-3 words.
• There is no need to type "research article" into the search box.

Use the "Advanced Search" feature of the database.

• This will allow you to limit your search to only peer reviewed articles or a certain time frame (for example: 2013 or later).
• Click the red tab above for tips on advanced search strategies .

Re-read the assignment guidelines often

• Does this article satisfy the scope of the assignment (e.g. a study focused on nutrition)?
• Does it meet the criteria for the assignment (e.g. an original research article)?

Not finding what you are looking for?

• Ask a Librarian!

Search and Find a Primary Research Article

Are you looking for a primary research journal article if so, that is an article that reports on the results of an original research study conducted by the authors themselves. .

You can use the library's databases to search for primary research articles.  A research article will almost always be published in a peer-reviewed journal. Therefore, it is a good idea to limit your results to peer-reviewed articles. Click on the  Advanced Search-Databases tab at the top of this guide for instructions. 

The following is _not_ primary research:

Review articles are studies that arrive at conclusions after looking over other studies. Therefore, review articles are not  primary (think "first") research.  There are a variety of review articles, including:

• Literature Reviews
• Systematic Reviews
• Meta-Analyses 
• Scoping Reviews
• Topical Reviews
• A review/assessment of the evidence

Having trouble?  Look for a  method section within the article. If the method section includes the process used to conduct the research, how the data was gathered and analyzed and any limitations or ethical concerns to the study, then it is most likely a primary research article. For example: a research article will describe the number of people (e.g. 175 adults with celiac disease) who participated in the study and who were used to collect data.

If the method section describes how the authors found articles on a topic using search terms or databases , then it is mostly likely a secondary review article and not primary research. If there is no method section, it is not a primary research article.

Other sections in a journal: 

Your search may yield these items, too. You can skip these because they are not full write-ups of research:

• Conference Proceedings 
• Symposium Publications

Example of a primary research article found in the Library's Academic Search Complete database : (these authors conducted an original research study)

• Lumia et al. (2015) Lumia, M., Takkinen, H., Luukkainen, P., Kaila, M., Lehtinen, J. S., Nwaru, B. I., Tuokkola, J., Niemelä, O., Haapala, A., Ilonen, J., Simell, O., Knip, M., Veijola, R., & Virtanen, S. M. (2015). Food consumption and risk of childhood asthma. Pediatric Allergy & Immunology, 26(8), 789–796. https://doi.org/10.1111/pai.12352

Example of a secondary article found in the Library's Academic Search Complete database : (these authors are reviewing the work of other authors)

• Rachmah et al. (2022) Rachmah, Q., Martiana, T., Mulyono, Paskarini, I., Dwiyanti, E., Widajati, N., Ernawati, M., Ardyanto, Y. D., Tualeka, A. R., Haqi, D. N., Arini, S. Y., & Alayyannur, P. A. (2022). The effectiveness of nutrition and health intervention in workplace setting: A systematic review. Journal of Public Health Research, 11(1), 1–8. https://doi.org/10.4081/jphr.2021.2312

How do I know if this article is primary?

You've found an article in the library databases but how do you know if it's primary .

Look for these sections: (terminology may vary)

• abstract  - summarizes paper in one paragraph, states the purpose of the study
• methods  - explaining how the experiment was conducted (note: if the method section discusses how a search was conducted that is _not_ primary research) 
• results  - detailing what happened and providing raw data sets (often as tables or graphs)
• conclusions  - connecting the results with theories and other research
• references  - to previous research or theories that influenced the research

Scan the article you found to see if it includes the sections above. You don't have to read the full article (yet). Look for the clues highlighted in the images below. 

Questions? Use Ask a Librarian

• Next: Advanced Search-Databases >>
• Last Updated: Feb 19, 2024 11:55 AM
• URL: https://libguides.polk.edu/primaryresearch

Polk State College is committed to equal access/equal opportunity in its programs, activities, and employment. For additional information, visit polk.edu/compliance .

• University of Michigan Library
• Research Guides

Epidemiology

• Searching PubMed
• Creating a Well-Focused Question
• PubMed - Article Abstract Page
• Searching Other Databases - Embase
• Searching Other Databases - Web of Science
• Literature Reviews This link opens in a new window
• Citation Management
• Evaluating Sources
• Statistical Sources
• Data Visualization
• Tests & Measurement Instruments This link opens in a new window
• Citation Management Programs This link opens in a new window
• Funding Sources for SPH Graduate Students This link opens in a new window
• Research Funding & Grants This link opens in a new window
• National Institutes of Health Public Access Policy (NIHPAP) This link opens in a new window

Why Search PubMed?

PubMed is the free interface for the premier biomedical database, MEDLINE.  It was created & is maintained by the National Library of Medicine.  PubMed contains both primary & secondary literature.  Because it's a free to access, you can use it even when you leave the University of Michigan.

Articles in PubMed are indexed by MeSH ( Me dical S ubject H eadings), terms that have specific definitions within the database & help you to create more focused searches.

Running a Search

Search Results

Your results are listed on the Search Results page.

You can see that there are many results, including some that are not related to the question.

Search Tip - "Search Details"

What if your search results are not quite what you expected or they seem really off-base?  On the Advanced page (link right below the search bar), check Search Details , to see how PubMed "translated" your search.

If at least one term for each concept in your search doesn't map to a MeSH term, you should rethink your search terms or contact the library for help.

Look at how some terms were "translated," for example,  dietary intake  mapped to eating .  This is why the search results are so far off topic.  We'll need to revise the search.

Revising Your Search

Putting dietary intake and food intake in quotation marks

( "dietary intake" OR "food intake") AND (dairy products OR milk OR cheese OR yogurt)

will restrict this part of the search to those phrases.  The phrases won't map to MeSH terms, but may provide a more focused set of results.

And that's exactly what happens.

Because there are still so many results, add United States to the search: 

("dietary intake" OR "food intake") AND (dairy products OR milk OR cheese OR yogurt) AND United States

What's on this page

Choose from this list, or scroll down:   Why Search PubMed? ; Running a Search ; Search Results ; Revising Your Search ; Focusing Your Search with Filters ; Search Tip - Keeping Recent Articles in Your Search ; and Search Tip - "Search Details" .

Search Tip - Keeping Recent Articles in Your Search

To be sure that you're seeing the most recent articles on your topic in PubMed, change the default (Best Match) to Most Recent.

Focusing Your Search with Filters

F ilters , which can be found on the left side of the Search Results page, can help you focus your search appropriately.  Categories include Article types , Publication dates , Species , Languages , & Ages .

• Two filters that are almost always useful are Species / Humans (unless you're looking specifically for animal research) and Language / English .  Ages/ Adolescent will also be useful in this search.
• Some filters are always readily available:  Article type, Text Availability (which you should ignore while you're at Michigan), Publication dates.  Others you must add to the filters list.
• To add Language , Age , & other types of filters, click on the Additional filters link below the filter list. In the box that opens, select the category of filter & then the specific filters. Click the Show button to make the filters appear on the screen.  Next choose the filter(s) you want to add.
• When you apply filters, they appear above your search results.  You can clear a filter by clicking the name of the filter or the Clear link, or clear all at the top of the results.
• Remember to clear all filters when you do a new search.

Finally, if you want to see more recent articles, add a date filter.  Limiting this search to the last 5 years gives 30 results, a reasonable set of results to look through.

A Guide to Biology: Find Primary Articles

• Find Primary Articles
• Find Books and Background Information
• Literature Reviews
• Citing Biology Sources/Citation Management

Journals List: Do We Have this Journal?

When you have a source with a bibliography, you can see if a particular article from the bibliography is available by looking the journal's name up at the link below. Then you can use the volume and date information to navigate to the article. If we don't have access to that journal, we usually can get it from another library.

• Search the Journals List: Do We Have this Journal?

Biology Journals in Print

These print-format journals all publish primary research and review articles in the field of biology.

American Midland Naturalist Genes and Development (most current year; earlier volumes in PMC)  Nature Science Wilson Journal of Ornithology

We also subscribe in print to the following biology-related journals and magazines Environment: Science and Policy for Sustainable Development Environmental Ethics Horticulture (current issue on main floor) Loon Minnesota Birding Minnesota Conservation Volunteer National Wildlife New Scientist (current issue on main floor)

Open Access to Biology Research

When searching PubMed, you can narrow the results to "free full text." 

For a single source of open access journal articles in the life sciences, this collection from the National Library of Medicine is hard to beat.

• PubMed Central (digital archive of journal literature) This link opens in a new window Free full text scholarly journal archive of literature in the life and health sciences, managed by the National Center for Biotechnology Information at the National Library of Medicine.

Biology Databases

Often you will hear the phrase "primary articles" when starting biology research, meaning articles written by scientists reporting new research. These typically introduce the research with a review of previous research in the introduction, methodology, results, and discussion and/or conclusion. Journals in biology also publish "review articles" that provide a roundup of recent research on a topic in biology. If you are looking for primary articles or review articles in biology and biomedical topics, these databases will be especially useful.

• Biological Science This link opens in a new window Covers research in all areas of biological science, including animal behavior, biomedicine, zoology, ecology, and others. Coverage is from 1982 to the present. Includes abstracts and citations, as well as access to thousands of full text titles.
• PubMed (citations from MEDLINE and other sources) This link opens in a new window PubMed contains more than 30 million citations and abstracts of biomedical literature. Click the "Find it at Gustavus" button to link to the full text or to make an interlibrary loan request. PubMed was developed and is maintained by the National Institutes of Health.
• Web of Science (Web of Knowledge) This link opens in a new window Science Citation Index Expanded, Social Sciences Citation Index, and Arts & Humanities Citation Index of the Institute for Scientific Information (ISI). Besides indexing a wide range of journals in the sciences, social sciences, and history, this resource allows you to search for articles that cite a specific author or published work. Coverage from 1997 to the present. Click on the "Web of Science" tab to limit your search to one or more specific citation databases.

Annual Reviews

These annual books publish review articles - detailed recaps of research on questions in the field. They are an excellent place to gain a sense of the various approaches to a topic and references to the literature that supports it.

Two series are shelved in the general collection under the following call numbers:

• ADVANCES IN MARINE BIOLOGY v. 1, 1963- (QH 91 .A1 A22)
• ADVANCES IN VIRUS RESEARCH v. 1, 1953- (QR 360 .A3)

Also of interest is WILDLIFE MONOGRAPHS. Current volumes are available online ; volumes from 1956 - 2009 are sheved at QL 1 .W54.

The Annual Reviews series online also includes biology-related review articles.

• Annual Review of Biochemistry
• Annual Review of Cell and Developmental Biology
• Annual Review of Ecology, Evolution, and Systematics
• Annual Review of Entomology
• Annual Review of Genetics
• Annual Review of Genomics and Human Genetics
• Annual Review of Immunology
• Annual Review of Neuroscience

Other Science Databases

• AGRICOLA This link opens in a new window Citations and abstracts for agricultural publications from the 15th century to the present, including articles from over 600 periodicals, USDA and state experiment station and extension publications, and selected books. Subjects include animal and veterinary sciences, entomology, plant sciences, food and human nutrition, and earth and environmental sciences. Many records are linked to full-text documents online. A resource of the National Agricultural Library.
• Google Scholar This link opens in a new window This search engine points toward scholarly research rather than all Web-based sources. It is stronger in the sciences than in the humanities, with social sciences somewhere in between. One interesting feature of Google Scholar is that in includes a link to sources that cite a particular item. Not all of the articles in Google Scholar are free; the library can obtain many of them for you through Interlibrary loan.

How Do I Get the Actual Articles?

If there isn't a PDF available, look for a "find it" link. That will check to see if it's available through another of our databases. If no full text is available, it will give you an opportunity to request the article from another library. You will have to log in using your Gustavus username and password. It usually takes a day or two. Look for an email that will explain how to download the PDF. 

If you're using Google Scholar, look for either a "find it @ Gustavus" link to the right or a "more" link under the reference you're interested in.  

• << Previous: Start
• Next: Find Books and Background Information >>
• Last Updated: Feb 15, 2024 3:49 PM
• URL: https://libguides.gustavus.edu/BIO

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Med Res Methodol

Primary versus secondary source of data in observational studies and heterogeneity in meta-analyses of drug effects: a survey of major medical journals

Guillermo prada-ramallal.

1 Department of Preventive Medicine and Public Health, University of Santiago de Compostela, c/ San Francisco s/n, 15786 Santiago de Compostela, A Coruña, Spain

2 Health Research Institute of Santiago de Compostela (Instituto de Investigación Sanitaria de Santiago de Compostela - IDIS), Clinical University Hospital of Santiago de Compostela, 15706 Santiago de Compostela, Spain

Fatima Roque

3 Research Unit for Inland Development, Polytechnic of Guarda (Unidade de Investigação para o Desenvolvimento do Interior - UDI/IPG), 6300-559 Guarda, Portugal

4 Health Sciences Research Centre, University of Beira Interior (Centro de Investigação em Ciências da Saúde - CICS/UBI), 6200-506 Covilhã, Portugal

Maria Teresa Herdeiro

5 Department of Medical Sciences & Institute for Biomedicine – iBiMED, University of Aveiro, 3810-193 Aveiro, Portugal

6 Higher Polytechnic & University Education Co-operative (Cooperativa de Ensino Superior Politécnico e Universitário - CESPU), Institute for Advanced Research & Training in Health Sciences & Technologies, 4585-116 Gandra, Portugal

Bahi Takkouche

7 Consortium for Biomedical Research in Epidemiology & Public Health (CIBER en Epidemiología y Salud Pública – CIBERESP), Santiago de Compostela, Spain

Adolfo Figueiras

Associated data.

All data generated or analysed during this study are included in this published article.

The data from individual observational studies included in meta-analyses of drug effects are collected either from ad hoc methods (i.e. “primary data”) or databases that were established for non-research purposes (i.e. “secondary data”). The use of secondary sources may be prone to measurement bias and confounding due to over-the-counter and out-of-pocket drug consumption, or non-adherence to treatment. In fact, it has been noted that failing to consider the origin of the data as a potential cause of heterogeneity may change the conclusions of a meta-analysis. We aimed to assess to what extent the origin of data is explored as a source of heterogeneity in meta-analyses of observational studies.

We searched for meta-analyses of drugs effects published between 2012 and 2018 in general and internal medicine journals with an impact factor > 15. We evaluated, when reported, the type of data source (primary vs secondary) used in the individual observational studies included in each meta-analysis, and the exposure- and outcome-related variables included in sensitivity, subgroup or meta-regression analyses.

We found 217 articles, 23 of which fulfilled our eligibility criteria. Eight meta-analyses (8/23, 34.8%) reported the source of data. Three meta-analyses (3/23, 13.0%) included the method of outcome assessment as a variable in the analysis of heterogeneity, and only one compared and discussed the results considering the different sources of data (primary vs secondary).

Conclusions

In meta-analyses of drug effects published in seven high impact general medicine journals, the origin of the data, either primary or secondary, is underexplored as a source of heterogeneity.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0561-3) contains supplementary material, which is available to authorized users.

Specific research questions are ideally answered through tailor-made studies. Although these ad hoc studies provide more accurate and updated data, designing a completely new project may not represent a feasible strategy [ 1 , 2 ]. On the other hand, clinical and administrative databases used for billing and other fiscal purposes (i.e. “secondary data”) are a valuable resource as an alternative to ad hoc methods (i.e. “primary data”) since it is easier and less costly to reuse the information than collecting it anew [ 3 ]. The potential of secondary automated databases for observational epidemiological studies is widely acknowledged; however, their use is not without challenges, and many quality requirements and methodological pitfalls must be considered [ 4 ].

Meta-analysis represents one of the most valuable tools for assessing drug effects as it may lead to the best evidence possible in epidemiology [ 5 ]. Consequently, its use for making relevant clinical and regulatory decisions on the safety and efficacy of drugs is dramatically increasing [ 6 ]. Existence of heterogeneity in a given meta-analysis is a feature that needs to be carefully described by analyzing the possible factors responsible for generating it [ 7 ]. In this regard, the results of a recent study [ 8 ] show that whether the origin of the data (primary vs secondary) is explored as a potential cause of heterogeneity may change the conclusions of a meta-analysis due to an effect modification [ 9 ]. Thus, considering the source of data as a variable in sensitivity and subgroup analyses, or meta-regression analyses, seems crucial to avoid misleading conclusions in meta-analyses of drug effects.

Given the evidence noted [ 8 , 9 ], we surveyed published meta-analyses in a selection of high-impact journals over a 6-year period, to assess to what extent the origin of the data, either primary or secondary, is explored as a source of heterogeneity in meta-analyses of observational studies.

Meta-analysis selection and data collection process

General and internal medicine journals with an impact factor > 15 according to the Web of Science were included in the survey [ 10 ]. This method has been widely used to assess quality as well as publication trends in medical journals [ 11 – 13 ]. The rationale is that meta-analyses published in high impact journals: (1) are likely to be rigorously performed and reported due to the exhaustive editorial process [ 12 , 14 ]; and, (2) in general, exert a higher influence on medical practice due to the major role played by these journals in the dissemination of the new medical evidence [ 14 , 15 ]. We searched MEDLINE on May 2018 using the search terms “meta-analysis” as publication type and “drug” in any field between January 1, 2012 and May 7, 2018 in the New England Journal of Medicine ( NEJM ), Lancet, Journal of the American Medical Association ( JAMA) , British Medical Journal ( BMJ ), JAMA Internal Medicine (JAMA Intern Med) , Annals of Internal Medicine ( Ann Intern Med ), and Nature Reviews Disease Primers (Nat Rev Dis Primers) .

Two investigators (GP-R, FR) independently assessed publications for eligibility. Abstracts were screened and if deemed potentially relevant, full text articles were retrieved. Articles were excluded if they met any of the following conditions: (1) were not a meta-analysis of published studies, (2) no drug effects were evaluated, (3) only randomized clinical trials were included in the meta-analysis (in order to consider observational studies), (4) less than two observational studies were included in the meta-analysis (since with a single study it would not have been possible to calculate a pooled measure). When a meta-analysis included both observational studies and clinical trials, only observational studies were considered.

A data extraction form was developed previously to extract information from articles. Two investigators (GP-R, FR) independently extracted and recorded the information and resolved discrepancies by referring to the original report. If necessary, a third author (AF) was asked to resolve disagreements between the investigators.

When available we extracted the following data from each eligible meta-analysis: first author, publication year, journal, drug(s) exposure and outcome(s); number of individual studies included in the meta-analysis based on each type of data source used (primary vs secondary), for both exposure and outcome assessment; and exposure- and outcome-related variables included in sensitivity, subgroup or meta-regression analyses. We extracted data directly from the tables, figures, text, and supplementary material of the meta-analyses, not from the individual studies.

Assessment of exposure and outcome

We considered “primary data” the information on drug exposure collected directly by the researchers using interviews –personal or by telephone– or self-administered questionnaires. The origin of the data was also considered primary when objective diagnostic methods were used for the determination of drug exposure (e.g. blood test). “Secondary data” are data that were formerly collected for other purposes than that of the study at hand and that were included in databases on drug prescription (e.g. prescription registers, medical records/charts) and dispensing (e.g. computerized pharmacy records, insurance claims databases). Regarding the outcome assessment, we considered primary data when an objective confirmation is available that endorses them (e.g. confirmed by individual medical ad hoc diagnosis, lab test or imaging results). These criteria are based on those commonly used in the risk assessment of bias for observational studies [ 16 – 19 ].

MEDLINE search results yielded 217 articles from the major general medical journals (3 from NEJM , 46 from Lancet , 26 from JAMA , 85 from BMJ , 19 from JAMA Intern Med, 38 from Ann Intern Med, and 0 from Nat Rev Dis Primers ) (see Fig. ​ Fig.1). 1 ). A total of 194 articles were excluded (see list of excluded articles with reasons for exclusion in Additional file 1 ) leaving 23 articles to be examined [ 20 – 42 ]. General characteristics of the 23 included meta-analyses are outlined in Table ​ Table1 1 .

Flow diagram of literature search results

Characteristics of the 23 included meta-analyses

Abbreviations : AABs antibodies against biologic agents, ACEIs , angiotensin converting enzyme inhibitors, Ann Intern Med Annals of Internal Medicine , ARBs angiotensin receptor blockers, BMJ British Medical Journal , DPP-4 Dipeptidyl Peptidase-4, GLP-1 glucagon like peptide-1, JAMA Journal of the American Medical Association , MIC minimum inhibitory concentration, NSAIDs non-steroidal anti-inflammatory drugs, SGLT-2 sodium–glucose cotransporter 2, SSRIs selective serotonin reuptake inhibitors

Source of exposure and outcome data

Table ​ Table2 2 summarizes the evidence regarding the type of data source included in each meta-analysis, according to the information presented in the data extraction tables of the article. The information was evaluated taking the study design into account. Only eight meta-analyses [ 21 , 24 , 26 , 31 , 32 , 34 , 38 , 41 ] reported the source of data, three of them [ 31 , 34 , 38 ] reporting mixed sources for both the exposure and outcome assessment. Five meta-analyses [ 21 , 24 , 26 , 32 , 41 ] reported only secondary sources for the exposure assessment, three of them [ 21 , 24 , 41 ] reporting as well only secondary sources for the outcome assessment, while in the other two [ 26 , 32 ] only primary and mixed sources for the outcome assessment were reported respectively.

Reporting of the data source in the data extraction tables of the included meta-analyses

Abbreviations : 1ry number of individual studies in each MA based on primary data sources, 2ry number of individual studies in each MA based on secondary data sources, NR number of individual studies in each MA with not reported data source

a Although the meta-analysis shows the results of methodological quality assessment based on a standardized scale, it does not indicate the type of data source used for each individual observational study included in the meta-analysis

b Cohort with nested case-control analysis

c The meta-analysis reports that most of the included observational studies assessed medication exposure through a review of medical records

d The meta-analysis reports only data from high-quality observational studies

Source of data in the analysis of heterogeneity

All but two [ 20 , 42 ] of the meta-analyses performed subgroup and/or sensitivity analyses. Although three of them [ 23 , 34 , 36 ] considered the methods of outcome assessment – type of diagnostic assay used for Clostridium difficile infection, method of venous thrombosis diagnosis confirmation, and type of scale for psychosis symptoms assessment respectively– as stratification variables, only the second referred to the origin of the data. Only five meta-analyses [ 22 , 28 , 33 , 35 , 39 ] included meta-regression analyses to describe heterogeneity, none of which considered the source of data as an explanatory variable. Other findings for the inclusion of the data source as a variable in the analysis of heterogeneity are presented in Table ​ Table3 3 .

Inclusion of the data source as a variable in the analysis of heterogeneity of the included meta-analyses

Abbreviations : APACHE acute physiology and chronic health evaluation, MIC minimum inhibitory concentration, SSRIs selective serotonin reuptake inhibitors, TNF tumor necrosis factor

We finally assessed if the influence of the data origin on the conclusions of the meta-analyses was discussed by their respective authors. We found that only four meta-analyses [ 21 , 31 , 32 , 34 ] noted limitations derived from the type of data source used.

The findings of this research suggest that the origin of the data, either primary or secondary, is underexplored as a source of heterogeneity and an effect modifier in meta-analyses of drug effects published in general medicine journals with high impact. Few meta-analyses reported the source of data and only one [ 34 ] of the articles included in our survey compared and discussed the meta-analysis results considering the different sources of data.

Although it is usual to consider the design of the individual studies (i.e. case-control, cohort or experimental studies) in the analysis of the heterogeneity of a meta-analysis [ 43 , 44 ], the type of data source (primary vs secondary) is still rarely used for this purpose [ 9 , 45 ]. In fact, the current reporting guidelines for meta-analyses, such as MOOSE (Meta-analysis Of Observational Studies in Epidemiology) [ 18 ] or PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [ 46 , 47 ], do not recommend that authors specifically report the origin of the data. This is probably due to the close relationship that exists between the study design and the type of data source used, despite the fact that each criterion has its own basis. Performing this additional analysis is a simple task that involves no additional cost. Failure to do so may lead to diverging conclusions [ 8 ].

Conclusions about the effects of a drug that are derived from studies based exclusively on data from secondary sources may be dicey, among other reasons, because no information is collected on consumption of over-the-counter drugs (i.e. drugs that individuals can buy without a prescription) [ 48 ] and/or out-of-pocket expenses for prescription drugs (i.e. costs that individuals pay out of their own cash reserves) [ 49 ]. In the health care and insurance context, out-of-pocket expenses usually refer to deductibles, co-payments or co-insurance. Figure ​ Figure2 2 shows the model that we propose to describe the relationship between the different data records according to their origin, including the possible loss of information (susceptible to be registered only through primary research).

Conceptual model of individual data recording. * Never dispensed. † Absence of dispensing of successive prescriptions (or self-medication) among patients with primary adherence, or inadequate secondary adherence

Failure to take these situations into account may lead to exposure measurement bias [ 48 , 49 ]. Consumption of a drug may be underestimated when only prescription data is used as secondary source without additionally considering unregistered consumption, such as over-the-counter consumption (e.g. oral contraceptives [ 34 , 50 ]), that may only be available from a primary database. Alternatively, this may occur when dispensing data for billing purposes (reimbursement) are used for clinical research, if out-of-pocket expenses are not considered (see Fig. ​ Fig.2). 2 ). The portion of the medical bill that the insurance company does not cover, and that the individual must pay on his own, is unlikely to be recorded. Data on the sale of over-the-counter drugs will also not be available in this scenario.

The reverse situation may also occur and consumption may be overestimated when only prescription data is used, if the prescribed drug is not dispensed by the pharmacist; or when dispensing data is used, if the drug is not really consumed by the patient. While primary non-adherence occurs when the patient does not pick up the medication after the first prescription, secondary non-adherence refers to the absence of dispensing of successive prescriptions among patients with primary adherence, or to inadequate secondary adherence (i.e. ≥20% of time without adequate medication) [ 51 ] (see Fig. ​ Fig.2). 2 ). In some diseases the medication adherence is very low [ 52 – 55 ], with percentages of primary non-adherence (never dispensed) that exceed 30% [ 56 ]. It should be noted that the impact of non-adherence varies from medication to medication. Therefore, it must be defined and measured in the context of a particular therapy [ 57 ].

Moreover, failing to take into consideration the portion of consumption due to over-the-counter and/or out-of-pocket expenses may lead to confounding , as that variable may be related to the socio-economic level and/or to the potential of access to the health system [ 58 ], which are independent risk factors of adverse outcomes of some medications (e.g. myocardial infarction [ 21 , 28 , 30 , 41 ]). Given the presence of high-deductible health plans and the high co-insurance rate for some drugs, cost-sharing may deter clinically vulnerable patients from initiating essential medications, thus negatively affecting patient adherence [ 59 , 60 ].

Outcome misclassification may also give rise to measurement bias and heterogeneity [ 61 ]. This occurs, for example, in the meta-analysis that evaluates the relationship between combined oral contraceptives and the risk of venous thrombosis [ 34 ]. In the studies without objective confirmation of the outcome, the women were classified erroneously regardless of the use of contraceptives. This led to a non-differential misclassification that may have underestimated the drug–outcome relationship, especially when the third generation of progestogen is analysed: Risk ratio (RR) primary data = 6.2 (95% confidence interval (CI) 5.2–7.4), RR secondary data = 3.0 (95% CI 1.7–5.4) [ 34 ].

On the one hand, medical records are often considered as being the best information source for outcome variables. However, they present important limitations in the recording of medications taken by patients [ 62 ]. On the other hand, dispensing records show more detailed data on the measurement of drug exposure. However, they do not record the over-the-counter or out-of-pocket drug consumption at an individual level [ 48 , 49 ], apart from offering unreliable data on outcome variables [ 62 , 63 ].

Limitations

The first limitation of this research is that its findings may not be applicable to journals not included in our survey such as journals with low impact factor. Despite the widespread use of the impact factor metric [ 64 ], this method has inherent weaknesses [ 65 , 66 ]. However, meta-analyses published in high impact general medicine journals are likely to be most rigorously performed and reported due to their greater availability of resources and procedures [ 12 , 14 ]. It is then expected that the overall reporting quality of articles published in other lesser-known journals will be similar. Another limitation would be related to the limited search period . In this sense, and given that the general tendency is the improvement of the methodology of published meta-analyses [ 67 , 68 ], we find no reason to suspect that the adverse conclusions could be different before the period from 2012 to 2018. Although it exceeds the objective of this research, one last limitation may be the inability to reanalyse the included meta-analyses stratifying by the type of data source since our study design restricts the conclusions to the published data of the meta-analyses, which were insufficiently reported , or the number of individual studies in each stratum was insufficient to calculate a pooled measure (see Table ​ Table2 2 ).

Owing to automated capture of data on drug prescription and dispensing that are used for billing and other administration purposes, as well as to the implementation of electronic medical records, secondary databases have generated enormous possibilities. However, neither their limitations, nor the risk of bias that they pose should be overlooked [ 69 ]. Thus, researchers should consider the link between administrative databases and medical records, as well as the advisability of combining secondary and primary data in order to minimize the occurrence of biases due to the use of any of these databases.

No source of heterogeneity in a meta-analysis should ever be considered alone but always as part of an interconnected set of potential questions to be addressed. In particular, the origin of the data, either primary or secondary, is insufficiently explored as a source of heterogeneity in meta-analyses of drug effects, even in those published in high impact general medicine journals. Thus, we believe that authors should systematically include the source of data as an additional variable in subgroup and sensitivity analyses, or meta-regression analyses, and discuss its influence on the meta-analysis results. Likewise, reviewers, editors and future guidelines should also consider the origin of the data as a potential cause of heterogeneity in meta-analyses of observational studies that include both primary and secondary data. Failure to do this may lead to misleading conclusions, with negative effects on clinical and regulatory decisions.

Additional file

Excluded articles. List of articles excluded with reasons for exclusion. (PDF 247 kb)

This study received no funding from the public, commercial or not-for-profit sectors.

Availability of data and materials

Abbreviations, authors’ contributions.

AF and GP-R contributed to study conception and design. GP-R, FR and AF contributed to searching, screening, data collection and analyses. GP-R was responsible for drafting the manuscript. FR, MTH, BT and AF provided comments and made several revisions of the manuscript. All authors read and approved the final version.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Competing interests.

The authors declare that they no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Guillermo Prada-Ramallal, Email: [email protected] .

Fatima Roque, Email: tp.gpi@euqorf .

Maria Teresa Herdeiro, Email: tp.au@oriedrehaseret .

Bahi Takkouche, Email: [email protected] .

Adolfo Figueiras, Phone: (+34) 981 95 11 92, Email: [email protected] .

Understanding Nursing Research

What is primary research, how can i tell if my article is "primary research", limiting your search to primary research.

• Qualitative vs. Quantitative Research
• Experimental Design
• Is it a Nursing journal?
• Is it Written by a Nurse?
• Systematic Reviews and Secondary Research
• Quality Improvement Plans

Your Team! College of Education and Human Development and College of Nursing and Health Sciences

Left to Right: Trisha Hernandez, Emily Murphy, Lorin Flores, Aida Almanza-Ferro.

We are the librarians for College of Education and Human Development, and the College of Nursing. We look forward to working with you! To contact us or to make an appointment:

Submit your request and we'll get right back to you!

Or, you can reach out directly. For our email addresses and phone numbers, see the list below:

Aida Almanza-Ferro  | [email protected] | 361-825-2356 Lorin Flores | [email protected] | 361-825-2609 Trisha Hernandez | [email protected]  |361-825-2687 Emily Sartorius Murphy  | [email protected] | 361-825-2610 Librarians are available M-F, 8-5.

The phrase "Primary" can mean something different depending on what subject you're in.

In History , for example, you might hear the phrase "primary sources." This means the researcher is looking for sources that date back to when an event occurred. Primary sources can be a diary, a photograph, or a newspaper clipping.

If this is the kind of research you're looking for, check out this research guide on how to find primary sources:

• Primary Sources

If you're in Nursing or another scientific field you're more likely to hear the phrase "Primary Research."

Primary Research refers to research that was conducted by the author of the article you're reading. So if you're reading an article and in the methodology section the author refers to recruiting participants, identifying a control group, etc. you can be pretty sure the author has conducted the research themselves.

When you're asked to find primary research, you're being asked to find articles describing research that was conducted by the authors.

Check out the video below for an explanation of the differences between primary and secondary research.

To determine if the article you're looking at is considered Primary Research, look for the following:

• In the Abstract, can you find a description of research being conducted?
• Were participants recruited?
• Were surveys distributed?

The main question to ask yourself is "Did the author conduct research, or did they read and synthesize other people's research?"

If you've found an article in CINAHL and you want to know if it's primary research, look under "Publication Type" to see if it's a research article.

This is not always 100% correct, though. To be sure, you should always read the Methodology section to understand what kind of article you're looking at.

If you're using PubMed, you can check the article's Keywords and Abstract for clues to see if the article is primary research, like in the article below:

Or you can check to see if the article includes a "Publication Type" section like this article:

The following Publication Types are usually considered Primary Research:

• Adaptive Clinical Trial
• Clinical Study
• Clinical Trial
• Controlled Clinical Trial
• Equivalence Trial
• Evaluation Studies
• Observational Study
• Pragmatic Clinical Trial
• Randomized Controlled Trial

Remember, you will always need to read the Methodologies section of an article to be sure the article is an example of primary research!

In certain databases you can specify that you're only interested in resources that are considered primary research.

Two of those databases are CINAHL and PubMed, which you can access here:

To limit your results to primary research in CINAHL, check the "Research Article" box on the homepage before you hit "Search"

This check box is helpful, but it isn't 100% correct, so always read the Methodology section of your article to determine what kind of article it is!

If you're conducting a search in PubMed and want to limit your results to a certain kind of article, you can enter your search terms on the homepage and click "Search."

Then, when you're on your results page, use the limiters on the left side of the screen to specify the "Article Type" you're interested in. Under "Article Types" click the "Customize..." link to see the full list of article types available to you.

Check any of the article types you're interested in (don't forget to scroll down on this list!) and then click the blue "Show" button at the bottom of the pop up window.

Now the Article Types you just selected should appear under the Article Types heading. Click on the article types you want to show up in your results list and your results will limit themselves to just those that meet your criteria.

Remember to read the article's Methodology section yourself before deciding whether or not it's Primary Research! These limits are great, but they aren't always 100% accurate.

• Next: Qualitative vs. Quantitative Research >>
• Last Updated: Feb 6, 2024 9:34 AM
• URL: https://guides.library.tamucc.edu/nursingresearch

• Open access
• Published: 16 March 2024

Impact of reimbursement systems on patient care – a systematic review of systematic reviews

• Eva Wagenschieber 1 &
• Dominik Blunck   ORCID: orcid.org/0000-0001-8843-2411 1  

Health Economics Review volume  14 , Article number:  22 ( 2024 ) Cite this article

295 Accesses

1 Altmetric

Metrics details

There is not yet sufficient scientific evidence to answer the question of the extent to which different reimbursement systems influence patient care and treatment quality. Due to the asymmetry of information between physicians, health insurers and patients, market-based mechanisms are necessary to ensure the best possible patient care. The aim of this study is to investigate how reimbursement systems influence multiple areas of patient care in form of structure, process and outcome indicators.

For this purpose, a systematic literature review of systematic reviews is conducted in the databases PubMed, Web of Science and the Cochrane Library. The reimbursement systems of salary, bundled payment, fee-for-service and value-based reimbursement are examined. Patient care is divided according to the three dimensions of structure, process, and outcome and evaluated in eight subcategories.

A total of 34 reviews of 971 underlying primary studies are included in this article. International studies identified the greatest effects in categories resource utilization and quality/health outcomes. Pay-for-performance and bundled payments were the most commonly studied models. Among the systems examined, fee-for-service and value-based reimbursement systems have the most positive impact on patient care.

Patient care can be influenced by the choice of reimbursement system. The factors for successful implementation need to be further explored in future research.

The health care system has a variety of payment and reimbursement systems that provide different financial incentives for patient care. Every payment system carries incentives to over- or underprovide care. There is no optimal solution, as there is constant pressure to adapt and reform in order to ensure the best possible quality of care. Health care systems are reaching their financial limits and therefore it is desirable to achieve an increase in efficiency in patient treatment and, for example, to avoid unnecessary interventions [ 1 ]. To achieve this, health policy must ensure a regulatory framework in which health status is also an economic incentive for all actors in the health system, promoting health benefits and reducing economic disincentives.

Physicians have a stronger position in the physician–patient relationship because of the knowledge and information advantage, and problems arise in the provision of care when physicians’ financial interest do not match the patients’ need for treatment [ 2 ]. In addition to medical necessity, economic and financial factors also play a key role in patient treatment. Medical decisions in the inpatient sector are influenced daily by economic requirements, economic considerations, and financial resources, potentially with negative consequences for the quality of treatment and patient safety. In the hospital setting, economization is exemplified in that physicians often feel ethical conflicts and economic goals occur at the expense of adjustments in length of stay, case numbers, and patient selection [ 3 ]. The influence on patient care is examined under four different reimbursement systems: Salary, bundled payment, fee-for-service (FFS), value-based reimbursement. With a fixed salary, remuneration is based solely on the duration of working hours, whereas the type and volume of service, as well as the number of treatment cases or patients enrolled, have no influence on financial income. At the same time, both an advantage and a disadvantage in this reimbursement system is the dependence of the quality of treatment on the intrinsic motivation of the provider [ 2 ]. Bundled payment is the term for payments such as capitation or disease related groups (DRGs). Services are combined and “bundled” for payment during a single patient contact or over a temporal episode. One disadvantage of this reimbursement system is the incentive for health care providers to treat as many patients as possible with as little effort as possible and thus to engage in risk selection. On the other hand, this can increase the incentive for preventive measures on the part of health care providers [ 4 ]. In FFS reimbursement, the provider’s fee is based on the volume of services rendered. Shared-savings payment models are a mix of FFS and a fixed salary where providers participate from savings they achieve in patient care. This creates the disadvantage of FFS reimbursement that service providers will unnecessarily expand the number of services for monetary reasons, resulting in unnecessary care at the expense of payers and potentially patients. On the other hand, (potentially expensive) diseases can be identified and treated earlier through increased preventive measures [ 2 , 5 ]. Value-based reimbursement additionally promotes the quality and success of medical procedures. Remuneration is expanded to the extent that it is linked to predefined quality targets at the levels of transparency, accessibility to care, indication, structure, process or outcome. While value-based reimbursement can promote the intrinsic motivation of providers, care must be taken to ensure that there is no risk selection for patients who can be treated well or that there are no negative spill-over effects into other areas of treatment. Another disadvantage of this reimbursement system is the large number of factors besides medical treatment that contribute to recovery, such as comorbidities or socioeconomic factors [ 1 ].

Other reviews have addressed effects on patient care in outpatient settings [ 6 ] or included studies from developing countries in their evaluations [ 7 ]. Previous studies only focus on specific areas of patient care [ 8 ], are not methodologically designed as a systematic review [ 9 ], focus only on individual specialties [ 10 ] or reimbursement systems [ 11 ] and do not compare the effect of different reimbursement systems. A comprehensive and structured overview, comparing the outcomes of several reimbursement systems on areas of patient care, is missing.

The objective of this paper, thus, is to provide a review of systematic reviews on the relationship between reimbursement systems and patient care. The research question is narrowed down using the PICOS algorithm: Physicians (Population), Reimbursement systems (Intervention), different reimbursement systems or differences over time (Comparison), effects on patient care divided into the parameters structure, process, outcome (Outcome), systematic reviews and meta-analyses (Study type). The aim is to analyze how reimbursement systems affect patient care across countries.

Materials and methods

The systematic review follows the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Metaanalyses) statement [ 12 ], has been performed via the databases PubMed, Web of Science and Cochrane Database of Systematic Reviews between 02/12/2021 and 22/12/2021 and has been complemented with an additional search on Google Scholar and in the reference lists of relevant studies. The search term was formed by linking keywords and their synonyms from previously published relevant studies on the three aspects of the research questions: impact, reimbursement systems, and patient care (see Table 1 for the full search term for each database).

Inclusion criteria are defined as (a) the paper must be a systematic review or meta-analysis, (b) the countries considered must be industrialized nations, and (c) the effect of payment/reimbursement systems on patient care was examined.

The search period is set to ten years and only studies published in German or English were included. All records were exported to EndNote 20 [ 13 ] and screened by the authors; disagreements were solved by discussion. All studies categorized as “relevant” or “uncertain” in this step were analyzed in full text.

Studies categorized as relevant after full text analysis were included in this work and assessed for study quality using the AMSTAR-2 score, which is a comprehensive questionnaire to assess systematic reviews of (non)randomized trials [ 14 ]. Using the framework of Donabedian, the results are divided into the three dimensions structure, process, outcome [ 15 ] (see Table  2 ). The structure dimension combines the following parameters: “unintended consequences” and “organizational changes”. Unintended consequences are mostly related to changes in risk selection or spill-over effects, whereas organizational changes are related to effects in personnel structures, for example. The dimension of structure is of particular interest for health care authorities as well as payers as it shapes the organizational characteristics of how care is delivered.

The categories “resource utilization”, “access”, and “behavior” are combined under the parameter process. While resource utilization mostly describes changes in readmission rates or length of stay, the access category reflects socioeconomic inequalities in the utilization of health care services. The behavior category includes effects related to intrinsic motivation, preventive services provided by physicians, or documentation of health parameters, among others. The dimension of process defines how providers deliver care as well as the points of contacts for patients.

The outcome dimension, on the other hand, combines the parameters “quality/health outcomes”, “efficiency”, and “economic effects”. Actual changes in mortality, treatment quality, screening or vaccination rates are mapped in the “quality/health outcomes” category. The “efficiency” category deals with the effects on direct savings in the provision of a specific medical service or effects on salaries, whereas the “economic effects” category records effects that are significant for society. The dimension of outcome could be regarded of the main value driver from a patient perspective as it answers to what extent patients’ original need for care is fulfilled. Furthermore, outcomes are of particular interest for payers, as payers commonly decide, for example, what services are reimbursed and therefore potentially have a high interest in a positive cost-outcome-relation.

For all reimbursement systems described, the number of included studies, as well as the examined medical specialties or physician groups and countries in which the interventions are carried out, are also transferred in each case. For each reimbursement system described, it is examined whether it improved or worsened the outcome categories of patient care, whether there were heterogeneous results, or whether no difference was found in the outcome categories before and after the intervention. The frequency reviews found an improvement, worsening, heterogeneous outcome, or no difference for each payment system per outcome category were summarized in a single table. In this study, increases in healthcare utilization, documentation of health parameters, and higher screening rates or lower mortality rates are defined as improvements. A measurable increase in risk selection, negative spill-over effects, longer hospital stays, or higher readmission rates are considered deteriorations in patient care. In the economic categories of efficiency and economic effects, savings in health care spending and total societal spending, respectively, are considered as improvements. Reviews finding heterogeneous results include studies with conflicting findings, because some of the included primary studies find positive results in one category, whereas other primary studies find negative effects or no significant effects at all, leaving the study or respective review with an overall heterogeneous result. It is assumed that health care is optimized by an increase in health care services, shorter lengths of stay, more efficient care, and lower overall societal health care expenditures.

A total of 1,213 hits were identified by the database search on 02/12/2021, with 2 additional hits identified by the search in Google Scholar. After duplicates were removed, 1,053 abstracts were screened by both authors, resulting in 943 hits being initially excluded. The remaining 110 hits were analyzed in full text, whereupon 34 hits were included in this work (see Fig.  1 ).

Overall, the 34 included systematic reviews describe the influences on patient care based on a total of 971 primary studies. Ten of the 34 included reviews are rated as high quality, 16 as moderate quality, and eight as low quality according to the assessment procedure using the AMSTAR-2 questionnaire (see Table  3 ). Some of the identified systematic reviews examined more than one reimbursement system. Therefore, for the sake of clarity, we refer to a total number of 60 studies in the following. Of these, the reimbursement system salary was investigated in four studies, bundled payments in 15, FFS payments in a further eleven studies and value-based reimbursement in a total of 29 studies. Out of the 60 studies 45 were conducted in the USA, 38 in European countries, 28 in the UK, 23 in other countries and 17 in Canada. An overview of the results is provided in Table  4. . In the following, we describe the results of the systematic review regarding Donabedian’s categories of quality: structure, process, and outcome.

Unintended consequences

No unintended consequences in patient care are found for the salary payment system. Studies find heterogeneous results for this category for bundled payments in form of a decrease in treatment volume while there is an increase in risk selection and case complexity [ 16 , 17 ]. An association was found between bundled payments and patient selection based on sociodemographic factors and comorbidities [ 16 ]. Positive changes were noted in indicators that were not included in the FFS model; these were, however, only short-term [ 18 ]. Some reviews find unintended changes after implementation of pay-for-performance models (P4P), a type of value-based reimbursement, in form of risk selection, spill-over effects, protocol-driven and less patient-centered care and neglect of non-incentive indicators [ 19 , 20 , 21 , 22 , 23 ]. Some studies find no evidence for a change in patient risk selection in their included primary studies [ 24 , 25 ].

Organizational changes

There are heterogeneous results on the impact on patient care after the introduction of different payment systems. One study reports effects in the form of increasing numbers of physicians per patient and decreasing numbers for bundled payments [ 26 ]. While one review finds heterogeneous results for salary, bundled payment, FFS, and value-based payment for the structural organization of patient care [ 27 ], others find both positive and negative effects for value-based payment as an improvement in care management processes or a worse organization of large hospitals [ 28 , 29 ].

Resource utilization

Reviews find heterogeneous effects for salary models differentiated by specialty. While induction time and total treatment time increase in anesthesiology, outpatient visits and surgical procedures decrease in gynecology [ 30 ]. When salary and FFS payments are combined, a decrease in clinical services per year and in hospital readmissions is noted [ 27 , 30 ]. Within models of bundled payments, heterogeneous results are found: While one source describes a decline in all-cause hospitalizations and readmissions [ 30 ], other sources find both improvements and deterioration in hospital facility use and the number of acute admissions [ 27 , 31 ]. Deteriorations are described in the following categories: use of patient care resources, number of services provided per patient, shorter lengths of stay, discharges to post-hospital facilities [ 16 , 18 , 24 , 30 , 32 ]. Some reviews find both differences and no differences in the use of health care resources after the introduction of bundled payments [ 17 , 27 , 30 ]. Within DRG models, evidence is heterogeneous and describes no change, an increase, or a decrease in hospital readmissions and in the length of stay [ 26 , 33 ]. For global-based payment, evidence is heterogeneous in terms of higher or lower utilization, and no change in resource utilization [ 34 ]. The heterogeneity of influences on health care resource utilization continues for FFS payments as sources find an increase in the number of physician visits per patient [ 18 , 24 ], a reduction in length of stay and computer tomography exams [ 30 , 31 , 32 ] or heterogeneous results for process indicators [ 27 ]. Negative effects include an increase in the number of patients per physician [ 35 ]. In P4P models, six studies report an improvement in resource utilization as an increase in health care services, physician visits and a shorter length of stay [ 20 , 25 , 28 , 32 , 36 , 37 ]. Other reviews come to very heterogeneous results regarding the change in resource utilization after the introduction of P4P models in the following categories: health care and resource utilization, length of stay, readmission rates, process indicators [ 10 , 11 , 27 , 38 ].

There is no research showing an impact of salary on access to health care. Bundled payments show heterogeneous results in form of changes of the patient structure with respect to insured status or a decline in patients with home dialysis [ 17 , 30 ]. Studies examining FFS payment may also measure the impact on access to care. Improvements are noted in waiting time and a reduction of patients who leave the health care provider without treatment [ 30 , 35 ]. No differences were found in the treatment of social or ethnic inequalities [ 18 , 24 ]. For value-based models, results are heterogeneous regarding the impact on access to patient care. Among them, three studies identify a positive impact after the introduction of P4P models in form of an increase in equity of access to care and a decrease in social inequalities [ 20 , 32 , 36 ]. Other results show no significant reduction in access for disadvantaged groups or no improved access to primary care [ 11 , 19 ].

Salary models lead to a decrease in hours worked per week [ 30 ]. For bundled payment models, the results show both increases and decreases, means heterogeneous results, in the number of preventive consultations [ 18 ] and services as well as increases in preventive consultations, reported illness severity and referral to post-acute care facilities after hospitalization [ 24 , 33 ]. An increased number of services provided were reported for FFS models [ 18 , 24 , 35 ]. Positive changes after the introduction of P4P models were noticed in some categories: increased use of computers and documentation of care, diabetes tests, physician behavior [ 11 , 20 , 35 , 39 ]. Other reviews find results that are more heterogeneous on effects on the behavior in patient care [ 10 , 22 , 23 , 25 , 36 , 40 ]. For example, an improved data collection leads to increased pressure on physicians and thereby provoke negative behavior change [ 36 ]. General heterogeneous effects in terms of a disruption of patient-centered care with less focus on patient needs are reported as well as an increase in blood pressure checks and an improvement in intrinsic motivation among care providers [ 10 , 23 , 25 , 40 ]. Both, an increase and no change in medication prescription is found in two value-based models [ 10 , 41 ].

Quality/health

One review finds a decrease in transfer rates out of hospitals for a salary-based payment [ 30 ]. The results for bundled payments are heterogeneous [ 18 , 27 , 30 , 31 ]. Heterogeneous results, which means improvements as well as decreases and no changes are found within the primary studies in the reviews for mortality, rehospitalization rates, quality of care and numbers of treatment cases [ 16 , 27 , 30 , 31 , 42 ]. Some reviews notice an improvement in the quality and number of screenings [ 30 , 42 ] or a decrease in the case complexity [ 16 ]. Evidence of the impact on quality of care and health outcomes associated with P4P is also examined in reviews. One review reports improvement in terms of an increase in immunization rates among children for FFS payments [ 35 ], whereas other sources find increases, decreases and no changes in number of treatment cases, treatment outcomes, mortality, and hospitalization rates [ 18 , 27 , 31 ]. The most influences on health outcomes or quality of care are found in models of value-based payment. Nine reviews find evidence of improvement with P4P models in these categories: immunization rates [ 35 , 43 ], specific clinical values (e.g., cholesterol, blood pressure, screening rates, birth weight) [ 21 , 39 , 42 , 44 ], quality of care [ 23 , 28 , 45 ]. Heterogeneous outcomes are found in another ten reviews [ 11 , 19 , 20 , 22 , 27 , 36 , 38 , 40 , 46 , 47 ]. Among these, positive as well as negative results are found in patient-related health outcomes [ 19 , 27 ], complication rates [ 38 ], health outcomes, quality of care and screening rates [ 22 , 47 ]. Other sources report heterogeneous effects in patient satisfaction, short-term health outcomes and mortality [ 20 , 22 , 40 , 47 ]. No effects on mortality, quality of care, health outcomes, rehospitalization or patient satisfaction after an implementation of value-based reimbursement are described in six reviews [ 11 , 20 , 31 , 37 , 38 , 46 ].

When providers are reimbursed with fixed salaries in combination with FFS elements, the annual salary increases [ 30 ]. Bundled payments have a positive impact on the efficiency in terms of a decrease in health care spending and hospitalizations [ 16 , 30 , 42 ]. Furthermore, heterogeneous results, means deterioration as well as improvement, in treatment costs are described in one review [ 26 ]. Shared-savings models were found to lead to a reduction in perinatal care spending [ 42 ]. An improvement in the cost-effectiveness of treatments in P4P models by reducing costs was found in one review [ 19 ]. Other sources present heterogeneous results in terms of both positive and negative effects on the (marginal) costs of care [ 29 , 38 ]. No evidence for changes in efficiency are determined in three other reviews [ 20 , 22 , 45 ].

Economic effects

For bundled payments, the results are very heterogeneous. Cuts in health spending as well as increases, no changes or unclear effects are noted [ 31 , 32 , 34 ]. When payment is based on FFS models, positive effects on health care spending are most often found [ 18 , 32 ]. One study, however, reports heterogeneous effects [ 31 ]. The results on the impact of value-based payment models on economic conditions are mostly positive, as they lead to a reduction in the growth of health care spending and costs [ 32 , 41 , 44 ].

Principal results

To answer the question of the relationship of different reimbursement systems and patient care, we conducted a systematic review of systematic reviews in order to structure the existing body of evidence in this topic. We identified 34 studies analyzing 60 reimbursement systems and structured the results from the perspective of the Donabedian framework.

For the reimbursement of health care providers via salary, the results show little to no influence on the subcategories of the dimension structure. For the dimension process, the results are heterogeneous with a tendency toward deterioration, manifested in a reduction in services rendered and hours worked. The classic disincentives of salary-based reimbursement, minimization of the quantity of services and treatments, are confirmed in the results. The categories of the outcome dimension, on the other hand, are clearly improved, with a decrease in hospital discharge rates and an increase in income. The certainty of these results is high due to the high study quality and the risk of bias is low, since three high-quality studies and one medium-quality study were included in the evaluation.

The studies on bundled payments show few and heterogeneous effects on the structural dimension of patient care. The resource utilization subcategory shows heterogeneous results, with most results being equally positive and negative. The remaining categories in the process dimension appear to have mostly heterogeneous effects. Overall, bundled payments are found to have more positive effects on patient care in the outcome dimension categories. The disincentives of bundled payments are confirmed in the form of reductions in services, but also refuted in the form of shorter lengths of stay and lower readmission rates in hospitals. When interpreting the results, the rather below-average study quality must be considered. Although five high-quality reviews examine the effects of the bundled payments, eight reviews with a medium quality and four papers with a low quality are also included in the evaluations, so that the certainty of results is limited and there is a risk of bias.

In the results for FFS models, especially the categories in the dimension process tend to be positively affected. While access to health care and provider behavior tend to be mostly positive, there are as many heterogeneous and negative effects for resource utilization as positive ones. Measured health impact is very heterogeneous and tend to be negative, while efficiency and economic impacts tend to be improved. An increase in the number of health care services, a classic disincentive, is directly confirmed by several studies. The quality of the included reviews and, thus, also the certainty of results tends to be high, since seven reviews with a low risk of bias, four with a medium and only one review with a high risk of bias are included in the evaluation.

For models of value-based reimbursement, results are inconclusive or more negative with respect to subcategories of the structural dimension, noting changes in risk selection, negative spillover effects, and a shift away from patient-centered care [ 19 , 20 , 21 , 22 , 23 ]. In contrast, these payment models achieve substantial improvements in the process dimension and specifically in resource utilization. Although the effects on health outcomes are heterogeneous for P4P models, they indicate a clear tendency toward improvement, whereas no clear improvements or deteriorations were found for the other two subcategories. The misaligned incentives of value-based payment in the form of patient selection described at the beginning are both confirmed [ 21 , 22 , 23 ] and refuted [ 24 , 25 ]. The quality of the included reviews and thus also the certainty of results is average overall. Although seven of the relevant reviews are of high quality, 15 have a medium risk and seven have a high risk of bias, which may affect the results.

Overall, the rate of identified improvements for FFS and VBP is the best compared to heterogeneous effects, deteriorations, or no identified changes. While about 50% of all identified results for FFS show improvements, it is 40% for VBP. On the other hand, only 25% of the identified outcomes for a salary are improvements and 21% for bundled payment. Across all reimbursement systems, most of the results were identified in the categories resource utilization and quality/health outcome. Especially the categories of the process and outcome dimension, specifically the subcategories resource utilization and health outcome are influenced by the choice of reimbursement models and cause a change in patient care. These categories therefore have a greater impact on the overall results than categories in which fewer results have been identified. Mainly models of bundled and value-based reimbursement are affected. The effects of FFS and value-based reimbursement are mostly positive in the results compared to the other two reimbursement systems. Both payment models tend to show positive effects in the categories of the process and outcome dimension, and cite an increase in health care services provided, a reduction in length of stay, an increase in screening rates of patients, and an improvement in health parameters. In the case of value-based reimbursement, however, many endpoints were found to have no or very heterogeneous effects following the introduction of these reimbursement models. Primarily, these endpoints are unintended consequences, resource use, behavior, health outcomes, and efficiency. Bundled payment models show more heterogeneous and more negative than positive results. These are found predominantly in the resource utilization and health outcome categories, indicating a more positive impact of FFS and value-based compensation. Salary receives heterogeneous results, with categories in the process dimension tending to worsen and those in the outcome dimension tending to improve. Although the disincentives of the respective reimbursement systems are confirmed for all models, refutations are found for bundled and value-based reimbursement regarding length of stay, readmission rates, negative spill-over effects and patient selection.

Implication

In particular, the categories of the process and outcome dimension, more precisely defined as the subcategories resource utilization and quality/health outcome, are reported to be influenced by the choice of reimbursement model and cause a change in patient care. Models of bundled and value-based reimbursement seem to be particularly affected. The effects are more positive for FFS and value-based reimbursement in comparison to both other reimbursement systems. FFS as well as VBP models show positive effects in the process and outcome dimension categories, frequently citing an increase in health care services provided, a reduction in length of stay, an increase in patient screening rates, and an improvement in health parameters. Judging by the results and comparison of the four reimbursement systems, it is therefore worthwhile to further expand models of FFS and value-based reimbursement in the health care system and to investigate their successful implementation as well as potential moderating factors.

Limitations

There are some limitations in this review. The AMSTAR-2 tool is only partly appropriate to evaluate the reviews because it also evaluates clinical studies and therefore might underestimate the actual quality of some reviews involved. Not all of the included reviews provide a clear definition of their view on improvement or deterioration of care. Individual primary studies may be integrated into the results of several studies of included reviews and have a greater influence on the analysis than other primary studies included in only one review which bears the risk of overestimation of certain results. When interpreting the results, it is important to note that FFS or P4P models cannot be applied to any health care system; rather, the exact conditions for successful implementation must be individually and critically examined. Finally, publication bias is a limitation and can lead to overrepresentation of improvements due to the implementation of the described reimbursement models. Future studies should also identify more relevant databases to increase the quality of the systematic review and the validity of the results. Additionally, future studies should analyze the monetarization of the effects and aim for a better comparability of study settings as difficulties arise from interpreting health policy analyses which were conducted in different settings as well as causal interpretation might be limited as most underlying studies were not conducted as randomized controlled trials.

Availability of data and materials

No new data generated/Not applicable.

Abbreviations

• Fee-for-service
• Pay-for-performance

Klauber J, Geraedts M, Friedrich J, Wasem J, Beivers A. Krankenhaus-Report 2020. Finanzierung und Vergütung Am Scheideweg. Berlin: Springer; 2020.

Book   Google Scholar  

Gerlinger, T.: Grundprobleme der Vergütung ärztlicher Leistungen. https://www.bpb.de/themen/gesundheit/gesundheitspolitik/252093/grundprobleme-der-verguetungaerztlicher-leistungen/ . Accessed: 2022–03–12 (2017).

Siewert AC, Wehkamp K-H, Krones CJ, Vogd W, Allemeyer E. Bewerbungsgespräche von Chefärzten: Ökonomie hat hohen Stellenwert. Dtsch Arztebl International. 2021;118(4):180–4.

Google Scholar  

Hussey, P.S., Mulcahy, A.W., Schnyer, C., Schneider, E.C.: Closing the quality gap: revisiting the state of the science (vol. 1: bundled payment: effects on health care spending and quality). Evidence report/technology assessment (208.1), 1–155 (2012). https://doi.org/10.23970/ahrqepcerta208.1 .

Bailit M, Hughes C. Key design elements of shared-savings payment arrangements. Issue Brief (Commonw Fund). 2011;20:1–16.

PubMed   Google Scholar  

Flodgren, G., Eccles, M., Shepperd, S., Scott, A., Parmelli, E., Beyer, F.: An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane database of systematic reviews (Online) 7, 009255 (2011). https://doi.org/10.1002/14651858.CD009255 .

Scott, A., Sivey, P., Ouakrim, D., Willenberg, L., Naccarella, L., Furler, J., Young, D.: The effect of financial incentives on the quality of health care provided by primary care physicians. Cochrane database of systematic reviews (Online) 9, 008451 (2011). https://doi.org/10.1002/14651858.CD008451.pub2 .

Tao W, Agerholm J, Burström B. The impact of reimbursement systems on equity in access and quality of primary care: A systematic literature review. BMC Health Serv Res. 2016;16:542. https://doi.org/10.1186/s12913-016-1805-8 .

O’Reilly J, Busse R, Hakkinen U, Or Z, Street A, Wiley M. Paying for hospital care: The experience with implementing activity-based funding in five european countries. Health Econ Policy Law. 2012;7:73–101. https://doi.org/10.1017/S1744133111000314 .

Article   PubMed   Google Scholar  

Mitchell AP, Rotter JS, Patel E, Richardson D, Wheeler SB, Basch E, Goldstein DA. Association between reimbursement incentives and physician practice in oncology: A systematic review. JAMA Oncol. 2019;5(6):893–9. https://doi.org/10.1001/jamaoncol.2018.6196 .

Article   PubMed   PubMed Central   Google Scholar  

Gupta N, Ayles H. Effects of pay-for-performance for primary care physicians on diabetes outcomes in single-payer health systems: a systematic review. Eur J Health Econ. 2019;20. https://doi.org/10.1007/s10198-019-01097-4 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372. https://doi.org/10.1136/bmj.n71 .

Endnote. 2021;20.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomized or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358. https://doi.org/10.1136/bmj.j4008 .

Donabedian, A.: The quality of care: How can it be assessed? JAMA 260(12) (1988). https://doi.org/10.1001/jama.1988.03410120089033 .

Agarwal R, Liao JM, Gupta A, Navathe AS. The impact of bundled payment on health care spending, utilization, and quality: A systematic review. Health Aff. 2020;39(1):50–7. https://doi.org/10.1377/hlthaff.2019.00784 .

Article   Google Scholar  

Bernstein DN, Reitblat C, van de Graaf VA, O’Donnell E, Philpotts LL, Terwee CB, Poolman RW. Is there an association between bundled payments and “cherry picking” and “lemon dropping” in orthopaedic surgery? a systematic review. Clin Orthop Realt Res. 2021;479(11):2430–43. https://doi.org/10.1097/CORR.0000000000001792 .

Brocklehurst P, Price J, Glenny A, Tickle M, Birch S, Mertz E, Grytten J. The effect of different methods of remuneration on the behaviour of primary care dentists. Cochrane Database Syst Rev. 2013;11:1465–858. https://doi.org/10.1002/14651858.CD009853.pub2 .

Eijkenaar F, Emmert M, Scheppach M, Schöffski O. Effects of pay for performance in health care: A systematic review of systematic reviews. Health Policy. 2013;110(2):115–30. https://doi.org/10.1016/j.healthpol.2013.01.008 .

Gillam SJ, Siriwardena AN, Steel N. Pay-for-performance in the United Kingdom: impact of the quality and outcomes framework: a systematic review. Ann Fam Med. 2012;10(5):461–8. https://doi.org/10.1370/afm.1377 .

Langdown C, Peckham S. The use of financial incentives to help improve health outcomes: is the quality and outcomes framework fit for purpose? A systematic review. J Public Health. 2014;36(2):251–8. https://doi.org/10.1093/pubmed/fdt077 .

Lee JY, Sand-II L, Jo M-W. Lessons from healthcare providers’ attitudes toward pay-for-performance: What should purchasers consider in designing and implementing a successful program? J Prev Med Public Health. 2012;45(3):137–47. https://doi.org/10.3961/jpmph.2012.45.3.137 .

Martin, B., Jones, J., Miller, M., Johnson-Koenke, R.: Health care professionals’ perceptions of pay-for-performance in practice: A qualitative metasynthesis. The Journal of Health Care Organization, Provisions, and Financing 57 (2020). https://doi.org/10.1177/0046958020917491 .

Carter R, Riverin B, Levesque J-F, Gariepy G, Quesnel-Vallee A. The impact of primary care reform on health system performance in canada: a systematic review. BMC Health Serv Res. 2016;16:324. https://doi.org/10.1186/s12913-016-1571-7 .

Kondo KK, Wyse J, Mendelson A, Beard G, Freeman M, Low A, Kansagara D. Pay-for-performance and veteran care in the VHA and the community: a systematic review. J Gen Intern Med. 2018;33(7):1155–66. https://doi.org/10.1007/s11606-018-4444-4 .

Barouni M, Ahmadian L, Anari H, Mohsenbeigi E. Investigation of the impact of drg based reimbursement mechanisms on quality of care, capacity utilization, and efficiency - a systematic review. International Journal of Healthcare Management. 2020;14:1–12. https://doi.org/10.1080/20479700.2020.1782663 .

Heider A-K, Mang H. Effects of monetary incentives in physician groups: A systematic review of reviews. Appl Health Econ Health Policy. 2020;18(5):655–67. https://doi.org/10.1007/s40258-020-00572-x .

de Bruin SR, Baan CA, Struijs JN. Pay-for-performance in disease management: a systematic review of the literature. BMC Health Serv Res. 2011;11:272. https://doi.org/10.1186/1472-6963-11-272 .

Markovitz AA, Ryan AM. Pay-for-performance: Disappointing results or masked heterogeneity? Med Care Res Rev. 2017;74(1):3–78. https://doi.org/10.1177/1077558715619282 .

Quinn AE, Trachtenberg AJ, McBrien KA, Ogundeji Y, Souri S, Manns L, Rennert-May E, Ronksley P, Au F, Arora N, Hemmelgarn B, Tonelli M, Manns BJ. Impact of payment model on the behaviour of specialist physicians: A systematic review. Health Policy. 2020;124(4):345–58. https://doi.org/10.1016/j.healthpol.2020.02.007 .

Brown K, El Husseini N, Grimley R, Ranta A, Kass-Hout T, Kaplan S, Kaufman BG. Alternative payment models and associations with stroke outcomes, spending, and service utilization: A systematic review. Stroke. 2022;53(1):268–78. https://doi.org/10.1161/STROKEAHA.121.033983 .

Feldhaus I, Mathauer I. Effects of mixed provider payment systems and aligned cost sharing practices on expenditure growth management, efficiency, and equity: a structured review of the literature. BMC Health Servies Research. 2018;18:996. https://doi.org/10.1186/s12913-018-3779-1 .

Palmer, K.S., Agoritsas, T., Martin, D., Scott, T., Mulla, S.M., Miller, A.P., Agarwal, A., Bresnahan, A., Hazzan, A.A., Jeffery, R.A., Merglen, A., Negm, A., Siemieniuk, R.A., Bhatnagar, N., Dhalla, I.A., Lavis, J.N., You, J.J., Duckett, S.J., Guyatt, G.H.: Activity-based funding of hospitals and its impact on mortality, readmission, discharge destination, severity of illness, and volume of care: A systematic review and meta-analysis. PLoS ONE 9(10) (2014). https://doi.org/10.1371/journal.pone.0109975 .

Cattel D, Eijkenaar F. Value-based provider payment initiatives combining global payments with explicit quality incentives: A systematic review. Med Care Res Rev. 2020;77(6):511–37. https://doi.org/10.1177/1077558719856775 .

Jia L, Meng Q, Scott A, Yuan B, Zhang L. Payment methods for healthcare providers working in outpatient healthcare settings. Cochrane Database Syst Rev. 2021;1. https://doi.org/10.1002/14651858 .

Ahmed, K., Hashim, S., Khankhara, M., Said, I., Shandakumar, A., Zaman, S., Veiga, A.: What drives general practitioners in the uk to improve the quality of care? a systematic literature review. BMJ Open Quality 10 (2021). https://doi.org/10.1136/bmjoq-2020-001127 .

Forbes LJ, Marchand C, Doran T, Peckham S. The role of the quality and outcomes framework in the care of long-term conditions: a systematic review. Br J Gen Pract. 2017;67(664):775–84. https://doi.org/10.3399/bjgp17X693077 .

Kim KM, Max W, White JS, Chapman SA, Muench U. Do penalty-based pay-for-performance programs improve surgical care more effectively than other payment strategies? a systematic review. Annals of Medicine and Surgery. 2020;60:623–30. https://doi.org/10.1016/j.amsu.2020.11.060 .

Huang J, Yin S, Lin Y, Jiang Q, He Y, Du L. Impact of pay-for-performance on management of diabetes: A systematic review. J Evid Based Med. 2013;6:173–84. https://doi.org/10.1111/jebm.12052 .

Mendelson, A., Kondo, K., Damberg, C., Low, A., Motuapuaka, M., Freeman, M., O’Neil, M., Relevo, R., Kansagara, D.: The effects of pay-for-performance programs on health, health care use, and processes of care: A systematic review. Annals of Internal Medicine 166 (2017). https://doi.org/10.7326/M16-1881 .

Vlaanderen F, Tanke M, Bloem B, Faber M, Eijkenaar F, Schut F, Jeurissen P. Design and effects of outcome-based payment models in healthcare: a systematic review. Eur J Health Econ. 2019;20(2):217–32. https://doi.org/10.1007/s10198-018-0989-8 .

Article   CAS   PubMed   Google Scholar  

De Vries E, Scheefhals Z, Bruin-Kooistra M, Baan C, Struijs J. A scoping review of alternative payment models in maternity care: Insights in key design elements and effects on health and spending. Int J Integr Care. 2021;21(2):6. https://doi.org/10.5334/ijic.5535 .

Benabbas R, Shan G, Akindutire O, Mehta N, Sinert R. The effect of pay-for-performance compensation model implementation on vaccination rate: A systematic review. Qual Manag Health Care. 2019;28(3):155–62. https://doi.org/10.1097/QMH.0000000000000219 .

Herbst, T., Emmert, M.: Characterization and effectiveness of pay-for-performance in ophthalmology: a systematic review. BMC Health Services Research 17 (2017). https://doi.org/10.1186/s12913-017-2333-x .

Emmert M, Eijkenaar F, Kemter H, Esslinger AS, Schöffski O. Economic evaluation of pay-for-performance in health care: a systematic review. Eur J Health Econ. 2012;13:755–67. https://doi.org/10.1007/s10198-011-0329-8 .

Mathes, T., Pieper, D., Morche, J., Polus, S., Jaschinski, T., Eikermann, M.: Pay for performance for hospitals. The Cochrane database of systematic reviews 7 (2019). https://doi.org/10.1002/14651858.CD011156.pub2 .

Mauro, M., Rotundo, G., Giancotti, M.: Effect of financial incentives on breast, cervical and colorectal cancerscreening delivery rates: Results from a systematic literature review. Health Policy 123 (2019). https://doi.org/10.1016/j.healthpol.2019.09.012 .

Download references

Acknowledgements

Supplementary data with this article can be provided by the authors.

We acknowledge financial support by Deutsche Forschungsgemeinschaft and Friedrich-Alexander-Universität Erlangen-Nürnberg within the funding programme “Open Access Publication Funding”.

Open Access funding enabled and organized by Projekt DEAL. We acknowledge financial support by Deutsche Forschungsgemeinschaft and Friedrich-Alexander-Universität Erlangen-Nürnberg within the funding programme “Open Access Publication Funding”.

Author information

Authors and affiliations.

Department of Healthcare Management, Institute of Management, Friedrich-Alexander-Universität Erlangen-Nürnberg, Lange Gasse 20, 90403, Nuremberg, Germany

Eva Wagenschieber & Dominik Blunck

You can also search for this author in PubMed   Google Scholar

Contributions

EW: Conceptualization, methodology, formal analysis, investigation, resources, writing–-original draft preparation, writing–-review and editing.

DB: Conceptualization, methodology, formal analysis, investigation, validation, writing–-original draft preparation, writing–-review and editing, supervision.

All authors read and approved the final manuscript.

Corresponding author

Correspondence to Dominik Blunck .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors do declare that they have no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wagenschieber, E., Blunck, D. Impact of reimbursement systems on patient care – a systematic review of systematic reviews. Health Econ Rev 14 , 22 (2024). https://doi.org/10.1186/s13561-024-00487-6

Download citation

Received : 27 March 2023

Accepted : 07 February 2024

Published : 16 March 2024

DOI : https://doi.org/10.1186/s13561-024-00487-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Reimbursement
• Bundled payment
• Patient treatment
• Systematic review

Health Economics Review

ISSN: 2191-1991

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 20 March 2024

Primary care involvement in clinical research – prerequisites, motivators, and barriers: results from a study series

• Julian Wangler 1 &
• Michael Jansky 1  

Archives of Public Health volume  82 , Article number:  41 ( 2024 ) Cite this article

74 Accesses

1 Altmetric

Metrics details

Long-term reinforcement in the role of primary care and improvement the healthcare system as a whole requires the involvement of GPs in clinical research processes. However, many clinical studies fail due to failure to achieve sample population targets amongst GPs and their patients. This issue has been identified and discussed, but effective strategies to overcome it are still lacking. One of the reasons is that the positions, requirements, and experiences of GPs on participating in clinical research have hardly been examined up to now.

The years 2021 and 2022 saw three quantitative and qualitative surveys amongst GPs in Germany with the aim of shedding light on the attitudes, experiences, and potential issues regarding the involvement of primary care in clinical research projects and participation in cluster-randomised controlled trials (cRCTs) in a general sense. This overview summarises and abstracts conclusions gained from the exploratory series of studies and compares the results with the current research situation. From here, this contribution will then develop an approach towards optimising the integration of GPs into clinical research.

Most of the GPs asked associated clinical research with opportunities and potential such as closing gaps in healthcare, using evidence-based instruments, optimising diagnostic and therapeutic management, and reinforcement of multiprofessional healthcare. Even so, many GPs unsure as to how far primary care in particular would stand to benefit from studies of this type in the long term. Respondents were also divided on willingness to participate in clinical research. GPs having already participated in Innovation Fund projects generally saw a benefit regarding intervention and cost–benefit relationship. However, some also reported major hurdles and stress factors such as excessive documentation and enrolment requirements, greater interference in practice routines, and sometimes poor integration into project processes such as in communication and opportunities to play an active role in the project.

Conclusions

Results from the studies presented provide indications as to how GPs perceive clinical research projects and cRCTs as a whole and from their existing project experience, and on the requirements that studies would have to meet for GPs to be willing to participate. In particular, making sure that clinical studies fully conform with GPs would play a major role; this especially applies to freedom to make medical decisions, limitation of documentation obligations, interference in regular practice routine, greater involvement in research planning, and long-term reinforcement in the role of primary care. Clinical research projects and cRCTs should be planned, designed, and communicated for clear and visible relevance to everyday primary care.

Peer Review reports

Primary care plays an indispensable role in ensuring a functioning healthcare system. This applies to continuous (long-term) healthcare across the entire range of clinical conditions and complaints as well as patient types. However, it also applies to GPs in guiding their patients through the healthcare system by specifically referring them to other levels of care. Primary care participation in clinical research processes will play a central role in expanding primary care and other healthcare roles in a consistent and methodical fashion while also testing novel forms of healthcare and improving the healthcare system as a whole [ 1 , 2 ]. New healthcare models – especially in the low-prevalence area – need to encompass sufficiently large patient cohorts for evidentially significant results, making primary care involvement inevitable in many cases [ 3 , 4 , 5 , 6 , 7 , 8 , 9 ].

However, GP-based interventions face significant hurdles in projects in clinical as well as healthcare research despite the significant role of primary care in clinical research and the potential benefits that may result. It is often a challenge to recruit a sufficient number of GPs for these studies, which usually involves a sophisticated cluster-randomised design in cRCTs [ 2 , 10 ]. Various research projects and application areas have indeed shown recruitment of GPs to be a limiting factor in performing clinical projects involving primary care [ 1 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ]. Especially cRCTs usually require sufficiently sized study cohorts with failure to achieve patient recruitment goals often leading to insufficient statistical significance and even premature study termination [ 17 , 18 ].

There are various reasons for insufficient overall recruitment and research participation amongst GPs [ 8 , 13 , 19 ]. Many studies have identified a lack of time and resources [ 11 , 14 , 15 , 20 , 21 ] and fear of administrative and documentation effort [ 21 , 22 , 23 ] as the main reasons for GPs to decide against participating in research projects. A lack of relationship with research and the problems this involves in understanding and implementing research methods have also been given as possible reasons [ 1 , 3 , 4 , 6 ]. One qualitative study found GPs sometimes facing problems in enrolling patients in clinical research projects and supporting them throughout the intervention as they did not see themselves as being equipped with the comprehensive clinical research competence necessary [ 20 ].

Another factor is that Germany lacks a longstanding tradition of involving GPs in clinical research activities, unlike other countries [ 16 , 24 ]. The healthcare and innovation systems may not be directly comparable, but studies in other Western countries have shown issues associated with recruiting and involving GPs in clinical research projects [ 5 , 10 , 13 , 14 , 16 , 22 , 23 , 25 , 26 , 27 ]. As an example, only every third health study in primary care achieves its target patient cohort in Anglo-American countries as insufficient numbers of GPs can be recruited and/or too many leave the research projects early [ 5 , 17 , 28 ].

Summarising, recruiting GPs for major clinical healthcare studies is one of the greatest challenges facing healthcare research. This issue has already been identified and discussed, but there is still a lack of effective strategies towards overcoming it [ 2 , 4 , 6 , 9 , 10 , 12 , 16 ]. One of the reasons is that the positions, requirements, and experiences of GPs on participating in clinical research have hardly been examined up to now.

Research interest and aim of study

Addressing the attitudes, experiences and potential issues involved in including primary care in clinical research projects and participation in cRCTs in a general sense crucially requires ascertaining the perspective of GPs.

The overview article summarises and abstracts the conclusions gained from an exploratory series of studies as well as the authors' own research experiences. The results are intended for comparison against the research situation up to now. This articles centres on the following issues:

What attitudes do GPs have towards clinical research and its benefits for primary care?

How far do GPs see barriers against participating in clinical research projects?

Under what conditions would GPs be willing to participate in clinical research projects and cRCTs?

What experiences have GPs had after participating in clinical research projects and cRCTs? Adding up the columns, what conclusions have they drawn from project participation?

What would GPs like to see in the way of optimisation to increase the attractiveness of participation in clinical studies or cRCTs in the future?

In principle, the series of studies was about all types and forms of clinical research projects, i.e. not necessarily just about therapeutic interventions, but also, for example, about questions of quality of life, drug therapy and drug therapy safety, cross-sectoral care, medical guidelines and application adherence, geriatric care, telemedicine and eHealth/mHealth, delegation and substitution of services, care for vulnerable groups (e.g. family caregivers), communication with patients and promotion of health literacy, care in structurally weak or rural areas etc.

Overall, we aim to contribute to a better understanding of barriers and facilitators of the recruitment of GPs and their patients. With this in mind, we used the findings presented as a synopsis towards developing approaches towards optimising integration of GPs in clinical research.

The studies included in this overall assessment include detailed surveys amongst GPs in Germany and in their willingness to participate clinical research activities and cRCTs as well as their experiences specifically in this regard. From the findings gathered together and presented in this contribution, we have drawn conclusions as to how clinical research projects might be designed towards making participation as attractive as possible amongst GPs.

Study design and recruitment

This analysis includes three surveys on German GPs posing a variety of questions with central areas of focus regarding participation in clinical research. All sub-studies were deliberately designed to be exploratory in nature, reflecting the paucity of research on this subject (see Fig.  1 ).

Order of the individual studies [ 29 , 30 , 31 ]

All 13,170 GPs with active practices in the federal states of Baden-Württemberg, Hesse, and Rhineland-Palatinate were invited to an online survey between July and November 2021 [ 30 ]; the survey was based on a smaller qualitative preliminary study that had already taken place [ 29 ]. This initial study served to collect general information on the topic in order to create the conditions for conducting a large quantitative study. The main study asked GPs for their attitudes, expectations on participation, and experiences from clinical research and especially the Innovation Fund, which serves as the central health policy instrument for promoting and financing new forms of healthcare in standard healthcare. Footnote 1

A third study was qualitative in nature and functioned as an in-depth study, specifically aiming to capture the perspective of general practitioners with research experience. A total of 36 semi-standardised individual interviews with GPs already having participated in clinical and Innovation Fund projects were conducted between September 2021 and February 2022 [ 31 ] alongside the quantitative survey. Eleven regional physicians’ networks in the federal states of Rhineland-Palatinate, Hesse, North Rhine-Westphalia, and Schleswig–Holstein were involved in the recruitment process. This study mainly focused on investigating actual experiences amongst GPs from participating in research studies on the health services. With the help of the mentioned regional doctors’ networks, contact was established with a total of 36 GPs; interviews were conducted with all of them.

None of the studies included used any form of incentives.

Development of survey instruments

Questionnaires and interview guidelines were developed for the quantitative and qualitative surveys on GPs as to their general participation and willingness to participate in clinical research and cRCTs; these questionnaires and guidelines took into account the authors’ previous research and recruitment experience in the Innovation Fund and evidence-based instruments [ 36 , 37 , 38 , 39 , 40 ] and general desk research (including Lech et al. [ 1 ] and Heytens et al. [ 34 ]). Both the quantitative main survey and the qualitative study contain the following main content areas: a) attitudes towards clinical research projects and their benefits; b) willingness to participate and corresponding prerequisites; c) experiences from taking part in specific projects; d) perceived optimisation potential.

The quantitative study contains a total of 25 questions. In addition to the standardized questions, which were often 4-point Likert scales, a series of open questions were used. The sociodemographic characteristics recorded were gender, age, practice environment, type of practice and patients per quarter. A pretest was carried out prior to data collection. For this purpose, the questionnaire was presented to 50 randomly selected GPs. The pretest showed that the questionnaire was easy to understand, structured and has complete answer categories.

The qualitative in-depth study included 20 questions. The focus was more on the experiences of GPs in clinical research projects. Here, too, a pretest was carried out in advance to check the comprehensibility and practicality of the guidelines.

Data analysis

The SPSS 23.0 statistical package was used for evaluating the data from the quantitative survey studies. Apart from the descriptive analysis, Student’s t -test for independent samples was used to analyse for significant differences between the two groups. STROBE was used as the reporting statement for the main study.

Qualitative content analysis according to Mayring [ 41 ] was used as a basis for evaluating the qualitative interviews and open questions in the questionnaires. After transcription, we evaluated the interviews in a team using the MAXQDA software. In preparation, the written consultations were summarised with the essential information to gain an overview of the fundamental material. The text was then extracted in individual sentences or paragraphs depending on importance and expressiveness with units to be used in analysis previously determined (context, interview code, original text, paraphrasing, generalisation). The most important core statements were isolated, abstracted and summarised before forming categories. The categorical system created (see Multimedia Appendix 1 ) was based on the priorities set in the guidelines, repeatedly checked, and modified as necessary in the course of evaluation. We used the COREQ methodology as reporting statement for the qualitative study.

Sample overview

The 3,556 fully completed questionnaires corresponding to a response rate of 27% were included in analysis [ 30 ]. Table 1 compares the sample obtained with reference data from the German Association of Statutory Health Insurance Physicians (KV) on the structure of GPs in Germany.

The qualitative sample [ 31 ] comprised the following (see Table  2 ):

General results

Table 3 summarises the salient findings of the studies mentioned. These findings will be discussed alongside the research issues listed in the following.

Attitudes towards clinical research and healthcare benefit

Around half the GPs surveyed had an explicitly favourable attitude towards clinical research in all studies covered; the other physicians saw this rather negatively or did not take a clear position, which was mostly due to their stated unfamiliarity with scientific research [ 29 , 30 , 31 ]. Notably, the proportion of those reporting a favourable verdict in the quantitative study was significantly higher among urban than rural physicians (60% vs. 38%, p  < 0.001). Around every third general practitioner associated clinical research with major benefit, while another third saw minor to moderate benefit [ 30 ].

Closer inspection reveals that a large proportion of those surveyed associated clinical research with considerable opportunities for the healthcare system, especially regarding identifying and closing gaps in care, using evidence-based instruments and procedures, and therefore optimising diagnostics and/or therapeutic management. Another benefit of clinical research projects according to respondents was their own contribution to reinforcing multiprofessional and cross-sectoral care, and therefore also the sequence of healthcare steps between the various medical and nursing protagonists.

“I do think it gives us an opportunity to benefit from targeted and sustainable improvements in taking care of our patients.” (I-8 m)

“Complex clinical research – Germany has long since been a bit of a developing country in getting general practitioners on board. This is where the vast majority of patients receive healthcare in everyday life. […] So, it’s definitely a step in the right direction.” (I-17f)

Even so, many GPs also doubted that clinical research would be an easy fit for the requirements of primary care, where pragmatic and social considerations (“talking medicine”) play a far greater role than a strictly research-based focus. Some therefore wondered how far primary care could benefit on a larger scale from involvement in this type of research project. Respondents especially mentioned addressing specific primary care needs and (sustainable) accuracy in interventions.

Apart from that, many respondents expressed concerns that clinical research could lead to issues in primary care in the long term with funds in the healthcare system being reallocated towards specialised structures even with the dependency of clinical research on primary care for studying larger patient cohorts and testing interventions. Some respondents during the interviews reported on their own experiences with projects involving new health protagonists such as special case managers with the concern that these new multiprofessional positions might ultimately come at the expense of primary care budgets and lead to “an over-engineered and bloated healthcare system” (I-24 m) [ 31 ].

“Clinical research encourages a kind of proliferation and chaos. New professional groups are constantly popping up, challenging the guiding role of general practitioners." (I-30 m)

With this in mind, the level of support from those surveyed was relatively low as to the prospect of clinical research projects and cRCTs leading to long-term reinforcement in the role of primary care. Physicians in urban areas anticipated this significantly more frequently than rural physicians (51% vs. 28%, p  < 0.001) [ 30 ].

“The whole thing could also have a negative side. […] For example, I see a risk that these studies might ultimately bypass the reality of general practitioners too much and be of little use to us, or even a burden in the worst case.” (I-11 m)

"We’ve already seen that happen. GPs are recruited, but they’re more of a means to an end […] to feed study planners with patients.” (I-14 m)

Apart from that, a number of GPs expressed concerns that clinical research “does not necessarily support projects that the healthcare system needs;” rather, that it often focused on “politically selected topics and issues” (I-11 m). Some of the respondents also expressed doubts as to whether new healthcare models, such as those being tested in cRCTs, would ultimately find their way into standard care in practice [ 29 , 31 ].

"Remember that these studies are subject to funding programmes lasting a few years. This is a high bar to overcome in successfully providing evidence of an intervention’s efficacy. I think many of these projects would just fizzle out for a whole variety of reasons." (I-25f)

Perceived barriers to the participation of primary care in clinical research

Respondents saw the various aspects of additional workload as the greatest barriers facing general practitioner involvement in clinical research activities [ 29 , 31 ]. This included increased amounts of work and significantly increased time and resource pressure for the entire practice team. The cost to flexibility in everyday practice due to research project commitments and intervention specifications was also seen as an issue.

"I’ve heard about this from a close colleague in general practice. He applied a clinically developed algorithm towards improving early diagnosis of liver disease. Sounds easy enough. But you can’t imagine the chaos that all the action guidelines caused in his medical practice. It sounded really awful.” (I-33m)

These perceptions are based on the fear of lasting detriment to established routine at the practice. Many respondents took the view that “general practitioners can’t afford to compromise on regular patient care for some special project especially in these times of high patient numbers and general practitioners in acutely short supply in some cases” (I-27f) [ 31 ]. A reduction in the total number of patient contacts and treatment programmes would therefore not be an acceptable condition for participation in research projects, according to many respondents.

“The thing is, you either join the project in full or not at all. That means either you’re willing to take on this added burden, or you’re not. But what if you want to contribute as a GP, but you can't get involved as much as the project requires in time or seasonality? Count me out. Because there’s no in-between in project participation, I mean as in flexibility." (I-29f)

This came with a high level of concern facing comprehensive and potentially escalating documentation and administrative obligations, such as in registering patients and filling out case files for the project. A few respondents also reported fearing substantial financial losses from participating in clinical research projects. Another key barrier was the lack of a research background amongst many GPs, so finding their way around the clinical procedure – especially in cRCTs – would mean a “transition and additional effort that shouldn’t be underestimated” (I-17f).

No fewer GPs saw a barrier in that those responsible for the project often failed to demonstrate any concrete benefit or added value for primary care from the intervention; practical implications for primary care remained unclear when recruiting from general practices for a study [ 31 ].

“Maybe it’s my lack of basic knowhow in research. But I'd like to know exactly what's in it for my patients and, of course, for me as a physician, before getting involved in something like this. I’m sure the project managers know what they have in mind, but they have a problem communicating it.” (I-11m)

Willingness to participate, prerequisites, and reasons for participating in clinical research projects

According to the large-scale written survey of GPs, 31% of respondents were generally willing to consider participating in a clinical study or cRCT in the future, and another 24% reported that they had already participated in at least one associated study [ 30 ]. In contrast, 45% were fundamentally unwilling to participate in any clinical research project. Comparing age groups, 47% of physicians younger than the median age of 55 saw participation in a clinical research project would as an option vs. 20% of physicians aged 55 and over ( p  < 0.001).

In an open question, the respective physicians explained their willingness to participate as mainly due to curiosity and involvement in scientific research (35%), interest in or prior knowledge in the specific topic (35%), and a desire to help improve healthcare and quality of life for patients (45%).

Respondents not willing to participate explained their stance with consistently high workloads (54%), concerns about excess burden when participating in research activities (44%), and doubts as to the benefits of clinical scientific research in some cases (29%). Amongst GPs for whom taking part in clinical studies or cRCTs was out of the question, most doubted that these studies would find their way into standard healthcare (57%) or that they would be of any substantial benefit to primary care (58%).

Prerequisites played an important role for physicians responding that they would consider participating in a study or had already participated in one or more projects. Apart from likely diagnostic or therapeutic benefit for patients, they mainly focused on issues regarding the (limited) additional burden (such as preparation and follow-up, documentation, patient registration), appropriate remuneration, and structural improvement to the primary care setting. Respondents also saw importance in projects contributing to breaking down sector boundaries in the healthcare system and, above all, not interfering with normal operations and responsibilities in their medical practice [ 29 , 30 , 31 ]. Rural physicians in the quantitative survey emphasised the prerequisite that the project must not cause changes in practice routines far more often than city physicians (64% vs. 30% amongst city physicians, p  < 0.001) [ 30 ].

“Committing yourself to studies like this isn’t trivial. They should see how they can accommodate general practitioners here. I think there’s still too little of that.” (I-6f)

Experiences from taking part in specific projects

According to their own replies, 24% (875) of those respondents in the quantitative survey had already been involved or were currently participating in at least one clinical research project or cRCT [ 30 ]. The respondents comprised 92% urban and 8% rural physicians. Of the 875 respondents, 33% were individual and 67% group practices. Regarding age, 73% were younger than the mean, and 35% were networked with other physicians.

The information gained from surveyed reveals that most of the projects in which the physicians were participating or had participated focused on optimising a specific area of patient care, drug therapy or drug therapy safety, polypharmacy, extending regional and multiprofessional care networks, or promoting evidence-based medicine or compliance with guidelines. Many of the projects also involved telemedicine as well as enabling the delegation of care services. Projects focused on care in vulnerable groups such as caring relatives or people with disabilities or on promoting health and communication skills were less frequent.

However, some respondents emphasised that they had initially weighed up the feasibility of taking part in a large research study against their heavy workload [ 31 ].

“You have to think carefully about whether you can afford to take part in a study like this. You have to play it out in your head, even if things don’t turn out to be that serious.” (I-19m)

Two-thirds of respondents involved in the project reported that they needed to train members of the medical practice staff due to participation [ 30 ]. This especially applied to physicians participating in projects focused on drug therapy or specific medical conditions. Of the respondents, 80% reported severe (27%) or moderate (53%) complications vs. 20% with no complications as a result of participating in one or more research projects at their medical practice.

“That was an issue. The practice staff had to undergo a huge amount of preparation, the short-term training requirement was heavy… we were not informed about the type and extent of training from the start, and the training was scheduled at too short notice. This made normal office routine more difficult.” (I-26f)

The physicians involved in the project reported they were especially impressed by the results (from treatment) and optimised patient care (69%), improvement in cooperation with other care providers and sectors (52%), and enhancement of their diagnostic and therapeutic skills (40%) in response to an open question in the quantitative survey [ 30 ].

In contrast, physicians saw increased time pressure (66%) as well as considerable documentation requirements such as in registering patients and heavy paperwork in many cases (64%) as negatives alongside interference with practice routines and established procedures arising from project participation (55%) as well as too little involvement in research processes and decisions relating to the project for some of the physicians (43%). A few reported pressure from the project management to “recruit an unrealistically large patient cohort” (I-2 m).

“The hurdles and additional burdens shouldn’t be underestimated. I can understand why not all doctors can take part.” (I-25 m)

Some GPs complained that they did not have the research skills for rapid quick integration into the project or easy grasp of the procedure. On the other hand, some criticised the apparent lack of priority in bringing physicians up to speed on the research requirements such as in corresponding preparation courses [ 29 , 31 ].

"Apart from that, we as general practitioners – especially in Germany – don’t have the academic background to keep up with these activities. This is a real problem that has to be addressed in medical studies in the long term if we really want to train general practitioners with an affinity for research.” (I-32f)

Verdicts on project participation

GPs having participated in clinical studies or cRCTs draw a favourable overall conclusion in the general quantitative survey [ 30 ] on the benefits of the intervention tested. Of the respondents, 72% reported that care and treatment for the patients involved benefited very highly at 13% or rather highly at 45% vs. 18% less highly, 16% not at all, 8% difficult to say. Likewise, 66% rated the project participation benefit as clearly (43%) or slightly (23%) outweighing the effort involved vs. 11% about the same, 12% effort slightly outweighing benefit, 11% effort clearly outweighing benefit. Respondents rated projects covering healthcare in economically underdeveloped areas, drug therapy/safety, delegation and substitution, and cross-sector healthcare favourably for added value.

Of all the respondents having already participated in clinical trials or cRCTs, 15% reported prematurely ceasing participation. The main reasons they gave were excessive additional burden, (documentation) effort, and interference with practice routine. Some also mentioned inadequate opportunity for decision-making and participation. The qualitative interviews came to the same result [ 31 ].

“This project just got out of hand. They were constantly increasing the requirements for me as a physician without asking me. At some point, it became too much of a burden.” (I-31f)

Even so, most of the respondents stated that they would generally consider participating in other projects in the future provided the project promised worthwhile benefits for primary care from their point of view.

Potential for optimisation

Physicians surveyed having taken part in clinical studies or cRCTs named several improvements they would like to see [ 30 ]. These involved strict limitations to documentation obligations (65%), a simple documentation system (62%), clearer organisation in project coordination (56%), making more flexibility possible in medical decision-making such as in calling in patients as well as decisions related to treatment, less severe interference with practice routines (49%), and reinforcement and improvement in structuring communication and cooperation between the physicians and other healthcare protagonists (37%). Finally, the physicians stated that they would appreciate (more) cost-based remuneration (34%).

The responses also demonstrate that the position of GPs should be reinforced further at various stages of a clinical study. Of all respondents, 57% saw importance in involving GPs more than before in study design and development. This would also include project-internal formats for structured participation such as research workshops as well as institutionalised exchange with colleagues and the research consortium.

“General practitioners simply just need to be more involved than before in designing and developing new studies and healthcare models. Once that happens, the studies will be more compatible with primary care and they’ll achieve their aims earlier." (I-23f)

GPs considered it particularly important for research projects to ensure the possibility of delegation allowing individual physicians to entrust practice staff members with project activities. Ensuring this across the board would save time and resources in the intervention. GPs also saw importance in integrated and coordinated training for the whole practice staff to prepare for workable project participation and avoid stressful individual situations. Flexible adjustment possibilities in project requirements such as varying levels and types of participation adjusted to capacity and degree of workload – such as reducing project obligations to account for seasonal factors – would also help prevent premature study cessation of GPs while also lowering barriers to entry [ 29 , 30 , 31 ].

GPs would appreciate more overall recognition of their commitment to clinical research. Some respondents suggested clinical practice status as recognition. A few respondents also raised the possibility of further academic titles as a result of years of involvement in clinical research.

Principal findings and comparison with prior work

We have presented the main results from various surveys amongst GPs on their pervious experiences and future willingness to participate in clinical studies and cRCTs as a synopsis in the course of this contribution. These findings show GPs to be divided on whether to participate in studies of this type.

Overall, the attitudes of many GPs were notably favourable with regard to the fundamental benefits and added value from clinical research, and that opportunities for corresponding research projects were being taken such as towards identifying and closing healthcare gaps, intensifying application-oriented healthcare research, using evidence-based research instruments and procedures, optimising diagnostic and therapeutic management, and reinforcing multiprofessional healthcare. The research literature repeatedly described methodically embedding primary care into cross-sector, interdisciplinary structures as a major asset in clinical care models [ 15 , 25 , 36 ].

However, some GPs took a more critical and distanced attitude towards long-term goal orientation in corresponding studies. Some respondents were unsure as to what extent structures created by clinical-scientific care models could contribute in practice towards making the healthcare system more effective in the long term. Some expressed uncertainty as to whether primary care could actually benefit from such research participation in the long term.

Urban physicians in the quantitative survey sample [ 30 ] identified clear benefits from clinical studies and cRCTs, but their rural counterparts took a more cautious stance. This tallies with the general research findings that GPs in rural areas perceive lower added value in evidence-based structures and instruments [ 39 , 40 ]. Likewise, most of the 24% of respondents having already taken part in cRCTs were located in urban areas with a greater variety of care services, which is often a prerequisite in effective clinical research [ 7 , 9 ]. We did not find any significant gender differences in the studies we carried out; this contrasts with other individual studies on willingness of GPs to participate in research networks such as Virnau et al. [ 2 , 42 , 43 ]. Apart from the difference between urban and rural physicians, age is a factor that this study has in common: Openness to clinical research projects amongst the respondents decreased with age [ 2 , 21 ].

Physicians fundamentally open to or already participating in research projects raised a number of requirements in this regard. Apart from added value for patient care, they emphasised manageable and plannable additional burden, impact on practice routines remaining tolerable, and structural reinforcement in the role of primary care. This tallies with results from previous surveys of primary care research networks (to be established) (see for example: [ 9 , 24 , 44 , 45 , 46 , 47 , 48 , 49 ]). A study by Güthlin et al. showed GPs to be especially interested in complex research projects if the topic seemed relevant to them and participation promised an actual benefit for the staff and patients of the practice. With this in mind, it hardly comes as a surprise that physicians having participated in clinical care models give especially favourable assessments of studies on topics such as rural care, drug therapy/safety, delegation, or sector cooperation. Other studies have also shown GPs to consider areas such as polypharmacy, drug safety and adverse drug effects, and multiprofessional cooperation models to be especially important [ 1 , 2 , 15 , 29 , 31 ]. Apart from that, many GPs currently would not want their medical practice just “researched,” but would rather help shape how these research projects are conducted [ 44 , 45 ].

The conclusion reached by most of the GPs involved from participation in the corresponding studies is clearly favourable. This applies to healthcare and increase in treatment quality for the patients involved and to the cost–benefit relationship. Physicians found it easier to assess care needs of patients and their relatives, and recommend assistance services. Finally, there was a noticeable increase in subjective capability to perform diagnostics and disease management, and to apply the S3 guideline. Even so, some respondents described negative experiences and stress factors as reflected in documentation requirements and administrative effort, temporary but substantial changes in practice routine, deficits in project communication, and remuneration not matching the effort involved.

The results from the survey may be seen as confirmation of increased willingness amongst GPs to participate in empirical, evidence-oriented studies with the aim of optimising healthcare [ 15 ]. Especially younger GPs in urban catchment areas are increasingly basing their work on standardised, evidence-based interventions [ 39 , 40 ]. Even so, a substantial proportion of general medical practices are fundamentally unwilling or remain reluctant regarding these research projects [ 1 , 7 , 8 , 9 ]. This has resulted in a regional shortage of GPs available for recruitment in complex studies as reflected by project experience from the Innovation Fund in existence in Germany since 2015, often failing to achieve the original target cohorts of physicians and patients [ 50 ]. Lech et al. [ 1 ] provided one example in a contribution reporting on a cluster-randomised study to optimise outpatient dementia care. The authors reported difficulties in recruiting GPs despite using a wide range of recruitment strategies.

There is mounting evidence that combining these projects with everyday general practice care is not a smooth process, although the reasons for barriers and challenges to recruiting GPs for clinical research have hardly been investigated to date. This involves, on the one hand, immediate difficulties from an underlying shortage of time and resources in general practices [ 45 ]. GPs need to make the time required for project activities during consultation hours, which represents a major barrier to any research interest [ 43 , 51 ]. This barrier may be raised further by the fear of a potentially escalating additional workload such as what GPs see as high-threshold registration of patients for the project, alongside documentation requirements and dealing with complex documentation systems such as digital case files. A low, tightly planned time investment for GPs and their staff always boosts the attractiveness of clinical research and may be of benefit to future recruitment [ 52 ]. On the other hand, GPs cast doubts on the match and fit of interventions in everyday primary care. This applies to project plans often conceived from a clinical-scientific perspective that then led to complications and limitations in primary care [ 44 , 45 , 49 , 50 , 53 ].

A review by Fletcher et al. [ 3 ] on GP-based clinical research identified barriers such as poor communication by study coordinators, difficulties experienced by GPs in understanding research methods, concerns about possible harm to patients, and the feeling of being overwhelmed by too many research requests without being perceived as genuine research partners. Routinised communication between all the stakeholders in every project phase plays an especially important role in enabling and improving practice-oriented research [ 54 ]. Apart from that, reliable and persistent contacts such as at university hospitals play a major part as an indispensable prerequisite for workable and cooperative relationships between resident GPs and clinical project management [ 55 ].

There are also indications that topics covered in clinical research projects do not always match the interests and perceived issues shared by gene, making it impossible to convince them to participate [ 2 , 21 , 43 , 56 , 57 , 58 ]. This points to the need for continuous interaction between hospital-based primary care and GPs for continuous identification of everyday topics related to healthcare as relevant to GPs and their patients [ 59 , 60 , 61 , 62 ].

Beyond the issues already covered, Lech et al. [ 1 ] also discussed requirements for a specific recruitment of GPs. The contribution emphasised the benefit of greater concentration on (regional) physicians’ networks to specific recruitment in cRCT studies due to increased research interest, specific topic reference, and close coordination between the participating physicians [ 47 , 63 , 64 ]. A substantial proportion of the physicians involved in the studies were also members of a physicians’ network in the surveys presented [ 29 , 30 , 31 , 36 , 37 ].

Finally, consideration should be given to remuneration for participating GPs. GPs and their staff would welcome some financial reward for participating in clinical research even if most do not anticipate major financial losses from time spent in participation. One possibility would be increasing the remuneration amount with the number of patients enrolled into the study [ 65 ]. Apart from that, many GPs would benefit from reimbursement of additional expenses; this would help to ensure continuity and sustainability in clinical research networks [ 63 ]. Norway provides an example of good practice where physicians participating in research projects receive an annual fee for ongoing administrative work in addition to an hourly fee for each study participation [ 66 ].

Most important takeaways from the studies presented

As shown, the findings obtained the from survey included in this contribution are largely consistent with existing research literature on primary care involvement in clinical research and cRCTs [ 55 , 67 , 68 ]. However, specific weightings and focal points in general practitioner positions as well as additional insights towards motivating GPs to take part in complex clinical research projects emerged during the studies. Figure  2  summarises the central takeaways.

Approaches developed towards optimising integration of primary care in clinical studies (own diagram)

GPs expressed a desire for a manageable and predictable additional workload such as in the complexity of the intervention to be used and in administrative and documentation tasks without excessively interfering with established practice routines [ 36 , 39 , 40 , 49 ]. GPs also wished for more individual flexibility in action and decision-making extending beyond participation in research activities to involvement in evidence-based structures and instruments such as disease management programmes and guidelines. Examples of this included authorisation to take alternative approaches considered beforehand in these research projects or temporarily cutting down on project commitment without having to withdraw from the study entirely.

In addition, many GPs also expressed a strong desire for more involvement in shaping project activities and more inclusion in clearly structured communications during research projects. Many GPs advocate constant updates on research-related matters from project management, but also institutionalised, multi-layered exchange and feedback opportunities within the research network. GPs also found it important to use integrated and methodical training programmes and, wherever possible, detailed delegation plans for practice teams to demonstrate possibilities for implementing interventions while saving time and resources as far as possible. All this indicates that clinical research projects have still not always been compatible with the salient primary care setting up to now [ 16 , 44 , 53 ].

GPs saw it as desirable to approach rewards for participation in clinical studies not only in terms of remuneration alone, but also as a form of academic and research recognition. Some respondents saw a definite motivational factor in the possibility of receiving official certification as a university-associated research practice or a specific academic title in recognition of years of commitment to clinical research [ 44 , 47 ].

Finally, training, and further training as a whole should undergo significant extension towards participation in research studies. The studies performed demonstrated that a sizable proportion of GPs were unsure about their research qualifications and current level of knowledge, leading to doubts as to their personal suitability for active research involvement. Germany has only seen increased efforts towards integrating research competence more firmly as a component of Medical Studies programmes in recent years [ 1 , 37 ].

The main findings have demonstrated how it might be possible to recruit more GPs in the future. In the opinion of the authors, consistent implementation of these resulting clusters will not only exert a favourable effect on motivation to join but also to remain in the project while also improving process and result quality in cRCT studies. This would also create more favourable general conditions for GPs to take an active part in clinical research in the future. It would also be important to align research projects with topics that GPs see as relevant for these optimisation approaches to materialise, and also to convey the specific benefit and added value for primary care in a clear fashion when addressing physicians [ 2 ].

Strengths and limitations

The studies presented in this contribution are to the best of our knowledge amongst the few empirical studies that have been published so far with an in-depth focus on attitudes, acceptance, expectations, and experiences of GPs towards participation in clinical research projects. However, the study cannot make any representative claims in the strict sense due to the limited number of cases and regional recruiting focus. We must also take into account that the focus of the surveys was also largely placed within the Innovation Fund context. Particularly extensive, complex, and also cost-intensive clinical research projects in Germany are financed by the Innovation Fund, which is not necessarily representative of any other type of clinical research.

In addition, the proportion of GPs involved in research is noticeably overrepresented in the quantitative survey sample compared to the total number of GPs, so selection bias needs to be considered. This implies that the survey addressed physicians with a greater interest or commitment to the topic at hand in contrast to physicians with no connection to clinical research, who have presumably participated to a lesser extent in clinical research. The responses from respondents on the main topics of respective research projects should also be seen within this context. The ranking order of responses listed reflects the topical interest of GPs, but the number of projects available for the respective subject areas may cause a bias in this information.

Even so, the heterogeneous random sample taken approximated to the general population of GPs in important aspects (see Table  1 ). The exploratory approach combining quantitative and qualitative components allowed a wide range of general practitioner perspectives, attitudes, and experiences to be documented.

The present study has not looked into how far the projects the responding GPs took part in were implemented, co-managed, or coordinated by primary care institutes. These institutes have gathered a wealth of experience in research collaboration with GPs. Future studies should therefore focus on whether the study conditions for GPs could be more favourable in cooperation with primary care institutes.

The present studies have also left aside the extent to which settings other than clinical studies may be more suited to primary care regarding willingness to become involved in scientific research. Studies from primary care suggest that the research practice model may potentially achieve more effective recruitment and participation [ 44 , 47 ]. In this respect, results from the present study may be compared with results from a survey to be suggested here documenting the experiences of GPs specifically in the research practice setting. This type of survey would be feasible on a larger scale in view of the recent emergence of larger research networks coordinated by primary care institutes.

Results from the studies presented provide indications as to how GPs perceive clinical research projects and cRCTs as a whole and from their existing project experience, and on the requirements that studies would have to meet for GPs to be willing to participate. Future research projects on primary care-based interventions should redouble their efforts at reflecting the positions, needs, and experiences of GPs. This would enable us to even out the hurdles and challenges perceived by GPs in participating in projects of this type. In particular, making sure that clinical studies fully conform with GPs would play a significant role; this especially applies to the medical decision-making freedom, limitation of documentation obligations, impediment to medical practice routine, greater involvement in research planning, and long-term reinforcement in the role of primary care. Clinical research projects and cRCTs should be planned, designed, and communicated for clear and visible relevance to everyday primary care.

Availability of data and materials

All major data generated or analyzed during this study are included in this published article. Additional information can be provided on request made to the corresponding author.

The year 2015 saw the Innovation Fund established as part of the Federal Joint Committee (G-BA) [ 32 ]. As a health policy instrument, the aim of the fund is to promote evidence-based development in pay-as-you-go healthcare by developing new healthcare concepts [ 32 , 33 , 34 , 35 ]. An annual funding volume of €200 million has been secured for the project in the current funding phase with funding provided by the statutory health insurance companies and health fund.

Lech S, O’Sullivan JL, Wellmann L, et al. Recruiting general practitioners and patients with dementia into a cluster randomised controlled trial: strategies, barriers and facilitators. BMC Med Res Methodol. 2021;21(1):1–3. https://doi.org/10.1186/s12874-021-01253-6 . PMID: 33784967.

Article   Google Scholar  

Virnau L, Braesigk A, Deutsch T, et al. General practitioners’ willingness to participate in research networks in Germany. Scand J Prim Health Care. 2022;40(2):237–45. https://doi.org/10.1080/02813432.2022.2074052 . PMID: 35770652.

Article   PubMed   PubMed Central   Google Scholar  

Fletcher B, Gheorghe A, Moore D, et al. Improving the recruitment activity of clinicians in randomised controlled trials: a systematic review. BMJ Open. 2012;2(1):e000496. https://doi.org/10.1136/bmjopen-2011-000496 . PMID: 22228729.

Ngune I, Jiwa M, Dadich A, et al. Effective recruitment strategies in primary care research: a systematic review. Qual Prim Care. 2012;20(2):115–23 PMID: 22824564.

PubMed   Google Scholar  

Bower P, Wallace P, Ward E, et al. Improving recruitment to health research in primary care. Fam Pract. 2009;26(5):391–7. https://doi.org/10.1093/fampra/cmp037 . PMID: 19549623.

Article   PubMed   Google Scholar  

Foster A, Horspool KA, Edwards L, et al. Who does not participate in telehealth trials and why? A cross-sectional survey. Trials. 2015;16(1):258. https://doi.org/10.1186/s13063-015-0773-3] . PMID: 26044763.

Williamson MK, Pirkis J, Pfaff JJ, et al. Recruiting and retaining GPs and patients in intervention studies: the DEPS-GP project as a case study. BMC Med Res Methodol. 2007;7(1):42. https://doi.org/10.1186/1471-2288-7-42 . PMID: 17875219.

Hummers-Pradier E, Bleidorn J, Schmiemann G, et al. General practice-based clinical trials in Germany – a problem analysis. Trials. 2012;13(1):205. https://doi.org/10.1186/1745-6215-13-205 . PMID: 23136890.

Hummers-Pradier E, Chenot J-F, Scherer M. Are primary care practices research infrastructure? [Sind Hausarztpraxen Forschungsinfrastruktur?]. Z Allg Med. 2014;90(7/8):317–22. https://doi.org/10.3238/zfa.2014.0317-0322 .

Page MJ, French SD, McKenzie JE, et al. Recruitment difficulties in a primary care cluster randomised trial: investigating factors contributing to general practitioners’ recruitment of patients. BMC Med Res Methodol. 2011;11(1):35. https://doi.org/10.1186/1471-2288-11-35 . PMID: 21453543.

Huibers MJ, Bleijenberg G, Beurskens AJ, et al. An alternative trial design to overcome validity and recruitment problems in primary care research. Fam Pract. 2004;21(2):213–8. https://doi.org/10.1093/fampra/cmh219 . PMID: 15020394.

Sahin D, Yaffe MJ, Sussman T, et al. A mixed studies literature review of family physicians’ participation in research. Fam Med. 2014;46(7):503–14 PMID: 25058542.

Voorhees JR, Xierali IM, Bazemore AW, et al. A small percentage of family physicians report time devoted to research. J Am Board Fam Med. 2013;26(1):7–8. https://doi.org/10.3122/jabfm.2013.01.120125 . PMID: 23288274.

Johnston S, Liddy C, Hogg W, et al. Barriers and facilitators to recruitment of physicians and practices for primary care health services research at one centre. BMC Med Res Methodol. 2010;10(1):109. https://doi.org/10.1186/1471-2288-10-109 . PMID: 21144048.

Leathem CS, Cupples ME, Byrne MC, et al. Identifying strategies to maximise recruitment and retention of practices and patients in a multicentre randomised controlled trial of an intervention to optimise secondary prevention for coronary heart disease in primary care. BMC Med Res Methodol. 2009;9(1):40. https://doi.org/10.1186/1471-2288-9-40 . PMID: 19545366.

Rosemann T, Szecsenyi J. General practitioners’ attitudes towards research in primary care: qualitative results of a cross sectional study. BMC Fam Pract. 2004;5(1):31. https://doi.org/10.1186/1471-2296-5-31 . PMID: 15613246.

van der Wouden JC, Blankenstein AH, Huibers MJH, et al. Survey among 78 studies showed that Lasagna’s law holds in Dutch primary care research. J Clin Epidemiol. 2007;60(8):819–24. https://doi.org/10.1016/j.jclinepi.2006.11.010 . PMID: 17606178.

van der Gaag WH, van den Berg R, Koes BW, Bohnen AM, et al. Discontinuation of a randomised controlled trial in general practice due to unsuccessful patient recruitment. BJGP Open. 2017;1(3):bjgpopen17X101085. https://doi.org/10.3399/bjgpopen17X101085 . PMID: 30564680.

Jowett S, Macleod J, Wilson S, et al. Research in primary care: extent of involvement and perceived determinants among practitioners from one English region. Br J Gen Pract. 2000;50(454):387–9 PMID: 10897537.

CAS   PubMed   PubMed Central   Google Scholar  

Mason V, Shaw A, Wiles N, et al. GPs’ experiences of primary care mental health research: a qualitative study of the barriers to recruitment. Fam Pract. 2007;24(5):518–25. https://doi.org/10.1093/fampra/cmm047 . PMID: 17698979.

Article   CAS   PubMed   Google Scholar  

FerrandDevouge E, Biard M, Beuzeboc J, et al. Motivations and willingness of general practitioners in France to participate in primary care research as investigators. Fam Pract. 2019;36(5):552–9. https://doi.org/10.1093/fampra/cmy126 . PMID: 30605509.

Loskutova NY, Smail C, Ajayi K, et al. Recruiting primary care practices for practice-based research: a case study of a group-randomized study (TRANSLATE CKD) recruitment process. Fam Pract. 2017;35(1):111–6. https://doi.org/10.1093/fampra/cmx064 . PMID: 28985294.

Article   PubMed Central   Google Scholar  

Messner DA, Moloney R, Warriner AH, et al. Understanding practice-based research participation: The differing motivations of engaged vs. non-engaged clinicians in pragmatic clinical trials. Contemp Clin Trials Commun. 2016;4:136. https://doi.org/10.1016/j.conctc.2016.08.003 . PMID: 29736476.

Peters-Klimm F, Freund T, Bentner M, et al. “In the Practice and for the Practice!” A Workshop Report of the Development of an Academic Practice-Based Research Net [„Aus der Praxis und für die Praxis!“ Aufbau eines Netzes von akademischen hausärztlichen Forschungspraxen – ein Werkstattbericht]. Z Allg Med. 2013;89(4):183–8. https://doi.org/10.3238/zfa.2013.0183-0188 .

Tan ACW, Clemson L, Mackenzie L, et al. Strategies for recruitment in general practice settings: the iSOLVE fall prevention pragmatic cluster randomised controlled trial. BMC Med Res Methodol. 2019;19(1):236. https://doi.org/10.1186/s12874-019-0869-7 . PMID: 31829133.

Groenewegen PP, Greß S, Schäfer W. General practitioners’ participation in a large, multicountrycombined general practitioner-patient survey: recruitment procedures and participation rate. Int J Family Med. 2016;2016:4929432. https://doi.org/10.1155/2016/4929432 . PMID: 27047689.

Tong SF, Ng CJ, Lee VKM, et al. Decision making process and factors contributing to research participation among general practitioners: a grounded theory study. PLoS One. 2018;13(4):e0196379. https://doi.org/10.1371/journal.pone.0196379 . PMID: 29694439.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gorringe J. Initial preparation for clinical trials. Princ Pract Clin Trials Edinburgh Livingstone. 1970;4146:38.

Google Scholar  

Wangler J, Jansky M. Real progress for care? viewpoints and experiences of family physicians with regard to the german innovation fund [Echter Fortschritt für die Versorgung? - Standpunkte und Erfahrungen von Hausärzt*innen in Bezug auf den Innovationsfonds]. Z Allg Med. 2022;98(4):148–53. https://doi.org/10.53180/zfa.2022.0148-0153 .

Wangler J, Jansky M. The German Innovation Fund and primary care – What expectations and experiences do general practitioners have with regard to participating in innovative care models? [Innovationsfonds und Primärversorgung – Welche Erwartungen und Erfahrungen vertreten Hausärzt*innen in Bezug auf die Teilnahme an innovativen Versorgungsmodellen?]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2022;65(6):697–705. https://doi.org/10.1007/s00103-022-03533-y . PMID: 35476151.

Wangler J, Jansky M: Experiences with Innovation Fund healthcare models in primary care: A qualitative study amongst German general practitioners. Wien Med Wochenschr 2022 [Online First]. PMID: 35503146 https://doi.org/10.1007/s10354-022-00935-0 .

Deutscher Bundestag: Drucksache 19/8500. Available from: URL: https://dipbt.bundestag.de/doc/btd/19/085/1908500.pdf . Cited 2023; May 26.

Schmitt J, Petzold T, Nellessen-Martens G, et al. Prioritization and consentation of criteria for the appraisal, funding and evaluation of projects from the german innovationsfonds: a multi-perspective delphi study [Priorisierung und Konsentierung von Begutachtungs-, Förder- und Evaluationskriterien für Projekte aus dem Innovationsfonds: Eine multiperspektivische Delphi-Studie]. Gesundheitswesen. 2015;77(8/9):570–9. https://doi.org/10.1055/s-0035-1555898 . PMID: 26270043.

Heytens H, Walther F, Keßler L, et al. Characteristics of innovation fund-supported intervention studies: review and document analysis of study protocols, publications and final reports [Charakteristika von durch den Innovationsfonds geförderten Interventionsstudien: Review und Dokumentenanalyse von Studienprotokollen, Publikationen und Abschlussberichten]. Gesundheitswesen. 2021;83(5):20–37. https://doi.org/10.1055/a-1448-2412 . PMID: 34015857.

Schmitt J, Geraedts M, Maier B, et al. On the status quo and the planned further development of the Innovation Fund (Version 3, February 4th, 2020) [Zum Status quo und der vorgesehenen Weiterentwicklung des Innovationsfonds (Version 3, 4.2.2020)]. Gesundheitswesen. 2020;82(5):374–7. https://doi.org/10.1055/a-1119-3984 .

Bablok I, Binder H, Stelzer D, et al. Primary dementia care based on the individual needs of the patient: study protocol of the cluster randomized controlled trial, DemStepCare. BMC Geriatrics. 2021;21(1):222. https://doi.org/10.1186/s12877-021-02114-z . PMID: 33794789.

Wangler J, Geschke K, Wuttke-Linnemann et al.: Prevention of dementia-related care crises in a general practitioner-based setting—the innovative care model DemStepCare [Prävention von demenzbedingten Versorgungskrisen im hausarztbasierten Setting – das Projekt DemStepCare als innovative Versorgungsform]. Präv Gesundheitsf 2022 [Online First]. https://doi.org/10.1007/s11553-022-00931-7 .

Wangler J, Jansky M. Evaluation of elevated liver values in primary care - a series of studies on the status quo of care in Germany with special reference to alcoholic liver disease. BMC Prim Care. 2022;23(1):104. https://doi.org/10.1186/s12875-022-01714-x] . PMID: 35501826.

Wangler J, Jansky M. Attitudes to and experience of disease management programs in primary care – an exploratory survey of general practitioners in Germany. Wien Med Wochenschr. 2021;171(13–14):310–20. https://doi.org/10.1007/s10354-021-00867-1 . PMID: 34338907.

Wangler J, Jansky M. What is the significance of guidelines in the primary care setting? Results of an exploratory online survey of general practitioners in Germany. Wien Med Wochenschr. 2021;171(13–14):321–9. https://doi.org/10.1007/s10354-021-00849-3 . PMID: 34101082.

Mayring P: Qualitative content analysis. Theoretical foundation, basic procedures and software solution. Available from: URL: https://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173 . Cited 2023; May 26.

Jones KM, Dixon ME, Dixon JB. General practice research–does gender affect the decision to participate? Aust Fam Physician. 2012;41(6):419–23 PMID: 22675685.

Tawo S, Gasser S, Gemperli A, et al. General practitioners’ willingness to participate in research: a survey in Central Switzerland. PLoS One. 2019;14(3):e0213358. https://doi.org/10.1371/journal.pone.0213358 . PMID: 30822332.

Güthlin C, Beyer M, Erler A, et al. Recruitment of family practitioners for research – experiences from five studies [Rekrutierung von Hausarztpraxen für Forschungsprojekte - Erfahrungen aus fünf allgemeinmedizinischen Studien]. Z Allg Med. 2012;88(4):173–81. https://doi.org/10.3238/zfa.2012.0173-0181 .

Hummers-Pradier E, Scheidt-Nave C, Martin H, et al. Simply no time? Barriers to GPs’ participation in primary health care research. Fam Pract. 2008;25(2):105–12. https://doi.org/10.1093/fampra/cmn015 . PMID: 18417465.

Poß-Doering R, Kunz A, Pohlmann S, et al. Engaging family physicians in a research project to pilot an electronic health record [Hausarztpraxen für ein Forschungsprojekt zur Erprobung einer elektronischen Patientenakte gewinnen]. Z Allg Med. 2019;95(12):515–9. https://doi.org/10.3238/zfa.2019.0515-0519 .

Bleidorn J, Heim S, Lingner J, et al. Family Practitioners’ View of Research in Practice Based Research Networks. A Focus Group Analysis [Wie sehen Hausärzte allgemeinmedizinische Forschung im Praxennetz? Eine Fokusgruppenanalyse]. Z Allg Med. 2014;90(9):348–53. https://doi.org/10.3238/zfa.2014.0348-0353 .

Bleidorn J, Voigt I, Wrede J, et al. Keeping the wire hot with calls? recruiting family practices for a health care research project [Anrufen ohne Ende? Über das Gewinnen hausärztlicher Praxen für ein Versorgungsforschungsprojekt]. Z Allg Med. 2012;88(2):61–8. https://doi.org/10.3238/zfa.2012.0061-0068 .

Wolf F, Kreuse M, Wiegand S, et al. Family physicians’ participation in a research practice network: which motivations play a decisive role? results of a survey in Thuringia [Was motiviert Hausärztinnen und Hausärzte, sich an einem Forschungspraxennetz zu beteiligen? Ergebnisse einer Befragung in Thüringen]. Z Allg Med. 2020;96(12):490–5. https://doi.org/10.3238/zfa.2020.0490-0495 .

Beerheide R: Projects with an uncertain future [Projekte mit ungewisser Zukunft]. Dtsch Arztebl 2020; 117: A 188–190.

Cadwallader J-S, Lebeau J-P, Lasserre E, et al. Patient and professional attitudes towards research in general practice: the RepR qualitative study. BMC Fam Pract. 2014;15:136. https://doi.org/10.1186/1471-2296-15-136 . PMID: 25047280.

Brodaty H, Gibson LH, Waine ML, et al. Research in general practice: a survey of incentives and disincentives for research participation. Ment Health Fam Med. 2013;10(3):163–73 PMID: 24427184.

PubMed   PubMed Central   Google Scholar  

Schmid S. Innovation Fund and standard care. Hardly any effects for general practitioners [Innovationsfonds und Regelversorgung. Kaum Effekte für Hausarztpraxen]. doctors|today. 2021;1(7):38.

Hoffmann AE, Leege EK, Plane MB, et al. Clinician and staff perspectives on participating in Practice-based Research (PBR): a report from the Wisconsin Research and Education Network (WREN). J Am Board Fam Med. 2015;28(5):639–48. https://doi.org/10.3122/jabfm.2015.05.150038 . PMID: 26355136.

Huas C, Petek D, Diaz E, et al. Strategies to improve research capacity across European general practice: the views of members of EGPRN and Wonca Europe. Eur J Gen Pract. 2019;25(1):25–31. https://doi.org/10.1080/13814788.2018.1546282 . PMID: 30607993.

Supper I, Ecochard R, Bois C, et al. How do French GPs consider participating in primary care research: the DRIM study. Fam Pract. 2011;28(2):226–32. https://doi.org/10.1093/fampra/cmq073 . PMID: 20829279.

Peters-Klimm F, Hermann K, Gagyor I, et al. Experiences and attitudes regarding practice-based clinical trials: results of a survey among German primary care physicians Erfahrungen und Einstellungen zu Klinischen Studien in der Hausarztpraxis. Ergebnisse einer Befragung von deutschen Hausärzten Gesundheitswesen. 2013;75(5):321–7. https://doi.org/10.1055/s-0032-1321742 . PMID: 22893207.

Tsiligianni I, Oikonomou N, Papaioannou A, et al. Exploring primary care physician experiences conducting practice-based research on adult vaccination: a qualitative evaluation study in Greece. Fam Pract. 2020;37(6):828–33. https://doi.org/10.1093/fampra/cmaa063 . PMID: 32779702.

Koskela TH. Building a primary care research network – lessons to learn. Scand J Prim Health Care. 2017;35(3):229–30. https://doi.org/10.1080/02813432.2017.1358439 . PMID: 28847229.

Souza DLB, Oliveras-Fabregas A, Minobes-Molina E, et al. Trends of multimorbidity in 15 European countries: a population-based study in community-dwelling adults aged 50 and over. BMC Public Health. 2021;21(1):76. https://doi.org/10.1080/02813432.2017.1358439 . PMID: 28847229.

Rieckert A, Trampisch US, Klaaßen-Mielke R, et al. Polypharmacy in older patients with chronic diseases: a cross-sectional analysis of factors associated with excessive polypharmacy. BMC Fam Pract. 2018;19(1):113. https://doi.org/10.1186/s12875-018-0795-5 . PMID: 30021528.

Calmbach WL, Ryan JG, Baldwin L-M, et al. Practice-based research networks (PBRNs): meeting the challenges of the future. J Am Board Fam Med. 2012;25(5):572–6. https://doi.org/10.3122/jabfm.2012.05.120064 . PMID: 22956692.

Pirotta M, Temple-Smith M. Practice-based research networks. Aust Fam Physician. 2017;46(10):793–5 PMID: 29036782.

O’Regan A, Hayes P, O’Connor R, et al. The university of Limerick education and research network for general practice (ULEARN-GP): practice characteristics and general practitioner perspectives. BMC Fam Pract. 2020;21(1):25. https://doi.org/10.1186/s12875-020-1100-y . PMID: 32024480.

Salmon P, Peters S, Rogers A, et al. Peering through the barriers in GPs’ explanations for declining to participate in research: the role of professional autonomy and the economy of time. Fam Pract. 2007;24(3):269–75. https://doi.org/10.1093/fampra/cmm015 . PMID: 1750477.

Bjorvatn B, Kristoffersen ES, Halvorsen PA et al. New infrastructure for research in general practice [Ny infrastruktur for allmennmedisinsk forskning]. Tidsskr nor Laegeforen 2019; 139(1). https://doi.org/10.4045/tidsskr.18.0689 . PMID: 30644691.

Goodyear-Smith F, Bazemore A, Coffman M, et al. Primary care research priorities in low-and middle-income countries. Ann Fam Med. 2019;17(1):31–5. https://doi.org/10.1370/afm.2329 . PMID: 30670392.

Beasley JW, Dovey S, Geffen LN, et al. The contribution of family doctors to primary care research: a global perspective from the international federation of primary care research networks (IFPCRN). Prim Health Care Res Dev. 2004;5(4):307–16. https://doi.org/10.1191/1463423604pc221oa .

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Centre for General Medicine and Geriatrics, University Medical Center of the Johannes Gutenberg, University Mainz, Am Pulverturm 13, Mainz, 55131, Germany

Julian Wangler & Michael Jansky

You can also search for this author in PubMed   Google Scholar

Contributions

JW prepared, coordinated and implemented the project. Both JW and MJ contributed to the project design, analysis of transcripts and drafting of the manuscript. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Julian Wangler .

Ethics declarations

Ethics approval and consent to partricipate.

All methods were carried out in accordance with relevant guidelines and regulations.

Since this is an overview article on the status of the topic under discussion, the Ethics Commission of the State of Rhineland-Palatinate, Germany, informed us that approval by an ethics committee was not necessary.

Written informed consent for participation was obtained from all participants before the start of the sub-studies [ 29 , 30 , 31 ]. The respondents received information about the aim and purpose of the respective study and were informed that it was an anonymous survey in accordance with the existing data protection standards. Furthermore, it was made clear that the data will only be used for scientific purposes.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: multimedia appendix 1..

Categorical system, qualitative in-depth study.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wangler, J., Jansky, M. Primary care involvement in clinical research – prerequisites, motivators, and barriers: results from a study series. Arch Public Health 82 , 41 (2024). https://doi.org/10.1186/s13690-024-01272-x

Download citation

Received : 31 August 2023

Accepted : 12 March 2024

Published : 20 March 2024

DOI : https://doi.org/10.1186/s13690-024-01272-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical research
• Health services research
• Cluster-randomised controlled trial
• Research network
• Innovation Fund
• General practitioner
• Primary care

Archives of Public Health

ISSN: 2049-3258

• General enquiries: [email protected]