find a research study that is reported in the news

Subscribe or renew today

Every print subscription comes with full digital access

Science News

Pelvic exams at hospitals require written consent, new U.S. guidelines say 

Hospitals must now get written consent to perform pelvic, breast, prostate and rectal exams on sedated patients or risk losing federal funding.

A new method of making diamonds doesn’t require extreme pressure 

A vaccine for bees has an unexpected effect

Glowing octocorals have been around for at least 540 million years

Plant ‘time bombs’ highlight how sneaky invasive species can be

Separating science fact from fiction in Netflix’s ‘3 Body Problem’ 

A rapid shift in ocean currents could imperil the world’s largest ice shelf

Trending stories.

A new look at Ötzi the Iceman’s DNA reveals new ancestry and other surprises

A new U.S. tool maps where heat will be dangerous for your health

Social media harms teens’ mental health, mounting evidence shows. What now?

Sign Up For the Latest from Science News

Headlines and summaries of the latest Science News articles, delivered to your inbox

Thank you for signing up!

There was a problem signing you up.

Spotlight on Health

Language models may miss signs of depression in Black people’s Facebook posts

Researchers hope to use social media posts to identify population-wide spikes in depression. That approach could miss Black people, a study shows.

What can period blood reveal about a person’s health?

Immune cells’ intense reaction to the coronavirus may lead to pneumonia, from the archives.

How to Stop a Biological Clock

March 9, 1974 Vol. 105 No. #10

Science News Magazine

April 20, 2024 Vol. 205 No. 8

A new study has linked microplastics to heart attacks and strokes. Here’s what we know 

Dogs know words for their favorite toys.

Featured Media

How brain implants are treating depression

This six-part series follows people whose lives have been changed by an experimental treatment called deep brain stimulation.

Explore the expected life spans of different dog breeds

Does this drone image show a newborn white shark? Experts aren’t sure

Parrots can move along thin branches using ‘beakiation’

How ghostly neutrinos could explain the universe’s matter mystery, follow science news.

• Follow Science News on X
• Follow Science News on Facebook
• Follow Science News on Instagram

More Stories

This marine alga is the first known eukaryote to pull nitrogen from air

During a total solar eclipse, some colors really pop. here’s why, this is the first egg-laying amphibian found to feed its babies ‘milk’.

These are the chemicals that give teens pungent body odor

Here’s why covid-19 isn’t seasonal so far, human embryo replicas have gotten more complex. here’s what you need to know.

‘On the Move’ examines how climate change will alter where people live

Waterlogged soils can give hurricanes new life after they arrive on land, cold, dry snaps accompanied three plagues that struck the roman empire.

How a 19th century astronomer can help you watch the total solar eclipse

Jwst spies hints of a neutron star left behind by supernova 1987a, astronomers are puzzled over an enigmatic companion to a pulsar.

Physicists take a major step toward making a nuclear clock

A teeny device can measure subtle shifts in earth’s gravitational field, 50 years ago, superconductors were warming up, health & medicine, aimee grant investigates the needs of autistic people, teens are using an unregulated form of thc. here’s what we know.

Polar forests may have just solved a solar storm mystery

Earth’s oldest known earthquake was probably triggered by plate tectonics, climate change is changing how we keep time, science & society.

In ‘Get the Picture,’ science helps explore the meaning of art

What  science news  saw during the solar eclipse, your last-minute guide to the 2024 total solar eclipse.

Subscribers, enter your e-mail address for full access to the Science News archives and digital editions.

Not a subscriber? Become one now .

Advertisement

Supported by

A Megaraptor Emerges From Footprint Fossils

A series of foot tracks in southeastern China points to the discovery of a giant velociraptor relative, paleontologists suggest in a new study.

By Jack Tamisiea

Could Eating Less Help You Live Longer?

Calorie restriction and intermittent fasting both increase longevity in animals, aging experts say. Here’s what that means for you.

By Dana G. Smith

In Coral Fossils, Searching for the First Glow of Bioluminescence

A new study resets the timing for the emergence of bioluminescence back to millions of years earlier than previously thought.

By Sam Jones

Howie Schwab, ESPN Researcher and Trivia Star, Dies at 63

He stepped out of his behind-the-scenes role in 2004 when he was cast as the ultimate sports know-it-all on the game show “Stump the Schwab.”

By Richard Sandomir

New Study Bolsters Idea of Athletic Differences Between Men and Trans Women

Research financed by the International Olympic Committee introduced new data to the unsettled and fractious debate about bans on transgender athletes.

By Jeré Longman

Why Are Younger Adults Developing This Common Heart Condition?

New research suggests that A-fib may be more prevalent, and more dangerous, in people under 65 than previously thought.

By Dani Blum

How Do We Know What Animals Are Really Feeling?

Animal-welfare science tries to get inside the minds of a huge range of species — in order to help improve their lives.

By Bill Wasik and Monica Murphy

Yellowstone’s Wolves: A Debate Over Their Role in the Park’s Ecosystem

New research questions the long-held theory that reintroduction of such a predator caused a trophic cascade, spawning renewal of vegetation and spurring biodiversity.

By Jim Robbins

Generative A.I. Arrives in the Gene Editing World of CRISPR

Much as ChatGPT generates poetry, a new A.I. system devises blueprints for microscopic mechanisms that can edit your DNA.

By Cade Metz

Scientists Fault Federal Response to Bird Flu Outbreaks on Dairy Farms

Officials have shared little information, saying the outbreak was limited. But asymptomatic cows in North Carolina have changed the assessment.

By Apoorva Mandavilli and Emily Anthes

1 in 5 U.S. Cancer Patients Join in Medical Research

HealthDay April 3, 2024

CDC: Tuberculosis Cases Increasing

While the U.S. has one of the lowest rates of tuberculosis in the world, researchers found that cases increased 16% from 2022 to 2023.

Cecelia Smith-Schoenwalder March 28, 2024

Researchers Find New Way to Curb Asthma Attacks

HealthDay March 26, 2024

Biden to Sign Order Expanding Health Research in Women

HealthDay March 18, 2024

Politics Hasn't Shaken Most Americans' Faith in Science: Study

HealthDay March 12, 2024

Jill Biden Announces $100 Million for Research on Women's Health

HealthDay Feb. 22, 2024

Study Links Living Alone to Depression

New research bound to influence conversations about America’s ‘loneliness epidemic’ suggests living alone could have implications for physical and mental health.

Steven Ross Johnson Feb. 15, 2024

Scientists Discover New Way to Fight Estrogen-Fueled Breast Cancer

HealthDay Feb. 14, 2024

Food Insecurity Tied to Early Death

An inability to get adequate food is shaving years off people’s lives in the U.S., a new study suggests.

Steven Ross Johnson Jan. 29, 2024

Dana Farber Cancer Center to Retract or Fix Dozens of Studies

HealthDay Jan. 23, 2024

America 2024

Climate change damage could cost $38 trillion per year by 2050, study finds

• Medium Text

Sign up here.

Reporting by Riham Alkousaa, Editing by Rachel More, Katy Daigle and Barbara Lewis

Our Standards: The Thomson Reuters Trust Principles. New Tab , opens new tab

Thomson Reuters

Riham Alkousaa is the energy and climate change correspondent for Reuters in Germany, covering Europe’s biggest economy's green transition and Europe’s energy crisis. Alkousaa is a Columbia University Journalism School graduate and has 10 years of experience as a journalist covering Europe’s refugee crisis and the Syrian civil war for publications such Der Spiegel Magazine, USA Today and the Washington Times. Alkousaa was on two teams that won Reuters Journalist of the year awards in 2022 for her coverage of Europe’s energy crisis and the Ukraine war. She has also won the Foreign Press Association Award in 2017 in New York and the White House Correspondent Association Scholarship that year.

Business Chevron

Oil eases as US demand concerns outweigh fears over Middle East conflicts

Oil prices eased in early trade on Thursday as concerns about a potential slowdown in the U.S. economy amid prospects for delayed interest rate cuts outweighed worries over the risk of expanding conflict in the Middle East.

Singapore's Keppel said on Thursday its first-quarter net profit excluding the effects of legacy offshore and marine assets was higher, boosted by strong performance in the global asset manager's infrastructure and connectivity segments.

• Skip to main content
• Keyboard shortcuts for audio player

• Your Health
• Treatments & Tests
• Health Inc.
• Public Health

How to Thrive as You Age

Got tinnitus a device that tickles the tongue helps this musician find relief.

Allison Aubrey

After using the Lenire device for an hour each day for 12 weeks, Victoria Banks says her tinnitus is "barely noticeable." David Petrelli/Victoria Banks hide caption

After using the Lenire device for an hour each day for 12 weeks, Victoria Banks says her tinnitus is "barely noticeable."

Imagine if every moment is filled with a high-pitched buzz or ring that you can't turn off.

More than 25 million adults in the U.S., have a condition called tinnitus, according to the American Tinnitus Association. It can be stressful, even panic-inducing and difficult to manage. Dozens of factors can contribute to the onset of tinnitus, including hearing loss, exposure to loud noise or a viral illness.

There's no cure, but there are a range of strategies to reduce the symptoms and make it less bothersome, including hearing aids, mindfulness therapy , and one newer option – a device approved by the FDA to treat tinnitus using electrical stimulation of the tongue.

The device has helped Victoria Banks, a singer and songwriter in Nashville, Tenn., who developed tinnitus about three years ago.

"The noise in my head felt like a bunch of cicadas," Banks says. "It was terrifying." The buzz made it difficult for her to sing and listen to music. "It can be absolutely debilitating," she says.

Shots - Health News

Tinnitus bothers millions of americans. here's how to turn down the noise.

Banks tried taking dietary supplements , but those didn't help. She also stepped up exercise, but that didn't bring relief either. Then she read about a device called Lenire, which was approved by the FDA in March 2023. It includes a plastic mouthpiece with stainless steel electrodes that electrically stimulate the tongue. It is the first device of its kind to be approved for tinnitus.

"This had worked for other people, and I thought I'm willing to try anything at this point," Banks recalls.

She sought out audiologist Brian Fligor, who treats severe cases of tinnitus in the Boston area. Fligor was impressed by the results of a clinical trial that found 84% of participants who tried Lenire experienced a significant reduction in symptoms. He became one of the first providers in the U.S. to use the device with his patients. Fligor also served on an advisory panel assembled by the company who developed it.

"A good candidate for this device is somebody who's had tinnitus for at least three months," Fligor says, emphasizing that people should be evaluated first to make sure there's not an underlying medical issue.

Tinnitus often accompanies hearing loss, but Victoria Banks' hearing was fine and she had no other medical issue, so she was a good candidate.

Banks used the device for an hour each day for 12 weeks. During the hour-long sessions, the electrical stimulation "tickles" the tongue, she says. In addition, the device includes a set of headphones that play a series of tones and ocean-wave sounds.

The device works, in part, by shifting the brain's attention away from the buzz. We're wired to focus on important information coming into our brains, Fligor says. Think of it as a spotlight at a show pointed at the most important thing on the stage. "When you have tinnitus and you're frustrated or angry or scared by it, that spotlight gets really strong and focused on the tinnitus," Fligor says.

"It's the combination of what you're feeling through the nerves in your tongue and what you're hearing through your ears happening in synchrony that causes the spotlight in your brain to not be so stuck on the tinnitus," Fligor explains.

A clinical trial found 84% of people who used the device experienced a significant reduction in symptoms. Brian Fligor hide caption

A clinical trial found 84% of people who used the device experienced a significant reduction in symptoms.

"It unsticks your spotlight" and helps desensitize people to the perceived noise that their tinnitus creates, he says.

Banks says the ringing in her ears did not completely disappear, but now it's barely noticeable on most days.

"It's kind of like if I lived near a waterfall and the waterfall was constantly going," she says. Over time, the waterfall sound fades out of consciousness.

"My brain is now focusing on other things," and the buzz is no longer so distracting. She's back to listening to music, writing music, and performing music." I'm doing all of those things," she says.

When the buzz comes back into focus, Banks says a refresher session with the device helps.

A clinical trial found that 84% of people who tried Lenire , saw significant improvements in their condition. To measure changes, the participants took a questionnaire that asked them to rate how much tinnitus was impacting their sleep, sense of control, feelings of well-being and quality of life. After 12 weeks of using the device, participants improved by an average of 14 points.

"Where this device fits into the big picture, is that it's not a cure-all, but it's quickly become my go-to," for people who do not respond to other ways of managing tinnitus, Fligor says.

One down-side is the cost. Banks paid about $4,000 for the Lenire device, and insurance doesn't cover it. She put the expense on her credit card and paid it off gradually.

Fligor hopes that as the evidence of its effectiveness accumulates, insurers will begin to cover it. Despite the cost, more than 80% of participants in the clinical trial said they would recommend the device to a friend with tinnitus.

But, it's unclear how long the benefits last. Clinical trials have only evaluated Lenire over a 1-year period. "How durable are the effects? We don't really know yet," says audiologist Marc Fagelson, the scientific advisory committee chair of the American Tinnitus Association. He says research is promising but there's still more to learn.

Fagelson says the first step he takes with his patients is an evaluation for hearing loss. Research shows that hearing aids can be an effective treatment for tinnitus among people who have both tinnitus and hearing loss, which is much more common among older adults. An estimated one-third of adults 65 years of age and older who have hearing loss, also have tinnitus.

"We do see a lot of patients, even with very mild loss, who benefit from hearing aids," Fagelson says, but in his experience it's about 50-50 in terms of improving tinnitus. Often, he says people with tinnitus need to explore options beyond hearing aids.

Bruce Freeman , a scientist at the University of Pittsburgh Medical Center, says he's benefitted from both hearing aids and Lenire. He was fitted for the device in Ireland where it was developed, before it was available in the U.S.

Freeman agrees that the ringing never truly disappears, but the device has helped him manage the condition. He describes the sounds that play through the device headphones as very calming and "almost hypnotic" and combined with the tongue vibration, it's helped desensitize him to the ring.

Freeman – who is a research scientist – says he's impressed with the results of research, including a study published in Nature, Scientific Reports that points to significant improvements among clinical trial participants with tinnitus.

Freeman experienced a return of his symptoms when he stopped using the device. "Without it the tinnitus got worse," he says. Then, when he resumed use, it improved.

Freeman believes his long-term exposure to noisy instruments in his research laboratory may have played a role in his condition, and also a neck injury from a bicycle accident that fractured his vertebra. "All of those things converged," he says.

Freeman has developed several habits that help keep the high-pitched ring out of his consciousness and maintain good health. "One thing that does wonders is swimming," he says, pointing to the swooshing sound of water in his ears. "That's a form of mindfulness," he explains.

When it comes to the ring of tinnitus, "it comes and goes," Freeman says. For now, it has subsided into the background, he told me with a sense of relief. "The last two years have been great," he says – a combination of the device, hearing aids and the mindfulness that comes from a swim.

This story was edited by Jane Greenhalgh

• ringing in ears
• hearing loss

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 September 2021

News media coverage of COVID-19 public health and policy information

• Katharine J. Mach 1 , 2 ,
• Raúl Salas Reyes   ORCID: orcid.org/0000-0002-1683-8516 3 ,
• Brian Pentz   ORCID: orcid.org/0000-0003-2713-6699 3 ,
• Jennifer Taylor   ORCID: orcid.org/0000-0002-8301-3434 4 ,
• Clarissa A. Costa 3 ,
• Sandip G. Cruz 3 ,
• Kerronia E. Thomas 3 ,
• James C. Arnott   ORCID: orcid.org/0000-0003-3989-6724 5 ,
• Rosalind Donald 1 ,
• Kripa Jagannathan   ORCID: orcid.org/0000-0003-4584-8358 6 , 7 ,
• Christine J. Kirchhoff   ORCID: orcid.org/0000-0002-2686-6764 8 ,
• Laura C. Rosella   ORCID: orcid.org/0000-0003-4867-869X 9 &
• Nicole Klenk   ORCID: orcid.org/0000-0001-8224-6992 3  

Humanities and Social Sciences Communications volume  8 , Article number:  220 ( 2021 ) Cite this article

38k Accesses

55 Citations

72 Altmetric

Metrics details

• Cultural and media studies
• Science, technology and society

During a pandemic, news media play a crucial role in communicating public health and policy information. Traditional newspaper coverage is important amidst increasing disinformation, yet uncertainties make covering health risks and efforts to limit transmission difficult. This study assesses print and online newspaper coverage of the coronavirus disease COVID-19 for March 2020, when the global pandemic was declared, through August 2020 in three countries: Canada (with the lowest per-capita case and death rates during the study timeframe), the United Kingdom (with a pronounced early spike), and the United States (with persistently high rates). Tools previously validated for pandemic-related news records allow measurement of multiple indicators of scientific quality (i.e., reporting that reflects the state of scientific knowledge) and of sensationalism (i.e., strategies rendering news as more extraordinary than it really is). COVID-19 reporting had moderate scientific quality and low sensationalism across 1331 sampled articles in twelve newspapers spanning the political spectrums of the three countries. Newspapers oriented towards the populist-right had the lowest scientific quality in reporting, combined with very low sensationalism in some cases. Against a backdrop of world-leading disease rates, U.S. newspapers on the political left had more exposing coverage, e.g., focused on policy failures or misinformation, and more warning coverage, e.g., focused on the risks of the disease, compared to U.S. newspapers on the political right. Despite the generally assumed benefits of low sensationalism, pandemic-related coverage with low scientific quality that also failed to alert readers to public-health risks, misinformation, or policy failures may have exacerbated the public-health effects of the disease. Such complexities will likely remain central for both pandemic news media reporting and public-health strategies reliant upon it.

Similar content being viewed by others

Newspapers’ coverage of the COVID-19 pandemic in Eswatini: from distanciated re/presentations to socio-health panics

Spread of awareness of COVID-19 between December 2019 and March 2020 in France

Anti-intellectualism and the mass public’s response to the COVID-19 pandemic

Introduction.

News media reporting is understood to play a central role during national security and health emergencies (Laing, 2011 ; Klemm et al., 2016 ; Pieri, 2019 ). News coverage communicates risks to readers and shapes public perceptions through the amount, content, and tone of reporting. It simultaneously frames ongoing public debates about policy responses, including conflicting priorities relevant to the timing or stringency of implemented policies (Laing, 2011 ; Pieri, 2019 ). Pandemic policy-making requires rapid, iterative responses under conditions of knowledge deficit, as well as the coordination of multi-level public-health agencies and sectors (e.g., hospitals, schools, and workplaces) (Laing, 2011 ; Rosella et al., 2013 ). In these complex circumstances, news media serve as a primary source of health information and uncertainties and connect health professionals, policymakers, and the public in critical ways (Laing, 2011 ; Hoffman and Justicz, 2016 ). The quality and balance of scientific coverage, such as through reporting that reflects the state of scientific knowledge and is not overstated, affect trust in science and accountability for decision-making (Laing, 2011 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ).

Inadequate scientific quality in news coverage of past pandemics has posed risks and limited capacities to disseminate public-health guidance and coordinate responses (Hoffman and Justicz, 2016 ). Reporting on the state of scientific knowledge during a novel, evolving pandemic is challenging. Low-quality scientific reporting of pandemics may overstate or understate disease risks or the efficacy of protective measures for different individuals or fail to communicate the nature of the evidence. Such reporting may constrain the feasibility or effectiveness of options for policymakers directing government action, miss opportunities to inform individuals making health decisions, and increase the exposure of health professionals to disease. It can both exacerbate disease outcomes and generate unnecessary fear, in combination with other factors shaping perceptions among the public (Laing, 2011 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ). For example, news media reporting may have overly emphasized the threat of the 2009 A/H1N1 influenza (H1N1) pandemic with insufficient indication of available protective measures, and in pairing trustworthy information from credible scientists with uninformed opinions, it may have promoted a “false balance” (Laing, 2011 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ). Further, news coverage rapidly waned after the initial pandemic declaration even though public-health risks persisted (Klemm et al., 2016 ; Reintjes et al., 2016 ). Similar issues with media reporting occurred during the 2003 severe acute respiratory syndrome (SARS) outbreak and the 2014 Ebola outbreak (Hoffman and Justicz, 2016 ; Pieri, 2019 ).

During the COVID-19 pandemic, media representations of complex, rapidly evolving epidemiological science shape public understandings of the risks, measures to limit disease spread, and associated political and policy discourses. Traditional newspaper media coverage may have particular importance given simultaneous misinformation and disinformation, social fragmentation, political polarization, and failures of policy coordination, and national newspapers influence how other outlets cover the same subject across media platforms (Ball and Maxmen, 2020 ; Holtz et al., 2020 ; Thorp, 2020 ; Grossman et al., 2020 ). The COVID-19 pandemic creates an opportunity to assess the strengths and limitations of the media’s pandemic coverage and provide insights for future news media coverage. Such assessment also informs the communication strategies of public-health institutions and policymakers towards clear public-health guidance and coordinated responses across health systems (Laing, 2011 ; Hoffman and Justicz, 2016 ; Pieri, 2019 ).

Canada, the United Kingdom, and the United States, our countries of focus, differ in how they govern public health, including pandemic responses. In its constitutionally determined role, the Canadian federal government sets healthcare standards and administers funding to support the healthcare system spanning provinces and territories (Government of Canada, 2016 ). Pandemic health-related policies are set and implemented predominantly by provinces with federal guidance from Health Canada and the Public Health Agency of Canada (Adeel et al., 2020 ). The U.K. central government funds healthcare throughout the United Kingdom yet only sets policies for England. Northern Ireland, Scotland, and Wales each govern their own National Health Service systems. By contrast, the healthcare system in the United States is a complex mixture of public and private health insurance programs. The U.S. federal government generally adopts a leading role during national crises, although during the COVID-19 pandemic states and municipalities have led adoption and implementation of most policy measures to contain the spread of COVID-19 (Adeel et al., 2020 ). Prior to the COVID-19 pandemic, the 2019 Global Health Security Index ranked the United States first, United Kingdom second, and Canada fifth among 195 countries for preparedness to manage a serious disease outbreak (Cameron et al., 2019 ).

In this paper, we systematically quantify the amount, scientific quality, and sensationalism of newspaper media coverage of COVID-19 in Canada, the United Kingdom, and the United States. Newspapers studied span the political spectrum of each case-study country (Table 1 ) (Gentzkow and Shapiro, 2010 ; Puglisi and Snyder, 2015 ; Anderson and Coletto, 2017 ; Mitchell et al., 2018 ; Hönnige et al., 2020 ; Jurkowitz et al., 2020 ; Austen, 2020 ). Our analysis begins two weeks prior to COVID-19’s official recognition as a pandemic and follows its development over the subsequent five months (i.e., from 1 March 2020 to 15 August 2020). Given the volume of COVID-19 news media articles published over the timeframe of this study, we created a manageable corpus for analysis by randomly sampling one day of media coverage per week for six consecutive 4-week periods; we then randomly selected five eligible articles from each news outlet on each sampled day for the evaluation of scientific quality and sensationalism. In our evaluation, scientific quality refers to the alignment between reporting and the state of scientific evidence and its uncertainties, and sensationalism is a discursive strategy rendering news as more extraordinary, interesting, or relevant than it really is (Oxman et al., 1993 ; Molek-Kozakowska, 2013 ; Hoffman and Justicz, 2016 ). We apply previously validated survey tools developed to measure scientific quality and sensationalism of pandemic-related health news records in combination with broader methods from policy analyses of pandemic responses (SI Coding Tool) (Oxman et al., 1993 ; Rosella et al., 2013 ; Molek-Kozakowska, 2013 ; Reintjes et al., 2016 ; Hoffman and Justicz, 2016 ). We analyze (1) the COVID-19 public-health outcomes and policies in each country and (2) the amount, scientific quality, sensationalism, and topics of COVID-19 news media coverage across the political spectrum of each country.

Public health contextualization of news media analyses

To contextualize our news media analyses, we analyzed and visualized existing data sets on the number of COVID-19 cases, deaths, and tests in each country (e.g., Roser et al., 2020 ; CBC News, 2020 ; Public Health England and NHSX, 2020 ; CDC, 2020 ). We also recorded the key public-health declarations, policies, and guidance during the study time period (e.g., drawing from WHO, 2020a , 2020b ; see also SI Table S1 ). We tracked these decisions at international scales through to subnational scales in each country studied. Media analyses outlined below thereby were considered with respect to the reported number of cases and confirmed deaths and policy actions taken (Reintjes et al., 2016 ).

News media search strategy and inclusion criteria

Print and online news media records were retrieved from the Factiva database for news outlets across the political spectrum of Canada, the United Kingdom, and the United States (see Table 1 ) (Gentzkow and Shapiro, 2010 ; Puglisi and Snyder, 2015 ; Anderson and Coletto, 2017 ; Mitchell et al., 2018 ; Hönnige et al., 2020 ; Jurkowitz et al., 2020 ; Austen, 2020 ). Selected news media outlets have primary news products in print and online media, rather than television broadcasting or social media, and full article entries available in Factiva. Search terms included “coronavirus,” “COVID-19,” “epidemic,” “outbreak,” “pandemic,” or “SARS-CoV-2.” Individual English-language news articles were retrieved for sampled dates between 1 March 2020 and 15 August 2020. This period captures news media coverage prior to the declaration of the COVID-19 pandemic and over the subsequent five months.

Individual news records were screened to identify original news reporting (i.e., news reporting and news analysis articles) relevant to our study objectives. First, eligible articles must have a direct focus on the public-health implications of COVID-19 or on attempts to control its spread—in some or all of an article’s text. By excluding articles without this focus, we ensured all articles included in the study could contain scientific information on the public health effects or spread of COVID-19 and associated policies. Second, eligible articles must be focused on the newspaper’s country of publication (e.g., an article reporting on COVID-19 transmission or mitigation efforts in only New Zealand or China, without discussion of implications for the newspaper’s country of publication, would be excluded). We included this eligibility criterion to analyze science–policy interfaces and science–society interactions most proximate to the news outlets, although we acknowledge that articles about other countries may influence perceptions of readers even without direct discussion of implications for them. Third, eligible articles must be original news reporting or analysis, meaning we excluded opinion pieces, editorials, interview transcripts, microblogs, front-page snippets, news roundups, obituaries, advertisements, corrections memos, and letters to the editor; these excluded article types would have required distinct question framings beyond the scope of our codebook. This third criterion, therefore, ensured that coded responses could be compared coherently across articles for the different measures of scientific quality and sensationalism.

Sampling of news media articles

As the evaluation of scientific quality and sensationalism through manual coding is time intensive, and a very large number of COVID-19 news media articles were published during the timeframe of our study, we used a random sample of news media articles for analysis, prioritizing sampling during each week over the course of the study timeframe. The sample design enabled a manageable analysis of newspaper media coverage and potential changes over the timeframe of the study. First, the sample of news media articles was constructed by sampling one day of media coverage per week in consecutive four-week periods. These four days of the week were randomly sampled without replacement (Monday through Saturday only, not including Sunday in the sampling), given cyclic variation in news media coverage (Lacy et al., 2001 ; Riffe et al., 2016 ). The study timeframe was divided into six four-week periods of equal duration from 1 March to 15 August 2020.

Second, for each randomly sampled day, all available news records were retrieved from Factiva for the 12 news outlets (Table 1 ). Randomly selected articles were screened for eligibility, with the goal of identifying 5 eligible articles for each news outlet on each sampled day. In some cases, fewer than 5 eligible articles were published by a given outlet on a sampled day. In these cases, the full set of eligible articles was included in the study.

Analysis of scientific quality and sensationalism of news articles

The coding tool for measuring scientific quality and sensationalism of news article records was adapted from the final tool of Hoffman and Justicz, designed for evaluating pandemic-related health news records (Hoffman and Justicz, 2016 ). Scientific quality, as defined in that study, is “a measure of an article’s reliability and credibility on a given topic” (Hoffman and Justicz, 2016 ). Importantly, scientific quality is linked to the state of scientific understanding and its uncertainties at specific moments in time rather than being an absolute or objective characteristic. The codebook we applied for measuring scientific quality is therefore designed to be flexible and responsive to the inevitable shifts in scientific understanding that occur through time, most especially during a novel disease outbreak and evolving pandemic. Sensationalism, as defined in that study, is “a way of presenting articles to make them seem more interesting or extraordinary than they actually are” (Hoffman and Justicz, 2016 ).

Our coding tool (SI Coding Tool) included six questions for scientific quality (each evaluated on a scale from 1 to 5—5 corresponding to highest quality) and six questions for sensationalism (each evaluated on a scale from 1 to 5—5 corresponding to highest sensationalism). The question categories (SI Coding Tool) for assessing scientific quality were as follows: applicability, opinion versus facts, validity, precision, context, and global assessment (i.e., an overall assessment of the article’s scientific quality based on the five preceding specific measures). For sensationalism, the question categories (SI Coding Tool) included exposing, speculating, generalizing, warning, extolling, and global assessment (i.e., an overall assessment of the degree of sensationalism in the article based on the five preceding specific measures) (Oxman et al., 1993 ; Molek-Kozakowska, 2013 ; Hoffman and Justicz, 2016 ).

In addition, metadata collected for each article included the coder’s identity, the article title, the article’s sample date, the news outlet (including if the article was originally written by another outlet such as the Associated Press), the societal sector (up to 2 selected per article), and public-health measures discussed (SI Coding Tool).

Coder assignments, training, reliability assessment, and analysis

For each sampled day, two independent coders assessed all relevant news media records based on the scientific quality and sensationalism questions and article-attribute metadata. Coders recorded scores for each article through a Google-form version of the codebook (SI Coding Tool).

To ensure consistent application of the coding tool, substantial training and calibration occurred over a six-week period. First, the three coders in coordination with the project leadership team read national and international public-health agency descriptions of the coronavirus disease and associated public-health policies and measures. Second, the coders completed multiple rounds of individual coding of example news articles, followed by group discussions of application of the codebook. The group discussions considered difficult judgments and common versus unusual examples. The goal was to ensure consistent application of the coding tool across question categories and the range of article examples that arose. During the training and calibration phases of coding, we updated the codebook to include examples specific to news records on COVID-19 (SI Coding Tool), and we tracked illustrative examples (news articles and specific quotes) across the scale (1–3–5) for the scientific quality and sensationalism question categories. This process led to development of example answers particularly representative of low versus high scientific quality and low versus high sensationalism under each category of response. Additionally, we developed “decision rules” for the more unusual or challenging categories of examples to ensure consistency across coders, especially where disagreements arose in individually assigned responses.

Interrater reliability was assessed during the training and calibration stage and throughout the duration of the study. Where coders assigned scores for a given question that were 3 or 4 units apart on the 1–5 scale, a reconciliation discussion occurred; the small fraction of question responses in this category following the training stage enabled the coders and project team to continue developing and ensuring shared understanding of coding approaches for unusual or challenging applications. Weighted Cohen’s Kappa, with quadratic weighting, was applied given the high-inference codebook and ordinal data collected via a Likert scale, as previously done for related measures (Cohen, 1960 ; Fleiss and Cohen, 1973 ; Oxman et al., 1993 ; Antoine et al., 2014 ; Hoffman and Justicz, 2016 ; Tran et al., 2020 ). Coded data were analyzed with Kruskal–Wallis one-way analysis of variance and post-hoc multi-comparison pairwise tests (kruskal.test and kruskalmc in pgirmess package in R) (Giraudoux et al., 2018 ; R Core Team, 2020 ).

Public health and policy contexts

From March through August 2020, Canada, the United Kingdom, and the United States differed substantially in their public-health responses to COVID-19 and in health outcomes from the novel coronavirus disease (Fig. 1 ). Beginning in early March, all three countries implemented a combination of policy measures to contain the spread of COVID-19, including emergency laws, stay-at-home orders, mask mandates, school and business closures, border and travel restrictions, social distancing measures, and quarantines upon entry (SI Table S1 ). These restrictions were followed by gradual phases of reopening measures allowing restricted social and economic activities to occur. Across the three countries, the role of national versus subnational governments differed with respect to authority and actions on public-health guidance and care, resulting in differing timing and levels of coordination for both restrictions and reopening measures (SI Fig. S1 ). From March to August 2020, the United Kingdom experienced the highest death rate from COVID-19 (maximum 7-day average of 13.9 deaths per million people; Fig. 1 ), whereas the United States had the highest case rate of the three countries (maximum 7-day average of 203.5 cases per million), as well as the greatest cumulative number of cases and deaths globally (SI Figs. S2 - S3 ). Of the three countries, Canada had the most effective public-health outcomes as measured by per capita COVID-19 case or death rates (Fig. 1 ).

COVID-19 cases, deaths, and national-level policies are indicated for ( A ) Canada, ( B ) the United Kingdom, and ( C ) the United States. 7-day rolling averages of cases (left vertical axis, solid black line) and deaths (right vertical axis, dotted black line) per one million people are shown for the timeframe of this media study, 1 March through 15 August 2020 (Roser et al., 2020 ). The timeline for each country specifies national-level public-health policies and guidance, especially emergency declarations, school and non-essential business closures, travel and border restrictions, quarantines and social distancing, mask usage, and reopening phases. Implementation of enforceable policies (solid) and non-enforced guidance (dotted) is specified with vertical red lines, and corresponding reopening and relaxation of policies and guidance are specified with vertical blue lines. Detailed descriptions of national-level policies within each panel are provided in SI Table S1 .

The amount of pandemic media coverage

The studied news outlets differed in the amount of news media coverage related to COVID-19 from 1 March through 15 August 2020 (Fig. 2 ). The amount of coverage increased notably in March as case rates climbed in each country, subsequently decreasing gradually in May and June while case rates also declined. Across the 24 randomly sampled days, the 12 studied news outlets published 18,430 articles related to COVID-19. Of these, an estimated 4321 articles (23.4%) were eligible for inclusion in this study—that is, as news reporting or analysis relevant to the country of publication and containing a direct focus on COVID-19 public health or policy information (SI Figs. S4 - S5 ). Articles with a direct focus on COVID-19 public health or policy information (to a small or large extent) could be coded for the scientific quality of the reporting of this information and its sensationalism.

For each randomly sampled day ( A ) and each news outlet ( B ), the total number of individual news records is shown, based on Factiva database searches for articles related to COVID-19 public health and policy information (Methods). News articles are partitioned across the following categories: articles eligible for inclusion in our study (eligible), articles not focusing on the newspaper’s country of publication (location out of scope), articles that are not original news reporting or analysis (opinion/editorial/letters), and articles that include COVID-19-relevant search terms, but do not include any direct focus on COVID-19 public health or policy information (no direct focus). Estimated totals for these categories are calculated using (i) the total number of Factiva returns and (ii) the rates at which articles were assigned to these categories during the eligibility screening process for each outlet and randomly sampled day (SI Fig. S4 ). On the stacked bars, percentages of articles falling into each category are specified for each day ( A ) and news outlet ( B ).

Content analysis of pandemic media coverage

We collected a manageable, well-defined random sample of 1331 news media articles satisfying our eligibility criteria (SI Fig. S4 ) for coding of scientific quality and sensationalism (SI Coding Tool and Dataset S1 ). Six questions each for scientific quality and for sensationalism were evaluated on a scale from 1 to 5 (5 corresponding to highest scientific quality or sensationalism, 1 corresponding to lowest scientific quality or sensationalism). Question categories included for scientific quality: applicability, opinion versus facts, validity, precision, context, and global assessment (i.e., an overall assessment of the article’s scientific quality); and for sensationalism: exposing, speculating, generalizing, warning, extolling, and global assessment (i.e., an overall assessment of the degree of sensationalism in the article) (SI Coding Tool). For this content analysis, interrater reliability was moderate to substantial for the summative “global” assessment of scientific quality and sensationalism (SI Table S2 ). Reliability was similarly high for specific scientific quality and sensationalism measures, with the exception of questions for which coded scores displayed restriction of range or unbalanced distributions (e.g., “generalizing” scores of mostly 1 and 2, rather than ranging from 1 through 5 with balance around 3; SI Coding Tool and Dataset S1 ) (Hallgren, 2012 ; Tran et al., 2020 ).

The scientific quality of pandemic media coverage

The scientific quality of news media articles differed among news outlets across the political spectrums of the respective countries (Fig. 3 ). Within each country, the overall scientific quality of news reporting and analysis was lowest on the populist-right of the political spectrum (mean summative “global” scientific quality of 2.58, n = 106 articles, for Toronto Sun ; 2.67, n = 115, for Daily Mail ; and 2.28, n = 118, for New York Post ; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for within-country pairwise comparisons except Daily Mail versus Times of London and Telegraph , SI Table S3 ). For these outlets, lower scientific quality was especially evident for validity, precision, and context as measures of scientific quality (e.g., articles reporting claims without fact checking, specificity, or background details) (Fig. 3 ).

Scores for six scientific quality questions (SI Coding Tool) are shown (mean, 95% confidence interval) for articles ( n  = 1331) communicating COVID-19 public health or policy information (Fig. 2 ): ( A ) applicability, ( B ) opinion versus facts, ( C ) validity, ( D ) precision, ( E ) context, and ( F ) global assessment (i.e., an overall assessment of the article’s scientific quality). Each question was evaluated on a scale from 1 to 5 (5 corresponding to highest scientific quality). Sampled articles were published between 1 March and 15 August 2020 (Methods). Kruskal–Wallis one-way analysis of variance and post-hoc multi-comparison test statistics are in SI Table S3 .

The sensationalism of pandemic media coverage

The sensationalism of news media articles was low overall for all news outlets, although somewhat greater for outlets on the left and middle of the political spectrum in Canada and the United States (Fig. 4F ). In both countries, news outlets at the populist-right combined low scientific quality with low sensationalism (Figs. 3 F and 4F ). In Canada, the overall sensationalism of news reporting and analysis was lowest for the Toronto Sun (mean summative “global” sensationalism of 1.77, n  = 106 articles; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for pairwise comparisons with Globe and Mail and National Post , SI Table S3 ). In the United States, overall sensationalism was lower in the Wall Street Journal (mean global sensationalism of 2.03, n  = 118 articles) and New York Post (mean of 2.13, n  = 118), as compared to the New York Times (mean of 2.40, n  = 120) and Washington Post (mean of 2.38, n  = 119; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for pairwise comparisons, SI Table S3 ). For these outlets, lower sensationalism was especially observed for exposing, speculating, and warning as measures of sensationalism (Fig. 4 ). In the United Kingdom, overall sensationalism did not vary across news outlets ( p  = 0.283 for Kruskal–Wallis, SI Table S3 ).

Scores for six sensationalism questions (SI Coding Tool) are shown (mean, 95% confidence interval) for articles ( n  = 1331) communicating COVID-19 public health or policy information (Fig. 2 ): ( A ) exposing, ( B ) speculating, ( C ) generalizing, ( D ) warning, ( E ) extolling, and ( F ) global assessment (i.e., an overall assessment of the degree of sensationalism in the article). Each question was evaluated on a scale from 1 to 5 (5 corresponding to highest sensationalism). Sampled articles were published between 1 March and 15 August 2020 (Methods). Kruskal–Wallis one-way analysis of variance and post-hoc multi-comparison test statistics are in SI Table S3 .

Syndicated versus original reporting

Across all outlets, the scientific quality of original reporting (mean global scientific quality of 2.93, n  = 1278 articles) was significantly higher than the scientific quality of syndicated articles (mean of 2.71, n  = 54; p  = 0.020, Kruskal–Wallis; SI Fig. S6 and Table S4 ). Additionally, the sensationalism of syndicated articles (mean global sensationalism of 1.82, n  = 54 articles) was significantly lower than the sensationalism of original reporting (mean of 2.14, n  = 1278; p  ≤ 0.001, Kruskal–Wallis; SI Fig. S6 and Table S4 ). The Toronto Sun published the highest proportion of syndicated news articles by far, with 34% of the paper’s 106 coded articles originating from syndicated sources. Other news outlets with more than 1% of coded articles drawing from syndicated sources included the Toronto Star (6% of articles) and the National Post (11%).

Neither scientific quality nor sensationalism varied substantially through time, with the exception of lower scientific quality on 3 July 2020 resulting from limited coverage of the healthcare sector that day (Fig. 5 , SI Fig. S7 ).

The topics of news media articles analyzed ( A ) over the timeframe of this study and ( B ) by news outlet are specified. Sampled articles were published on randomly sampled days between 1 March and 15 August 2020 (Methods). The topic of each article ( n  = 1331) was categorized by societal sectors (up to 2 selected per article) related to healthcare, leisure and entertainment, economics and commerce, government and politics, and other social services.

The topics of pandemic media coverage

News media articles were categorized based on the societal sectors (up to 2 per article) that were the primary focus of each article (Fig. 5 ). The sectors, related to healthcare, leisure and entertainment, economics and commerce, government and politics, and other social services, are listed in full in Figs. 5 and 6 . Although all analyzed articles contained information on the public-health effects of COVID-19 or measures to limit its spread (SI Fig. S4 ), topics of focus differed widely, for example including recreation, the arts, transportation, or daycare, not just medical facilities or vaccine research.

Scientific quality ( A ) and sensationalism ( B ) of news media articles are indicated by the topics of articles. Overall global assessment scores for scientific quality and sensationalism (SI Coding Tool) are shown (mean, 95% confidence interval) for articles communicating COVID-19 public health or policy information (Fig. 2 ). For each article, scientific quality and sensationalism were each evaluated on a scale from 1 to 5 (5 corresponding to highest scientific quality or to highest sensationalism). Sampled articles ( n  = 1331) were published between 1 March and 15 August 2020 (Methods). The topic of each article was categorized across the following societal sectors (up to two selected per article): healthcare and institutions; health-related medical and technology research; family, lifestyle, and social groups; professional or high-level sports; public parks and recreation; culture and the arts; private sector impacts and measures; employment impacts and benefits; macroeconomics and economy-wide fiscal or stimulus measures; politics and elections; law enforcement and court systems; foreign affairs and international aid; transportation, shipping, and border closures; education and daycare; public services; and energy and the environment.

The topics of news media articles corresponded to scientific quality and sensationalism of news reporting and analysis to some degree (Fig. 6 ). News media articles related to healthcare, health institutions, and health-related research were most common (Fig. 5 ), and they had significantly greater scientific quality compared to articles on other topics (mean global scientific quality of 3.23 for healthcare and institutions and 3.72 for health-related research; p  ≤ 0.001 for Kruskal–Wallis, p  ≤ 0.05 for pairwise comparisons except with energy and the environment; Fig. 6A ). News media articles during the first four-week period studied, starting 1 March 2020, included the greatest focus (50.2% of coverage) on healthcare and related institutions and research (Fig. 5A ).

Sensationalism of articles related to politics and foreign affairs was greatest (mean global sensationalism of 2.53 for politics; and of 2.49 for foreign affairs; p  < 0.001 for Kruskal–Wallis, p  < 0.05 for pairwise comparisons of politics versus all sectors except foreign affairs, employment, and energy and the environment; Fig. 6B ). For example, sensational statements related to politics and foreign affairs could include exposing disinformation from political leaders or extolling political leaders for border closures as a pandemic or broader policy response. News outlets in the United States published the most articles related to politics and elections (63.8% of coverage across all outlets; Fig. 5B ).

Public-health policies consistently covered through time included measures related to social distancing, testing and tracing, and protective equipment and disinfection practices, while coverage of mask guidance and reopening policies increased over the course of the study (Fig. 7 ).

Sampled articles ( n  = 1331) were published on randomly sampled days between 1 March and 15 August 2020 (Methods). Public-health policies and measures in each article were coded under specific categories related to social distancing, testing and tracing, protective equipment and disinfection practices, reopening policies, vaccines and treatments, and more (all relevant categories selected for each article).

Managing the public health and societal risks of a pandemic requires iterative, informed decision-making by governments, individuals, and the private sector. News media play a central role in communicating public health and policy information, establishing accountability for decision-making, and shaping public perceptions through the number of news reports, their content, and their tone (Klemm et al., 2016 ; Reintjes et al., 2016 ). For news outlets spanning the political spectrum of three countries with contrasting public-health outcomes and policy responses (Fig. 1 ), based on a random sample of days, coverage related to COVID-19 increased substantially in March 2020 and declined gradually thereafter in May and June (Fig. 2 ), not rebounding even during the dramatic increase in U.S. COVID-19 cases in June and July (SI Figure S5 ). Understanding this news media reporting in the early stages of COVID-19 response provides important lessons for ensuring the accessibility of information in support of public health and gauging its degree of effectiveness in creating accountability for policy decisions.

News media reporting grappled with complications of scientific understanding and its uncertainties during the timeframe of our study, as assessed through our measures of validity, precision, and overall scientific quality. For example, the mechanisms of disease transmission, especially airborne transmission, were slow to be recognized, leading to dynamic adjustments of public-health guidance (e.g., for mask usage by the general public) (Zhang et al., 2020 ). Despite such uncertainties and frequent knowledge updates over time, the scientific quality of reporting was highest for the healthcare sector, also the most commonly occurring article topic (Fig. 6 ). The scientific quality of reporting overall did not improve as the pandemic proceeded and knowledge of COVID-19 increased, which may be attributed to shifts from healthcare to other topics of news media reporting (Fig. 5 and SI Fig. S7 ).

We did, however, identify major differences in the degree to which newspaper reporting of COVID-19 presented high-quality scientific information about the public-health effects of the coronavirus disease and measures to limit its spread. News media articles generally had moderate scientific quality overall (Fig. 3F ). Outlets on the populist-right of the political spectrum of each country, though, had significantly lower scientific quality in reporting related to COVID-19 (Fig. 3F ). Scientific quality was low especially for validity, precision, and context as measures of scientific quality, as well as for the distinction between opinion versus facts in some cases (e.g., articles reporting claims without fact checking, specificity, background details, or sourcing) (Fig. 3 ). These findings pertain to news reporting and analysis, rather than opinion pieces, editorials, or letters, which were excluded from the scope of news media articles we evaluated. The differences across outlets suggest that, in reading news reporting and analysis in different newspapers, readers access reporting of varying scientific quality related to the health risks and effectiveness of available measures to limit disease transmission.

Further, patterns of U.S. media reporting were correlated with failures of national leadership under the Trump Administration, and they may have both reflected and contributed to politicization of COVID-19 in the United States. During this study’s timeframe, the United States led the world in cases and deaths despite its pre-pandemic ranking as the country best equipped to manage a pandemic such as COVID-19 (Cameron et al., 2019 ). These public-health outcomes occurred against a backdrop of disinformation and failures of national leadership (Evanega et al., 2020 ; Ball and Maxmen, 2020 ; Holtz et al., 2020 ; Lincoln, 2020 ; Thorp, 2020 ). Lack of national leadership was observed in the relative dearth of national-level public-health policies and guidance (Fig. 1 ) and the divergence of subnational policy responses, correlated with partisan politics (SI Fig. S1 and Table S1 ). Elites and incumbent governments have outsize influence on public opinion and media coverage, which likely contributed to polarization and politicization of pandemic media coverage (Green et al., 2020 ; Hart et al., 2020 ). Linked to these trends, we observed higher sensationalism related to politics and elections topics and greater coverage of these sectors among U.S. newspapers (Figs. 5 – 6 ). Additionally, news outlets on the political left in the United States (i.e., New York Times , Washington Post ) published articles with more exposing and warning coverage, for example discussing disinformation on the part of government leaders and the risks of disease (Fig. 4 ). Although most Americans believe the media are fulfilling key roles during the pandemic, the majority of these individuals identify as Democrats, and Democrats trust many more new sources than individuals identifying as Republican (Jurkowitz et al., 2020 ; Gottfried et al., 2020 ).

In both Canada and the United States, low scientific quality was paired with lower-than-average sensationalism in news outlets on the populist-right (Figs. 3 F and 4F ). Sensationalism was low overall for all news outlets, but within Canada and the United States, it was lowest for the Toronto Sun and New York Post , as well as the Wall Street Journal . Although low sensationalism is generally considered beneficial, very low sensationalism combined with low scientific quality may have failed to alert readers to public-health risks and policy failures in some cases (e.g., per the measures of exposing and warning coverage in Fig. 4 ). Such trends also resulted, in part, from higher reliance on syndicated articles, especially in Canada, potentially related to structural and economic changes in news media (SI Fig. S6 ). Across the political spectrum, our results demonstrate that existing ideological perspectives may influence how information is used in reporting (Rosella et al., 2013 ). For example, news outlets at the populist-right in the United Kingdom and the United States may tend towards support of populist-right governments, demonstrating preference for those governments’ interpretation of the science, implemented policies, and use of science to justify choices made (Bennett et al., 2008 ; Grundmann and Stehr, 2012 ).

The studied news media outlets—traditional, national-level print media—have disproportionate influence on the content of other media platforms and on how that content is covered (Project for Excellence in Journalism, 2010 ; Denham, 2014 ). A better understanding of the effects of news media—or lack thereof—on public-health decision-making and public sentiment in the early stages of this pandemic can, for future pandemics or other public-health crises, increase public-health officials’ capacity to adapt communication strategies in disseminating guidance and coordinating responses of health system stakeholders (Laing, 2011 ; Rosella et al., 2013 ; Klemm et al., 2016 ; Hoffman and Justicz, 2016 ; Pieri, 2019 ). Such understanding is crucial as the impacts of the policy actions themselves accumulate. The findings of this study point to complex interactions among scientific evidence on public-health risks and response measures, societal politicization of the science, and the scientific quality and sensationalism of media reporting. An inherent tension may exist: tendencies towards low sensationalism, especially combined with low scientific quality, may in some cases lead to characterization of public-health threats and policy failures as less extraordinary and relevant than they actually are.

Data availability

All data generated or analyzed during this study are included in this published article and its Supplementary Information .

Adeel AB, Catalano M, Catalano O, et al (2020) COVID-19 policy response and the rise of the sub-national governments. Can Public Policy. https://doi.org/10.3138/cpp.2020-101

Anderson B, Coletto D (2017) Canadian news media and “fake news” under a microscope. In: Abacus Data. https://abacusdata.ca/canadian-news-media-and-fake-news-under-a-microscope/ . Accessed 25 Aug 2020

Antoine J-Y, Villaneau J, Lefeuvre A (2014) Weighted Krippendorff’s alpha is a more reliable metrics for multi- coders ordinal annotations: experimental studies on emotion, opinion and coreference annotation. In: Proceedings of the 14th conference of the European chapter of the Association for Computational Linguistics, Association for Computational Linguistics, Gotenborg, Sweden, pp. 550–559

Austen I (2020) Canada’s largest newspaper changes hands amid vow to keep liberal voice. N. Y. Times

Ball P, Maxmen A (2020) The epic battle against coronavirus misinformation and conspiracy theories. Nature 581:371–374. https://doi.org/10.1038/d41586-020-01452-z

Article   ADS   CAS   PubMed   Google Scholar  

Bennett WL, Lawrence RG, Livingston S (2008) When the press fails: political power and the news media from Iraq to Katrina. University of Chicago Press

Cameron EE, Nuzzo JB, Bell JA (2019) GHS index: global health security index: building collective action and accountability. NTI and Johns Hopkins Bloomberg School of Public Health

CBC News (2020) Tracking the spread of coronavirus in canada and around the world. https://newsinteractives.cbc.ca/coronavirustracker/ . Accessed 25 Aug 2020

Centers for Disease Control and Prevention (CDC) (2020) CDC COVID Data Tracker. https://covid.cdc.gov/covid-data-tracker . Accessed 30 Aug 2020

Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46. https://doi.org/10.1177/001316446002000104

Article   Google Scholar  

Denham BE (2014) Intermedia attribute agenda setting in the New York Times: the case of animal abuse in U.S. horse racing. Journal Mass Commun Q 91:17–37. https://doi.org/10.1177/1077699013514415

Evanega S, Lynas M, Adams J, Smolenyak K (2020) Coronavirus misinformation: quantifying sources and themes in the COVID-19 ‘infodemic.’ Cornell Alliance for Science

Fleiss JL, Cohen J (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 33:613–619. https://doi.org/10.1177/001316447303300309

Gentzkow M, Shapiro JM (2010) What drives media slant? Evidence from U.S. daily newspapers. Econometrica 78:35–71. https://doi.org/10.3982/ECTA7195

Article   MathSciNet   Google Scholar  

Giraudoux P, Antonietti J-P, Beale C, et al (2018) pgirmess: spatial analysis and data mining for field ecologists. R package version 1.6.9 https://CRAN.R-project.org/package=pgirmess

Gottfried J, Walker M, Mitchell A (2020) Americans’ views of the news media during the Coronavirus outbreak. Pew Research Center

Government of Canada (2016) Canada’s health care system. https://www.canada.ca/en/health-canada/services/canada-health-care-system.html . Accessed 18 Oct 2020

Green J, Edgerton J, Naftel D et al. (2020) Elusive consensus: polarization in elite communication on the COVID-19 pandemic. Sci Adv 6:eabc2717. https://doi.org/10.1126/sciadv.abc2717

Article   ADS   PubMed   PubMed Central   Google Scholar  

Grossman G, Kim S, Rexer JM, Thirumurthy H (2020) Political partisanship influences behavioral responses to governors’ recommendations for COVID-19 prevention in the United States. Proc Natl Acad Sci USA 117:24144–24153. https://doi.org/10.1073/pnas.2007835117

Article   CAS   PubMed   PubMed Central   Google Scholar  

Grundmann R, Stehr N (2012) The power of scientific knowledge: from research to public policy. Cambridge University Press, Cambridge

Book   Google Scholar  

Hallgren KA (2012) Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol 8:23–34. https://doi.org/10.20982/tqmp.08.1.p023

Article   PubMed   PubMed Central   Google Scholar  

Hart PS, Chinn S, Soroka S (2020) Politicization and polarization in COVID-19 news coverage. Sci Commun 42:679–697. https://doi.org/10.1177/1075547020950735

Article   PubMed Central   Google Scholar  

Hoffman SJ, Justicz V (2016) Automatically quantifying the scientific quality and sensationalism of news records mentioning pandemics: validating a maximum entropy machine-learning model. J Clin Epidemiol 75:47–55. https://doi.org/10.1016/j.jclinepi.2015.12.010

Holtz D, Zhao M, Benzell SG et al. (2020) Interdependence and the cost of uncoordinated responses to COVID-19. Proc Natl Acad Sci USA 117:19837–19843. https://doi.org/10.1073/pnas.2009522117

Hönnige C, Nyhuis D, Meyer P et al. (2020) Dominating the debate: visibility bias and mentions of British MPs in newspaper reporting on Brexit. Polit Res Exch 2:1788955. https://doi.org/10.1080/2474736X.2020.1788955

Jurkowitz M, Mitchell A, Shearer E, Walker M (2020) U.S. media polarization and the 2020 election: a nation divided. Pew Research Center

Klemm C, Das E, Hartmann T (2016) Swine flu and hype: a systematic review of media dramatization of the H1N1 influenza pandemic. J Risk Res 19:1–20. https://doi.org/10.1080/13669877.2014.923029

Lacy S, Riffe D, Stoddard S et al. (2001) Sample size for newspaper content analysis in multi-year studies. J Mass Commun Q 78:836–45

Google Scholar  

Laing A (2011) The H1N1 crisis: roles played by government communicators, the public and the media. J Prof Commun 1:123–149. https://doi.org/10.15173/jpc.v1i1.88

Article   ADS   Google Scholar  

Lincoln M (2020) Study the role of hubris in nations’ COVID-19 response. Nature 585:325–325. https://doi.org/10.1038/d41586-020-02596-8

Mitchell A, Simmons K, Matsa KE, et al (2018) In Western Europe, public attitudes toward news media more divided by populist views than left-right ideology. Pew Research Center

Molek-Kozakowska K (2013) Towards a pragma-linguistic framework for the study of sensationalism in news headlines. Discourse Commun 7:173–197. https://doi.org/10.1177/1750481312471668

Oxman AD, Guyatt GH, Cook DJ et al. (1993) An index of scientific quality for health reports in the lay press. J Clin Epidemiol 46:987–1001. https://doi.org/10.1016/0895-4356(93)90166-x

Article   CAS   PubMed   Google Scholar  

Pieri E (2019) Media framing and the threat of global pandemics: the Ebola crisis in UK media and policy response. Sociol Res Online 24:73–92

Project for Excellence in Journalism (2010) How news happens: a study of the news ecosystem of one American city. Pew Research Center

Public Health England, NHSX (2020) Coronavirus (COVID-19) in the UK. https://coronavirus.data.gov.uk . Accessed 2 Oct 2020

Puglisi R, Snyder JM (2015) The balanced US press. J Eur Econ Assoc 13:240–264. https://doi.org/10.1111/jeea.12101

R Core Team (2020) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria

Reintjes R, Das E, Klemm C et al. (2016) “Pandemic public health paradox”: time series analysis of the 2009/10 influenza A / H1N1 epidemiology, media attention, risk perception and public reactions in 5 European countries. PLoS ONE 11:e0151258. https://doi.org/10.1371/journal.pone.0151258

Riffe D, Aust CF, Lacy SR (2016) The effectiveness of random, consecutive day and constructed week sampling in newspaper content analysis. Journal Q 70:133–139. https://doi.org/10.1177/107769909307000115

Rosella LC, Wilson K, Crowcroft NS et al. (2013) Pandemic H1N1 in Canada and the use of evidence in developing public health policies–A policy analysis. Soc Sci Med 83:1–9. https://doi.org/10.1016/j.socscimed.2013.02.009

Roser M, Ritchie H, Ortiz-Ospina E, Hasell J (2020) Coronavirus pandemic (COVID-19). Our World in Data. https://ourworldindata.org/coronavirus . Accessed 30 Aug 2020

Thorp HH (2020) Trump lied about science. Science 369:1409. https://doi.org/10.1126/science.abe7391

Tran D, Dolgun A, Demirhan H (2020) Weighted inter-rater agreement measures for ordinal outcomes. Commun Stat-Simul Comput 49:989–1003. https://doi.org/10.1080/03610918.2018.1490428

World Health Organization (WHO) (2020a) Coronavirus disease (COVID-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 . Accessed 7 Jun 2020

World Health Organization (WHO) (2020b) Timeline of WHO’s response to COVID-19. https://www.who.int/news/item/29-06-2020-covidtimeline . Accessed 5 Jun 2020

Zhang R, Li Y, Zhang AL et al. (2020) Identifying airborne transmission as the dominant route for the spread of COVID-19. Proc Natl Acad Sci USA 117:14857–14863. https://doi.org/10.1073/pnas.2009637117

Download references

Acknowledgements

S. Damouras provided advising on methods of statistical analysis, and J. Niemann formatted references. Funding for this work was provided by the University of Toronto Scarborough Department of Physical and Environmental Sciences and the University of Miami Rosenstiel School of Marine and Atmospheric Science.

Author information

Authors and affiliations.

Department of Environmental Science and Policy, Rosenstiel School of Marine and Atmospheric Science, University of Miami, Miami, FL, USA

Katharine J. Mach & Rosalind Donald

Leonard and Jayne Abess Center for Ecosystem Science and Policy, University of Miami, Coral Gables, FL, USA

Katharine J. Mach

Department of Physical and Environmental Sciences, University of Toronto, Toronto, ON, Canada

Raúl Salas Reyes, Brian Pentz, Clarissa A. Costa, Sandip G. Cruz, Kerronia E. Thomas & Nicole Klenk

Department of Geography and Planning, University of Toronto, Toronto, ON, Canada

Jennifer Taylor

Aspen Global Change Institute, Basalt, CO, USA

James C. Arnott

Earth and Environmental Sciences Area, Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Kripa Jagannathan

School for Environment and Sustainability, University of Michigan, Ann Arbor, MI, USA

Department of Civil and Environmental Engineering, University of Connecticut, Storrs, CT, USA

Christine J. Kirchhoff

Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

Laura C. Rosella

You can also search for this author in PubMed   Google Scholar

Contributions

All authors conceived the analysis. KJM, RSR, BP, JT, CAC, SGC, KET, and NK designed the methods of analysis with review by all authors. RSR, BP, JT, CAC, SGC, and KET collected data. KJM, RSR, BP, and JT performed analysis of data and developed visualizations of data. KJM, RSR, BP, JT, and NK drafted the manuscript with review and edits from all authors.

Corresponding author

Correspondence to Katharine J. Mach .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, si appendix dataset s1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Mach, K.J., Salas Reyes, R., Pentz, B. et al. News media coverage of COVID-19 public health and policy information. Humanit Soc Sci Commun 8 , 220 (2021). https://doi.org/10.1057/s41599-021-00900-z

Download citation

Received : 06 January 2021

Accepted : 15 September 2021

Published : 28 September 2021

DOI : https://doi.org/10.1057/s41599-021-00900-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Covid-19 pandemic and vaccination skepticism.

• Abdul Latif Anas
• Mashudu Salifu
• Hanan Lassen Zakaria

Human Arenas (2023)

Local TV News Coverage of Racial Disparities in COVID-19 During the First Wave of the Pandemic, March–June 2020

• Elizabeth K. Farkouh
• Jeff Niederdeppe

Race and Social Problems (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Research News

NYU ranked #8 research institution in North America—up from #25 in 2016—based on increase in articles in top science journals

Labor trafficking affects children in industries ranging from domestic work to forced criminality, entertainment, and agriculture.

Leading scholars in engineering and biostatistics receive lifetime honor recognizing extraordinary contributions to science

There’s new evidence from a researcher at NYU Silver that nonstandard hours, a growing phenomenon, can take a serious toll over the course of a person's working years.

NYU historian Stefanos Geroulanos says we need to ‘take responsibility for what humanity is becoming,’ rather than looking to prehistory for easy answers.

Study reveals how small environmental changes can have a major impact on the shapes of cells and organisms

Study is fourth published in past seven months by NYU Wagner’s Patricia Satterstrom on interdisciplinary teamwork and collaborating across power dimensions

New “quasar catalog” serves as a 3D history book of the universe

Carter Journalism Institute undergrads research historical coverage of racial killings to improve present-day reporting on today’s bias crimes

Identifying animals resilient to DNA damage may provide clues for human risk factors

AI Index Report

Welcome to the seventh edition of the AI Index report. The 2024 Index is our most comprehensive to date and arrives at an important moment when AI’s influence on society has never been more pronounced. This year, we have broadened our scope to more extensively cover essential trends such as technical advancements in AI, public perceptions of the technology, and the geopolitical dynamics surrounding its development. Featuring more original data than ever before, this edition introduces new estimates on AI training costs, detailed analyses of the responsible AI landscape, and an entirely new chapter dedicated to AI’s impact on science and medicine.

Read the 2024 AI Index Report

The AI Index report tracks, collates, distills, and visualizes data related to artificial intelligence (AI). Our mission is to provide unbiased, rigorously vetted, broadly sourced data in order for policymakers, researchers, executives, journalists, and the general public to develop a more thorough and nuanced understanding of the complex field of AI.

The AI Index is recognized globally as one of the most credible and authoritative sources for data and insights on artificial intelligence. Previous editions have been cited in major newspapers, including the The New York Times, Bloomberg, and The Guardian, have amassed hundreds of academic citations, and been referenced by high-level policymakers in the United States, the United Kingdom, and the European Union, among other places. This year’s edition surpasses all previous ones in size, scale, and scope, reflecting the growing significance that AI is coming to hold in all of our lives.

Steering Committee Co-Directors

Ray Perrault

Steering committee members.

Erik Brynjolfsson

John Etchemendy

Katrina Ligett

Terah Lyons

James Manyika

Juan Carlos Niebles

Vanessa Parli

Yoav Shoham

Russell Wald

Staff members.

Loredana Fattorini

Nestor Maslej

Letter from the co-directors.

A decade ago, the best AI systems in the world were unable to classify objects in images at a human level. AI struggled with language comprehension and could not solve math problems. Today, AI systems routinely exceed human performance on standard benchmarks.

Progress accelerated in 2023. New state-of-the-art systems like GPT-4, Gemini, and Claude 3 are impressively multimodal: They can generate fluent text in dozens of languages, process audio, and even explain memes. As AI has improved, it has increasingly forced its way into our lives. Companies are racing to build AI-based products, and AI is increasingly being used by the general public. But current AI technology still has significant problems. It cannot reliably deal with facts, perform complex reasoning, or explain its conclusions.

AI faces two interrelated futures. First, technology continues to improve and is increasingly used, having major consequences for productivity and employment. It can be put to both good and bad uses. In the second future, the adoption of AI is constrained by the limitations of the technology. Regardless of which future unfolds, governments are increasingly concerned. They are stepping in to encourage the upside, such as funding university R&D and incentivizing private investment. Governments are also aiming to manage the potential downsides, such as impacts on employment, privacy concerns, misinformation, and intellectual property rights.

As AI rapidly evolves, the AI Index aims to help the AI community, policymakers, business leaders, journalists, and the general public navigate this complex landscape. It provides ongoing, objective snapshots tracking several key areas: technical progress in AI capabilities, the community and investments driving AI development and deployment, public opinion on current and potential future impacts, and policy measures taken to stimulate AI innovation while managing its risks and challenges. By comprehensively monitoring the AI ecosystem, the Index serves as an important resource for understanding this transformative technological force.

On the technical front, this year’s AI Index reports that the number of new large language models released worldwide in 2023 doubled over the previous year. Two-thirds were open-source, but the highest-performing models came from industry players with closed systems. Gemini Ultra became the first LLM to reach human-level performance on the Massive Multitask Language Understanding (MMLU) benchmark; performance on the benchmark has improved by 15 percentage points since last year. Additionally, GPT-4 achieved an impressive 0.97 mean win rate score on the comprehensive Holistic Evaluation of Language Models (HELM) benchmark, which includes MMLU among other evaluations.

Although global private investment in AI decreased for the second consecutive year, investment in generative AI skyrocketed. More Fortune 500 earnings calls mentioned AI than ever before, and new studies show that AI tangibly boosts worker productivity. On the policymaking front, global mentions of AI in legislative proceedings have never been higher. U.S. regulators passed more AI-related regulations in 2023 than ever before. Still, many expressed concerns about AI’s ability to generate deepfakes and impact elections. The public became more aware of AI, and studies suggest that they responded with nervousness.

Ray Perrault Co-director, AI Index

Our Supporting Partners

Analytics & Research Partners

Stay up to date on the AI Index by subscribing to the  Stanford HAI newsletter.

Is this study legit? 5 questions to ask when reading news stories of medical research

Associate Professor, La Trobe University

Disclosure statement

Hassan Vally does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

La Trobe University provides funding as a member of The Conversation AU.

View all partners

• Bahasa Indonesia

Who doesn’t want to know if drinking that second or third cup of coffee a day will improve your memory, or if sleeping too much increases your risk of a heart attack?

We’re invested in staying healthy and many of us are interested in reading about new research findings to help us make sense of our lifestyle choices.

But not all research is equal, and not every research finding should be interpreted in the same way. Nor do all media headlines reflect what was actually studied or found.

So how can you tell? Keep these five questions in mind when you’re reading media stories about new studies.

1. Has the research been peer reviewed?

Peer review is a process by which a study is checked by experts in the discipline to assess the study’s scientific validity.

This process involves the researcher writing up their study methods and results, and sending this to a journal. The manuscript is then usually sent to two to three experts for peer review.

If there are major flaws in a study, it’s either rejected for publication, or the researchers are made to address these flaws.

Although the peer-review process isn’t perfect, it shows a study has been subjected to scrutiny.

Read more: Peer review has some problems – but the science community is working on it

Any reported findings that haven’t been peer reviewed should be read with a degree of reservation.

2. Was the study conducted in humans?

Findings from studies conducted in animals such as mice or on cells in a lab (also called in vitro studies) represent the earliest stage of the scientific discovery process.

Regardless of how intriguing they may be, no confident claims about human health should ever be made based on these types of study alone. There is no guarantee that findings from animal or cell studies will ever be replicated in humans.

3. Are findings likely to represent a causal relationship?

For a study to have relevance to our day-to-day health, the findings need to reflect a causal relationship rather than just a correlation .

If a study showed that coffee drinking was associated with heart disease, for example, we want to know if this was because coffee actually caused heart disease or whether these to things happened to occur together.

In a number of studies that found this association, researchers subsequently found that coffee drinkers were more likely to be smokers and therefore, these results were more likely to reflect a true causal relationship between smoking and heart disease.

In observational studies, where researchers observe differences in groups of people, it can sometimes be difficult to disentangle the relationship between variables.

The highest level of evidence regarding causality comes from double-blind placebo controlled randomised controlled trials (RCTs). This experimental type of study, where people are separated into groups to randomly receive either an intervention or placebo (sham treatment), is the best way we can determine if a something causes disease. However it, too, is not perfect.

Although other types of studies in humans play an important role in our understanding of health and disease, they may only highlight associations that are not indicative of causal relationships.

Read more: Clearing up confusion between correlation and causation

4. What is the size of the effect?

It’s not enough to know that an exposure (such a third cup of coffee or more than nine hours of sleep a night) causes an outcome, it’s also important to clearly understand the strength of this relationship. In other words, how much is your risk of disease going to increase if you are exposed?

If your risk of disease is reported to increase by 50% (which is a relative risk), this sounds quite frightening. However, if the original risk of disease is low, then a 50% increase in your risk may not represent a big actual increased risk of disease. A 50% increased risk of disease could mean going from a 0.1% risk of disease to your risk being 0.15%, which doesn’t sound quite so dramatic.

Read more: What you need to know to understand risk estimates

5. Is the finding corroborated by other studies?

A single study on its own, even if it’s a well-conducted randomised controlled trial, can never be considered definitive proof of a causal relationship between an exposure and disease.

As humans are complex and there are so many variables in any study, we can’t be confident we understand what is actually going on until findings are replicated in many different groups of people, using many different approaches.

Until we have a significant body of evidence that is in agreement, we have to be very careful about our interpretation of the findings from any one study.

What if these questions aren’t answered?

If the media report you’re reading doesn’t answer these questions, consider changing news sites or looking at the original paper. Ideally this would be linked in the news article you’re reading, or you can search PubMed for the article using a few keywords.

The journal article’s abstract should tell you the type of study, whether it was conducted on humans and the size of the effect. If you’re not blocked by a paywall, you may be able to view the full journal article which should answer all of the questions you have about the study.

Read more: Where's the proof in science? There is none

• Medical research
• Health literacy
• Correlation
• Randomised control trial
• Evidence-based medicine
• Media reporting

Project Offier - Diversity & Inclusion

Senior Lecturer - Earth System Science

Sydney Horizon Educators (Identified)

Deputy Social Media Producer

Associate Professor, Occupational Therapy

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

• PMC8756300.1 ; 2021 Jun 1
• ➤ PMC8756300.2; 2022 Jan 31

Quality of information in news media reports about the effects of health interventions: Systematic review and meta-analyses

1 Centre for Informed Health Choices, Norwegian Institute of Public Health, Oslo, Norway

2 Faculty of Health Sciences, Oslo Metropolitan University, Oslo, Norway

Lillebeth Larun

3 Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway

Giordano Pérez Gaxiola

4 Cochrane Associated Centre and Evidence‐based Medicine Department, Sinaloa's Pediatric Hospital, Culiacan, Mexico

Dima Alsaid

5 Institute for Evidence in Medicine (for Cochrane Germany Foundation), Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany

Anila Qasim

6 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada

Christopher James Rose

Karin bischoff, andrew david oxman, associated data, underlying data.

All data underlying the results are available as part of the article and its supporting information files and tables.

Extended data

Zenodo: Quality of news reports about the effects of health interventions - Supporting Information. http://doi.org/10.5281/zenodo.4781182 . 105

This project contains the following extended data:

• - S1 File. Protocol. “Quality of news media reports about the effects and costs of health interventions: Systematic review protocol”.
• - S2 File. Protocol Appendix 1. “Appendix 1: Reference list of potentially eligible studies”.
• - S3 File. Protocol Appendix 2. “Appendix 2: Description of search for an existing review”.
• - S4 File. Dissertation. “Criteria used to measure the quality of news media reports about the effects of health interventions: Systematic review”.
• - S5 File. Detailed search strategy.
• - S6 File. Informed Health Choices Key Concepts. 2019 version.
• - S7 File. Detailed information about deviations from the protocol.
• - S8 File. Forest plots for individual criteria groups.
• - S9 File. Individual risk of bias assessments.
• - S10 File. Individual criteria included in meta-analyses. Sample characteristics, verbatim and reworded criteria, related IHC Key Concepts, overall risk of bias, estimates and confidence intervals, and sample sizes.
• - S1 Table. Included studies . References for all studies included in qualitative summary and reason for exclusion from meta-analyses or number of criteria included in meta-analyses.
• - S2 Table. Sample characteristics and tools by study. Sample size, medium/media, country/countries, time period(s), intervention category/categories, and tool(s).
• - S3 Table. Objectives of included studies.
• - S4 Table. Sampling frames and methods.
• - S5 Table. Reported subgroup analyses.
• - S6 Table. Number of quality criteria included in the meta-analysis per relevant IHC Key Concept.

Reporting guidelines

Zenodo: PRISMA checklist for ‘Quality of information in news media reports about the effects of health interventions: systematic review and meta-analyses’. http://doi.org/10.5281/zenodo.4781182 . 105

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Version Changes

Revised. amendments from version 1.

There are no major differences between this and the previous version of the article. Most of the changes are based on the reviews. The only other substantive changes are adding two references to the discussion, under "Underlying problems": Glasziou and Chalmers 2018, and O'Keefe et al. 2021. Based on the reviews, we replaced "inappropriate [criteria]" with "ineligible". In the abstract, we removed "qualitative summary", and we now explain the term at first mention in the main text. At the end of the introduction, we added a paragraph about the Informed Health Choices (IHC) project, including the authors' involvement. In the methods section, we added numbers to the example of a reframed criterion, and we explain our focus on detection bias. In the results, we revised the formatting of Table 2, so the columns alternate colour, rather than the rows, and we removed reference to the checklist for communicating evidence-based information about health care. We moved up the bullet-point list with overall estimates from the end of "Key findings", in the discussion, to the start of "Synthesis of results", in the results. Under "Key findings", we specified that the samples were diverse in terms of English-speaking and high-income countries (not countries in general), and we adjusted the end of the section where the aforementioned bullet-point list had been. Further, in the discussion, we removed the second clause in the sentence about Heaner's findings; we clarified that IHC is a separate initiative from the Informed Health site; we specified that it is possible for news outlets to improve their reporting "with enough resources"; and we deleted the last paragraphs of the discussion, about IHC. The remaining changes are corrections of typos.

Peer Review Summary

Many studies have assessed the quality of news reports about the effects of health interventions, but there has been no systematic review of such studies or meta-analysis of their results. We aimed to fill this gap (PROSPERO ID: CRD42018095032).

We included studies that used at least one explicit, prespecified and generic criterion to assess the quality of news reports in print, broadcast, or online news media, and specified the sampling frame, and the selection criteria and technique. We assessed criteria individually for inclusion in the meta-analyses, excluding ineligible criteria and criteria with inadequately reported results. We mapped and grouped criteria to facilitate evidence synthesis. Where possible, we extracted the proportion of news reports meeting the included criterion. We performed meta-analyses using a random effects model to estimate such proportions for individual criteria and some criteria groups, and to characterise heterogeneity across studies. 

We included 44 primary studies in the review, and 18 studies and 108 quality criteria in the meta-analyses. Many news reports gave an unbalanced and oversimplified picture of the potential consequences of interventions. A limited number mention or adequately address conflicts of interest (22%; 95% CI 7%-49%) (low certainty), alternative interventions (36%; 95% CI 26%-47%) (moderate certainty), potential harms (40%; 95% CI 23%-61%) (low certainty), or costs (18%; 95% CI 12%-28%) (moderate certainty), or quantify effects (53%; 95% CI 36%-69%) (low certainty) or report absolute effects (17%; 95% CI 4%-49%) (low certainty). 

There is room for improving health news, but it is logically more important to improve the public’s ability to critically appraise health information and make judgements for themselves.

Introduction

For decades, researchers have studied and criticised the quality of health information in the mass media. In 1972, The New England Journal of Medicine—the “most prestigious” general medical journal in the world 1 —published an “examination” of health information on American television (TV). 2 The paper ended in reproach: “The potential of television to inform, instruct and educate should not continue to be wasted and abused, but rather should be used as an integral part of a plan designed to deliver better health care.”

Misleading health information in the mass media continues to be a hot topic. In 2020, the World Health Organization reported that the Coronavirus disease (COVID-19) had been accompanied by an “infodemic”: “an over-abundance of information—some accurate and some not—that makes it hard for people to find trustworthy sources and reliable guidance when they need it.”. 3

Many studies of health news have focused on “traditional” news media, including print, broadcast, and news websites. With the advent of social media, traditional news media have increased competition as a source of lay health information. However, people still get health information from the latter, directly or via social media. Chew and Eysenbach found that about 60% of more than 5000 tweets about the H1N1 outbreak in 2009 included links and about a quarter of those links were to news websites, the most common destination. 4 In comparison, less than 5% of links were to government and public health agencies or to other social media, combined.

Many of the studies focusing on news media have further focused on news reports about the effects of health interventions. This includes “modern” medicine (aka. “academic”, “conventional” or “Western” medicine); “alternative” medicine (aka. “complementary”, “traditional” or “natural” medicine); screening; surgery; devices; diet; exercise; lifestyle interventions; and health systems and policies. We were unable to find a systematic review of such studies (see extended data - S1, S2 and S3 Files 105 ). 5 We then aimed to fill this gap, providing meta-analytical estimates of the quality of news reports about the effects of health interventions, and potentially informing further and related research.

The first part of this study was submitted by the first author, MO, to the University of Oxford as his dissertation, in partial fulfilment of the requirement for the award of Master of Science (MSc) in Evidence-Based Health Care, under the title “Criteria used to measure the quality of news media report about the effects of health interventions: Systematic review” (see extended data - S4 File 105 ).

The study is related to the Informed Health Choices (IHC) project, in which MO, GG, CR and AO have been involved. The project goal is to help people think critically about information about the effects of health interventions, as well as choices about health interventions, so they avoid waste and unnecessary suffering. A premiss for the project is that there is a large amount of unreliable information about the effects of health intervention. This review helped test this premiss, and takes advantage of the IHC Key Concepts framework, which MO and AO have helped develop.

• • To assess the quality of information in news media reports about the effects of health interventions.
• • To assess, map and group criteria used to measure the quality of information in news media reports about the effects of health interventions.

This review was registered in the International Prospective Register of Systematic Reviews ( PROSPERO ) (ID: CRD42018095032 ). The protocol was published on the IHC project website as “Quality of news media reports about the effects and costs of health interventions: Systematic review protocol” (see extended data - S1, S2 and S3 Files 105 ). 5 Appendixes to the protocol were published in a separate compressed folder, on the same website. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Checklist 6 has been included in the Reporting guidelines for this review.

Eligibility criteria

Primary studies had to satisfy four eligibility criteria to be included in the review ( Table 1 ): two related to the sample of news reports included in the study; one related to the outcome measure; and one related to reporting of the study. We included studies in any language in the qualitative summary, but we only included studies in English or Norwegian in the meta-analyses (the two languages spoken fluently by the primary investigator, MO). Note that the qualitative summary is not a synthesis of qualitative studies, but a structured synthesis of results excluded from the meta-analyses. We did not place any explicit limitation on study design, but—in effect—case studies of a single news report and purely qualitative studies were ineligible.

The first eligibility criterion was that the sample include content labelled by the researchers as “news”, taken from newspapers or magazines (print); television, radio or podcasts (broadcast); or dedicated news websites (online). We placed no limits on where in the world or when the news reports were published.

Second, the sample had to include news reports about the effects of health interventions. Other types of health news include non-reports (e.g. features or opinion pieces) and other categories of news reports about health (e.g. reports about the health effects of exposures or about health-related ethical or legal issues). We defined “health intervention” as any action intended to improve the health of individuals or communities, as per the Glossary of Evaluation Terms for Informed Treatment choices ( GET-IT ). 7 We defined “effect” as a negative or positive change or difference in a health outcome, again as per GET-IT.

Third, the researchers had to have used at least one explicit, prespecified and generic criterion to measure quality, herein called a quality criterion. “Quality” was defined as how conducive the report is to informed decisions or, conversely, how misleading it is. Information about the effects of health interventions can be directly misleading—such as claims that an intervention causes an outcome when it is only associated with it—or it can be misleading by omission, such as omitting absolute effect estimates, particularly when the baseline risk is small.

“Criterion” was defined as a standard of quality that could be satisfied or not. “Generic” meant that the quality criterion was not specific to an intervention (e.g. a drug), category of interventions (e.g. “modern” medicine), or condition (e.g. cancer), or specific to any evidence (e.g. findings from a trial). We excluded studies if they only measured the quality of the evidence cited in the news reports; only measured the factual accuracy of the reports; or only described the reports, as opposed to explicitly assessing their quality. We included quality criteria related to cost of the intervention.

The fourth and final eligibility criterion was that the study specified the sampling frame (where the news reports were sampled from), the selection criteria for the news reports, and the selection technique (how the reports were sampled).

To clarify the eligibility criteria, three reviewers piloted them on a sample of five studies from a list of potentially eligible studies of which we were aware before finalising the protocol (Appendix 1 of the protocol; extended data - S2 File 105 ). 5

Search strategy

We searched for eligible primary studies indexed in PubMed on May 24, 2018, and Google Scholar on June 21 and 22, 2018. We searched directly in Open Grey and Grey Literature Report on June 22, 2018, and in ProQuest Dissertations & Theses (Global Full text plus UK and Ireland abstracts) on June 25, 2018. We updated the PubMed search on Aug. 30, 2019.

We conducted citation searches for papers describing the development of tools for assessing the quality of news reports about health: the Index of Scientific Quality; 8 the Quality Index for health-related Media Reports; 9 and a checklist for improving drug information in the general press. 10 We also checked the reference lists of those three papers. Finally, we conducted citation searches and reference lists checks for an arbitrary selection of eligible studies. S5 File in the extended data 105 is a detailed description of the search strategy, including search strings.

Study selection

Screening search results for inclusion in the qualitative summary consisted of two rounds. First, two reviewers screened titles and abstracts. Second, two reviewers screened full texts of studies included after the first round. The two discussed and resolved disagreements after each round. Where two reviewers could not reach a consensus, a third was brought in to arbitrate. Screening reference lists and the results of citation searches included a third, initial round, wherein only one reviewer screened titles. We also screened all of the potentially eligible studies of which we were aware before finalising the protocol (Appendix 1 of the protocol; extended data – S2 file 105 ). We screened search results using Rayyan . 11

If a study included a mix of news reports about the effects of health interventions and other types of health news, we excluded the study from the meta-analyses unless results for the news report about the effects of health interventions were reported separately for at least one eligible quality criterion. Further, we excluded studies from the meta-analyses if they did not include any eligible quality criteria or any such criteria for which results were adequality reported (see the section “Assessing criteria”). Judgements about excluding studies from the meta-analyses were made independently by two reviewers, who then discussed and resolved disagreements.

Data extraction

Data were extracted by a single reviewer and entered into spreadsheets (Microsoft Excel, Version 16.49). We only extracted reported data. For example, if a study did not include the country or countries in which the sample of news reports were published, we did not attempt to code the individual news reports by country or contact the study authors for that information. A second reviewer checked a sample of the extracted descriptive data and checked all the extracted quality criteria that were included in the meta-analyses, as well as results for those criteria.

For the qualitative summary, we extracted descriptive data about the samples of news reports and the tools (sets of quality criteria) used to assess the reports. Data about the sample of news reports included: the categories of media in which the reports were published; the countries in which they were published; the time period in which they were published; the categories of health interventions that were the subject of the reports; and the financial models of the news outlets that published the reports. We also extracted study objectives, sampling frame, selection criteria and technique, and reported subgroup analyses.

All categories and the time periods were predetermined in the protocol. The categories of media were newspaper; magazine; radio; podcast; television; and news website. The categories of intervention were: “modern” medicine; “alternative” medicine; screening; surgery; devices; diet; exercise; lifestyle; and systems and policies. The two categories of financial model were non-commercial (public or independent) and commercial.

We extracted quality criteria verbatim. For the meta-analyses, we extracted all response options for each included quality criterion, the number of relevant news reports (reports about the effects of health intervention) to which the quality criterion was applied, and the results for those reports. Some quality criteria were not applied by researchers to the total sample of relevant reports in a given study, meaning the researchers did not consider the quality criterion universally applicable.

Again, if a study included a mix of news reports about the effects of health interventions and other types of health news, we only included results for the former. We excluded studies if those results were not reported separately.

For the meta-analyses, we were interested in the proportion of relevant news reports that satisfied a given quality criterion. If this was not reported explicitly, we imputed the proportion using reported data, if possible. We excluded quality criteria if the proportion was not explicitly reported and it was impossible to calculate the proportion based on the reported data—for example, if only mean scores were reported.

Assessing criteria

After selecting studies to be included in the meta-analyses ( Figure 1 ), two authors assessed criteria used in those studies: both quality criteria and other criteria, such as descriptive criteria or criteria measuring factual accuracy, using spreadsheets. We excluded ineligible criteria, as well as criteria for which it was not possible to extract results for only news reports about the effects of health interventions.

We assessed each criterion in each study, even if the researchers used a tool used in a previous study, in case it had been modified. For example, the Media Doctor Australia tool is used in several studies, including those by Smith et al . 12 and Bonevski et al . 13 However, while Smith et al . asked whether there is “objective evidence to support the treatment”, Bonevski et al . asked whether the study methodology is reported, which is more specific.

Two reviewers independently assessed criteria, then discussed and resolved any differences. We excluded criteria if they were non-generic or if they assessed factual accuracy, if they were descriptive, or if they were not clear or sensible (i.e. if their face validity was inadequate 14 ). We also excluded global scores and composite quality criteria because they can be difficult to interpret, and they are difficult to compare across studies using different scoring systems.

Mapping and grouping quality criteria

We mapped and grouped quality criteria, using spreadsheets, to be able to synthesise results across studies using different tools. For the mapping, we used the IHC Key Concepts framework. The most recent version of the framework was published on IHC website in 2019 and includes nine high-level concepts and 49 low-level concepts organised in three overarching groups (see extended data - S6 File 105 ). 15 The IHC Key Concepts are essential concepts for assessing information about the effects of health interventions and making informed choices. 15 – 17 The framework is based on a mix of evidence and logic. It has a combination of characteristics that make it unique, as well as uniquely useful for this study. First, it has been developed systematically, transparently and iteratively. Second, it only includes concepts that are intended to be relevant to the general public (typically the target audience of health news). And third, it is intended for mapping and assessing measurement tools (amongst other purposes). Moreover, the starting point for the framework included checklists for journalists and input from journalists. 18

Two reviewers independently mapped the included quality criteria against the low-level IHC Key Concepts, then discussed and resolved any disagreements. Finally, one reviewer grouped the criteria, informed by the results of the mapping, and a second reviewer assessed whether the groups were sensible (i.e. face validity 14 ).

Meta-analysis

Where an included quality criterion had more than two response options, one reviewer dichotomised the categories and a second reviewer assessed the dichotomisation. For example, for studies that used the Index of Scientific Quality, where response options range from a score of one to five, we considered a score of four or five as satisfying the quality criterion.

Furthermore, we reframed negative quality criteria as positive, and paraphrased all quality criteria, for the data to be consistently framed and worded, and for the criteria to be succinctly labelled. For example, Bubela et al . report the number of news reports that “did not” specify whether a study was randomized (503 of 553, 91%), which was reframed as the number of news reports that did specify whether a study was randomized (50 of 553, 9%). 19

For each individual quality criterion and group of criteria, we estimated mean proportions with 95% confidence intervals (CIs), using a random effects model, quantifying precision using 95% confidence intervals. Where criteria groups included more than one criterion from the same study, we included results for both criteria. We performed statistical analysis using Stata 16 (StataCorp LLC, College Station, Texas, USA). Each of the included studies provided data on the number of relevant news reports assessed and how many of those reports met a given criterion. For each study and quality criterion for which necessary data were available, we estimated the proportion of relevant news reports meeting the criterion. These point estimates and 95% confidence intervals were transformed to the logit scale for meta-analysis (Stata's default scale for meta-analysis of proportions). We had anticipated substantial between-study heterogeneity and therefore used random effects meta-analyses. We estimated the percentage of variance that can be attributed to between-study heterogeneity rather than chance (I 2 , expressed as a percentage) and calculated P-values to test the null hypotheses of no between-study heterogeneity using Cochran's Q statistic. We back-transformed meta-analytical estimates from the logit scale to present estimates as percentages (in forest plots summarizing meta-analytical results from different criteria groups) or proportions (in forest plots for individual groups).

Two reviewers assessed whether what the different quality criteria in each group measured was similar enough that it was sensible to report an overall estimate. If a group included a single quality criterion, we did not report an overall estimate.

Risk of bias

Two reviewers independently assessed the risk of detection bias for each criterion included in the meta-analyses, then compared judgements and resolved disagreements. Detection bias was assessed using three questions:

• 1. Was the criterion applied by at least two researchers independently?
• 2. Were the researchers blinded in terms of the journalist and news outlet?
• 3. Does the criterion require substantial judgement?

As explained in the protocol (extended data – S2 file), we decided to focus on detection and selection bias based on the critical appraisal tool developed by Munn et al . for systematic reviews of observational epidemiological studies reporting prevalence data, 20 the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement, 21 and methods used by the Cochrane Methodology Review Group to systematically review the prevalence of an outcome in a population of information (as opposed to people). 22 As explained in this article, in the section on deviations from the protocol, we later found that the item we had developed to assess selection bias told us more about applicability than bias, and therefore only assessed detection bias.

If the authors did not state how many researchers applied the criterion or whether the researcher(s) did so independently, we recorded the answer to the first question as “unclear” and counted it as a “no”. If from looking at the criterion in and of itself, it was unclear how much judgement was required, we looked at whether the researchers reported or referenced any guidance for applying it. If they did, we looked at how detailed and specific the guidance was. If the researcher(s) applying the criterion received training, but the authors did not report what this entailed—i.e. what guidance was provided—we considered the criterion to require substantial judgement.

We considered the risk of detection bias as low if the criterion did not require substantial judgement. We considered the risk of detection bias as moderate if the criterion required substantial judgement, but at least two researchers applied the criterion independently and they were blinded in terms of the journalist and outlet. We considered the risk of detection bias as high if the criterion required substantial judgement and only one researcher applied it or the researchers were not blinded.

If there was a high risk of bias for criteria providing ≥50% of the weight in a given criterion group, we considered there to a be a high risk of bias for the overall estimate.

Rating the quality of the evidence

We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach as a framework for rating the quality of evidence. 23 GRADE considers five factors that can lower the certainty of evidence for effect estimates: risk of bias, inconsistency of results, indirectness of evidence, imprecision, and publication bias. We did not consider publication bias, as we are not aware of any research documenting publication bias in studies of the quality of news reports or similar studies. We also did not consider directness, which was irrelevant.

Again, if there was a high risk of bias for criteria providing ≥50% of the weight in a given group, we considered there to be a high risk of bias for the overall estimate. If a CI is wider than 0.25 (a quartile), we considered there to be important imprecision, based on guidance from the Cochrane Effective Practice and Organization of Care (EPOC) group. 24 We used an I 2 of >75% as a rule of thumb for there being substantial inconsistency.

GRADE factors for increasing the certainty of the evidence were irrelevant.

Deviations from the protocol

We report the main deviations from the protocol here. In S7 File (see extended data 105 ), we provide detailed information about all deviations. Some of the deviations were in part due to the originality of our research question in terms of a systematic review and meta-analysis, meaning there was lack of methodological guidance or precedence for the study as a whole. In other words, we were in “unchartered waters”. Other deviations were due to limited time and resources. This review had no funding and as noted in the background section, the first part of the review was MO's MSc dissertation, meaning he was required to do as much as possible of that work himself.

In terms of search strategy, we did not search Scopus as planned, aside from citation searches, since the time saved outweighed the likelihood of identifying additional eligible studies, based on the results of searching the other databases. Also, we only conducted reference list checks and citation searches for arbitrary samples of included studies, due to limited time and resources, as well as decreasing yield (extended data – S5 File 105 ).

In terms of eligibility, we made explicit that at least one of the criteria that researchers in a given study used to assess quality had to be prespecified and generic, and used specifically to assess the quality of news reports, not describe their content or assess the quality of underlying evidence. As mentioned, we included studies in any language in the qualitative summary, but only included studies in English and Norwegian (the two languages spoken fluently by the primary investigator, MO) in the meta-analyses.

In terms of data extraction, a second reviewer checked all data extracted for the meta-analyses, but only a sample of data extracted for the qualitative summary due to limited time and resources. There were no errors in the sample. In terms of assessing risk of bias, we were originally going to assess selection bias by looking at how news reports were sampled: randomly, sequentially or other. However, we found this told us more about applicability than bias, since samples typically consisted of all news reports that met the researchers’ eligibility criteria, rather than a sample of those reports. Therefore, we only assessed the risk of detection bias, as described earlier.

In terms of synthesising results and rating the quality of the evidence, we have not reported overall estimates for groups of quality criteria if what those criteria measure is too different for a synthesis to be sensible, despite the criteria being clearly related. Nor have we reported overall estimates for groups that include a single criterion. Further, we did not rate the quality of the evidence in these cases. Where we do report an overall estimate and rate the certainty of the evidence, if we assessed the risk of bias as high for criteria providing ≥50% of the weight in a given group, we assessed the risk of bias as high for the group as a whole. We did not consider assessing risk of bias for whole criteria groups in the protocol.

We screened 2063 unique records and retrieved and assessed 140 full texts for eligibility, as shown in the PRISMA flow diagram ( Figure 2 ). We identified 44 primary studies for inclusion in the qualitative summary (see extended data - S1 Table 105 ): 31 in English, five in German, four in Spanish, three in Norwegian and one in Slovenian. We excluded most (78) full-text articles from the qualitative summary based on one or more of the four eligibility criteria. We excluded another 16 full-text articles that were not research (e.g. commentaries), and two duplicates: one doctoral thesis that is a composite of other studies that we assessed 25 and one working paper 26 superseded by the final publication. 27 The third eligibility criterion—that the researcher had used at least one explicit, prespecified and generic quality criterion—was least often satisfied. We excluded 47 full-text articles solely for failing to satisfy that eligibility criterion.

Of the 44 studies included in the qualitative summary, we included 18 (41%) in the meta-analyses. Of the 26 studies only included in the qualitative summary, we excluded nine (20%) from the meta-analyses for being in a language other than English or Norwegian; 16 for inadequate reporting (36%); and one (2%) where the sample of news reports 28 was included in a larger sample in a later study. 29 Only one of the 18 studies included in the meta-analyses was in Norwegian, 30 meaning the rest were in English.

Study characteristics

Table 2 is an overview of the samples of news reports in the studies included in the qualitative summary. Descriptive data for each study can be found in S2 Table in the extended data.

Sample sizes varied from 17 to 3096 units. We use “units” here since some samples include content besides news reports, such as non-news websites, 31 or include health news that is not news reports about the effects of health interventions, for example news features 32 or news reports about the effects of exposures, such as recreational cannabis (as opposed to medical cannabis). 33

Newspaper was the most common medium, with newspaper content explicitly included in 35 of the 39 studies (90%) in which a medium was specified ( Table 2 ). However, the number of studies that included newspaper content versus content from news websites should be interpreted with caution, since newspaper content may also have been published online. In the study with the second-largest sample, Walsh-Childers et al . write that news reports “must have been published on the news organization's website,” but categorise those reports only as newspaper content. 29

The United States (US) was the most common country in terms of where content assessed in the included studies was published. Of the 40 studies where at least one country was specified, 12 samples (30%) included content published in the US ( Table 2 ). Twenty-five (63%) of those 40 studies included content published in English-speaking countries (see extended data - S2 Table 105 ).

We categorised time periods according to what authors reported. This was in some cases the period in which news reports were published and in others the period in which reports were assessed. Differences between the two are likely minor, since all studies focused on the recent past (relative to the time that the study was conducted). The earliest year of publication was 1996. 34

“Modern” medicine was included in 24 of 32 samples (75%) where a category of interventions is specified. In some cases, it was not possible to use the categories pre-specified in the protocol for this review. For example, Haneef et al . included news reports about “non-pharmacological” interventions, 35 which could include “Alternative” medicine, devices, diet, etc. In these cases, we used the category “Other”.

Financial model was not specified in any of the included studies—i.e. it was not explicitly reported whether the news outlets that published news reports included in the sample were commercial or non-commercial (public or independent).

The tools used to assess the quality of news reports in each study are presented with the descriptive data in S2 Table in the extended data. 105 Most studies used a tool that was, to varying degrees, original or modified. Only one tool was used in >4 of the 44 studies included in the qualitative summary: the Index of Scientific Quality, which was used in seven studies (16%). 31 , 33 , 34 , 36 – 39 However, even more studies used tools that were to some extent, directly or indirectly, based on the criteria developed by Moynihan et al . 40 This includes the four studies that used the Media Doctor Australia tool 12 , 13 , 41 , 42 and the 3 studies that used the Health News Review tool, 28 , 29 , 43 which was based on the Media Doctor Australia tool. 28 Two studies used two separate tools. 41 , 44 We did not identify any studies that used the Quality Index for health-related Media Reports 9 or the checklist for improving drug information in the general press. 10

Objectives, sampling frames, selection criteria and technique, and reported subgroup analyses

The explicit or implicit objectives of studies included in the qualitative summary are presented in S3 Table, the sampling frames and selection criteria and techniques in S4 Table, and the variables used in the reported subgroup analyses in S5 Table (see extended data 105 ). Detail and clarity varied. For example, some studies reported explicit inclusion and exclusion criteria for news reports, while others simply stated that they assessed news reports that included health or medical information.

Thirty-one studies (70%) reported at least one subgroup analysis, exploring differences in quality between subgroups of news reports. The most common overarching variables were ones related to time, medium, and outlet. How subgroups were categorised for the same overarching variable, as well as what categories were included, varied. For example, some studies included subgroup analyses based on specific intervention (e.g., comparing the quality news reports about one medicine with news reports about another medicine). Others included subgroup analyses based on the category of intervention (e.g., comparing news reports about “modern” medicines versus news reports about other categories of interventions). Furthermore, the specific interventions or categories of interventions varied.

Assessment, mapping and grouping of criteria

Two reviewers assessed all 208 criteria applied to news reports in the 18 studies included in the meta-analyses. Not all criteria were measures of quality. Of the 208 criteria, 77 (37%) were excluded: 66 (32%) for being non-generic or descriptive, for measuring factual accuracy, or for not being clear or sensible (having inadequate face validity); and 11 (5%) for being global scores or composites. We provide examples in Table 3 . Quality criteria were judged not to be sensible if they were in conflict with the IHC Key Concept and research evidence, for example a criterion assessing whether research presented in a news report is peer-reviewed. 43 , 45 The same two reviewers mapped the remaining 131 of 208 quality criteria (63%) against the IHC Key Concepts. Based on the results of the mapping, we established 19 groups of quality criteria (criteria groups), as well as eight subgroups.

In S6 Table in the extended data, 105 we present the number of quality criteria included in the meta-analyses per relevant IHC Key Concept. We were unable to identify or include quality criteria related to several IHC Key Concepts that are important for communicating information about the effects of health interventions in the news media or otherwise. For example, there were no criteria related to the concept “ Average differences between treatments can be misleading ” (2.3c).

Synthesis of results

Of the 208 criteria used in the 18 studies included in the meta-analysis, we included 108 (52%) in the meta-analyses. We calculated overall estimates for five of the 19 main criteria groups (26%) and 6 of the 8 subgroups (75%). In those cases, we found:

• • 86% of news reports do not commit “disease mongering” (95% CI 78%-91%) (low certainty),
• • 22% adequately address conflicts of interest (95% CI 7%-49%) (low certainty),
• • 93% do not selectively report “statistically significant” results (95% CI 87%-97%) (high certainty),
• • 53% quantify effects (95% CI 36%-69%) (low certainty),
• • 68% do not use misleading language (95% CI 35%-90%) (low certainty),
• • 17% report absolute effects (95% CI 4%-49%) (low certainty),
• • 36% include adequate information about alternative interventions (95% CI 26%-47%) (moderate certainty),
• • 82% do not focus on surrogate outcomes or extrapolate from surrogate to important outcomes (95% CI 76%-86%) (high certainty),
• • 40% mention, discuss or explain potential harms of the intervention (95% CI 23%-61%) (low certainty),
• • 18% mention or discuss the cost of the intervention (95% CI 12%-28%) (moderate certainty), and
• • 97% mention at least one benefit (95% CI 56%-100%) (moderate certainty).

Of the 131 quality criteria included in the 19 groups, we excluded 22 (17%) from the meta-analyses for inadequate reporting of results, meaning it was impossible to impute the proportion of relevant news reports that satisfied the criterion based on the reported data. The distribution of the remaining 108 criteria amongst the 19 main groups and eight subgroups, is presented in Table 4 , with the order of the groups based on the IHC Key Concept framework. In one case, we included a criterion in two groups: the third criterion in the study by Stassen, 43 which we included in Criteria related to qualitative descriptions of effects and Criteria related to pros and cons. Several criteria from the same study were included in three of the 19 main groups (16%) and four of the eight subgroups (50%). Criteria related to “disease mongering” 12 stem from a paper by Moynihan et al. 46

N = criteria included in the meta-analyses.

The primary meta-analyses are summarised in Figure 3 , excluding subgroups. The samples of news reports in two of the studies included in the meta-analyses overlap. 12 , 13 For groups that included a criterion from both of these studies, we performed primary analyses that included only the largest and most-recent study by Bonevski et al ., 13 and an ad hoc sensitivity analysis that also included results from the study by Smith et al. 12 The sensitivity analyses are summarised in Figure 4 , again excluding subgroups. There were no important differences between the overall estimates and CIs for the primary analyses versus the sensitivity analyses. Forest plots for each criteria group, including individual estimates and CIs for each criterion, are presented in S7 File in the extended data. 105

For most of the main groups (14 of 19) and a minority of the subgroups (two of eight), we did not consider it sensible to report overall estimates because—although the criteria in the group are clearly related—what they each measure was too different for it to be sensible to synthesise results. For example, in the group Criteria related to associations and randomization, the ninth criterion from the study by Bubela et al . measured whether the news report specifies that a trial was randomized or not, whereas the 12 th criterion from the study by Haneef et al . measures whether there is a causal claim based on a non-randomized study. The two are clearly related, but at the same time, synthesising results would clearly not be sensible. Leaving out whether a study was randomized might be misleading by omission, but this can be compensated for by distinguishing causation and association and pointing out potential confounding. On the other hand, making a causal claim based on a non-randomized study is directly misleading. Further, we do not report overall estimates for the groups that include a single criterion.

Due to heterogeneity, as well as inadequate reporting and small sample sizes in some studies, we did not conduct any of the six subgroup analyses considered in the protocol, exploring differences in quality between news reports:

• 1. in different media (broadcast vs. other),
• 2. from different decades,
• 3. by commercial vs. non-commercial outlets,
• 4. by health or science journalists vs. others,
• 5. in low-income countries vs. middle or high-income countries, or
• 6. in broadsheet vs tabloid newspapers. 5

The individual risk of bias assessments for each of the 108 criteria included in the meta-analyses are presented in S9 File in the extended data. 105 We considered there to be a high risk of bias in 13 cases (11%), including in the case of the one criterion included in two criteria groups (the third criterion in the study by Stassen 43 ). The 13 criteria for which we considered there to be a high risk of bias are the 13 that required substantial judgement to be applied, according to our assessment. In 12 of those 13 cases, it was—in addition—unclear whether researchers were blinded to the publication and journalist behind the news report. In the remaining case, it was clear that they were not blinded. In four of the 13 cases, it was also unclear whether the criterion was applied independently by at least two researchers.

Quality of the evidence

We only assessed the quality of the evidence for the five main groups and six subgroups where we report an overall estimate. Quality varied from low to high. In all but two of 10 cases, we downgraded quality for substantial inconsistency. In half of the cases, we downgraded for important imprecision. Finally, for the group criteria related to “disease mongering”, we downgraded for overall high risk of bias.

Results by criteria group

We summarise the main findings here. Details of these analyses are reported in S10 File in the extended data. 105

Criteria related to associations and randomization

Many news reports included causal claims based on associations, and few said whether trials were randomized. We included three estimates, from three studies, 19 , 35 , 47 related to the IHC Key Concepts that “An outcome may be associated with a treatment but not caused by it” (1.2d) and “Comparison groups should be as similar as possible” (2.1a). Based on estimates for single criteria with a low risk of bias, 51% of news reports make causal claims based on non-randomized studies (95% CI 40%-62%) 32 and 9% specify whether a given trial was randomized (95% CI 7%-12%). 19

Criteria related to the need for comparisons

Some news reports concluded that an intervention has an effect despite a lack of a comparator, and many did not include evidence to support the effectiveness of the intervention. We included three estimates, from three studies, 12 , 35 , 41 related to the IHC key concept that “Identifying effects of treatments depends on making comparisons” (1.2f). Based on an estimate from a single study, three quarters (74%) of news reports avoid concluding there is a beneficial effect when there is a lack of a comparator (95% CI 63%-83%). 35 Two studies found that only a third (34%) 12 to half (55%) 41 of news reports include evidence about the effects of the intervention (95% CIs 25%-43% and 38%-72%). The risk of bias for these two criteria was high.

Criteria related to consistency between studies

Few news reports considered and soundly assessed the consistency of evidence. We included a single criterion, from the study by Krauth and Apollonio, 31 related to the IHC Key Concept that “The results of one study considered in isolation can be misleading” (1.2g) and “Consider how certain you can be about each advantage and disadvantage” (3.3d). The risk of bias was low for this criterion. The study found that 13% (95% CI 7%-26%) of news reports published in American newspapers and magazines about tobacco cessation therapy “consider” and make a “well-founded” assessment of consistency between studies.

Criteria related to “disease mongering”

Some news reports committed disease mongering. We included five estimates, from five studies, 12 , 13 , 29 , 41 , 43 related to disease mongering, which is addressed by the IHC Key Concept that “Earlier detection of ‘disease’ is not necessarily better” (1.2k). All five criteria had a high risk of bias. Overall, 86% of news reports do not commit disease mongering (95% CI 78%-91%) ( Figure 3 ). The certainty of the evidence for that estimate is low due to substantial inconsistency (I 2 = 92%) and the high risk of bias.

Criteria related to conflicts of interest

Many news reports failed to address conflicts of interest. We included seven estimates, from seven studies, 19 , 29 , 30 , 40 , 41 , 43 , 48 related to the IHC Key Concept that “Competing interests may result in misleading claims” (1.3b) All criteria had a low risk of bias. Overall, 22% (95% CI 7%-49%) in some way disclose potential conflicts of interest ( Figure 3 ). The certainty of the evidence for that estimate is low, due to substantial inconsistency (I 2 = 99%) and important imprecision.

Criteria related to opinions

Some news reports failed to adequately distinguish opinion and fact. We included a single estimate, from the study by Krauth and Apollonio, 31 related to the IHC Key Concept that “Opinions alone are not a reliable basis for claims” (1.3d). The study evaluated American newspaper and magazine reports about tobacco cessation therapy. It found that three quarters (73%) of news reports adequately distinguish opinion and fact (95% CI 6%-83%). The criterion had a low risk of bias.

Criteria related to study design and risk of bias in general

Many news reports did not address the “quality of evidence”. We included five estimates, from five studies, 13 , 29 , 31 , 41 , 43 related to the risk of bias (“type”, “credibility” or “quality” of the evidence), which is addressed by multiple low-level IHC Key Concepts under the high-level concepts “Comparisons of treatments should be fair” (2.1) and “Syntheses of studies need to be reliable” (2.2). Three of the criteria had a high risk of bias. The proportion of the news reports that meet the criterion used in each study varied from 8% (95% CI 3%-20%) 31 to 77% (95% CI 73%-81%). 43

Criteria related to blinding

Few news reports said whether trials were “double blinded”. We included a single criterion, from the study by Bubela et al ., 19 related to the IHC Key Concepts that “The people being compared should be cared for similarly apart from the treatments being studied” (2.1c), “If possible, people should not know which of the treatments being compared they are receiving” (2.1d) and “Outcomes should be assessed in the same way in all the groups being compared” (2.1e). The study evaluated reports in newspapers in English-speaking countries about herbal remedy and conventional pharmaceutical trials. It found that 7% (95% CI 6%-9%) of news reports specify if a trial was “double-blinded”. The criterion had a low risk of bias. The term “double blind” is a problematic term since it can have several meanings. 40 The study did not address this.

Criteria related to loss to follow-up

Few news reports said anything about follow-up of participants in trials. We included a single criterion, from the study by Bubela et al ., 19 related to the IHC Key Concept that “It is important to assess outcomes in all (or nearly all) the people in a study” (2.1g). This was the same study that evaluated whether news reports specify whether a trial was “double blind”. 18 The risk of bias was low for this criterion. The study found that 7% (95% CI 5%-9%) of news reports specify withdrawals or dropouts from a trial.

Criteria related to selective reporting

Few news reports selectively reported “statistically significant” results. We included two estimates, from two studies, 35 , 47 related to IHC Key Concepts that “Failure to consider unpublished results of fair comparisons may result in estimates of effects that are misleading” (2.2b) and “Deeming results to be ‘statistically significant’ or ‘nonsignificant’ can be misleading” (2.3g). Overall, 93% (95% CI 87%-97%) of news reports avoid selectively reporting “statistically significant” results ( Figure 3 ). The certainty of the evidence for that estimate is high.

Criteria related to qualitative descriptions of effects

Many news reports used misleading language to describe intervention effects and did not quantify effects. We included seven estimates, from five studies, 29 , 30 , 35 , 40 , 43 related to the IHC Key Concept that “Verbal descriptions of the size of effects alone can be misleading ” (2.3a) Overall, four studies that reported six criteria related to quantification of effects found that 53% (95% CI 36%-69%) of news reports quantify effects, as opposed to only describing them qualitatively. There was a high risk of bias for two of the criteria. The certainty of the evidence for this estimate is also low, due to substantial inconsistency (I 2 = 99%) and important imprecision. Overall, two studies, found that 68% (95% CI 35%-90% of news reports do not use misleading language to describe intervention effects. One of the criteria had a risk of bias. The certainty of the evidence for that estimate is low, due to substantial inconsistency (I 2 = 98%) and important imprecision.

Criteria related to absolute effects

Most news reports did not report absolute effects. We included five estimates, from five studies, 12 , 13 , 40 , 41 , 49 related to the IHC Key Concept that “Relative effects of treatments alone can be misleading” (2.3b). All five criteria had a low risk of bias. Overall, 17% (95% CI 4%-49%) of news reports report absolute effects ( Figure 3 ). The certainty of the evidence for that estimate is low, due to substantial inconsistency (I 2 = 97%) and important imprecision.

Criteria related to the play of chance

Few news reports claimed that an intervention has a benefit despite a small sample size, and more than half specified the sample size of trials, but few clearly and soundly “assessed the precision” 8 of reported estimates of effect. We included four estimates, from four studies, 19 , 31 , 35 , 39 related to the IHC Key Concepts that “Small studies may be misleading” (2.3d) and “The use of p-values may be misleading; confidence intervals are more informative” (2.3f). Based on estimates from single studies 95% (95% CI 88%-98%) of news reports do not claim there was a benefit despite a small sample size, 35 and 63% (95% CI 59%-67%) specify the sample size of trials. 19 Two studies found that 6% (95% CI 1%-38%) 39 and 8% (95% CI 3%-20%) 31 of news reports “clearly and soundly” assess the precision of estimates or risk of random error.

Criteria related to subgroup analyses

Most news reports did not focus on inappropriate subgroups. We included a single estimate, from the study by Yavchitz et al ., 47 related to the IHC Key Concept that “Results for a selected group of people within a study can be misleading” (2.3e). The study found that 94% (95% CI 85%-98%) of news reports avoid focusing on an “inappropriate subgroup”. The criterion had a low risk of bias.

Criteria related to lack of evidence

Most news reports avoided claiming that interventions have equivalent effects based on a difference that was not “statistically significant”. We included a single estimate, from the study by Yavchitz et al ., 47 related to the IHC Key Concept that “Lack of evidence of a difference is not the same as evidence of ‘no difference’” (2.3h). The study found that 93% (95% CI 83%-97%) of news reports do not claim equivalence when there was a “statistically nonsignificant” difference. 47 The criterion had a low risk of bias.

Criteria related to options

Most news reports did not include information about alternative interventions. We included seven estimates, from six studies, 13 , 29 , 41 , 43 , 50 related to the IHC Key Concept “Be clear about what the problem or goal is and what the options are” (3.1a). Overall, 36% (95% CI 26%-47%) of news reports include information about alternative interventions ( Figure 3 ). The certainty of the evidence for that estimate is moderate, due to substantial inconsistency (I 2 = 96%).

Criteria related to extrapolations

Some news reports focused on surrogate outcomes or extrapolated from surrogate to important outcomes. Some news reports did not report indications for interventions and most did not report contraindications. Few news reports did not clearly report to whom the information applied. Some extrapolated from study participants to a larger or different population. Many news reports did not consider the availability of the intervention or the dosage of the intervention. Some interventions extrapolated from the study intervention to a different intervention.

Altogether, we included 21 criteria, from 12 studies, 12 , 13 , 19 , 29 , 31 , 35 , 38 , 41 , 43 , 47 , 50 , 51 related to the IHC Key Concepts that “Attention should focus on all important effects of treatments, and not surrogate outcomes” (3.2a), “Fair comparisons of treatments in animals or highly selected groups of people may not be relevant” (3.2b), “The treatments compared should be similar to those of interest” (3.2c) and “There should not be important differences between the circumstances in which the treatments were compared and those of interest” (3.2d). There was a low risk of bias for all the criteria.

We included two criteria, from two studies 35 , 50 related to the IHC Key Concept that “Attention should focus on all important effects of treatments, and not surrogate outcomes” (3.2a). Overall, 82% (95% CI 76%-86%) of news reports do not solely focus on surrogate outcomes or extrapolate from surrogate to important outcomes. The certainty of the evidence for this estimate is high.

We included eight criteria, from six studies, 31 , 35 , 38 , 41 , 50 , 51 related to IHC Key Concept that “Fair comparisons of treatments in animals or highly selected groups of people may not be relevant” (3.2b). Five of these criteria, from three of the studies, 41 , 50 , 51 addressed reporting of indications and contraindications. Indications are reported in 70% (95% CI 64%-76%) to 97% (95% CI, 95%-98%) of news reports, whereas contraindications are only reported in 4% (95% CI 2%-8%) to 13% (95% CI 11%-16%) of news reports. Two criteria, from two studies, 31 , 38 addressed whether it was clear to whom the information applies. These criteria are met in 94% (95% CI 81%-98%) to 100% (95% CI 98%-100%) of news reports. The other criterion, from the study by Haneef et al ., 35 assessed whether results were extrapolated from study participants to a larger or different population. The study found that 85% (95% CI 72%-92%) of news reports do not extrapolate to a larger or different population.

We included eight criteria, from eight studies, 13 , 19 , 29 , 35 , 41 , 43 , 51 related to the IHC Key Concept that “The treatments compared should be similar to those of interest” (3.2c). Five of these criteria addressed availability of the intervention. The availability of the intervention is reported in 41% (95% CI 37%-45%) to 79% (95% CI 66%-88%) of news reports. Two of the criteria addressed dosage. Those studies found that 14% (95% CI 11%-17%) 19 to 50% (95% CI 46%-54%) 51 of news reports specify or mention dosage. The other criterion addressed extrapolation from the study intervention to a different intervention. That study 35 found that 85% (95% CI 72%-92%) of news reports do not extrapolate to a different intervention.

Criteria related to animal studies

Some news reports extrapolated a beneficial effect from an animal study to humans. We included a single estimate, from the study by Haneef et al ., 35 related to the IHC Key Concept that “Fair comparisons of treatments in animals or highly selected groups of people may not be relevant” (3.2b). The study, which had a sample of 29 news reports, found that 79% (95% CI 61%-90%) of reports do not extrapolate beneficial effects in humans from animal studies. 35 The criterion had a low risk of bias.

Criteria related to pros and cons

Few news reports failed to mention at least one benefit of an intervention. On the other hand, few mentioned or discussed the cost of the intervention, and most did not mention or adequately discuss or explain potential harms.

Altogether, we included 32 estimates, from 17 studies, 12 , 13 , 29 – 31 , 35 , 38 – 41 , 43 , 47 – 52 related to IHC Key Concepts that “Treatments can cause harms as well as benefits” (1.1a) and “Weigh the benefits and savings against the harms and costs of acting or not” (3.3a). One criterion was also included with criteria related to qualitative descriptions of effects, related to the IHC Key Concept that “Verbal descriptions of the size of effects alone can be misleading” (2.3a). That criterion was the only one that had a high risk of bias.

We included 14 criteria, from 13 studies, 12 , 13 , 30 , 35 , 40 , 41 , 43 , 47 – 52 related to harms. Overall, 40% (95% CI 23%-61%) of news reports about the effects of health interventions mention or adequately discuss or explain potential harms of the intervention. The certainty of the evidence for that estimate is low, due to substantial inconsistency (I 2 = 99%) and important imprecision.

We included 10 criteria, from nine studies, 13 , 29 , 30 , 40 , 41 , 43 , 49 – 51 related to cost. Overall, 18% (95% CI 12%-28%) mention or discuss the cost of the intervention. The certainty of the evidence for that estimate is moderate, due to substantial inconsistency (I 2 = 98%).

We included two criteria, from two studies, 41 , 50 related to benefits. Overall, 97% (95% CI 56%-100%) of news reports mention at least one benefit. The certainty of the evidence for that estimate is moderate, due to imprecision.

Key findings

Overall, we found that researchers have conducted many empirical studies of the quality of news reports about the effects of health interventions, assessing diverse samples of reports in terms of medium, country (although primarily English-speaking and high-income), and categories of interventions, using many different tools and criteria. Some criteria did not make sense and some relevant aspects of quality were not covered. We were able to synthesis results for some criteria groups, showing that there were several prevalent and important problems with the quality of news reports about the effects of health interventions. Many of the reports gave an unbalanced and oversimplified picture of the potential consequences of the interventions, leaving out information about conflicts of interest, alternative interventions, potential harms, and costs, and failing to quantify effects in absolute terms or at all.

Specifically, we identified 44 primary studies for inclusion in the qualitative summary (see extended data - S1 Table 105 ). Newspaper was the most common medium. Of the 39 studies that specified a medium, newspaper content was explicitly included in 35 (90%). The United States (US) was the most common country in which content was published. Of the 40 studies where at least one country was specified, 12 (30%) included content published in the US, while 25 (63%) included content published in English-speaking countries. “Modern” medicine was the most common category of health interventions. In the 32 samples where a category of intervention was specified, 24 (75%) included news reports about “modern” medicine. No study specified whether sampled reports were published by commercial or non-commercial outlets. Most studies used a tool that was original or modified.

The Index of Scientific Quality was the most common tool (set of criteria). It was used in seven studies (16%). 31 , 33 , 34 , 36 – 39 However, a larger number of studies used tools directly or indirectly based on the criteria developed by Moynihan et al. , 40 including the four studies that used the Media Doctor Australia tool 12 , 13 , 41 , 42 and the three studies that used the Health News Review tool. 28 , 29 , 43

Of the 44 studies included in the qualitative summary, we included 18 (41%) in the meta-analyses. We mapped all 208 quality criteria used in those studies against the IHC Key Concepts. Some of the criteria were in conflict with IHC Key Concepts, such as those that asked whether research discussed in a news report was peer-reviewed. We were unable to identify or include quality criteria related to several highly relevant IHC Key Concepts, for example “ Average differences between treatments can be misleading ” (2.3c).

In the end, we included 108 of the 208 criteria (52%) in the meta-analyses, organised in 19 groups, as well as eight subgroups. We calculated overall estimates and rated the certainty of the evidence for five of the 19 main criteria groups (26%) and 6 of the 8 subgroups (75%). Those estimates ranged from 17% of news reports satisfying criteria related to reporting absolute effects (95% CI 4%-49%) (low certainty), to 97% satisfying criteria related to reporting on benefits (95% CI 56%-100%) (moderate certainty).

Media doctor Australia and health news review

Some of the studies included in our review report partial results from initiatives aimed at helping journalists and the public assess health information in the news and improving health journalism. This includes studies related to the Media Doctor Australia initiative 12 , 13 , 42 and its American counterpart Health News Review (or HealthNewsReview.org ). 28 , 29 These initiatives involved an editorial team rating news reports about the effects of health interventions on a running basis and posting individual ratings with lay explanations on a dedicated website. Both have been shut down. We were unable to include the most complete published results from either initiative in our meta-analyses, but those results were consistent with the earlier results that we were able to include, as well as our overall findings.

In terms of Media Doctor Australia, we were able to include the studies by Smith et al. 12 and Bonevski et al . 13 in the meta-analyses. The former includes results from the first 104 ratings, published between Feb. 1 and Sept. 1, 2004, while the latter includes results from the first 222 ratings of news reports about “complementary” and “alternative” medicine, published between Jan. 1, 2004, and Sept. 1, 2007. Meanwhile, Wilson et al . include results from 1230 ratings, posted between March 2004 and June 2008, in a paper titled “Media reporting of Health Interventions: Signs of Improvement, but Major Problems Persist”. 42 We included the study by Wilson et al. in our qualitative summary, but not in the meta-analyses, since the authors only reported mean scores and not the sample sizes for each quality criterion. We knew from the two other Media Doctor Australia studies that sample sizes varied between criteria because a criterion was not always considered relevant to a given news report and, in those cases, not applied. In other words, we could not extract the exact proportions of news reports satisfying each criterion from the study by Wilson et al . That said, as the title of paper suggests, there were no notable differences between the new Media Doctor Australia results and the previous results reported by Smith et al . (see Table 2 in the study by Wilson and colleagues 42 ).

In terms of Health News Review, we were able to include the studies by Schwitzer 28 and Walsh-Childers et al. 29 in the qualitative summary. The former included results from 500 Health News Review ratings, published between April 2006 and February 2008, while the latter included results from 1889 ratings published from July 2005 through March 2013. Hence, we only included the study by Walsh-Childers et al. in the meta-analyses. The Health News Review team continued to publish ratings until the end of 2019, finishing at 2610. 53 In a blog post published Dec. 20, 2018, the founder and publisher, Schwitzer, shared a final “report card” with some of the results from all 2610 ratings. 53 Like with the Media Doctor Australia studies, there were no notable differences between the most recent results, published in Schwitzer's blog post, and those previously reported and included in this review, from the study by Walsh-Childers et al .

Underlying problems

Problems with the quality of information in health news reports could be explained by underlying problems occurring at different stages of the information pipeline, from when scientists plan, conduct and report studies, to when academic institutions and journals promote studies, to when news outlets cover studies. Not to mention deliberate misinformation by commercial industries and organisations. 54

Ioannidis has estimated that most published research findings are false due to small studies and small effect sizes, testing many relationships without preselecting those tests, flexibility in study designs, definitions, outcomes and analyses, financial and other conflicts of interest, and competition. 55 Similarly, Chalmers and Glasziou have estimated that 85% of research is wasted due to wrong research questions, unnecessary or poorly designed studies, failure to publish relevant research promptly or at all, and biased or unusable reports of research. 56 , 57 Meanwhile, Chi and colleagues have found problems with the quality of information in biomedical literature that correspond with problems found in health news reporting, for example inappropriate use of causal language. 58 A vast amount of poorly planned, conducted and reported research provides a terrible starting point for health and science journalism. Parenthetically, these studies provide additional evidence that peer review is a poor indicator of quality.

Several studies have identified problems with press releases about health research. Wang et al . found randomised trials were less likely to be the subject of a medical journal press release compared to observational studies. 59 Woloshin et al. assessed press releases from academic medical centres and found many were directly misleading or misleading by omission. 60 Yavchitz et al. found press releases for two-arm, parallel-group randomised trials often included some form of “spin”, for example conflating lack of evidence of a difference with no difference. 47 Sumner et al. have found exaggerations in press releases from both academic institutions 61 and scientific journals, 62 such as causal claims based on observational studies, were associated with exaggerations in related news reports. This finding was supported by a replication study conducted by Bratton et al . 63

Amend and Secko conducted a qualitative synthesis of studies exploring journalists’ experience of covering health and science. 64 They included 21 studies involving 788 journalists in total. Many of the themes cutting across studies were barriers to high-quality health and science journalism, including deadline pressure and limited time for researching and preparing stories; limited time or space for telling stories; limited budgets and staff, including a lack of specialised health and science journalists; competition and commercialization; pressure from advertisers and interest groups; and lack of education or training. More recently, O’Keeffe et al . found some of the same barriers to reliable reporting when they interviewed 22 Australian journalists, most of whom were specialised in health. 65

Other mass media sources of health information

Other studies have focused on other mass media sources of lay information about the effects of health interventions. This includes both other forms of journalism, besides news reports, and other mass media, besides news media. Some of those studies were included in this review. Overall, it appears quality varies depending on the source, but that there are problems across the board, granted we have not systematically reviewed or critically appraised studies focusing on other sources.

Newspaper features and advice columns – Heaner assessed news features and news reports about nutrition and physical activity. 32 She found features were generally of higher quality. Molnar et al. assessed medical advice columns for elderly readers and found that recommendations were often inappropriate or potentially dangerous. 66

Entertainment – Wilson et al . assessed the quality of advice popular magazine articles and found especially those with “health” in the title of the publication presented poor-quality, unreliable advice. 67 Korownyk et al . and Mishori et al. assessed recommendations made on medical talk shows. 68 , 69 Both studies found the recommendations were often unreliable or misleading.

Social media – Moorhead et al. conducted a systematic review of the uses, benefits and limitations of social media for health communication and found that concern about the quality of information was common across studies. Li et al. found that over a quarter (19) of the 69 most-viewed YouTube videos about Covid-19 contained misleading information, including “inappropriate recommendations”, consistent with previous studies of YouTube videos about other pandemic diseases. 70 Haber et al. found many of the health science articles most often shared on Twitter and Facebook in 2015, as well as media coverage of those articles, often used causal language that was too strong for the strength of the evidence. 27

Advertising – There is a long tradition of misleading advertisements for health interventions. Dushman has provided 10 examples from the early 20 th century, including an ad for Dr. William O. Coffee's “world-famous”, “marvelous” and “remarkable” treatment for deafness. 71 Frosch et al . and Faerber and Kreling have found problems with modern advertisements for both prescription and over-the-counter medicines. 72 , 73 In sales materials for herbal dietary supplements, the United States Government Accountability Office has found “improper” and illegal claims that the supplements can cure cancer, among other conditions. 74

Other consumer and patient sources – In a systematic review of studies assessing the quality of online health information for consumers, Eysenbach et al. found 55 of 79 studies (70%) had concluded that quality was a problem. 75 In more recently published studies, others have found problems with the information on anti-vaccination websites, 76 cancer and oncological industry websites, 77 websites for clinics offering weight loss surgery, 78 and websites for fertility centres. 79 In their study of media coverage of practice-changing clinical trials in oncology, Andrew et al . found the cancer and industry websites provided higher-quality information than newspaper and cable news reports. 77

Studies focusing on patient sources of information have found problems with patient health portals, 80 patient materials, 81 and brochures from a health fair. 34 Glenton and Oxman (AO) surveyed and interviewed leaders of organisations representing health care users. 82 They concluded that the organisations did not appear to promote evidence-based health care and that when they did promote scientific information, they appeared to do so uncritically, relying on limited sources and traditional authorities.

Last but not least, in a review published in 2019, Oxman and Paulsen assessed free, online sources of “trustworthy” information about the effects of health interventions. 83 They found that patients and the public could access trustworthy information using two websites: Cochrane Evidence ( www.cochrane.org/evidence ) 84 and Informed Health ( www.informedhealth.org ). 85 However, they concluded that both websites could be improved by consistently reporting information about the size of both benefits and harms, and the certainty of the evidence. They noted that many websites excluded from their review claim to provide evidence-based or reliable information about the effects of health interventions, but that it was difficult to assess the reliability of the information on those websites since the information was not explicitly based on systematic reviews.The Informed Health website is a separate initiative from the Informed Health Choices project.

Other problems with health news

Other studies have found other problems with health news, besides the quality of information. Wang et al . found that observational studies and small randomised trials reporting surrogate outcomes generated as much news coverage as large, randomised trials reporting important outcomes. 59 Dumas-Mallet et al. found that newspaper reports about biomedical research were typically about findings from initial studies. 86 Such findings were often contradicted by a meta-analysis, but this was rarely reported by news outlets.

As far as we are aware, this is the first systematic review of the quality of any type of health news. It provides a larger, more detailed picture of the quality of health news than any primary study that we identified. It also provides a starting point for similar reviews and alternative analyses, and it highlights important issues for interpreting and planning new studies. We conducted an expansive search and identified several studies outside the peer-reviewed literature (see extended data - S1 Table 105 ). We assessed records and full texts in any language for inclusion in the review, and included studies in five different languages in the qualitative summary (English, German, Norwegian, Slovenian and Spanish) (see extended data - S1 Table 105 ). We synthesised results from up to 13 different studies, using different tools and criteria, and pooled results for up to 4116 unique news reports (see extended data - S8 and S10 Files 105 ). The review provides an accessible summary of what is known about the quality of news reports about the effects of health interventions, in print, broadcast and online news media, for those aspects of quality that have been measured and reported. It also identifies gaps in what is known and directions for future research.

Limitations

The findings of this review are limited to information about the effects of health interventions and “traditional” news media. In the section “Other mass media source of health information”, we reference and discuss findings from research focusing on other news media and other non-news media, but we have not systematically reviewed or critically appraised such studies. Furthermore, we did not explore applicability of the synthesised results to different types of traditional news media due to limited sample sizes, as well as inadequate reporting in the included studies.

Examples of other types of health information include the accuracy of diagnostic tests and information about the health effects of exposures. News reports about the health effects of exposures (e.g. fatty foods) can be difficult to distinguish from news reports about the effects of health interventions (e.g. a low-fat diet). Some primary studies included in this review conflated the two, such as the study by Hackman and Moe, which assessed news about nutrition-related research. 87 We included this study in the qualitative summary, but not the meta-analyses. We did this because key concepts for assessing or communicating information about the effects of interventions may not apply to the same degree, or at all, to information about associations between exposures and outcomes. For example, Woodruff and Sutton point out that randomised trials are “virtually precluded” from evidence about hazardous environmental exposures because of ethical considerations”. 88 It is possible, if not likely, that there are important differences in quality between news reports about the effects of exposures versus interventions.

We excluded studies or criteria that were only descriptive, even though criteria that were descriptive sometimes provided the same data as criteria explicitly intended to assess quality. For example, Mercurio and Eliott describe Australian news about “alternative” medicine for cancer, but do not explicitly assess the quality of those news reports, 89 so we excluded the study. However, one of their coding variables was “Mention of risks, benefits and costs”. The data for this variable would have fit with data in our meta-analyses, in the group Criteria related to pros and cons. We consider it unlikely that inclusion of studies or criteria that were only descriptive would have substantially altered the findings of this review. Similarly, we only included studies in the meta-analyses if they were in English or Norwegian (one of the languages spoken fluently by the primary investigator, MO), but consider it unlikely that the studies reported in other languages would have substantially altered the findings of this review either, given that the underlying problems discussed earlier on are unrelated to language.

We did not code raw data ourselves or contact authors for help where data were inadequately reported for our purposes. Finally, we did not consider which criteria were more informative, only whether they were eligible and sensible, nor did we consider statistical measurement properties. 90

Implications

Implications for different stakeholders are summarised in Table 5 and explained in more detail in the subsequent sections.

Implications for journalists, editors, and news outlets

There is clearly room for improving the quality of health and science journalism. Based on our meta-analyses, journalists and editors should pay special attention to the following IHC Key Concepts (see extended data - S6 File 105 ) when communicating information about the effects of health interventions:

• • “Treatments can cause harms as well as benefits” (1.1a),
• • “Competing interests may result in misleading claims” (1.3b),
• • “Verbal descriptions of the size of effects alone can be misleading” (2.3a),
• • “Relative effects of treatments alone can be misleading” (2.3b),
• • “Be clear about what the problem or goal is and what the options are” (3.1a), and
• • “Weigh the benefits and savings against the harms and costs of acting or not” (3.3a).

Based on individual estimates from primary studies included in this review (see extended data - S8 and S10 Files 105 ), there is likely also room for improvement related to other highly relevant IHC Key Concepts, such as “ The treatments compared should be similar to those of interest ” (3.2c). Moreover, there is likely room for improvement related to highly relevant IHC Key Concepts that were not captured by any quality criteria, such as “ Average differences between treatments can be misleading ” (2.3c) (see extended data - S10 File 105 ).

Journalists and editors should take a critical approach to reports of research, especially single studies and observational studies, and to press releases (see “Underlying problems”). When reporting on the effects of interventions, they can search the Cochrane Evidence ( www.cochrane.org/evidence ) 84 and Informed Health ( www.informedhealth.org ) 85 websites for summaries of relevant systematic reviews. If possible, news outlets should provide or support special training for health and science reporters, as well as provide adequate time for researching and preparing stories, and adequate time or space to tell stories.

Journalists and editors who are interested in learning more about IHC Key Concepts can do so for free on the “That's a claim!” website ( www.thatsaclaim.org ), 91 where all of the concepts are explained in plain language, and there are references to relevant learning resources for each individual concept. They can also find free learning resources for respective concepts via the Teachers of Evidence-Based Health Care portal ( www.teachingebhc.org ). 92

Implications for researchers, scientific journals, academic institutions, and research funders

Researchers, journals, and academic institutions should pay special attention to the same IHC Key Concepts as journalists and editors, as well as other highly relevant concepts (see previous section), when reporting the results of studies assessing the effects of interventions in research papers and press releases, as well other contexts, such as conferences. More fundamentally, they should work to reduce false findings and research waste. This also goes for organisations that fund research.

The results of this review have a series of implications for investigators planning new studies of the quality of information about the effects of health interventions in mass media (as well as peer reviewers assessing new studies), from objectives, to methods, to reporting. First, they should think carefully about their goal. New studies can be designed to have practical value, for example informing interventions to improve a particular population's ability to think critically about health information by identifying the IHC Key Concepts that are most relevant for that population—e.g. a study of health claims in advertisements targeted at young people. In addition, they should consider research gaps that we have revealed. Again, there does not appear to be studies assessing the quality of health news reports in terms of several highly relevant IHC Key Concepts (see extended data - S6 Table 105 ). Neither does there appear to be studies that compare the quality of reports published by commercial outlets versus public or independent outlets. Others have pointed out that research is needed about how best to communicate information about the effects of health interventions, 93 including how to communicate uncertainty. 94

Second, they should think carefully about what tools and quality criteria to use, whether using a previously developed tool, adapting one, developing a new tool, or some combination. They should avoid criteria that are overly general (e.g. global scores and composites) or unclear (e.g. criteria assessing whether “double blinding” is mentioned), and criteria that do not make sense (e.g. criteria assessing whether research was peer-reviewed). They should be deliberate about the trade-offs between how straightforward it is to apply a criterion and report the results versus how informative it is. For example, it is relatively easy to see whether a news report mentions the design of a study. It is more difficult, but also more informative, to assess whether the report adequately discusses strengths and limitations of said design.

Third, regarding samples of news reports, they should consider applicability and potential explanatory factors, for example those we highlighted in the protocol for this review (see extended data - S1 File 105 ) as possible variables for subgroup analyses: medium; time period; financial model (commercial vs. non-commercial); specialisation (specialised health or science journalist vs. other); country income level; and newspaper type (broadsheet vs. tabloid). Fourth, they should consider risk of bias. If possible, two researchers should assess each news report or other unit of mass media content, and the researchers should be blinded to factors that could introduce bias, such as the name of the journalist or publication. This is especially important for criteria that require substantial judgement.

Fifth, when reporting new studies, researchers should be explicit, clear and comprehensive. They should include the study objective, the sampling frame, and the selection criteria and technique, with concern for applicability and comparing results across studies, including results for subgroups. They should include the time period in which the content was published, as well as when it was assessed; the country in which it was published or, if online content, the country that the outlet primarily targets or where it is based; the categories of interventions that the content focuses on; and the financial models of the outlets. If using a previously developed tool, they should provide reference to where the development is described, note whether they made any adjustments and, if so, explain them. If using a new tool, they should clearly describe how it was developed and what informed it, possibly in a separate paper. If the researchers received any training in applying the criteria, they should describe what this entailed. Finally, they should report results for each quality criterion separately, not just mean scores, and report absolute numbers (the total number of news reports or other units to which the criterion was applied and the number of units that satisfied the criterion), including for subgroups.

Implications for consumers and patients

The implications for consumers and patients are similar to those for journalists and editors. They should pay special attention to the same IHC Key Concepts, as well as other highly relevant concepts (see “Implications for journalists”); they should approach health information in the mass media critically; they can refer to the Informed Health and Cochrane Evidence websites for summaries of relevant systematic reviews; and they can take advantage of the “That's a claim!” website and Teachers of Evidence-Based Health Care portal to learn more about IHC Key Concepts.

Implications for citizens and policymakers

The results of this review, as well as the results of related studies referenced in the section “Other mass media sources of health information”, show that the global “infodemic” was compounded by Covid-19, but started long before the World Health Organization coined the term in their situation report on the virus published Feb. 13, 2020. 3 The infodemic is so dangerous because we know so many people are unable to critically assess claims or evidence about the effects of health interventions. 95 , 96 When people act on those claims or beliefs that are unreliable or fail to act on reliable advice, they may suffer unnecessarily or waste resources. It follows that infodemic is probably an important factor in the enormous, worldwide overuse of ineffective or harmful interventions 97 and underuse of effective interventions. 98 Even one misleading news report might contribute to wasteful, harmful, and even deadly choices. A dramatic example of misleading health news is coverage of the baseless suggestion that the measles, mumps and rubella vaccine causes autism. 99 Incidentally, news reports often focus on the benefits of interventions while ignoring or downplaying the harms, but this example illustrates that unbalanced reporting can also go in the opposite direction.

If the only goal of news about the effects of health interventions was to facilitate well-informed health choices, the content and format might look more like Cochrane plain language summaries, which have been systematically developed towards exactly said end. 100 , 101 The reality is that news outlets existentially depend on getting people's attention, not informing them. Commercial media rely on advertising, not least advertisements for health interventions, which provides incentive to sensationalise. In turn, this can create pressure on competing public and independent outlets to do the same. Along the same lines, the raison d'être of press releases is inciting media coverage, while scientists are forced to “publish or perish”. Setting aside systemic barriers to reliable health news (see the section “Underlying problems”), it may be difficult if not impossible for journalists to produce news reports that satisfy all sensible, applicable criteria identified in this review in a report that people would understand and want to read, not to mention other potential criteria related to other highly relevant IHC Key Concepts. Even if a report includes and appropriately presents all relevant information about an intervention, it is possible that many people will not read more than the headline, missing important details, such as potential side effects.

With enough resources, it is possible for news outlets, including commercial ones, to improve their health and science coverage without shrinking their audiences and perhaps even increasing their numbers of readers, listeners, or viewers in some cases. But putting an end to unreliable health news is a Sisyphean task, not to mention health misinformation from other sources. In 2009, during a debate titled “Does the media support or sabotage health?”, Goldacre suggested: “The future of people getting information about health does not lie in journalism but people having direct access to information or the world of blogs where people can link direct to primary sources. There is no role for the health journalists of the future other than as entertainers". 102 However, it seems unlikely that giving people more health information—even if it is reliable—will solve the problem, when they are already overwhelmed and lack the skills needed to “separate the wheat from the chaff”.

As the saying goes, give someone a fish and you will feed them for a day; teach them to fish and you feed them for a lifetime. In other words, you can provide people with reliable health information or advice, and maybe they will believe you. If you help them understand why it is reliable, or why a claim is unreliable, you are preparing them to assess information and make informed choices for themselves in the future, inoculating them against unreliable claims and uninformed choices. Hence, we should be helping people gain basic critical appraisal and decision-making skills, in addition to providing access to reliable information. Moreover, by improving people's ability to assess evidence or recognise a lack of reliable evidence, the public may start to demand reliable studies, reducing false findings and research waste.

Focusing on the education system allows for reaching large groups of people, gathered specifically to learn. It is important to start young, when people have time to reinforce and build upon what they have learned, and they have less to unlearn. The IHC network, which includes authors of this review, has developed respective interventions to help primary school children 103 and university students 104 master select IHC Key Concepts.

Data availability

Author contributions.

MO drafted the protocol and this paper. AO provided feedback on the draft protocol. All co-authors provided feedback on drafts of this paper. MO, LL and AQ piloted the eligibility criteria for primary studies. MO conducted all database searches, including citation searches. MO, LL, GG, DA, AQ and KB checked reference lists. All authors apart from CR contributed to study selection. All authors apart from AO and CR contributed to data extraction. MO and AO assessed, mapped and grouped the quality criteria. CR conducted the meta-analyses. MO and LL assessed risk of bias. MO and AO rated the certainty of the evidence.

Acknowledgements

Mike Clarke provided early guidance in a course on systematic reviews. Carl Heneghan and Kamal Mahtani were MO's MSc supervisors and provided feedback on the protocol and the first part of the review (MO's dissertation). Marit Johansen provided feedback on the search strategy. Anette Blüme, Tina Poklepović Peričić, Rob J.P.M. Scholten and Michel Wensing helped assess full texts in German, Slovenian and Dutch for inclusion in the qualitative summary. We are grateful to Mark Oette for providing a copy of his study, included in the qualitative summary. This review is dedicated to Lisa Schwartz who was a pioneer in the field 106 and a dear colleague and friend.

[version 2; peer review: 4 approved]

Funding Statement

The author(s) declared that no grants were involved in supporting this work.

Reviewer response for version 2

David henry.

1 Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Australia

My only remaining concern is the way the conclusion is framed: "it is logically more important to improve the public’s ability to critically appraise health information and make judgements for themselves". While education is preferable to regulation I don't think that improving media coverage of health is an alternative to educating the public to be more critical of health claims. We should be trying to encourage both.

Are the rationale for, and objectives of, the Systematic Review clearly stated?

Is the statistical analysis and its interpretation appropriate?

Are sufficient details of the methods and analysis provided to allow replication by others?

Are the conclusions drawn adequately supported by the results presented in the review?

Reviewer Expertise:

clinical epidemiology, pharmacoepidemiology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Reviewer response for version 1

The other reviewers of this article have highlighted its considerable strengths and I won’t repeat their comments. This is a solid attempt to summarise studies that have made structured attempts to quantify and report key characteristics of medical news stories that report therapeutic claims. It is very useful to have this summary.

Many of these studies were carried out to highlight the weaknesses in the reporting of medical advances in the mainstream media in the hope that this would raise standards. But there is no strong evidence that this has resulted from these efforts.

The studies included in this review preceded the news coverage of COVID-19, its preventive and therapeutic interventions. It would be very instructive to measure how the media performed during the epidemic and whether reporting evolved. My informal impression is that respect for science, and possibly the reports themselves, have improved. However, this has been countered by ignorant and some willfully and politically motivated inaccurate reporting in a few mainstream outlets and social media. My point is that the media landscape may have changed because of Covid-19.

I only have a few specific comments:

• I don’t think that conventional meta-analysis produces meaningful data in this situation. Meta-analysis is at its best when we can assume that there is an underlying single effect of interest that is measured imprecisely by individual studies. For instance, the relative effects of many important medications (e.g, statins, anticoagulants), which seem constant across different settings and background risks. Meta-estimates of prevalence are less helpful as there is usually an assumed variation. That is the case here where the study methods, story topics, settings, and reporters vary. There is no reason to assume a common effect to be meta-analysed. Some of the I 2 values are huge. In my view it would be preferable to present more as a scoping review, displaying and discussing the range of values reported by the studies.
• The investigators, appropriately, have mapped the varying scales and items to a common framework. They have used their own Informed Health Choices (IHC) Key Concepts framework. That’s fine, but the paper seems to assume a familiarity with the framework that many readers will not possess. I think a simplified version of the framework needs to be presented in the main text. At present, it seems to be buried in Supplementary data.
• In the Abstract, the authors conclude: ‘There is room for improving health news, but it is logically more important to improve the public’s ability to critically appraise health information and make judgements for themselves.’ The paper does not provide data to support this conclusion. The authors have done important ground-breaking work with IHC but the extensive coverage they provide themselves in the Discussion of an already long paper seems excessive. A separate editorial/opinion piece making these points would be welcome.
• I think there may be a mistake in the dot point summary in the Discussion which states ‘• 86% of news reports commit “disease mongering” (95% CI 78%-91%) (low certainty).’ In the paper they state ‘Overall, 86% of news reports do not commit disease mongering (95% CI 78%-91%)’. I think the latter is correct.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Norwegian Institute of Public Health, Norway

We are grateful to you and the other reviewers for your time, effort, and constructive criticism. Thank you! We have pasted your comments below, in italics, with responses in turn.

1. I don’t think that conventional meta-analysis produces meaningful data in this situation. Meta-analysis is at its best when we can assume that there is an underlying single effect of interest that is measured imprecisely by individual studies. For instance, the relative effects of many important medications (e.g, statins, anticoagulants), which seem constant across different settings and background risks. Meta-estimates of prevalence are less helpful as there is usually an assumed variation. That is the case here where the study methods, story topics, settings, and reporters vary. There is no reason to assume a common effect to be meta-analysed. Some of the I2 values are huge. In my view it would be preferable to present more as a scoping review, displaying and discussing the range of values reported by the studies.

We assume that this comment is suggesting the studies were too heterogeneous for estimating average proportions across studies to be meaningful. We were conservative about synthesising results this way, and cautious in our interpretation. We used a random effects analysis, which explicitly assumes there is not a common effect, but rather that the studies estimate distinct effect sizes. The large I2 values are a property of the literature included. The range and results for individual studies are available in the supplementary material, with the forest plots. We elected to exclude this information from the text because it would make the text harder to read, and it would not substantially alter the results—i.e., despite substantial heterogeneity, the overall estimates and confidence intervals provide an appropriate indication of what was found across the studies in each meta-analysis, and a better indication than the median and range, or range alone.

2. The investigators, appropriately, have mapped the varying scales and items to a common framework. They have used their own Informed Health Choices (IHC) Key Concepts framework. That’s fine, but the paper seems to assume a familiarity with the framework that many readers will not possess. I think a simplified version of the framework needs to be presented in the main text. At present, it seems to be buried in Supplementary data.

The article is already long, as pointed out in Reviews 1 and 3. We have included the framework as S6 File and reference the file at first mention of the framework in the methods section, where we also summarise its content, and explain why and how we used it.

3. In the Abstract, the authors conclude: ‘There is room for improving health news, but it is logically more important to improve the public’s ability to critically appraise health information and make judgements for themselves.’ The paper does not provide data to support this conclusion. The authors have done important ground-breaking work with IHC but the extensive coverage they provide themselves in the Discussion of an already long paper seems excessive. A separate editorial/opinion piece making these points would be welcome.

We have removed the last three paragraphs of the discussion, which were about IHC.

4. I think there may be a mistake in the dot point summary in the Discussion which states ‘• 86% of news reports commit “disease mongering” (95% CI 78%-91%) (low certainty).’ In the paper they state ‘Overall, 86% of news reports do not commit disease mongering (95% CI 78%-91%)’. I think the latter is correct.

Correct. We have fixed this typo.

Thank you again for the valuable feedback!

Amanda Wilson

1 School of Nursing and Midwifery, University of Technology Sydney, Sydney, NSW, Australia

This is a systematic review of studies assessing the quality of health news articles which provides a meta-analysis of the results. This is an interesting paper which provides a quantitative approach to what is likely to be considered a difficult area to quantify – that is quality of reporting. The influence of news reporting on health behaviours is well known and the current climate of pandemic and vaccination information makes this a highly pertinent area to explore.

The methodology is sound and rigorous with the authors following the gold standard approach of registering the review and publishing the protocol. All steps of the literature review are described in detail and the analyses and interpretation are sound.

The paper is extremely well written and I have little critical feedback on the structure of the research or presentation of the paper. My only criticism is that it is extremely long and possibly overly detailed. The length and detail are worthy of a Cochrane review but without the saving grace of a plain language summary. I wonder if some ruthless editing might enhance the number of readers who make it the whole way through. It feels like a PhD chapter rather than a journal article. Having said that, in the Discussion on page 18, I was interested to know which criteria did not make sense and which aspects of quality were not covered.

Public health, health literacy, health reporting in the media, health education

My only criticism is that it is extremely long and possibly overly detailed. The length and detail are worthy of a Cochrane review but without the saving grace of a plain language summary. I wonder if some ruthless editing might enhance the number of readers who make it the whole way through. It feels like a PhD chapter rather than a journal article. 

Please see our response to Review 1. 

Having said that, in the Discussion on page 18, I was interested to know which criteria did not make sense and which aspects of quality were not covered.

Regarding criteria that did not make sense, some conflict with IHC Key Concepts, such as the example in Table 3, or criteria that conflate a “statistically significant” effect with an important effect, including respective criteria used by Haneef et al. and Yavchitz et al. Others are imprecise measures of quality, for example criteria focusing on the novelty of the health intervention, or how much of a news report was based on a press release, since a report can be low or high-quality regardless of these characteristics. 

Regarding the aspects of quality not covered, S6 Table shows which IHC Key Concepts were not related to any criteria included in the meta-analyses.

Joel R. Lexchin

1 School of Health Policy & Management, Faculty of Health, York University, Toronto, Canada

2 Department of Family and Community Medicine, Faculty of Medicine, University of Toronto, Toronto, Canada

3 Department of Emergency Medicine, University Health Network, Toronto, Canada

This is an extremely comprehensive and well-done systematic review on an important topic. The authors have obviously put an enormous amount of effort into the project and the results show that. The study is well written and easy to follow. However, there are some areas where I believe that improvements can be made.

• There is a potentially relevant article that the authors appear to have overlooked: Cassels et al . Open Medicine 2008;2(1):e20–23. Another study (Cassels et al.,  2003) is a longer version of one that the authors use – reference 48.
• Page 3: The authors use a 2009 paper to claim that most people get health information either directly or indirectly from traditional news media as opposed to material that appears only on social media. However, studies that evaluate information sourced from the two different types of media may no longer be relevant to the present-day environment.
• Page 18: The authors say that their findings come from a diverse range of countries but don’t they mean primarily western countries?
• Page 20: The length of a newspaper story is not necessarily a measure of quality. Cassels et al.  have shown that there are few significant differences in the overall quality between short news briefs and longer stories. (Cassels et al ., 2003)
• Page 22: The recommendations for improving the reporting of journalists, editors and news outlets at least partially assume the existence of resources and time to educate journalists and editors about science journalism. However, the authors do not assess whether the necessary resources are actually present and available especially in smaller media outlets.
• Page 22: Informed Health is only available in English and German. Therefore, it is limited in terms of usefulness to those speaking other languages. Cochrane Evidence is superior in that regard.
• Page 23 (last line): Is it Informed Choices or Informed Health? Are these the same or different?
• Page 24: Earlier the authors refer to Informed Health and Informed Choices. Now there is a third name Informed Health Choices. Are all three the same?
• Page 24: Have any of the educational interventions that the authors mention (references 99-104) looked at the long-term (e.g., more than 1 year) effects?

Pharmaceutical policy, intellectual property rights

1. There is a potentially relevant article that the authors appear to have overlooked: Cassels et al. Open Medicine 2008;2(1):e20–23. 

We excluded Cassels et al. 2008 based on the fourth eligibility criterion, specifically for inadequate specification of the selection criteria for news reports and missing specification of the selection technique.

Another study (Cassels et al., 2003) is a longer version of one that the authors use – reference 48. 

We assume you are referring to the paper “Drugs in the News: How well do Canadian newspapers report the good, the bad and the ugly of new prescription drugs?” published by the Canadian Centre for Policy Alternatives. We were unaware of this version. However, it does not include data that we were missing from the shorter version. 

2. Page 3: The authors use a 2009 paper to claim that most people get health information either directly or indirectly from traditional news media as opposed to material that appears only on social media. However, studies that evaluate information sourced from the two different types of media may no longer be relevant to the present-day environment.

Our claim is that “people still get health information from the [news media],” not that most people get information directly or indirectly from traditional news media. We agree that distinguishing news and social media is challenging, and not necessarily sensible, which is an important point for new primary studies of the quality of health information in mass media.

3. Page 18: The authors say that their findings come from a diverse range of countries but don’t they mean primarily western countries?

Yes, except we mean English-speaking and high-income countries, not Western. We have specified this. 

4. Page 20: The length of a newspaper story is not necessarily a measure of quality. Cassels et al. have shown that there are few significant differences in the overall quality between short news briefs and longer stories. (Cassels et al., 2003)

We agree and we have removed the second clause of the sentence, which was a possible explanation for the finding that features were generally of higher quality than news reports. 

5. Page 22: The recommendations for improving the reporting of journalists, editors and news outlets at least partially assume the existence of resources and time to educate journalists and editors about science journalism. However, the authors do not assess whether the necessary resources are actually present and available especially in smaller media outlets.

In the section “Implications for citizens and policymakers”, we now specify that news outlets can improve their reporting “with enough resources”. In the section “Implications for journalists, editors, and news outlets”, we had already specified that news outlets should provide or support training, and provide adequate time and resources, “if possible”. And in the section “Underlying problems”, we had already pointed out that in their synthesis, Amend and Secko found limited time or space for telling stories, and limited budgets and staff were barriers to high-quality health and science journalism. 

In fact, we assume most outlets either cannot provide training or resources to improve their health and science journalism, or will not prioritise such spending, which is part of why we argue that it is so important to help all people, not just journalists, learn how to think critically about health information for themselves. That said, we already referred to free learning resources that journalists can use to improve their skills, without necessarily spending a lot of time. 

6. Page 22: Informed Health is only available in English and German. Therefore, it is limited in terms of usefulness to those speaking other languages. Cochrane Evidence is superior in that regard.

We agree, but it is still helpful to be aware of both sites.

7. Page 23 (last line): Is it Informed Choices or Informed Health? Are these the same or different?

It is “Informed Health”. We have corrected the typo.

8. Page 24: Earlier the authors refer to Informed Health and Informed Choices. Now there is a third name Informed Health Choices. Are all three the same?

At first mention of the Informed Health website, we now note that it is a separate initiative from the Informed Health Choices (IHC) project. Again, where we have referred to the site as “Informed Choices”, it should have said “Informed Health”, and we have fixed this. 

9. Page 24: Have any of the educational interventions that the authors mention (references 99-104) looked at the long-term (e.g., more than 1 year) effects?

No, and we believe that a series of interventions is necessary, reinforcing and building on what people have learned. However, this has not been tested. Note that we have removed the last three paragraphs, in response to Review 1.

Raymond Moynihan

1 Institute for Evidence-Based Healthcare, Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia

General Comments.

This review is extremely valuable – the first summarizing evidence about the quality of news reports about health interventions.

I have a few short general comments and some specific comments.

In relation to the use of the word  “qualitative” - I wonder if the authors might consider using a different word – or adding more explanation – perhaps saying “qualitative summary of results”  – as it sounds to the casual reader like when you refer to “qualitative” you may have done a SR of “qualitative” studies of news coverage (i.e. focus groups with journalists etc.) – whereas I think you have a different meaning in mind. (e.g.  P8 “qualitative summary”, which is also mentioned in Abstract).

Re: IHC. Given the importance of IHC concepts for this SR, and the role of the authors in IHC world, and the advocacy of IHC in the Discussion, I think the IHC concepts etc. needs to be mentioned in the Introduction and expanded on in the Methods (as to how and why exactly the IHC concepts interacted with your analysis in the SR) - and the authors link to IHC can be mentioned in Methods somewhere. This is important and will help explain how the Discussion section becomes a little like advocacy for IHC materials.

The paper is long and complex – but with much valuable information. If any more material can be moved into Supp files, that will be helpful.

I have no biostats training so I have not assessed the stats.

Specific comments.

• Search strategy: It seems almost 2 years since it was updated. Authors might consider updating before indexing.
• P5:  “We excluded inappropriate criteria…’   Perhaps “ineligible” might be better?

While this makes sense, I presume that sometimes that re-framing was not straight forward – and I wonder if there is anything more to say on any complexities associated with this approach. Also, if you can give a little more detail in the Bubela example, with numbers, that could help the reader.

• P7: “Risk of Bias”. Can the authors add a few words in text explaining why the "risk of bias" was solely about the risk of detection bias? A reader of an SR might normally expect a few different domains, so it would be good to just explain why this single domain was most relevant.
• P10: Table 2. I think Table 2 needs re-thinking – as it took me some time to work out what was going on. I was treating each row as a discreet coherent entity, but in fact you have multiple parts to each row. I am sure you can make it clearer somehow.
• P11: “In contrast, a checklist for people communicating evidence-based information about the effects of healthcare interventions includes the item “Help your audience to avoid misinterpreting continuous outcome measures,” granted the checklist is not for journalists.” I don’t fully understand meaning of this sentence - perhaps be a little clearer?

Also – a related point - I find the three paragraphs on P12 (and into P13) complex and hard to follow. I wonder whether this level of detail is necessary in the main text – and whether it might be moved to a supplementary – but that is for authors to decide. At the very least – if there is any way to make it all a little clearer around this point – that would be great for the reader. An example of the mention of the “ninth criteria” from Bubela and 12 th from Haneef.  I think this is way too much detail for main text and authors run the risk of losing readers, for what is a very valuable SR.  

Now I have read further I realise you have a long section  - starting on P14 – on the results – as they relate to IHC concepts. After having read that very long section, I still feel the need for some kind of punchy summary of the key findings (e.g. harms, COIs, costs, other options not often mentioned…) somewhere in the results section.  This kind of summary doesn’t appear until start of Discussion – but I think could be included earlier.   

• P18: I wonder if the dot points at bottom of P18 might be better in the Results – and help to drive the kind of summary I am suggesting above.
• Discussion: the paper feels like it morphs in the Discussion into a passionate advocacy for IHC materials – with a sometimes moralistic tone. I think the tone could be addressed. And I also think that in order to justify the IHC material in the Discussion, more needs to be said about IHC in the Introduction – and the authors relations with IHC need to be made more explicit in the Methods – as noted above.  

I cannot comment. A qualified statistician is required.

Medicine and Media; Conflicts of Interest; Overdiagnosis

In relation to the use of the word  “qualitative” - I wonder if the authors might consider using a different word – or adding more explanation – perhaps saying “qualitative summary of results”  – as it sounds to the casual reader like when you refer to “qualitative” you may have done a SR of “qualitative” studies of news coverage (i.e. focus groups with journalists etc.) – whereas I think you have a different meaning in mind. (e.g.  P8 “qualitative summary”, which is also mentioned in Abstract).

We have replaced “qualitative summary” with “review” in the abstract. And at first mention of “qualitative summary” in the main text, we explain that it is, not a synthesis of qualitative studies, but a structured synthesis of results excluded from the meta-analyses.

We have added a paragraph about IHC to the end of the introduction, where we mention the IHC Key Concepts and the relevant authors’ involvement in the project. In the methods section, on pages six to seven in the first version of the article, we already explained why and how we used the IHC Key Concepts framework in the review. We do not think it is necessary to mention the authors’ involvement in IHC in the methods section. We have removed the last three paragraphs of the discussion, which were about IHC.

We agree that the article is long and complex, which follows the study being large and complex. There are already 10 supplementary files and six supplementary tables, and we do not think it would be helpful to have more. We appreciate that the article was a demanding read for the reviewers, however we do not expect many other people will read it start-to-finish. The findings are summarised in the abstract and “Key findings” section, and the implications are summarised in Table 5. Moreover, there will be a short, plain-language post on the F1000Research blog summarising the findings and discussing the implications. Although we include implications for other groups, the target audience for the article itself is primarily researchers. Communicating the findings and implications to other groups is a separate task. 

Search strategy: It seems almost 2 years since it was updated. Authors might consider updating before indexing.

It is possible that there have been new, eligible studies since the last search, and we agree that an updated review could be of value, at some point. Others planning or reporting new studies may, indeed, want to conduct updated searches. However, that is beyond the scope of this review, which summarises relevant research up to August 2019.There are good reasons for ensuring that reviews of interventions, which inform healthcare decisions are up to date when they are published (and are kept up to date). However, this is a review of descriptive studies and we do not believe those reasons are relevant to this review. 

• P5:  “We excluded inappropriate criteria…’   Perhaps “ineligible” might be better?

We have replaced “ineligible [criteria]” with “inappropriate”, in the main text and Figure 1.

We found reframing the criteria straightforward, and there were few criteria that needed to be reframed as positive. As noted in the main text, each criterion included in the meta-analyses is presented in S10 File verbatim, as reported in the primary study, together with the reframed and paraphrased version that we used in the review. We have added numbers to the example with the criterion from the study by Bubela et al.

• P7: “Risk of Bias”. Can the authors add a few words in text explaining why the "risk of bias" was solely about the risk of detection bias? A reader of an SR might normally expect a few different domains, so it would be good to just explain why this single domain was most relevant.

We have added a paragraph about why we only assessed detection bias to the relevant section, with reference to the protocol and the section on deviations from the protocol, where we provide a detailed explanation.

• P10: Table 2. I think Table 2 needs re-thinking – as it took me some time to work out what was going on. I was treating each row as a discreet coherent entity, but in fact you have multiple parts to each row. I am sure you can make it clearer somehow.

We have asked the journal to change the formatting of Table 2 so that the columns are alternate colours, rather than the rows. 

• P11: “In contrast, a checklist for people communicating evidence-based information about the effects of healthcare interventions includes the item “Help your audience to avoid misinterpreting continuous outcome measures,” granted the checklist is not for journalists.” I don’t fully understand meaning of this sentence - perhaps be a little clearer?

Mentioning the item in the checklist was meant to emphasise that the concept is important for laypeople. Upon reconsideration, we have simply removed the sentence since it was unclear and inessential.

• P12: This is a structural suggestion. By the sub-heading “Synthesis of Results” I am really wanting to hear some of the key findings – in terms of content (i.e. a strong compelling prose version of the data in Figure 3)  – but I feel I am still getting a lot of complex fine detail about the characteristics of studies and elements of them. I understand there is a lot to report – but some clear statements of the key findings – even summarised and abstracted a little – would be helpful around here to orient the reader.

We have added a summary to the start of the “Synthesis of results” section, by moving up content from the “Key findings” section, like you suggest further down.

• Also – a related point - I find the three paragraphs on P12 (and into P13) complex and hard to follow. I wonder whether this level of detail is necessary in the main text – and whether it might be moved to a supplementary – but that is for authors to decide. At the very least – if there is any way to make it all a little clearer around this point – that would be great for the reader. An example of the mention of the “ninth criteria” from Bubela and 12th from Haneef.  I think this is way too much detail for main text and authors run the risk of losing readers, for what is a very valuable SR.  

Please see our response to your comment further up, about moving more material into supplementary files.

• Now I have read further I realise you have a long section  - starting on P14 – on the results – as they relate to IHC concepts. After having read that very long section, I still feel the need for some kind of punchy summary of the key findings (e.g. harms, COIs, costs, other options not often mentioned…) somewhere in the results section.  This kind of summary doesn’t appear until start of Discussion – but I think could be included earlier.

We have added a summary to the start of the “Synthesis of results” section, by moving up content from the “Key findings” section, including the dot points.

• Discussion: the paper feels like it morphs in the Discussion into a passionate advocacy for IHC materials – with a sometimes moralistic tone. I think the tone could be addressed. And I also think that in order to justify the IHC material in the Discussion, more needs to be said about IHC in the Introduction – and the authors relations with IHC need to be made more explicit in the Methods – as noted above.

Overstated findings in science and health reporting on social media

Spin. Clickbait. Exaggerated headlines. The rise of social media such as Facebook and Twitter has changed how health-related research and news is presented to audiences around the world, and it is not unheard of for researchers and reporters to overstate the findings of a study.

To better understand this issue, a May 30, 2018 study in PLOS One took a detailed look at the 50 most-shared academic journal articles linking any exposure with a health outcome, and media stories that covered those articles. The multidisciplinary research team was led by Noah Haber, who recently completed his ScD in health economics in the Department of Global Health and Population  at Harvard T.H. Chan School of Public Health. They assessed the studies’ strength of causal inference, or whether the study could determine that the exposure itself changed the health outcome, using a novel systematic review tool. They then compared them with the strength of causal language used to describe results in both academic journal articles and media articles.

The study found that 34% of the academic studies reviewed used language that reviewers considered too strong given their strength of causal inference, and 48% of media articles used stronger language than their associated academic articles. Moreover, 58% of media articles inaccurately reported the question, results, intervention, or population of the academic study. The team is now researching how academia, media, and social media contribute to this issue, and interventions to help fix it.

In addition to the PLOS paper, the team created a website called MetaCausal that includes a public explainer of its research, the full dataset, full protocol, review tool, analysis code, reviewer profiles, and results from the study.

Read the PLOS One paper: Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review

Psychology Research News

Top headlines, latest headlines.

• Beta Rhythms Implement Cognitive Control
• Glial Hyper-Drive for Triggering Seizures
• Miniature Brain Stimulator in Humans
• Protecting Brain Cells With Cannabinol
• How Working Memory Reallly Works
• Exercise Cuts Stress-Related Brain Activity
• Teen Stress and Depression in Adults
• Parkinson's and Lewy Body Dementia
• Your Brain ... On a Junk Food Diet
• Epilepsy Drug May Prevent Brain Tumors

Earlier Headlines

Thursday, april 11, 2024.

• Two Key Brain Systems Are Central to Psychosis
• Study Finds Increased Anxiety and PTSD Among People Who Remained in Ukraine
• Study Helps Explain Why Childhood Maltreatment Continues to Impact on Mental and Physical Health Into Adulthood
• Parkinson's Disease: New Theory on the Disease's Origins and Spread

Wednesday, April 10, 2024

• Researchers Discover How We Perceive Bitter Taste
• Brain Stimulation Treatment May Improve Depression, Anxiety in Older Adults
• Connecting Lab-Grown Brain Cells Provides Insight Into How Our Own Brains Work

Monday, April 8, 2024

• Are Lab-Grown Brain Tissues Ethical? There Is No No-Brainer Answer
• Tiny Brain Bubbles Carry Complete Codes
• New Study Highlights the Benefit of Touch on Mental and Physical Health

Thursday, April 4, 2024

• Prairie Voles Display Signs of Human-Like Depression
• Feeding the Lonely Brain

Wednesday, April 3, 2024

• Developing a Vaccine for the 'zombie Drug' Xylazine
• Researchers Map How the Brain Regulates Emotions

Tuesday, April 2, 2024

• How the Brain Senses Body Position and Movement

Monday, April 1, 2024

• Exposure to Common Environmental Carcinogens Linked to Decreased Lifespan Happiness
• Universal Brain-Computer Interface Lets People Play Games With Just Their Thoughts

Thursday, March 28, 2024

• New Imaging Method Illuminates Oxygen's Journey in the Brain
• For Younger Women, Mental Health Now May Predict Heart Health Later
• Reverse Effects of Trauma? Older Brain Cells Linger Unexpectedly Before Their Death

Wednesday, March 27, 2024

• Researchers Introduce Enhanced Brain Signal Analysis Technique
• Could AI Play a Role in Locating Damage to the Brain After Stroke?
• Making Long-Term Memories Requires Nerve-Cell Damage
• Risk Factors for Faster Aging in the Brain Revealed in New Study

Tuesday, March 26, 2024

• Study Finds High Prevalence of Hidden Brain Changes in People With Heart Disease
• Common Degenerative Brain Disease May Begin to Develop in Middle Age
• Large-Scale Animal Study Links Brain pH Changes to Wide-Ranging Cognitive Issues

Monday, March 25, 2024

• Human Brains Are Getting Larger: That May Be Good News for Dementia Risk

Friday, March 22, 2024

• The Aging Brain: Protein Mapping Furnishes New Insights
• Movement Disorder ALS and Cognitive Disorder FTLD Show Strong Molecular Overlaps, New Study Shows
• Early Intervention After the First Seizure May Prevent Long-Term Epilepsy and Associated Cognitive Deficits

Thursday, March 21, 2024

• Research Finds a Direct Communication Path Between the Lungs and the Brain
• Research Offers Hope for Preventing Post-COVID 'brain Fog' By Targeting Brain's Blood Vessels

Wednesday, March 20, 2024

• Treating Anxiety, Depression in People With Heart Disease Reduced ER Visits, Hospitalizations

Tuesday, March 19, 2024

• Brain Recordings in People Before Surgery Reveal How All Minds Plan What to Say Prior to Speaking
• Is the Secret to Anxiety in Young Women Hidden in Our Brain Chemistry?
• Biomarkers of the Middle-Aged Brain Predict Cognitive Health in Old Age
• How Neurochemicals Affect fMRI Readings

Monday, March 18, 2024

• Five Factors to Ensure an Infant Thrives
• Researchers Find Unanticipated Complexity in Aging Brain's Memory Decline

Friday, March 15, 2024

• Fatty Food Before Surgery May Impair Memory in Old, Young Adults
• Gut Bacteria Make Neurotransmitters to Shape the Newborn Immune System
• Abnormal Brain Structure Identified in Children With Developmental Language Problems

Thursday, March 14, 2024

• How Fear Unfolds Inside Our Brains

Wednesday, March 13, 2024

• Blast-Related Concussions Linked to Higher Alzheimer's Risk
• Researchers Identify Brain Connections Associated With ADHD in Youth

Tuesday, March 12, 2024

• Researchers Expand Our Understanding of How the Body and Brain Communicate

Monday, March 11, 2024

• Researchers Identify Gene Involved in Neuronal Vulnerability in Alzheimer's Disease

Wednesday, March 6, 2024

• Patience Pays Off
• A Noninvasive Treatment for 'chemo Brain'

Thursday, March 7, 2024

• Does Iron Accumulate in Brain After Concussions?

Monday, March 4, 2024

• A Better Way to Deliver Fetal Therapy for Serious Genetic Disorders

Friday, March 1, 2024

• Link Between Adversity, Psychiatric and Cognitive Decline

Thursday, February 29, 2024

• Tiny Magnetic Particles in Air Pollution Linked to Development of Alzheimer's
• Molecular Clusters on Glial Cells Show They Are More Than Our Brain's 'glue'

Wednesday, February 28, 2024

• Continued Cocaine Use Disrupts Communication Between Major Brain Networks
• Neurons Help Flush Waste out of Brain During Sleep
• In Fight Against Brain Pathogens, the Eyes Have It
• Could We Assess Autism in Children With a Simple Eye Reflex Test?
• More Than Just Neurons: A New Model for Studying Human Brain Inflammation

Tuesday, February 27, 2024

• Addressing Societal Concerns of Genetic Determinism of Human Behavior by Linking Environmental Influences and Genetic Research
• New Study Links Placental Oxygen Levels to Fetal Brain Development
• Visual Prosthesis Simulator Offers a Glimpse Into the Future
• Learning and Memory Problems in Down Syndrome Linked to Alterations in Genome's 'dark Matter'

Monday, February 26, 2024

• Gut-Brain Communication Turned on Its Axis
• Intervention Reduces Likelihood of Developing Postpartum Anxiety and Depression by More Than 70%
• Yoga Provides Unique Cognitive Benefits to Older Women at Risk of Alzheimer's Disease

Sunday, February 25, 2024

• Revolutionary Brain Stimulation Technique Shows Promise for Treating Brain Disorders

Friday, February 23, 2024

• Hearing Relaxing Words in Your Sleep Slows Your Heart Down

Thursday, February 22, 2024

• Cracking the Code of Neurodegeneration: New Model Identifies Potential Therapeutic Target
• Uncovering Anxiety: Scientists Identify Causative Pathway and Potential Cures
• Researchers Use Deep Brain Stimulation to Map Therapeutic Targets for Four Brain Disorders
• Living in Violent Neighborhoods Affects Children's Brain Development
• Newly Discovered Brain Cells Play a Key Role in Right and Left Turns

Wednesday, February 21, 2024

• Air Pollution Linked to More Signs of Alzheimer's in Brain
• How Does the Brain Make Decisions?
• Possible Trigger for Autoimmune Diseases Discovered : B Cells Teach T Cells Which Targets Must Not Be Attacked

Tuesday, February 20, 2024

• Fixing Rogue Brain Cells May Hold Key to Preventing Neurodegeneration
• Modifying Brain Molecule Relaxin-3 Can Potentially Reduce Side Effects in Treating Anxiety, Depression and More
• Blocking Key Protein May Halt Progression of Alzheimer's Disease
• Stress During Pregnancy Can Lead to Early Maturation of First-Born Daughters
• Mapping Potential Pathways to MND Treatment

Wednesday, February 14, 2024

• Exposure to Agent Orange Damages Brain Tissue in Ways Similar to Alzheimer's Disease

Tuesday, February 13, 2024

• Neural Prosthetic Device Can Help Humans Restore Memory
• Are You Depressed? Scents Might Help

Friday, February 9, 2024

• Immune Genes Are Altered in Alzheimer's Patients' Blood

Thursday, February 8, 2024

• How Emotions Affect Word Retrieval in People With Aphasia
• Patterns of Brain Connectivity Differ Between Pre-Term and Term Babies
• Pharmacological Inhibitor Protects Nerve Cells in ALS Disease
• Researchers Identify Potential Way to Treat Genetic Epilepsy by Replacing 'lost' Enzyme

Wednesday, February 7, 2024

• Researchers Make Progress Toward Developing Blood Tests for Psychiatric and Neurological Disorders
• New Direct Links Discovered Between the Brain and Its Surrounding Environment
• Stress Influences Brain and Psyche Via Immune System
• Things We Can Do to Help Cope With Traumatic Loss

Tuesday, February 6, 2024

• Study Finds Strongest Evidence to Date of Brain's Ability to Compensate for Age-Related Cognitive Decline
• Bullied Teens' Brains Show Chemical Change Associated With Psychosis

Monday, February 5, 2024

• Scientist Shows Focused Ultrasound Can Reach Deep Into the Brain to Relieve Pain
• World's Largest Childhood Trauma Study Uncovers Brain Rewiring

Thursday, February 1, 2024

• Researchers 3D-Print Functional Human Brain Tissue
• Scientists Discover a Potential Way to Repair Synapses Damaged in Alzheimer's Disease
• LATEST NEWS
• Health & Medicine
• Diseases & Conditions
• Alzheimer's Research
• Amyotrophic Lateral Sclerosis
• Attention Deficit Disorder
• Back and Neck Pain
• Birth Defects
• Bladder Disorders
• Blood Clots
• COVID and SARS
• Cervical Cancer
• Bladder Cancer
• Multiple Myeloma
• Pancreatic Cancer
• Brain Tumor
• Colon Cancer
• Breast Cancer
• Ovarian Cancer
• Lung Cancer
• Mesothelioma
• Skin Cancer
• Prostate Cancer
• Cerebral Palsy
• Chikungunya
• Chronic Fatigue Syndrome
• Cold and Flu
• Crohn's Disease
• Cystic Fibrosis
• Dengue Fever
• Down Syndrome
• Eating Disorder Research
• Encephalitis
• Epilepsy Research
• Erectile Dysfunction
• Fibromyalgia
• Gastrointestinal Problems
• HIV and AIDS
• Headache Research
• Hearing Loss
• Heart Health
• Cholesterol
• Stroke Prevention
• Heart Disease
• Hormone Disorders
• Hypertension
• Infectious Diseases
• Insomnia Research
• Irritable Bowel Syndrome
• Kidney Disease
• Liver Disease
• Lung Disease
• Lyme Disease
• Mental Health Research
• Multiple Sclerosis Research
• Mumps, Measles, Rubella
• Muscular Dystrophy
• Osteoporosis
• Parkinson's Research
• Prostate Health
• Restless Leg Syndrome
• Sickle Cell Anemia
• Sleep Disorder Research
• Thyroid Disease
• Triglycerides
• Tuberculosis
• Medical Topics
• Accident and Trauma
• Alternative Medicine
• Birth Control
• Bone and Spine
• Chronic Illness
• Controlled Substances
• Dietary Supplements and Minerals
• Epigenetics
• Food Additives
• Foodborne Illness
• Foot Health
• Gene Therapy
• Health Policy
• Human Biology
• Immune System
• Joint Health
• Medical Imaging
• Nervous System
• Pain Control
• Personalized Medicine
• Pharmacology
• Psychology Research
• Wounds and Healing
• PHYSICAL/TECH
• ENVIRONMENT
• SOCIETY & EDUCATION
• Holographic Displays: An Immersive Future
• Harvesting Energy Where River Meets Sea
• Making Diamonds at Ambient Pressure
• Eruption of Mega-Magnetic Star
• Clean Fuel Generation With Simple Twist
• Bioluminescence in Animals 540 Million Years Ago
• Fossil Frogs Share Their Skincare Secrets
• Fussy Eater? Most Parents Play Short Order Cook
• Precise Time Measurement: Superradiant Atoms
• Artificial Cells That Act Like Living Cells

Trending Topics

Strange & offbeat.

• Archived Calls
• Volumes and Issues
• Special Issues
• All Publications

Scientific research in news media: a case study of misrepresentation, sensationalism and harmful recommendations

Accurate news media reporting of scientific research is important as most people receive their health information from the media and inaccuracies in media reporting can have adverse health outcomes. We completed a quantitative and qualitative analysis of a journal article, the corresponding press release and the online news reporting of a scientific study. Four themes were identified in the press release that were directly translated to the news reports that contributed to inaccuracies: sensationalism, misrepresentation, clinical recommendations and subjectivity. The pressures on journalists, scientists and their institutions has led to a mutually beneficial relationship between these actors that can prioritise newsworthiness ahead of scientific integrity to the detriment of public health.

1 Introduction

1.1 media and scientific research.

Clear, balanced and accurate representation of scientific research in news media is important. Media both shape and reflect public opinion [Caulfield et al., 2014 ]. The public receive a significant amount of their health information from the media [Caulfield et al., 2014 ; Phillips et al., 1991 ]. Those who receive their health information from the media are not limited to general audiences but include content experts such as healthcare professionals and policy makers [Geller, Bernhardt and Holtzman, 2002 ]. Media coverage of health issues can influence government policy [King, Schneer and White, 2017 ] and impact healthcare decision making [Johnson, 1998 ]. Health information in news media can have a greater impact on public health behaviour than government led and supported public health campaigns [Seale, 2003 ]. Whilst scientific research includes vast fields that encompass many disciplines of investigation in both in the natural (biology, chemistry, physics) and social world (sociology, anthropology, psychology), in this paper, we refer to ‘scientific research’ as a short-hand way of referring to lab-based and clinical research with clear translations and implications for human health.

Research in natural scientific fields is generally considered positivist. Positivist research, like that undertaken in the case that is described in this study, is viewed as researchers working from a paradigm in which objective truths about the world can be developed through rigorous adherence to the scientific method. Scientific research uses rigorous methods to ensure researcher objectivity and minimise bias [O’Connor and Joffe, 2014 ]. However, a subtle shift occurs when scientific research is written about in public domains such as mainstream news media [O’Connor and Joffe, 2014 ]. Given the goals of media communication, the overall complexity, phrasing, language, and the relatability of the science needs to be adapted for a mainstream audience. Researchers’ goal of reporting high quality scientific research in media and the need for scientific research to be comprehensible and newsworthy presents competing priorities. As social science researchers, we view this under-investigated tension as important and worthy of study. Throughout this article we use a social constructivist perspective to investigate and explain the tensions that emerge when scientists communicate the outcomes of positivist science outside of the strict confines of academic publishing. We acknowledge the socially constructed nature of the journal article, the press release and news media reporting and aim to explore the processes, structures and activities that create these different modes of communication.

Since the 1990’s there have been significant changes to news media environments that have impacted both on the way science is communicated to the public and the way consumers of news engage with, and receive information about science. Recent changes include that ownership of media organisations has become more concentrated and media has become more digitized with convergence across platforms [Erdal, 2019 ]. In contemporary society, the public engages with news across multiple platforms using both traditional and digital sources. In 2018, the Pew Research Centre reported that people in the U.S. are most likely to receive their news from television followed by news websites, radio, social media and print newspapers [Shearer, 2018 ]. In 2019, Ofcom reported that people in the United Kingdom (U.K.) are most likely to receive their news from television followed by internet sources, radio and then print newspapers [Ofcom, 2019 ]. In 2019, Reuters Institute for the Study of Journalism reported that Australians are most likely receive their news from online sources followed by television, print and social media [Newman et al., 2019 ]. Important to note is that the sources that people receive their news from are shifting with online content being of increasing importance, especially for younger audiences such as those aged 18–29. In the U.S., for example, most young people report consuming news via social media followed by news websites [Shearer, 2018 ].

In addition to the change in ways that society consume news, there has been a steady decline in employment of ‘traditional’ journalists globally. In Australia around a third of all print journalist positions were lost in the twenty years from 1996 to 2016 [O’Regan and Young, 2019 ]. In the U.S., newsroom employment dropped by 23% from 2008 to 2019 [Walker, 2021 ]. These job losses have coincided with a steady decline in the circulation, readership and advertising revenue of print newspapers [Barthel, 2017 ]. As in many other countries, Australia has also seen a steep decline in specialist science journalists, with general journalists now covering science-related news without necessarily having any science training [Watkins, 2019 ]. In addition, the speed of today’s news production has resulted in the disappearance of scrutinised information and considered reflection [Le Masurier, 2015 ]. The pressure to produce real time news has resulted in greater inaccuracy [Hargreaves, 2003 ] and a dependence on press releases that are written by the public relations professionals employed by universities and research institutes [Lewis et al., 2008 ]. Even if journalists had the time to read journal articles, the majority of those articles remain behind journal paywalls [Butler, 2016 ]. Journalists are also under increasing pressure to generate ‘click bait’ and are therefore driven by headlines that include words such as “breakthrough”. In combination with a lack of science training and time pressures this results in inaccuracies and sensationalist stories being published [Watkins, 2019 ]. Research has shown that inaccurate or exaggerated scientific reporting has, in part, been a result of the information in the press releases [Sumner et al., 2016 ].

Researchers have reported that the desire to create newsworthy stories about science led to a perverse situation where poorer quality research can garner more news coverage than robust research based on a strong priori hypothesis, as the poorer research is more likely to yield surprising and newsworthy results. For example, Selvaraj and colleagues investigated study designs of medical research published in news media and found that newspapers were less likely to cover randomised controlled trials than observational studies and therefore preferentially reported on medical research with weaker study designs [Selvaraj, Borkar and Prasad, 2014 ]. Another example of this is when the poorly designed and subsequently retracted and debunked study led by Andrew Wakefield and published in the Lancet that described an association between the measles/mumps/rubella (MMR) vaccine and autism was widely published in news media and resulted in a reduced vaccination rate of children for years following the publication of the article [Godlee, Smith and Marcovitch, 2011 ]. Research designed to quantify the effect of this paper has demonstrated that this one study alone has been a primary cause of childhood vaccine scepticism in the U.S. highlighting that media attention of inaccurate scientific research can undermine public trust in vaccines [Motta and Stecula, 2021 ]. This case of the MMR vaccine is an example of widespread and damaging news coverage from a poorly design scientific study. The consequences of communicating scientific research via media when it involves misinformation, like the MMR vaccine, can lead to public misunderstanding, distrust in science and harmful health behaviours [Kata, 2010 ].

Other researchers have highlighted, that unlike for scientists, for the media, communicating the limitations and risks of a study may be of a lower priority. Omission of limitations and risk has been reported in a number of studies, Caulfield and colleagues found that vitamin D when reported on in news media was linked to a variety of health conditions for which there is no definitive scientific evidence in addition to under reporting the risks associated with vitamin supplementation [Caulfield et al., 2014 ]. Cassels and colleagues analysed the representation of five specific drugs in Canadian newspapers with the main findings being that the majority of articles did not mention potential side effects or harms [Cassels et al., 2003 ]. Schwitzer summarised the work of independent health news reviewing organisation ‘healthnewsreview.org’ which evaluated 1,800 health news stories across many U.S. news organisations. Findings showed that 70% of health news articles were deemed unsatisfactory when assessed for attributes such as quantifying potential harms and benefits and reporting on costs [Schwitzer, 2013 ]. Researchers have theorised that the omission of limitations and risks in the reporting of scientific studies in news media is to increase their newsworthiness or conversely, as described by Mellor, reporting on attributes such as limitations is considered a non-news value [Mellor, 2015 ].

In addition to the omission of limitations and risks, writing techniques used in journal articles, press releases and news media to make scientific research more newsworthy include the use spin and positive framing. In the context of scientific research, spin has been described as communicating findings so that the benefits of an intervention seem stronger or more positive than they actually are [Haneef et al., 2015 ]. The motivations to use spin to increase newsworthiness when writing about scientific research in news media have been linked to scientists, public relations specialists and journalists. In an analysis of randomised controlled trials reported in news media, Yavchitz and colleagues reported that the key predictor of ‘spin’ in a press release was the use of ‘spin’ in the conclusion of the abstract of the journal article [Yavchitz et al., 2012 ]. Even before the journal article is published, researchers have found that spin can be present at the beginning of the research process from grant applications in addition to academic journal articles and consequentially any material that is based on these documents [Landhuis, 2016 ]. Others have argued that spin can be introduced in the press release. Sumner and colleagues found that exaggerations and warnings in news reports mirrored those in press releases [Sumner et al., 2016 ]. Others have found fault with the practice of journalists. Taylor and colleagues [Taylor et al., 2015 ] investigated the accuracy of news media coverage of a meta-analysis (a complex statistical method that combines results across multiple studies) investigating the link between pancreatic cancer and processed meat. The authors found that most news reports were derived from secondary sources such as the journal press release and that the quality of the news reports was dependent on the quality of the secondary sources from which the news reports were derived [Taylor et al., 2015 ].

Framing is another technique that, when a news article is produced, will highlight and downplay certain elements of a story to promote a specific predetermined understanding [Entman, 2007 ]. News frames, therefore, can exert power over readers’ beliefs, attitudes and behaviours [Oliver, Raney and Bryant, 2019 ]. Furthermore, sense making theory suggests that readers consume news media portrayed in specific frames, as a short cut to understand complex topics [Scheufele and Lewenstein, 2005 ; Shih, Wijaya and Brossard, 2008 ]. Framing can therefore yield problematic representation and sense making interpretations of science if a study has been inaccurately portrayed for the purpose of newsworthiness for the benefit of media, scientists or both. Given that science needs to be both understandable and relatable to be newsworthy [Fuoco, 2021 ], it makes sense that there are shared motivations of scientists, public relations professionals and journalists that may result in techniques such as spin and framing to make scientific research more newsworthy. However, it also makes sense that, to garner interest in scientific research, research findings may be exaggerated and their implications inflated [Vinkers, Tijdink and Otte, 2015 ] via mechanisms such as spin and framing.

1.2 Science communication theory in the context of news media

The reasons that scientists increasingly prioritise public engagement are complex [Besley and Nisbet, 2013 ]. In addition to publishing in academic journal articles, there is an expectation that academics participate in public engagement [Glynn, 2016 ; Rawat and Meena, 2014 ]. Research from the U.K. has highlighted that the most important reasons for academics to engage with public audiences are to increase funding success by demonstrating research impact and to increase their institution’s competitiveness [Watermeyer and Lewis, 2018 ]. The relationships that exist between scientists and the public can be understood using the theoretical models of science communication [Metcalfe, 2019 ]. Over time, there have been many theoretical models of communication proposed, each based on different assumptions and definitions of communication [Burns, O’Connor and Stocklmayer, 2003 ]. The three main theoretical models of science communication described in academic literature include the deficit, dialogue and participation models [Metcalfe, 2019 ]. These three models underpin the communication strategies within two of the most commonly described paradigms of science communication. The deficit model belongs to the public understanding of science (PUS) paradigm and the dialogue and participation models belong to the public engagement with science and technology (PEST) paradigm [Schäfer, 2009 ]. The deficit model assumes that the public’s lack of understanding of science leads to the public being sceptical about science [Sturgis and Allum, 2004 ] and that public doubts and uncertainty about science are a result of ignorance about science [Gross, 1994 ; Sturgis and Allum, 2004 ]. In contrast to the deficit model, the dialogue and participation models emphasise informing and communicating diverse views and critical reflections about scientific issues to public audiences [Kamenova, 2017 ]. A PUS paradigm can oversimplify information in an attempt to facilitate public understanding. In contrast, the PEST paradigm does not assume the public are deficient in knowledge and thus seeks to communicate critical reflections about science. Evolving from the PEST paradigm, medialisation is a theory that seeks to understand the mutually beneficial relationship between science and the media, specifically; scientists’ awareness of the strategic benefits of direct media engagement and in turn, media’s increased science coverage [Rödder, 2011 ; Vestergård, 2015 ]. These models are idealistic and potentially also unrealistic in a world in which there are clear incentives for scientists, their institutions and news media organisations to generate newsworthy scientific stories that may be achieved through omission of risks and limitations and exaggerations and relevance of research findings.

Despite there being competing interests for newsworthiness, accuracy and relevance of scientific news stories [Cassels et al., 2003 ; Caulfield et al., 2014 ; Haneef et al., 2015 ; Schwitzer, 2013 ], the responsibility for the production of inaccurate reporting is not straightforward. Science communication researchers have attributed misrepresentation of scientific research to a complex relationship between scientists, science communicators and journalists [Caulfield, 2005 ]. Facilitating the dynamic between scientists and journalists are communication specialists who work at universities, research institutes, academic journals and other organisations. These professionals are responsible for the production of press releases and media engagement activities. As research findings are one of the main commodities for research institutions, they have the potential to impact the institution’s financial status and competitive rankings [Autzen and Weitkamp, 2020 ]. Additionally, institutions that publish the most press releases tend to have the highest rankings [Autzen, 2014 ]. There is a clear incentive for institutions to publish high volumes of press releases about research findings that garner as much news coverage as possible. Additionally, exaggerating research findings in press releases is incentivised when the outcome is increased news media coverage of scientific research which has the potential to benefit researchers, their institution and the news media. Furthermore, experts have noted that the reliance on one source of information, such as an institution press release, grants a level of control of the news agenda to the researchers and their institution [Weitkamp and Eidsvaag, 2014 ].

As the scientific research and media landscapes continue to evolve including the increasing pressures on scientists to engage with the public and the demands on journalists to publish newsworthy stories about science with fewer resources, the interactions between these two fields require continual investigation. Additionally, the interdependencies between scientists, science communicators and journalists, including the complexities of communicating positivist lab-based science in a socially constructed environment, there is a need to conduct a detailed examination of the process and consequences of translating scientific research from academic journal articles to press releases to news media reporting.

1.3 This case study

As a significant proportion of news media is derived from press releases [Lewis et al., 2008 ] and the press release impacts on the accuracy of scientific news, [Sumner et al., 2016 ], this study sought to investigate in detail, the communication process in a well-known case of significant misrepresentation of scientific research in news media. This study was published in one of the most highly cited scientific journals worldwide and was the subject of a substantial number of international news reports at the time; many of which had the potential to influence health behaviours. The study was the subject of media scrutiny and featured in ‘SBS News’ which reported it as harmful, “Vitamin B3 claims slammed by obstetricians” [SBS News, 2017 ].

The case at the centre of this paper is a journal article published in the New England Journal of Medicine in August 2017 titled “NAD deficiency, congenital malformations and niacin supplementation” [Shi et al., 2017 ] and the press release published by the researchers’ institution “Historic discovery has the potential to prevent miscarriages and birth defects globally” [Victor Chang Cardiac Research Institute, 2017 ]. The journal article described a study that investigated the role of gene variations and niacin supplementation in the prevention of congenital malformations. Of note, mice bred with specific genetic mutations were used to assess the impact of niacin supplementation in the prevention of congenital malformations. The genetic mutations were modelled on genetic mutations found in human families that underwent genetic sequencing where there existed a history of congenital malformations.

While a major component of the study design was investigating the effects of niacin supplementation in mice, many news media reports implied the research had been undertaken in humans with direct health implications for women during pregnancy. As the niacin supplementation component of the study was undertaken in mice, the recommendations about vitamin supplementation in pregnant women were outside the scope of the findings of the research study. Additionally, recommendations made about niacin supplementation had potentially harmful consequences as an excessive consumption of niacin can be harmful to both pregnant women and their babies [The Royal Australian and New Zealand College of Obstetricians and Gynaecologists, 2017 ].

To explore in detail, the communication process that resulted in this scientific study being misrepresented in news reports, we analysed the journal article, the corresponding press release and all of the subsequent online news reports available through Google News to address the following question: how and where did misrepresentation of the scientific study take place? Additionally, we sought to address one research question that was specifically related to the news reporting: what communication techniques were used in the news reports that resulted in misrepresentation?

We searched Google News for the online news reports for a five-month period from August 2017 to December 2017 using key words such as “niacin”, “vitamin B3”, “Vegemite”, “congenital malformations”, “birth defects”. The press release was issued on the 1 0 th of August and the vast majority of reports were published between 1 0 th and 1 2 th of August 2017. We restricted our search to Google News because it covers a vast range of news media sources [Filloux, 2013 ] and has been used previously in media analysis research as the single source of online news media coverage [Haneef et al., 2015 ; Young Lin and Rosenkrantz, 2017 ]. Google Chrome, Safari and Firefox were used to search for articles on Google News, all with refreshed browsers histories to ensure that all relevant articles were found and searching history did not affect the articles retrieved. After sourcing the journal article from the New England Journal of Medicine website [Shi et al., 2017 ], the press release from the Victor Chang website [Victor Chang Cardiac Research Institute, 2017 ] and the news media reports from Google News, each document was downloaded and imported into NVivo version 12. After reading each document, a preliminary coding framework was developed by the first author and refined through preliminary analysis and discussions with the other authors. The first author completed the quantitative and qualitative coding. For the quantitative coding, 10% of articles were double coded by another author (LK) and codes and coding definitions were adjusted until agreement reached 80%. For the qualitative coding, 10% of articles were double coded by LK and any disagreements were discussed and resolved and the same logic was applied to the rest of the qualitative coding by the first author.

2.1 Development of coding framework for quantitative content analysis

The coding framework involved developing preliminary codes to guide the analysis. This was based on reviewing the literature on the representation of scientific research in media (including the coding framework used by ‘healthnewsreview.org’ [HealthNewsReview.org, 2018 ] and by reading the journal article, press release and a subset of news media reports to tailor the coding to this specific study. An inductive approach followed the development of the preliminary codes and allowed for unexpected themes or the refinement of codes that developed during the analysis.

2.2 Quantitative coding and analysis

The coding framework included the following items: spin, buzz words, framing (positive, negative, balanced), a description of the study design, a description of the study population (mice and humans), description of the niacin supplementation trialled in mice, description of genetic sequencing undertaken in humans, a statement that study findings could not be translated to humans, clinical recommendations about vitamin supplementation, advice to consult a doctor for further information, the use of independent and non-independent expert commentators, the use of a patient narrative, the inclusion of funding information and a link to the journal article. Each of these items was coded either yes or no.

Spin has been defined in multiple ways in academic research [Bero, Chiu and Grundy, 2019 ]. We chose to use the following definition of spin: a way of reporting, for any motive whether intentional or unintentional, that emphasises that the beneficial effect of the intervention is greater than the actual results [Haneef et al., 2015 ]. We chose to use the following definition of buzzwords from the Oxford Dictionary: a word or phrase, often jargon, that is trendy in a particular context or at a specific time [Oxford English Dictionary, 2020 ]. Examples of buzzwords and phrases used in the press release and news media reporting included; ‘historic medical breakthrough’, ‘landmark discovery’, ‘Australia’s greatest ever medical achievements’. Framing can obfuscate objective reporting by highlighting and downplaying certain elements of stories in media which can impact the way readers interpret and relate to information [Birnbrauer, Frohlich and Treise, 2017 ; Entman, 1993 ] and impact readers’ understanding of a story [Caulfield et al., 2014 ]. We chose to analyse whether each article was framed positively, negatively or in a balanced way.

For each article we also recorded whether there was a description of the study design, a description of what component of the research was undertaken in mice and what component was undertaken in humans and whether these specific research findings could be translated to humans. The type of clinical recommendations regarding vitamin supplementation that we analysed were both those that were directly related to this study and those that related to pregnancy in general. We chose to include both types of recommendations as they both have the potential to impact readers’ health behaviour. We also recorded whether there was advice for readers to contact their doctor for more information and health advice about vitamin supplementation during pregnancy. Additionally, we recorded whether each article had independent expert commentators (i.e., those that were not involved with the study but who are experts in the area) or non-dependent expert commentators (those that were involved with the study either as authors or representatives from the researchers’ institute). We counted information about the funding sources as any information about what organisations funded the research. Information about how to access the journal article was coded as ‘yes’ if a link to the article was included, not just mentioning the name of the journal. We also coded whether news reports used a patient narrative. Narratives are important for storytelling and for readers’ understanding of the relevance of an issue.

2.3 Qualitative analysis

The qualitative analysis investigated in more depth, the data coded for the quantitative content analysis. The coded data was further analysed to determine, for example, in what context and for what effect: spin, buzz words and framing were used, whether the omissions or inclusions about the study design, the study population and what components of the research were done in mice and humans resulted in misrepresentation, the extent to which: information about study findings could be translated to humans, clinical recommendations about vitamin supplementation during pregnancy and advice to consult a doctor may contribute to potentially harmful clinical behaviours or outcomes for readers. The impact of independent and non-dependent commentators, patient narratives, funding information and access to the journal article were also reviewed to understand the role these played in relation in the subjectivity of the story.

We identified 60 unique news reports from 48 separate news organisations and websites. The news sources included organisations such as the British Broadcasting Corporation (BBC) and the Australian Broadcasting Corporation (ABC), as well as lesser-known technology-focused and health-related websites such as Gizmodo and Body and Soul. The journal article, the press release and the news reports were coded by the first author. The results of content analysis are presented in four groups of themes that emerged in the qualitative analysis. Table 1 summarises findings from the content analysis. The qualitative analysis is presented as themes and illustrated with quotes.

3.1 Content analysis

3.1.1 theme 1, sensationalism.

The journal article did not include spin in its title, or in the article itself, nor did it include buzz words and presented a balanced frame. The press release used spin in both the headline and body of the press release, included buzz words from experts and introduced positive framing. The majority of news reports included spin in the body of the article (68%) and buzz words (87%). The majority (71%) of news reports were framed positively.

3.1.2 Theme 2, Misrepresentation

The journal article contained a description of the: study design; study population as including both mice and humans; niacin supplementation being undertaken in mice and genetic sequencing being undertake in humans. The press release contained a description of the study design but did not describe the study design as including both mice and humans. It described the niacin supplementation as being undertaken in mice but did not describe the genetic sequencing being undertaken in humans. The majority (87%) of news reports described the study design and most (62%) described the niacin supplementation being undertaken in mice. Around half (57%) of the news reports described the study population as including both mice and humans. A similar proportion (56%) described the genetic sequencing being undertaken in humans.

3.1.3 Theme 3, Clinical recommendations

The journal article did not explicitly state that study findings could not be translated to humans and included clinical recommendations relevant to human health. The press release did not state that findings were not directly transferrable to humans. Clinical recommendations were made about human health and there was no advice for people to seek professional advice if readers wanted more information. In the news reports, while most (60%) stated that the study findings could not be translated to humans, the vast majority (88%) of articles included clinical recommendations about vitamin supplementation. Few news reports (7%) advised readers to consult their doctor for more information.

3.1.4 Theme 4, Subjectivity

The journal article did not contain commentary or patient narratives and there was a disclosure about study funding. The press release included non-independent commentators only (i.e., those with a direct connection to the study), no patient narrative, disclosure of study funding and a link to the journal article. In the news reports, around half (47%) included both independent and non-independent expert commentators. A similar proportion (45%) included non-independent expert commentators only. Seventeen percent of news reports included a patient narrative. The same proportion (17%) included funding information and a quarter (25%) included a link to the journal article.

Table 1 represents the results of the content analysis. The results of the content analysis were grouped into themes that were explored in more detail in the qualitative analysis.

3.2 Qualitative analysis

In the qualitative analysis, we explored each theme in more detail based on further analysis of the coded quantitative data.

3.2.1 Sensationalism

In the translation of information from the journal article to the press release to the news media reporting, the use of spin, buzzwords and positive framing were introduced in the press release and were in many cases, directly translated in news media reports. This direct translation is evident by the direct quoting of slabs of text from the press release to the news reports. In the example below, the extrapolation of the research findings to reduce miscarriages and birth defects in the press release is an example of spin. The word ‘landmark’ is an example of a buzz word and the general positivity without any discussion of limitations, such as the study being undertaken in mice, is an example of positive framing.

“The ramifications are likely to be huge. This has the potential to significantly reduce the number of miscarriages and birth defects around the world, and I do not use those words lightly,” says Professor Dunwoodie. The landmark study found that a deficiency in a vital molecule, known as NAD, can prevent a baby’s organs from developing correctly in the womb. (Historic discovery has the potential to prevent miscarriages and birth defects globally, Victor Chang press release, August 2017) “The ramifications are likely to be huge,” said the study’s senior researcher Professor Sally Dunwoodie at the Victor Chang Institute… “This has the potential to significantly reduce the number of miscarriage and birth defects around the world, and I do not use those words lightly.” (Sydney Morning Herald, 17 August 2017)

However even with spin, buzzwords and positive framing used in the press release, not all news media reports employed these literary techniques. Some news articles (32%) presented information with no spin and roughly half (47%) of articles had both non-independent and independent expert commentators. The news reports that were framed negatively focused on the potentially harmful health consequences of the misleading information. Below is an excerpt from a news report with negative framing.

The Royal Australian and New Zealand College of Obstetricians and Gynaecologists says the “extraordinary” suggestions by researchers at the Victor Chang Institute were based on a small mouse study and have the potential to do more harm than good. (SBS News, 11 August 2017)

3.2.2 Misrepresentation

The description of the study design in the journal article was clear and included both the human and mouse components of the research. The journal article described the human and mouse components of the research:

We used genomic sequencing to identify potentially pathogenic gene variants in families in which a person had multiple congenital malformations. We tested the function of the variant by using assays of in vitro enzyme activity and by quantifying metabolites in patient plasma. We engineered mouse models with similar variants using the CRISPR (clustered regularly interspaced short palindromic repeats)–Cas9 system. (NAD deficiency, congenital malformations and niacin supplementation, New England Journal of Medicine 2017)

However, the description of the study design in the press release did not reflect the journal article as the human component of the research was omitted. Additionally, the press release included information about how the study would have direct human health benefits without describing any limitations of extrapolating mouse research to humans. The press release indicates that the findings from mouse research will have human translations:

Scientists at the Victor Chang Institute have discovered simply boosting levels of this nutrient during pregnancy can potentially prevent recurrent miscarriages and birth defects. (Historic discovery has the potential to prevent miscarriages and birth defects globally, Victor Chang press release, August 2017)

One news report indicated that niacin supplementation may reduce birth defects in humans:

The study, published in the New England Journal of Medicine, found that deficiency in a key molecule among pregnant women stopped embryos and babies’ organs from developing correctly in the womb, but could be treated by taking the dietary supplement vitamin B3, also known as niacin. (Business Insider, 10 August 2017)

However, despite the implied direct translation of mouse research to humans, more than half of the news media reports included information about both the human and mice components of the research. Additionally, more than half of the news media reports included information about how the research findings cannot be directly translated to humans.

The study was a preclinical trial, and the results will need to be replicated in humans before doctors can recommend vitamin B3 supplements to pregnant women, but the results are certainly promising. (IFL Science, 10 August 2017)

3.2.3 Clinical recommendations

Toward the end of the journal article, there is a “theorisation” made about the use of vitamin supplementation, but it is clearly relating to the specific families who were involved in the genetic sequencing component of the research rather than the population more generally.

We theorize that supplementation with high-dose niacin (140 mg per day, which is 10 times the U.S. recommended daily allowance for women) before and during pregnancy might prevent recurrence of disease in these four families. It is also possible that niacin supplementation may benefit the speech and developmental delays in the surviving patients. (NAD deficiency, congenital malformations and niacin supplementation, New England Journal of Medicine 2017)

However, the information in the press release about vitamin supplementation could be interpreted as relevant to the population more broadly and could be interpreted as immediately applicable to human health.

Just like we now use folate to prevent spina bifida, Professor Dunwoodie’s research suggests that it is probably best for women to start taking vitamin B3 very early on, even before they become pregnant. (Historic discovery has the potential to prevent miscarriages and birth defects globally, Victor Chang press release, August 2017)

Although most reports made recommendations of some sort about vitamin supplementation, other reports did make it clear that this research study could not be translated directly into recommendations about vitamin supplementation.

Although this is a potentially exciting finding in a very emotive area, it is important to bear in mind that this result is based on studies in mice, and we will need a full research project in women to evaluate the cause and effect of any lack of this vitamin in humans. (Huffington Post U.K., 10 August 2017)

However, like the press release, some news media reports did make recommendations that could have harmful consequences.

The results published in the New England Journal of Medicine suggested giving women niacin supplements before and during pregnancy could significantly cut the risk of miscarriage and congenital defects. (Irish Times, 12 August 2017)

3.2.4 Subjectivity

As with framing, patient narratives can add weight to certain aspects of a story which can resonate with the reader. Patient narratives can be helpful if they assist readers to understand issues, but they can be misleading if they do not accurately represent the facts of a story. In this case study, the use of a patient narrative might assist readers in understanding the types of congenital malformations potentially prevented with niacin supplementation. However, because this research was undertaken in mice and not directly translatable to humans, a patient narrative might be misleading, suggesting to readers that all congenital malformations are prevented via niacin supplementation. Additionally, subjectivity was present in news reports where journalists used comments from non-independent experts. Without independent expert commentary, there is a lack of objectivity and critical reflection about the potential translation of the research findings.

Charlotte Scaife was just one day old when her parents found out the heartbreaking news — the middle part of their baby’s heart hadn’t formed properly and there were multiple holes in her heart… [parent of child (Charlotte) with congenital birth defect] “I wish they’d known about it and the information had been released two years ago or three years ago, and then maybe we wouldn’t be going through this.” (Huffington Post Australia, 11 August 2017)

Despite the press release only including non-independent expert commentators, both non-independent and independent expert commentators were included in almost half of the news media reports, providing evidence that journalists sought additional information to that which was provided in the press release and original journal article.

The press release provided a comment from a non-independent expert:

“We believe that this breakthrough will be one of our country’s greatest medical discoveries. It’s extremely rare to discover the problem and provide a preventive solution at the same time. It’s actually a double breakthrough,” said Professor Graham. (Historic discovery has the potential to prevent miscarriages and birth defects globally, Victor Chang press release, August 2017)

Some news reports sought independent experts to comment on the study:

Dr Katie Morris, an expert in maternal foetal medicine at the University of Birmingham, said: “While exciting, this discovery cannot be translated into recommendations for pregnant women, who at most may be deficient in vitamin B3. (BBC, 10 August 2017)

4 Discussion

In this study, we used quantitative and qualitative content analysis to investigate the translation of information from a scientific journal article, to the corresponding press release to the subsequent online news reporting of a known case of misrepresentation of scientific research in news media. Specifically, we sought to understand how and where misrepresentation of the scientific study took place and what communication techniques were used by journalists in media reports.

Results showed that sensationalism was present in the press release and was reflected in a large proportion of the news reporting via the use of reporting techniques such as spin, buzz words and positive framing. Misrepresentation of information in the form of inadequate descriptions of the study design and the study populations was translated from the press release to the news reports. In addition, potentially harmful clinical recommendations that featured in the press release were present in a large proportion of the news reports by way of unrealistic extrapolation of findings from mice to humans, a lack of discussion around the limitations of the research and a lack of further advice to consult a doctor for additional information.

The press release included commentary by non-independent experts, and this was reflected in many of the news reports. However, many journalists also sourced independent expert comment. Additionally, given the press release contained spin, buzz words, positive framing, non-independent expert commentators, a brief and inaccurate description of the study design, implied that the study findings in mice could be translated to humans, it is noteworthy that many journalists sought additional information and presented a more balanced account of the research than what was contained in the press release. Therefore, some journalists made deliberate efforts to avoid the misrepresentation that was present in the press release.

These findings highlight that in this case, mechanisms that may result in exaggerations and misrepresentation of scientific research can be directly traced back to the press release. The findings were that the press release and a proportion of the news reports had exaggerated the benefits via the extrapolation of a mouse study to humans and the absence of limitations such as the need for further research in humans and discussion about the potential risks resulting from excessive consumption of vitamin supplementation during pregnancy. This is in line with prior science communication research which has highlighted that scientific studies when written about it media, often exaggerate findings and downplay risks and limitations [Cassels et al., 2003 ; Caulfield et al., 2014 ; Haneef et al., 2015 ; Schwitzer, 2013 ]. Although exaggeration of findings and downplaying limitations and risks are unsurprising, the instances of journalists seeking diverse views and critical reflections of the study from independent sources are noteworthy.

In the context of research findings being a core commodity that impacts an institution’s financial and ranking successes [Autzen and Weitkamp, 2020 ] it is significant that the press release was produced by the scientific researchers’ institution and that this press release is where the exaggerations about findings and lack of information about risks originated. When thinking about the medialisation of science, there is both a clear and mutually beneficial relationship between scientists and the media. As the study findings were exaggerated, the story was able to be framed as a “breakthrough” garnering significant media attention for the potential benefit of the researchers, their institution and the media with the publication of many “click-bait” articles with headlines such as “Vegemite and pregnancy: niacin could prevent miscarriages” (Daily Telegraph, August 2017). As the public look to media to make sense of complex topics [Scheufele and Lewenstein, 2005 ], the framing of this scientific research in the press release and in the news media yielded some potentially harmful sense making interpretations followed by responsive backlash from experts in the field who labelled the researchers suggestions as having “the potential to do more harm than good” [SBS News, 2017 ].

From a theoretical perspective, both the press release and those news reports that used non-independent commentators and omitted key information required to understand the study were in line with a PUS paradigm of science communication. As an example, in some cases the description of the study was oversimplified to the point where it was not possible to understand how the study was conducted or what the implications might be for pregnant women. Despite the omission of information about the research study, specifically the lack of description of the study design and how both mice and humans were used, it is important to note that a proportion of the news media reports did seek information from sources outside of the press release to achieve a more informed, objective and accurate account of the scientific study. For example, some news media reports included both independent and non-dependent commentaries in addition to a detailed explanation of the study design that explained the role of both humans and mice in the study in addition to an explanation about how the study cannot yet be translated to human health and that further research is needed to before advice about niacin supplementation can be made. A portion of journalists wrote news reports in line with PEST theory by providing readers with sufficient and objective information which gave them the opportunity to understand the scientific study and make their own judgements about what the findings could mean. This more investigative and critical work by the journalists added a more objective and contextualised aspect to the story. These journalists were not just informing audiences about the ‘wonders of science’ but communicating diverse views and critical reflections. This is especially remarkable given that journalists have a strong trust in science, their scientific sources and are pressured to adhere to scientific values [Vogler and Schäfer, 2020 ] which is in addition to being under resourced and there being few science journalists with specialised skills to critique a scientific study [Barel-Ben David, Garty and Baram-Tsabari, 2020 ]. However, just as journalists critique politicians and policy, they too can critique scientists and science [Rensberger, 2009 ]. This would be made easier if journalists regained some of the scientific expertise and resourcing that has been lost as newsrooms have declined in overall staff including science journalists [Brüggemann, Lörcher and Walter, 2020 ].

A challenge exists in communicating via news media the relevance to human health of positivist lab based pre-clinical science where the scientific environment is highly controlled, and the research subjects are animals. Pre-clinical research can have direct relevance to human health in the long-term otherwise it would not be undertaken. However, making this relevance obvious without explaining all the caveats and further steps in the research process would likely result in pre-clinical discoveries becoming less newsworthy. Pre-clinical lab-based studies are an essential step in the formulation of evidence and are imperative to building the case for the next phase of research which, in this case study example, could be in humans. Therefore, if pre-clinical lab science is to be reported in news media, there exists a challenge whereby the findings need to be comprehensible and accurate but at the same time, relatable to readers. It is this tension, that could in part, be responsible for some of the misrepresentation of the study in the press release. On one hand, the researchers need to demonstrate ‘real-world’ impact to make their future research possible and therefore, an incentive to minimise the caveats of their research findings to make their research newsworthy. Conversely, demonstrating ‘real-world’ impact could be more difficult if press releases include detailed information about the limitations of the research and the additional research required to determine the relevance of findings to human health. Therefore, a potential interpretation of the motivations of the researchers in the misrepresentation of the findings in the press release, is that they may not have been aware of the dangers of misleading the public that can occur whilst trying to communicate the future potential of their research. In other words, attempting to strike a balance between the conservative language of scientists and the importance of media attention for the goal of generating further research funding and opportunities.

Additionally, the medialisation of science is important amidst the current global pandemic with COVID-19 receiving extensive and ongoing media coverage across the world since January 2020. COVID-19 has seen the world’s population rely on media for the dissemination and sense making of constantly evolving scientific information with news reports about the pandemic having major impacts on readers’ beliefs about its origins and their country’s policy responses and crisis politicisation [Pearman et al., 2021 ]. Some changes to scientific publishing that have ongoing consequences for science journalism that have occurred since January 2020 include: a dramatic increase in the number of published academic journal articles (not just on COVID-19 but on all topics and especially those in health related disciplines) and a significant increase in articles being made available prior to peer review [Else, 2020 ]. Both of these outcomes add more challenges for journalists who are overloaded with information to report on and who are now critiquing research that has not yet been through peer review.

5 Limitations

The use of one case study as a means of investigating a phenomenon provides rich data but means that the results may not be generalisable for understanding where and how misrepresentation of scientific information occurs in communication pathways in all cases. The use of Google News as a single source of online news means that some online news reports about this study may not have been captured. Whilst we developed the coding framework collaboratively and double coded 10% of reports and reached an 80% agreement, there is still some subjectivity to interpretation of the variables that were coded.

6 Conclusion

Science communication, and especially science journalism is about reporting truthfully. It is about going beyond hypotheses, data and breakthroughs and looking at the scientists, their conflicts, their funding and other issues that impact the production of science [Borel, 2015 ]. In an ideal world, there should be no need for scientists, science communicators or science journalists to oversell research findings, exaggerate benefits, omit limitations and risks and fail to describe scientific research in a way that readers can understand. However, there are pressures on scientists to demonstrate the ‘real-world’ impact of their work, on science communicators to generate media attention and on journalists to produce newsworthy content about science. This ‘pressure cocktail’ can result in misrepresentation of science that could lead to harmful health behaviours and public misunderstandings and distrust in science. It is for these reasons that those producing the science, the press releases and the news must work together to communicate truthful and objective science to society. Utilising the PEST paradigm, journalists would synthesise and scrutinise research findings, interview independent experts and present science in more than one simplistic science-dominated side to a story but in a contextualised-scientific way in which readers have enough information to judge the scientific research for themselves. However, given the constraints on journalists in both time and resources, it is unrealistic to expect this to be possible for every scientific study that is reported in news media. Given that it is a reality that journalists will need to rely, at least in part, on press releases, it is imperative that press releases are written with the same level of journalistic integrity as the PEST paradigm idealises.

This case study highlights the implications of what can happen when the translation of science from a journal article to a press release to the news media reporting is confounded by pressures faced by scientists, their institutions and news media. We hypothesise the lack of objectivity in this case to be a result of the pressures on journalists, scientists and their institutions which has led to a mutually beneficial relationship between these actors that can prioritises newsworthiness ahead of scientific objectivity to the detriment of public health. There must be an ongoing priority for scientific information to be represented in media in a way that is helpful, not harmful as entire populations try to make sense of the constantly evolving scientific advice related to COVID-19 and future public health crises. In the current scientific, science communication and journalistic climates, in combination with the way that populations are relying on media for their sense making of COVID-19, we acknowledge the following tensions faced by scientists, science communicators and journalists: not to exaggerate, oversimplify and or omit essential information for the sake of media attention and to equip the audience with the information required to understand a scientific study including contextualised information and independent commentary. This approach is especially important in areas of public mistrust such as those that have serious consequences for public health for example, COVID-19 vaccinations. Scientists, science communicators and journalists have an obligation to frame science as interesting and newsworthy without jeopardizing the truth.

Autzen, C. (2014). ‘Press releases — the new trend in science communication’. JCOM 13 (03), C02. https://doi.org/10.22323/2.13030302 .

Autzen, C. and Weitkamp, E. (2020). ‘Science communication and public relations: beyond borders’. In: Science Communication. Ed. by A. Leßmöllmann, M. Dascal and T. Gloning. Vol. 17. Handbooks of Communication Science. Boston, MA, U.S.A. and Berlin, Germany: De Gruyter Mouton, pp. 465–484. https://doi.org/10.1515/9783110255522-022 .

Barel-Ben David, Y., Garty, E. S. and Baram-Tsabari, A. (2020). ‘Can scientists fill the science journalism void? Online public engagement with science stories authored by scientists’. PLoS ONE 15 (1), e0222250. https://doi.org/10.1371/journal.pone.0222250 .

Barthel, M. (2017). ‘Despite subscription surges for largest U.S. newspapers, circulation and revenue fall for industry overall’. Pew Research Center . URL: https://www.pewresearch.org/fact-tank/2017/06/01/circulation-and-revenue-fall-for-newspaper-industry/ .

Bero, L., Chiu, K. and Grundy, Q. (2019). ‘The SSSPIN study — spin in studies of spin: meta-research analysis’. BMJ 367, l6202. https://doi.org/10.1136/bmj.l6202 .

Besley, J. C. and Nisbet, M. (2013). ‘How scientists view the public, the media and the political process’. Public Understanding of Science 22 (6), pp. 644–659. https://doi.org/10.1177/0963662511418743 .

Birnbrauer, K., Frohlich, D. O. and Treise, D. (2017). ‘Inconsistencies in reporting risk information: a pilot analysis of online news coverage of West Nile Virus’. Global Health Promotion 24 (3), pp. 14–22. https://doi.org/10.1177/1757975915594603 .

Borel, B. (2015). ‘The problem with science journalism: we’ve forgotten that reality matters most’. The Guardian . URL: https://www.theguardian.com/media/2015/dec/30/problem-with-science-journalism-2015-reality-kevin-folta .

Brüggemann, M., Lörcher, I. and Walter, S. (2020). ‘Post-normal science communication: exploring the blurring boundaries of science and journalism’. JCOM 19 (03), A02. https://doi.org/10.22323/2.19030202 .

Burns, T. W., O’Connor, D. J. and Stocklmayer, S. M. (2003). ‘Science communication: a contemporary definition’. Public Understanding of Science 12 (2), pp. 183–202. https://doi.org/10.1177/09636625030122004 .

Butler, D. (2016). ‘Dutch lead European push to flip journals to open access’. Nature 529 (7584), p. 13. https://doi.org/10.1038/529013a .

Cassels, A., Hughes, M. A., Cole, C., Mintzes, B., Lexchin, J. and McCormack, J. P. (2003). ‘Drugs in the news: an analysis of Canadian newspaper coverage of new prescription drugs’. CMAJ 168 (9), pp. 1133–1137. PMID: 12719316 . URL: https://www.cmaj.ca/content/168/9/1133 .

Caulfield, T. (2005). ‘Popular media, biotechnology, and the “cycle of hype”’. Houston Journal of Health Law & Policy 5 (2), pp. 213–233.

Caulfield, T., Clark, M. I., McCormack, J. P., Rachul, C. and Field, C. J. (2014). ‘Representations of the health value of vitamin D supplementation in newspapers: media content analysis’. BMJ Open 4 (12), e006395. https://doi.org/10.1136/bmjopen-2014-006395 .

Else, H. (2020). ‘How a torrent of COVID science changed research publishing — in seven charts’. Nature 588 (7839), p. 553. https://doi.org/10.1038/d41586-020-03564-y .

Entman, R. M. (1993). ‘Framing: toward clarification of a fractured paradigm’. Journal of Communication 43 (4), pp. 51–58. https://doi.org/10.1111/j.1460-2466.1993.tb01304.x .

— (2007). ‘Framing bias: media in the distribution of power’. Journal of Communication 57 (1), pp. 163–173. https://doi.org/10.1111/j.1460-2466.2006.00336.x .

Erdal, I. J. (2019). ‘Convergence in/of journalism’. In: Oxford Research Encyclopedia of Communication. Oxford, U.K.: Oxford University Press. https://doi.org/10.1093/acrefore/9780190228613.013.793 .

Filloux, F. (2013). ‘Google News: the secret sauce’. The Guardian . URL: https://www.theguardian.com/technology/2013/feb/25/1 .

Fuoco, R. (2021). ‘How to get media coverage and boost your science’s impact’. Nature . https://doi.org/10.1038/d41586-021-02067-8 .

Geller, G., Bernhardt, B. A. and Holtzman, N. A. (2002). ‘The media and public reaction to genetic research’. JAMA 287 (6), p. 773. https://doi.org/10.1001/jama.287.6.773-JMS0213-3-1 . PMID: 11851549 .

Glynn, D. (2016). ‘Why early career researchers should care about public engagement’. Times Higher Education . URL: https://www.timeshighereducation.com/blog/why-early-career-researchers-should-care-about-public-engagement .

Godlee, F., Smith, J. and Marcovitch, H. (2011). ‘Wakefield’s article linking MMR vaccine and autism was fraudulent’. BMJ 342, c7452. https://doi.org/10.1136/bmj.c7452 .

Gross, A. G. (1994). ‘The roles of rhetoric in the public understanding of science’. Public Understanding of Science 3 (1), pp. 3–23. https://doi.org/10.1088/0963-6625/3/1/001 .

Haneef, R., Lazarus, C., Ravaud, P., Yavchitz, A. and Boutron, I. (2015). ‘Interpretation of results of studies evaluating an intervention highlighted in Google Health News: a cross-sectional study of news’. PLoS ONE 10 (10), e0140889. https://doi.org/10.1371/journal.pone.0140889 .

Hargreaves, I. (2003). Journalism: truth or dare? Oxford, U.K.: Oxford University Press.

HealthNewsReview.org (2018). Our review criteria . URL: https://www.healthnewsreview.org/about-us/review-criteria/ .

Johnson, T. (1998). ‘Shattuck lecture — Medicine and the media’. The New England Journal of Medicine 339 (2), pp. 87–92. https://doi.org/10.1056/NEJM199807093390206 .

Kamenova, K. (2017). ‘Media portrayal of stem cell research: towards a normative model for science communication’. Asian Bioethics Review 9 (3), pp. 199–209. https://doi.org/10.1007/s41649-017-0026-8 .

Kata, A. (2010). ‘A postmodern Pandora’s box: anti-vaccination misinformation on the Internet’. Vaccine 28 (7), pp. 1709–1716. https://doi.org/10.1016/j.vaccine.2009.12.022 .

King, G., Schneer, B. and White, A. (2017). ‘How the news media activate public expression and influence national agendas’. Science 358 (6364), pp. 776–780. https://doi.org/10.1126/science.aao1100 .

Landhuis, E. (2016). ‘Scientific literature: information overload’. Nature 535 (7612), pp. 457–458. https://doi.org/10.1038/nj7612-457a .

Le Masurier, M. (2015). ‘What is slow journalism?’ Journalism Practice 9 (2), pp. 138–152. https://doi.org/10.1080/17512786.2014.916471 .

Lewis, J., Williams, A. E., Franklin, R. A., Thomas, J. and Mosdell, N. (2008). The quality and independence of British journalism. Tracking the changes over 20 years . Cardiff, U.K.: Cardiff University.

Mellor, F. (2015). ‘Non-news values in science journalism’. In: Absence in science, security and policy: from research agendas to global strategy. Ed. by B. Rappert and B. Balmer. London, U.K.: Palgrave Macmillan, pp. 93–113. https://doi.org/10.1057/9781137493736_5 .

Metcalfe, J. (2019). ‘Comparing science communication theory with practice: an assessment and critique using Australian data’. Public Understanding of Science 28 (4), pp. 382–400. https://doi.org/10.1177/0963662518821022 .

Motta, M. and Stecula, D. (2021). ‘Quantifying the effect of Wakefield et al. (1998) on skepticism about MMR vaccine safety in the U.S.’ PLoS ONE 16 (8), e0256395. https://doi.org/10.1371/journal.pone.0256395 .

Newman, N., Fletcher, R., Kalogeropoulos, A. and Kleis Nielsen, R. (2019). Reuters Institute Digital News Report 2019 . Oxford, U.K.: Reuters Institute for the Study of Journalism, University of Oxford. URL: http://www.digitalnewsreport.org/ .

O’Connor, C. and Joffe, H. (2014). ‘Gender on the brain: a case study of science communication in the new media environment’. PLoS ONE 9 (10), e110830. https://doi.org/10.1371/journal.pone.0110830 .

O’Regan, T. and Young, C. (2019). ‘Journalism by numbers: trajectories of growth and decline of journalists in the Australian census 1961–2016’. Media International Australia 172 (1), pp. 13–32. https://doi.org/10.1177/1329878x19862935 .

Ofcom (2019). News consumption in the UK: 2019 report . URL: https://www.ofcom.org.uk/__data/assets/pdf_file/0027/157914/uk-news-consumption-2019-report.pdf .

Oliver, M. B., Raney, A. A. and Bryant, J., eds. (2019). Media effects: advances in theory and research. 4th ed. New York, NY, U.S.A.: Routledge. https://doi.org/10.4324/9780429491146 .

Oxford English Dictionary (2020). Buzzword . URL: https://www.lexico.com/en/definition/buzzword .

Pearman, O., Boykoff, M., Osborne-Gowey, J., Aoyagi, M., Gammelgaard Ballantyne, A., Chandler, P., Daly, M., Doi, K., Fernández-Reyes, R., Jiménez-Gómez, I., Nacu-Schmidt, A., McAllister, L., McNatt, M., Mocatta, G., Petersen, L. K., Simonsen, A. H. and Ytterstad, A. (2021). ‘COVID-19 media coverage decreasing despite deepening crisis’. The Lancet Planetary Health 5 (1), E6–E7. https://doi.org/10.1016/S2542-5196(20)30303-X .

Phillips, D. P., Kanter, E. J., Bednarczyk, B. and Tastad, P. L. (1991). ‘Importance of the lay press in the transmission of medical knowledge to the scientific community’. The New England Journal of Medicine 325 (16), pp. 1180–1183. https://doi.org/10.1056/NEJM199110173251620 .

Rawat, S. and Meena, S. (2014). ‘Publish or perish: where are we heading?’ Journal of Research in Medical Sciences 19 (2), pp. 87–89. PMID: 24778659 .

Rensberger, B. (2009). ‘Science journalism: too close for comfort’. Nature 459 (7250), pp. 1055–1056. https://doi.org/10.1038/4591055a .

Rödder, S. (2011). ‘Science and the mass media — ‘Medialization’ as a new perspective on an intricate relationship’. Sociology Compass 5 (9), pp. 834–845. https://doi.org/10.1111/j.1751-9020.2011.00410.x .

SBS News (2017). ‘Vitamin B3 claims slammed by obstetricians’. URL: https://www.sbs.com.au/news/article/vitamin-b3-claims-slammed-by-obstetricians/kyua75v1f .

Schäfer, M. S. (2009). ‘From public understanding to public engagement: an empirical assessment of changes in science coverage’. Science Communication 30 (4), pp. 475–505. https://doi.org/10.1177/1075547008326943 .

Scheufele, D. A. and Lewenstein, B. V. (2005). ‘The public and nanotechnology: how citizens make sense of emerging technologies’. Journal of Nanoparticle Research 7 (6), pp. 659–667. https://doi.org/10.1007/s11051-005-7526-2 .

Schwitzer, G. (2013). ‘Addressing tensions when popular media and evidence-based care collide’. BMC Medical Informatics and Decision Making 13 (Suppl 3), S3. https://doi.org/10.1186/1472-6947-13-s3-s3 .

Seale, C. (2003). ‘Health and media: an overview’. Sociology of Health & Illness 25 (6), pp. 513–531. https://doi.org/10.1111/1467-9566.t01-1-00356 .

Selvaraj, S., Borkar, D. S. and Prasad, V. (2014). ‘Media coverage of medical journals: do the best articles make the news?’ PLoS ONE 9 (1), e85355. https://doi.org/10.1371/journal.pone.0085355 .

Shearer, E. (2018). ‘Social media outpaces print newspapers in the U.S. as a news source’. Pew Research Center . URL: https://www.pewresearch.org/fact-tank/2018/12/10/social-media-outpaces-print-newspapers-in-the-u-s-as-a-news-source/ .

Shi, H., Enriquez, A., Rapadas, M., Martin, E. M. M. A., Wang, R., Moreau, J., Lim, C. K., Szot, J. O., Ip, E., Hughes, J. N., Sugimoto, K., Humphreys, D. T., McInerney-Leo, A. M., Leo, P. J., Maghzal, G. J., Halliday, J., Smith, J., Colley, A., Mark, P. R., Collins, F., Sillence, D. O., Winlaw, D. S., Ho, J. W. K., Guillemin, G. J., Brown, M. A., Kikuchi, K., Thomas, P. Q., Stocker, R., Giannoulatou, E., Chapman, G., Duncan, E. L., Sparrow, D. B. and Dunwoodie, S. L. (2017). ‘NAD deficiency, congenital malformations, and niacin supplementation’. The New England Journal of Medicine 377 (6), pp. 544–552. https://doi.org/10.1056/nejmoa1616361 .

Shih, T.-J., Wijaya, R. and Brossard, D. (2008). ‘Media coverage of public health epidemics: linking framing and issue attention cycle toward an integrated theory of print news coverage of epidemics’. Mass Communication and Society 11 (2), pp. 141–160. https://doi.org/10.1080/15205430701668121 .

Sturgis, P. and Allum, N. (2004). ‘Science in society: re-evaluating the deficit model of public attitudes’. Public Understanding of Science 13 (1), pp. 55–74. https://doi.org/10.1177/0963662504042690 .

Sumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Bott, L., Adams, R., Venetis, C. A., Whelan, L., Hughes, B. and Chambers, C. D. (2016). ‘Exaggerations and caveats in press releases and health-related science news’. PLoS ONE 11 (12), e0168217. https://doi.org/10.1371/journal.pone.0168217 .

Taylor, J. W., Long, M., Ashley, E., Denning, A., Gout, B., Hansen, K., Huws, T., Jennings, L., Quinn, S., Sarkies, P., Wojtowicz, A. and Newton, P. M. (2015). ‘When medical news comes from press releases — A case study of pancreatic cancer and processed meat’. PLoS ONE 10 (6), e0127848. https://doi.org/10.1371/journal.pone.0127848 .

The Royal Australian and New Zealand College of Obstetricians and Gynaecologists (2017). ‘New research suggesting vitamin B3 can prevent miscarriage and fetal abnormality should be taken with caution’. URL: https://ranzcog.edu.au/news/new-research-suggesting-vitamin-b3-can-prevent-mis .

Vestergård, G. L. (2015). ‘Where does science news come from?’ Ph.D. thesis. Aarhus, Denmark: Aarhus University.

Victor Chang Cardiac Research Institute (2017). ‘Historic discovery has the potential to prevent miscarriages and birth defects globally’. URL: https://www.victorchang.edu.au/news/pregnancy-breakthrough .

Vinkers, C. H., Tijdink, J. K. and Otte, W. M. (2015). ‘Use of positive and negative words in scientific PubMed abstracts between 1974 and 2014: retrospective analysis’. BMJ 351, h6467. https://doi.org/10.1136/bmj.h6467 .

Vogler, D. and Schäfer, M. S. (2020). ‘Growing influence of university PR on science news coverage? A longitudinal automated content analysis of university media releases and newspaper coverage in Switzerland, 2003–2017’. International Journal of Communication 14, pp. 3143–3164. URL: https://ijoc.org/index.php/ijoc/article/view/13498 .

Walker, M. (2021). ‘U.S. newsroom employment has fallen 26% since 2008’. Pew Research Center . URL: https://www.pewresearch.org/fact-tank/2020/04/20/u-s-newsroom-employment-has-dropped-by-a-quarter-since-2008/ .

Watermeyer, R. and Lewis, J. (2018). ‘Institutionalizing public engagement through research in UK universities: perceptions, predictions and paradoxes concerning the state of the art’. Studies in Higher Education 43 (9), pp. 1612–1624. https://doi.org/10.1080/03075079.2016.1272566 .

Watkins, E. (2019). ‘What we lose with the decline of mainstream science journalists’. Crikey . URL: https://www.crikey.com.au/2019/01/24/science-journalism-denialism/ .

Weitkamp, E. and Eidsvaag, T. (2014). ‘Agenda building in media coverage of food research’. Journalism Practice 8 (6), pp. 871–886. https://doi.org/10.1080/17512786.2013.865966 .

Yavchitz, A., Boutron, I., Bafeta, A., Marroun, I., Charles, P., Mantz, J. and Ravaud, P. (2012). ‘Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study’. PLoS Medicine 9 (9), e1001308. https://doi.org/10.1371/journal.pmed.1001308 .

Young Lin, L. L. and Rosenkrantz, A. B. (2017). ‘The U.S. online news coverage of mammography based on a Google News search’. Academic Radiology 24 (12), pp. 1612–1615. https://doi.org/10.1016/j.acra.2017.05.011 .

Georgia is a Ph.D. candidate and Research Assistant at the Melbourne School of Population and Global Health within the Faculty of Medicine, Dentistry and Health Sciences at the University of Melbourne. Georgia’s Ph.D. is in the fields of science communication and public health. E-mail: [email protected] .

Georgina is a Senior Research Fellow at the Melbourne School of Population and Global Health within the Faculty of Medicine, Dentistry and Health Sciences at the University of Melbourne. Georgina has a background in health psychology with a major focus on social epidemiology. She has extensive experience in quantitative research across a broad range of content areas including disability, women and children’s health, public health law, mental health and wellbeing, suicide prevention and violence against women. E-mail: [email protected] .

Louise is a Professor at the Melbourne School of Population and Global Health within the Faculty of Medicine, Dentistry and Health Sciences at the University of Melbourne. Louise is a health sociologist researching lay and expert perceptions of risk and health decision-making, particularly in relation to the use of health technology. She is an expert in qualitative research methodology and the translation of evidence to clinical practice. E-mail: [email protected] .

Table of Contents

Non specialist summary.

This article has no summary

• Newsletters

Site search

• Israel-Hamas war
• Home Planet
• 2024 election
• Supreme Court
• TikTok’s fate
• All explainers
• Future Perfect

Filed under:

Study: half of the studies you read about in the news are wrong

And yes, this could be one of them.

Share this story

• Share this on Facebook
• Share this on Twitter
• Share this on Reddit
• Share All sharing options

Share All sharing options for: Study: half of the studies you read about in the news are wrong

There’s an oft-repeated mantra among scientists: A single study is rarely the final answer. And yet for science reporters, new studies are irresistible — a bold new finding makes a great headline.

Which explains how we get into confusing situations like this:

Why don't people pay attention to health advice? It's a mystery. pic.twitter.com/UQ0J8e580Y — Christopher Snowdon (@cjsnowdon) February 23, 2017

The problem isn’t necessarily that these studies are poorly designed (although some of them may be). The problem is that each headline gives an incomplete glimpse of how science works. One lab produces a result. Another lab — ideally — tries to replicate that result. Rinse and repeat. Eventually someone needs to do a meta-review of the totality of the evidence on the question to reach a conclusion. That meta-review, rather than any one study in isolation, is likely to get closer to the true answer.

Yet as researchers in PLOS O ne recently found , journalists typically only cover those initial papers — and skip over writing about the clarifying meta-reviews that come later on.

What’s more, the study finds, journalists “rarely inform the public when [initial studies] are disconfirmed” — despite the fact around half of the studies journalists write about are later rebutted by follow-up studies.

Journalists cover initial studies far more often than follow-ups or meta-reviews

The authors of the PLOS One paper assembled a huge database of studies in biomedical science, follow-ups to those studies, and meta-studies on those follow-ups. And then they searched the Dow Jones Factiva newspaper database to see how often each type of study was covered.

They found that initial studies were around five times more likely to be reported on than follow-up studies. And meta-reviews were barely covered at all.

What’s more, journalists really, really like to report on studies that deliver positive results — even though studies that deliver negative results are equally valuable. Of the 1,475 newspaper articles in the data set, only 75 articles reported on null findings.

Lastly, journalists seem to flock to studies that concern lifestyle choices like diet or exercise (and especially those published in the most prestigious journals). Non-lifestyle studies — on topics like brain imaging or genetics — were much less common. This “preferential coverage,” the researchers surmise, is due to the fact that readers can take direct action on lifestyle choices.

Nearly half of the single studies that get reported on turn out to be wrong

Here’s why this is a problem. The PLOS One analysis paper found that only 48.7 percent of 156 studies reported by newspapers were confirmed by a subsequent meta-review.* The percentage dropped to 34 when the researchers focused on initial studies only.

(*To be sure, meta-reviews aren’t perfect. Publication bias — the tendency for only papers with positive results to get into journals — can skew even the most careful meta-review. But, in general, they provide a more comprehensive answer than a single study.)

And although journalists gravitate toward covering single studies concerning lifestyle choices such as diet or exercise, these were actually the least likely to be confirmed by a meta-review (as opposed to non-lifestyle papers on topics like genetics).

Overall, the authors conclude:

Our study shows that many biomedical findings reported by newspapers are disconfirmed by subsequent studies. This is partly due to the fact that newspapers preferentially cover "positive" initial studies rather than subsequent observations, in particular those reporting null findings. Our study also suggests that most journalists from the general press do not know or prefer not to deal with the high degree of uncertainty inherent in early biomedical studies.

Why do reporters give undue weight to single studies?

A few possible reasons:

1) Journalists have a need for digestible headlines that convey simple, accessible, and preferably novel lessons. This is fundamentally in tension with how science works, which stresses a slow accumulation of knowledge, nuance, and doubt.

2) It’s not all the journalists’ fault. University press shops are less likely to put out press releases on meta-reviews than they are on a striking and dramatic single study. What’s more the meta studies themselves can be dense and difficult to parse.

3) It’s often the scientific papers or press releases themselves that spread hype about initial findings.

The PLOS One authors have some advice for reporters writing about new studies. Namely: Pick up the phone, and ask researchers whether it is an initial finding, and, if so, they should inform the public that this discovery is still tentative and must be validated by subsequent studies. Indeed, study co-author our result only refers to a small sample of the scientific research.

Also note that this PLOS One study has a few limitations itself. For one, it only looked at newspaper articles. In reality, the science media ecosystem is much, much bigger. There are web-only general interest news outlets like Vox, specialty science magazines such as New Scientist and Scientific American, news operations run by journals like Science and Nature, and television news programs — all of which report on science. “Our result only refers to a small sample of the scientific research,” Estelle Dumas Mallet, the study’s lead author, writes in an email. “Also, we cannot extrapolate these results to other domains such as physics and chemistry.”

The study also only included news articles published within a month of the publication of the scientific papers they cite. It’s possible newspapers do a better job when they cite research when it’s not breaking news.

That said, my guess is that the findings still stand for the broader media environment. As reporters, we’re biased toward what’s new and exciting. But in science, truth takes time.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

We accept credit card, Apple Pay, and Google Pay. You can also contribute via

Next Up In Science

Sign up for the newsletter today, explained.

Understand the world with a daily explainer plus the most compelling stories of the day.

Thanks for signing up!

Check your inbox for a welcome email.

Oops. Something went wrong. Please enter a valid email and try again.

Bird flu in cows — and now in milk. How worried should we be?

Challengers forces us to ask: Is tennis sexy?

Summer Lee’s primary puts Democrats’ divides on Israel on display

A new Supreme Court case seeks to make it much easier for criminals to buy guns

Students protested for Palestine before Israel was even founded

The Supreme Court’s likely to make it more dangerous to be pregnant in a red state

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Growing public concern about the role of artificial intelligence in daily life

A growing share of Americans express concern about the role artificial intelligence (AI) is playing in daily life, according to a new Pew Research Center survey.

Pew Research Center conducted this study to understand attitudes about artificial intelligence and its uses. For this analysis, we surveyed 11,201 U.S. adults from July 31 to Aug. 6, 2023.

Everyone who took part in the survey is a member of the Center’s American Trends Panel (ATP), an online survey panel that is recruited through national, random sampling of residential addresses. This way, nearly all U.S. adults have a chance of selection. The survey is weighted to be representative of the U.S. adult population by gender, race, ethnicity, partisan affiliation, education and other categories. Read more about the ATP’s methodology .

Here are the questions used for this analysis , along with responses, and its methodology .

Overall, 52% of Americans say they feel more concerned than excited about the increased use of artificial intelligence. Just 10% say they are more excited than concerned, while 36% say they feel an equal mix of these emotions.

The share of Americans who are mostly concerned about AI in daily life is up 14 percentage points since December 2022, when 38% expressed this view.

Concern about AI outweighs excitement across all major demographic groups. Still, there are some notable differences, particularly by age. About six-in-ten adults ages 65 and older (61%) are mostly concerned about the growing use of AI in daily life, while 4% are mostly excited. That gap is much smaller among those ages 18 to 29: 42% are more concerned and 17% are more excited.

Rising awareness, and concern, about AI

The rise in concern about AI has taken place alongside growing public awareness. Nine-in-ten adults have heard either a lot (33%) or a little (56%) about artificial intelligence. The share who have heard a lot about AI is up 7 points since December 2022.

Those who have heard a lot about AI are 16 points more likely now than they were in December 2022 to express greater concern than excitement about it. Among this most aware group, concern now outweighs excitement by 47% to 15%. In December, this margin was 31% to 23%.

Similarly, people who have heard a little about AI are 19 points more likely to express concern today than they were in December. A majority now express greater concern than excitement (58%) about AI’s growing role in daily life, while just 8% report the opposite feeling.

Our previous analyses have found that Americans’ concerns about AI include a desire to maintain human control over these technologies , doubts that AI will improve the way things are now, and caution over the pace of AI adoption in fields like health and medicine .

Opinions of whether AI helps or hurts in specific settings

Despite growing public concern over the use of artificial intelligence in daily life, opinions about its impact in specific areas are more mixed. There are several uses of AI where the public sees a more positive than negative impact.

For instance, 49% say AI helps more than hurts when people want to find products and services they are interested in online. Just 15% say it mostly hurts when used for this purpose, and 35% aren’t sure.

Other uses of AI where opinions tilt more positive than negative include helping companies make safe cars and trucks and helping people take care of their health.

In contrast, public views of AI’s impact on privacy are much more negative. Overall, 53% of Americans say AI is doing more to hurt than help people keep their personal information private. Only 10% say AI helps more than it hurts, and 37% aren’t sure. Our past research has found majorities of Americans express concern about online privacy generally and a lack of control over their own personal information.

Public views on AI’s impact are still developing, though. Across the eight use cases in the survey, 35% to 49% of Americans say they’re not sure what impact AI is having.

Demographic differences in views of AI’s impact

There are significant demographic differences in the perceived impact of AI in specific use cases.

Americans with higher levels of education are more likely than others to say AI is having a positive impact across most uses included in the survey. For example, 46% of college graduates say AI is doing more to help than hurt doctors in providing quality care to patients. Among adults with less education, 32% take this view.

A similar pattern exists with household income, where Americans with higher incomes tend to view AI as more helpful for completing certain tasks.

A big exception to this pattern is views of AI’s impact on privacy. About six-in-ten college graduates (59%) say that AI hurts more than it helps at keeping people’s personal information private. Half of adults with lower levels of education also hold this view.

Men also tend to view AI’s impact in specific areas more positively than women. These differences by education, income and gender are generally consistent with our previous work on artificial intelligence .

Note: Here are the questions used for this analysis , along with responses, and its methodology .

• Artificial Intelligence

Alec Tyson is an associate director of research at Pew Research Center

Emma Kikuchi is is a research assistant focusing on science and society research at Pew Research Center

Many Americans think generative AI programs should credit the sources they rely on

Americans’ use of chatgpt is ticking up, but few trust its election information, q&a: how we used large language models to identify guests on popular podcasts, striking findings from 2023, what the data says about americans’ views of artificial intelligence, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

• International edition
• Australia edition
• Europe edition

Only 40% of people in England trust their police force, research reveals

Metropolitan police scores lowest in public confidence, with women trusting London officers even less than men

Only four out of 10 people in England say they trust the police, with the UK’s biggest force, the Metropolitan police , getting the lowest confidence score, research has found.

The poll surveyed nine English regions, in eight of which female respondents had greater trust in the police than male respondents. But for the Met in London , hit by a succession of scandals, women trusted Britain’s largest force less than men.

The poll, which revealed significantly lower trust among ethnic minorities in policing than among white people, comes before a general election where law and order and crime are expected to be a big issue.

The poll followed work by academics commissioned by the Economic and Social Research Council , and is published in the journal Policing and Society. It concludes that the task facing the Met, in terms of its reputation, is “monumental”.

Eight thousand people were surveyed from July 2022 to September 2023, a period after Wayne Couzens was convicted of abusing his powers as a Met officer to kidnap and murder Sarah Everard, and which included the unmasking of his colleague, David Carrick, as one of the worst ever serial sexual attackers, who was allowed to stay in the Met despite a slew of allegations against him.

Respondents were asked to rate how much they trusted the police, and across all of England 41% of people said they did so, including 42% of women and 40% of men.

The poll revealed a potentially large “race gap”: while 42.6% of white British respondents said they trusted the police, only 32.1% of other people said they did.

The sample size was large enough to show regional differences. In the south-east, trust in policing was highest, at 43.5% for both men and women, and stood at 45% for white British people in that region – but only 31.6% for those of other ethnicities.

The Met scored the lowest percentages, with 34.6% of Londoners saying they trusted it, made up of 35.5% of men and 33.8% of women.

One of the academics who led the study, Steve Pickering of the University of Amsterdam, said: “It looks like policing has lost legitimacy and that it has been undermined by a succession of high-profile scandals.”

Other polling, carried out on behalf of forces and their police and crime commissioners, has put trust and confidence scores at 60% to 80%. Pickering said that polling was likely to have used a different methodology, and that he and his colleagues were confident of their findings: “We have been asking this question repeatedly, and over a long time, and are getting the same results.”

Last year, the Met was condemned for wholesale failings in the Casey review . Pickering said: “Our research suggests that public trust in the Met is even lower than Baroness Casey found.”

A Met police spokesperson said: “We are determined to earn back the trust of women and girls whose confidence in policing has been shaken by events of recent years.

“Through our violence against women and girls action plan we are transforming how we protect women and girls including seeing more predators brought to justice, more victims protected from harm and safer spaces for women and girls to enjoy.

“We are creating bigger teams to reduce caseloads of officers, provide a better service to victims and survivors and improving training.”

• Metropolitan police
• Sarah Everard
• Wayne Couzens

More on this story

Police officer found not guilty of raping woman in Plymouth while on duty

Scotland’s first minister defends Hate Crime Act amid barrage of criticism

Police suspected of on-duty offences to be dealt with faster, Home Office says

Nottinghamshire police placed in special measures

Elizabeth Emblem to honour UK public workers who die in line of duty

Independent inquiry into 2005 murder of Emma Caldwell announced

Hunt announces funds for police to use drones as ‘first responders’

Greater Manchester police admit footage of officer dragging homeless man is ‘unacceptable’

Mass protests in London put other police priorities at risk, MPs warn

West Midlands PCC applies for judicial review over scrapping of his role

Most viewed.

The rise of the inclusive consumer

The American consumer is undeniably becoming more inclusive. Responding to our survey 1 McKinsey Inclusive Consumer Survey, October 2021. in October 2021, two out of three Americans told us their social values now shape their shopping choices. And 45 percent—likely representing well over a hundred million shoppers 2 Extrapolated data: 45 percent of 270 million US residents over 18 years old would be about 120 million. —believe retailers should actively support Black-owned businesses and brands. This 45 percent represents the inclusive consumer.

About the authors

This article is a collaborative effort by Pamela Brown, Tiffany Burns , Tyler Harris, Charlotte Lucas, and Israe Zizaoui, representing views from McKinsey’s Retail Practice.

Large consumer and retail players from Nordstrom to Yelp are moving quickly to serve this group of influential customers. Indeed, while inclusive consumers tend to be younger, female, and more racially diverse, they include men and women across ethnic backgrounds, income levels, and age groups. Given this diverse and ubiquitous representation, the inclusive consumer holds the unique power to influence all demographic groups (Exhibit 1).

The inclusive consumer is more likely than other shoppers to buy Black-owned brands out of a desire to support diverse entrepreneurs on their growth journeys and small businesses in general (Exhibit 2). Like many other consumers today, they base their shopping choices less on traditional advertising and more on social media, friends’ recommendations, and the stories of those who founded the brands they admire. They are also more likely than other shoppers to care about sustainability and quality. 3 McKinsey Inclusive Consumer Survey, October 2021.

Where the inclusive consumer may be headed

Inclusive consumers are a large and influential population, and they want to spend more on Black-owned brands, which they know are underrepresented on store shelves. However, finding the brands and the products remains a barrier. Indeed, one in five inclusive consumers cited not finding the products they wanted as a reason for not buying from Black-owned brands. This plays out in the bigger picture of US retail spending as well. While about 14 percent of the US population identifies as Black, 4 “Quick facts,” United States Census Bureau, July 1, 2019. Black-owned brands rang up only about $83 billion in sales in 2020 5 Estimated with consumer-reported proportion of holiday shopping with Black-owned brands and National Retail Federation data on 2020 holiday spending of $790 billion. —less than 1.5 percent of $5.4 trillion 6 “Sales for U.S. retail trade sector over $5,411.0 billion,” United States Census Bureau, January 28, 2021. of retail spending. In our survey, about 21 percent of inclusive consumers said they had pledged to devote at least 15 percent of their retail spending to Black-owned brands in an effort to turn the tides. 7 McKinsey Inclusive Consumer Survey, October 2021.

Most retailers will need to make changes to meet the needs of inclusive consumers. Fewer than half of these shoppers say they know which products on retail shelves are from Black-owned brands, and a third don’t know where to go to purchase Black-owned brands. 8 McKinsey Inclusive Consumer Survey, October 2021. About 45 percent of the inclusive consumers we surveyed said they would value relevant labeling on store shelves and the ability to filter website results to find Black-owned brands. About a third would value additional sources of information, such as curated lists of products, a database of Black-owned brands, and recommendations from influencers.

Major retailers are now joining social influencers and other stakeholders to spread the word. Growing support for Black-owned brands is evident in social media and traditional media: a wide range of publications, from New York magazine and GQ to Harper’s Bazaar 9 “The 42 best gifts to shop from Black-owned businesses now and forever,” Harper’s Bazaar , November 17, 2021. and whowhatwear.com, published a curated holiday list of products from Black-owned brands in November 2021. In the same month, Oprah’s “favorite things” list 10 “Oprah’s favorite things 2021 list is here!,” Oprah Daily, November 23, 2021. focused on products from small businesses, many of which are owned by women and people of color.

How retailers could attract the inclusive consumer

In a marketplace being transformed by digitization, social change, and a global pandemic, leading retailers and brand managers know that they must keep meeting evolving consumer preferences to stay relevant. Major retailers are now reviewing how they work—from operations and merchandising to hiring and HR practices—and starting to move the needle. Retailers should consider actions in five main areas to bring the inclusive consumer through their doors and onto their websites.

Action 1: Reshape shelves. Retailers can first and foremost demonstrate a commitment to the inclusive consumer by evaluating and reimagining the brands on their shelves. Many are already taking strong action, reimagining their merchandising mix to make shelves and the brands that sit on them more representative. For example, more than 28 national retailers, including Gap, Macy’s, and Sephora, have signed the Fifteen Percent Pledge’s call for 15 percent of retail shelf space to be dedicated to Black-owned brands. Some of these retailers have doubled their assortments of Black-owned brands in just a year, opening their doors to almost 400 brands. 11 Elizabeth Segran, “In one year, the 15% Pledge got 385 Black-owned brands on the shelves of Macy’s, Gap, Sephora, and more,” Fast Company , July 7, 2021. In addition to introducing Black-owned brands, retailers could also review their planograms and product placement: for example, Ulta Beauty is intentional about placing Black-owned brands in prime locations in stores and creates dedicated efforts to increase Black-owned brands’ features in circular advertising and email marketing. 12 Liz Flora, “Camille Rose expands to Ulta Beauty as retailer makes Black-owned brand push,” Glossy, April 12, 2021.

Retailers can first and foremost demonstrate a commitment to the inclusive consumer by evaluating and reimagining the brands on their shelves.

Action 2: Switch up sourcing. Retailers, especially those who are vertically integrated, could also demonstrate commitment to the inclusive consumer by doubling down on diversity in their supply chain. For example, Best Buy has committed to spending at least $1.2 billion with businesses owned by members of the Black, Indigenous, and people of color (BIPOC) community and other diverse businesses by 2025. 13 “Best Buy commits to spending $1.2 billion with BIPOC and diverse businesses by 2025,” Business Wire, June 24, 2021. The commitment includes plans to increase all forms of spending with businesses from nearly every corner of the company—from how it brings goods and services to stores to where and how it advertises. Supplier diversity is not limited to retailers and can also be considered broadly by most brands and companies. For example, in 2020, the Coca-Cola Company committed to spending an additional $500 million over five years with Black-owned suppliers, more than doubling the company’s previous commitment. 14 “Coca-Cola Company commits $500 million in additional spending with Black-owned suppliers,” Coca-Cola Company, October 10, 2020. These actions show an increasing commitment to diversity across the supply chain, and these types of commitments will likely grow in future years as consumers begin calling for similar levels of diversity in the supply chain as they are calling for on shelves.

Action 3: Map it out. Inclusive consumers consistently say that not knowing which brands are Black-owned or where to shop for them are barriers to spending as much as they would like on brands from diverse founders. And wayfinding is their resounding answer. Retailers could help consumers identify Black-owned brands with clear labeling on shelves and websites, the ability to filter website search results, and curated lists of products, for example.

Sephora responded to these needs by creating a dedicated tab on its website to share Black-owned brands’ stories and products and by enhancing its online site, which features Black-owned brands on its browsing pane, provides a filter for shoppers looking for Black- and BIPOC-owned brands, and provides curated lists of recommended products, including a “Sephora Favorites” bundle in Black-owned beauty. 15 Kori Hale, “Sephora’s planning to double down on Black-owned brands,” Forbes , January 25, 2021. Google is also actively working on multiple initiatives to support Black-owned brands, including a tool that allows Black-owned businesses to identify themselves in maps, listings, and Google business-profile searches. 16 Keyword , “Standing with the Black community,” blog entry by Sundar Pichai, Google, June 3, 2020; “Supporting Black-owned businesses,” Google for Small Business, 2021. Additionally, Google has launched the ByBlack platform, which consists of both a national Black business directory (powered by the US Black Chambers) and a national certification as a Black-owned business (in partnership with American Express). This tool provides Black entrepreneurs access to a community with valuable business resources (including Google trainings) and a way to help reach new customers.

Action 4: Share the stories. Inclusive consumers care about what is on shelves, but they want more. They want to learn about founders’ stories, for example, and support the missions of small businesses. Further, our research shows that inclusive consumers are more likely than other shoppers to buy Black-owned brands based on recommendations from friends, and retailers that incorporate founder stories into marketing and digital placements are more likely than those that don’t to attract the inclusive consumer’s eye. Nordstrom, for example, introduced “Concept 012: Black_Space,” which includes a dedicated shop developed, designed, and curated by Black voices to amplify Black representation through in-store buildouts and merchandising. The concept is further supported with an online site experience that includes video content created by Black curators to represent their perspectives. 17 “New Concepts @Nordstrom launches Concept 012: Black_Space,” Nordstrom, February 22, 2021.

Action 5: Ditch the big-brand playbook. Introducing and cultivating smaller brands takes different capabilities and mindsets  than retailers may be accustomed to when working with larger, more established brands. Unlike big brands, small brands come with more variability in experience and know-how, as well as growing pains as they enter the bigger retail stage. Retailers that are committed to fostering smaller brands will need to work differently, creating teams that understand how these brands operate. For example, those with experience working with small brands know that everyday requirements from retailers, such as buy sizes and inventory systems, can have large-scale repercussions on small brands with limited working capital or team capacity to meet these terms. A retailer that seeks to help smaller brands can boost the odds of success by adapting its playbook and its teams to suit their needs.

The time is now

In the years ahead, millions more consumers will likely join the ranks of inclusive consumers, rewarding businesses that pursue inclusion and avoiding those that don’t.

Businesses that meet the needs of inclusive consumers will likely do more than raise revenues and loyalty—they may also earn dividends in other areas of the business, including attracting and retaining talent. About 70 percent of the US-based employees we surveyed in August 2020 said their sense of purpose is largely defined by work, and nearly two-thirds said COVID-19 had caused them to reflect on their purpose in life, and nearly half were reconsidering the kind of work they do. (For more on this topic, please see Naina Dhingra, Andrew Samo, Bill Schaninger, and Matt Schrimper “ Help your employees find purpose—or watch them leave .”)

Whether employees or customers, the inclusive consumer is changing the imperative for retail. The evidence is overwhelming: the inclusive consumer is leading the pack, influencing consumers across demographics, and voting with their pocketbooks for retailers that support diverse entrepreneurs and their products. (For more on this topic, please see “ The Black unicorn: Changing the game for inclusivity in retail .”)

Pamela Brown is a partner in McKinsey’s New Jersey office, where Charlotte Lucas and Israe Zizaoui are consultants; Tiffany Burns is a partner in the Atlanta office; and Tyler Harris is an associate partner in the Washington, DC, office.

Explore a career with us

Related articles.

Help your employees find purpose—or watch them leave

The Black unicorn: Changing the game for inclusivity in retail

Reimagining marketing in the next normal