importance of research questions and objectives

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

1-Research Questions

1. The Purpose of Research Questions

Both professional researchers and successful student researchers develop research questions. That’s because research questions are more than handy tools; they are essential to the research process.

By defining exactly what the researcher is trying to find out, these questions influence most of the rest of the steps taken to conduct the research. That’s true even if the research is not for academic purposes but for other areas of our lives.

For instance, if you’re seeking information about a health problem in order to learn whether you have anything to worry about, research questions will make it possible for you to more effectively decide whether to seek medical help–and how quickly.

Or, if you’re researching a potential employer, having developed and used research questions will mean you’re able to more confidently decide whether to apply for an internship or job there.

The confidence you’ll have when making such decisions will come from knowing that the information they’re based on was gathered by conscious thought rather than serendipity and whim.

Choosing & Using Sources: A Guide to Academic Research Copyright © 2015 by Teaching & Learning, Ohio State University Libraries is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Research Question 101 📖

Everything you need to know to write a high-quality research question

By: Derek Jansen (MBA) | Reviewed By: Dr. Eunice Rautenbach | October 2023

If you’ve landed on this page, you’re probably asking yourself, “ What is a research question? ”. Well, you’ve come to the right place. In this post, we’ll explain what a research question is , how it’s differen t from a research aim, and how to craft a high-quality research question that sets you up for success.

Research Question 101

What is a research question.

• Research questions vs research aims
• The 4 types of research questions
• How to write a research question
• Frequently asked questions
• Examples of research questions

As the name suggests, the research question is the core question (or set of questions) that your study will (attempt to) answer .

In many ways, a research question is akin to a target in archery . Without a clear target, you won’t know where to concentrate your efforts and focus. Essentially, your research question acts as the guiding light throughout your project and informs every choice you make along the way.

Let’s look at some examples:

What impact does social media usage have on the mental health of teenagers in New York?

How does the introduction of a minimum wage affect employment levels in small businesses in outer London?

How does the portrayal of women in 19th-century American literature reflect the societal attitudes of the time?

What are the long-term effects of intermittent fasting on heart health in adults?

As you can see in these examples, research questions are clear, specific questions that can be feasibly answered within a study. These are important attributes and we’ll discuss each of them in more detail a little later . If you’d like to see more examples of research questions, you can find our RQ mega-list here .

Research Questions vs Research Aims

At this point, you might be asking yourself, “ How is a research question different from a research aim? ”. Within any given study, the research aim and research question (or questions) are tightly intertwined , but they are separate things . Let’s unpack that a little.

A research aim is typically broader in nature and outlines what you hope to achieve with your research. It doesn’t ask a specific question but rather gives a summary of what you intend to explore.

The research question, on the other hand, is much more focused . It’s the specific query you’re setting out to answer. It narrows down the research aim into a detailed, researchable question that will guide your study’s methods and analysis.

Let’s look at an example:

Research Aim: To explore the effects of climate change on marine life in Southern Africa.

Research Question: How does ocean acidification caused by climate change affect the reproduction rates of coral reefs?

As you can see, the research aim gives you a general focus , while the research question details exactly what you want to find out.

Need a helping hand?

Types of research questions

Now that we’ve defined what a research question is, let’s look at the different types of research questions that you might come across. Broadly speaking, there are (at least) four different types of research questions – descriptive , comparative , relational , and explanatory . 

Descriptive questions ask what is happening. In other words, they seek to describe a phenomena or situation . An example of a descriptive research question could be something like “What types of exercise do high-performing UK executives engage in?”. This would likely be a bit too basic to form an interesting study, but as you can see, the research question is just focused on the what – in other words, it just describes the situation.

Comparative research questions , on the other hand, look to understand the way in which two or more things differ , or how they’re similar. An example of a comparative research question might be something like “How do exercise preferences vary between middle-aged men across three American cities?”. As you can see, this question seeks to compare the differences (or similarities) in behaviour between different groups.

Next up, we’ve got exploratory research questions , which ask why or how is something happening. While the other types of questions we looked at focused on the what, exploratory research questions are interested in the why and how . As an example, an exploratory research question might ask something like “Why have bee populations declined in Germany over the last 5 years?”. As you can, this question is aimed squarely at the why, rather than the what.

Last but not least, we have relational research questions . As the name suggests, these types of research questions seek to explore the relationships between variables . Here, an example could be something like “What is the relationship between X and Y” or “Does A have an impact on B”. As you can see, these types of research questions are interested in understanding how constructs or variables are connected , and perhaps, whether one thing causes another.

Of course, depending on how fine-grained you want to get, you can argue that there are many more types of research questions , but these four categories give you a broad idea of the different flavours that exist out there. It’s also worth pointing out that a research question doesn’t need to fit perfectly into one category – in many cases, a research question might overlap into more than just one category and that’s okay.

The key takeaway here is that research questions can take many different forms , and it’s useful to understand the nature of your research question so that you can align your research methodology accordingly.

How To Write A Research Question

As we alluded earlier, a well-crafted research question needs to possess very specific attributes, including focus , clarity and feasibility . But that’s not all – a rock-solid research question also needs to be rooted and aligned . Let’s look at each of these.

A strong research question typically has a single focus. So, don’t try to cram multiple questions into one research question; rather split them up into separate questions (or even subquestions), each with their own specific focus. As a rule of thumb, narrow beats broad when it comes to research questions.

Clear and specific

A good research question is clear and specific, not vague and broad. State clearly exactly what you want to find out so that any reader can quickly understand what you’re looking to achieve with your study. Along the same vein, try to avoid using bulky language and jargon – aim for clarity.

Unfortunately, even a super tantalising and thought-provoking research question has little value if you cannot feasibly answer it. So, think about the methodological implications of your research question while you’re crafting it. Most importantly, make sure that you know exactly what data you’ll need (primary or secondary) and how you’ll analyse that data.

A good research question (and a research topic, more broadly) should be rooted in a clear research gap and research problem . Without a well-defined research gap, you risk wasting your effort pursuing a question that’s already been adequately answered (and agreed upon) by the research community. A well-argued research gap lays at the heart of a valuable study, so make sure you have your gap clearly articulated and that your research question directly links to it.

As we mentioned earlier, your research aim and research question are (or at least, should be) tightly linked. So, make sure that your research question (or set of questions) aligns with your research aim . If not, you’ll need to revise one of the two to achieve this.

FAQ: Research Questions

Research question faqs, how many research questions should i have, what should i avoid when writing a research question, can a research question be a statement.

Typically, a research question is phrased as a question, not a statement. A question clearly indicates what you’re setting out to discover.

Can a research question be too broad or too narrow?

Yes. A question that’s too broad makes your research unfocused, while a question that’s too narrow limits the scope of your study.

Here’s an example of a research question that’s too broad:

“Why is mental health important?”

Conversely, here’s an example of a research question that’s likely too narrow:

“What is the impact of sleep deprivation on the exam scores of 19-year-old males in London studying maths at The Open University?”

Can I change my research question during the research process?

How do i know if my research question is good.

A good research question is focused, specific, practical, rooted in a research gap, and aligned with the research aim. If your question meets these criteria, it’s likely a strong question.

Is a research question similar to a hypothesis?

Not quite. A hypothesis is a testable statement that predicts an outcome, while a research question is a query that you’re trying to answer through your study. Naturally, there can be linkages between a study’s research questions and hypothesis, but they serve different functions.

How are research questions and research objectives related?

The research question is a focused and specific query that your study aims to answer. It’s the central issue you’re investigating. The research objective, on the other hand, outlines the steps you’ll take to answer your research question. Research objectives are often more action-oriented and can be broken down into smaller tasks that guide your research process. In a sense, they’re something of a roadmap that helps you answer your research question.

Need some inspiration?

If you’d like to see more examples of research questions, check out our research question mega list here .  Alternatively, if you’d like 1-on-1 help developing a high-quality research question, consider our private coaching service .

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Ingham Institute

click here to donate please

Home / Why the research question is so important

Why the research question is so important

04/05/2018 at 5:42 am

A research question is a sentence that defines what you will examine, within which population, and what the outcome of interest will be. Defining a clear research question is the first and most important part of the project. Though it sounds simple, writing a research question is tricky even for experienced researchers. This video will build your understanding of where to start looking for research questions, how to write them, and why it is important to work with a solid research question from the beginning of the project.

Share this post

CALD clinical clinical data clinical practice clinical trials clinicians conducting surveys data database degree degrees diverse communities engagement guidelines linguistic masters PhD question research research journey research project statistical statistics study design support survey SWSLHD

Additional Content

Links to useful research papers.

Farrugia P, Petrisor BA, Farrokhyar F, Bhandari M. Research questions, hypotheses and objectives. Canadian Journal of Surgery. 2010;53(4):278-281. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2912019/

Aslam, S., & Emmanuel, P. (2010). Formulating a researchable question: A critical step for facilitating good clinical research. Indian J Sex Transm Dis, 31(1), 47-50. https://www.ncbi.nlm.nih.gov/pubmed/21808439

Beitz, J. M. (2006). Writing the researchable question. J Wound Ostomy Continence Nurs, 33(2), 122-124. https://www.ncbi.nlm.nih.gov/pubmed/16572009

Links to useful websites

Centre for Clinical Effectivenbess

http://www.monashhealth.org/page/Resources

Centre for Evidence Based Medicine

https://www.cebm.net/category/ebm-resources/tools/

Libraries in SWSLHD

https://www.swslhd.health.nsw.gov.au/liverpool/library/default.html

https://www.swslhd.health.nsw.gov.au/ccq/library/

https://www.swslhd.health.nsw.gov.au/Fairfield/library/

https://www.swslhd.health.nsw.gov.au/bankstown/library/

Useful Information

Research questions examples

Useful resources to download

Help us improve!

Would you please take a moment to answer these 2 quick questions, to what extent did this resource increase your understanding of the topic, to what extent did this resource help you find further information on this topic , wow this is awesome.

Many people are getting problem in using bootstrap modal, so here I'm with custom modal which you can use anywhere.

100% of your donation goes to medical research

Setting a research question, aim and objective

Affiliations.

• 1 Department of Nursing and Midwifery, University of Limerick, Limerick, Republic of Ireland.
• 2 Department of Nursing and Midwifery, University of Limerick, Republic of Ireland.
• PMID: 26997231
• DOI: 10.7748/nr.23.4.19.s5

Aim: To describe the development of a research question, aim and objective.

Background: The first steps of any study are developing the research question, aim and objective. Subsequent steps develop from these and they govern the researchers' choice of population, setting, data to be collected and time period for the study. Clear, succinctly posed research questions, aims and objectives are essential if studies are to be successful.

Discussion: Researchers developing their research questions, aims and objectives generally experience difficulties. They are often overwhelmed trying to convert what they see as a relevant issue from practice into research. This necessitates engaging with the relevant published literature and knowledgeable people.

Conclusion: This paper identifies the issues to be considered when developing a research question, aim and objective. Understanding these considerations will enable researchers to effectively present their research question, aim and objective.

Implications for practice: To conduct successful studies, researchers should develop clear research questions, aims and objectives.

Keywords: novice researchers; nursing research; research aim; research objective; research question; study development.

• Nursing Research / methods
• Nursing Research / organization & administration*
• Organizational Objectives

• Privacy Policy

Buy Me a Coffee

Home » Purpose of Research – Objectives and Applications

Purpose of Research – Objectives and Applications

Table of Contents

Purpose of Research

Definition:

The purpose of research is to systematically investigate and gather information on a particular topic or issue, with the aim of answering questions, solving problems, or advancing knowledge.

The purpose of research can vary depending on the field of study, the research question, and the intended audience. In general, research can be used to:

• Generate new knowledge and theories
• Test existing theories or hypotheses
• Identify trends or patterns
• Gather information for decision-making
• Evaluate the effectiveness of programs, policies, or interventions
• Develop new technologies or products
• Identify new opportunities or areas for further study.

Objectives of Research

The objectives of research may vary depending on the field of study and the specific research question being investigated. However, some common objectives of research include:

• To explore and describe a phenomenon: Research can be conducted to describe and understand a phenomenon or situation in greater detail.
• To test a hypothesis or theory : Research can be used to test a specific hypothesis or theory by collecting and analyzing data.
• To identify patterns or trends: Research can be conducted to identify patterns or trends in data, which can provide insights into the behavior of a system or population.
• To evaluate a program or intervention: Research can be used to evaluate the effectiveness of a program or intervention, such as a new drug or educational intervention.
• To develop new knowledge or technology : Research can be conducted to develop new knowledge or technologies that can be applied to solve practical problems.
• To inform policy decisions: Research can provide evidence to inform policy decisions and improve public policy.
• To improve existing knowledge: Research can be conducted to improve existing knowledge and fill gaps in the current understanding of a topic.

Applications of Research

Research has a wide range of applications across various fields and industries. Here are some examples:

• Medicine : Research is critical in developing new treatments and drugs for diseases. Researchers conduct clinical trials to test the safety and efficacy of new medications and therapies. They also study the underlying causes of diseases to find new ways to prevent or treat them.
• Technology : Research is crucial in developing new technologies and improving existing ones. Researchers work to develop new software, hardware, and other technological innovations that can be used in various industries such as healthcare, manufacturing, and telecommunications.
• Education : Research is essential in the field of education to develop new teaching methods and strategies. Researchers conduct studies to determine the effectiveness of various educational approaches and to identify factors that influence student learning.
• Business : Research is critical in helping businesses make informed decisions. Market research can help businesses understand their target audience and identify trends in the market. Research can also help businesses improve their products and services.
• Environmental Science : Research is crucial in the field of environmental science to understand the impact of human activities on the environment. Researchers conduct studies to identify ways to reduce pollution, protect natural resources, and mitigate the effects of climate change.

Goal of Research

The ultimate goal of research is to advance our understanding of the world and to contribute to the development of new theories, ideas, and technologies that can be used to improve our lives. Some more common Goals are follows:

• Explore and discover new knowledge : Research can help uncover new information and insights that were previously unknown.
• Test hypotheses and theories : Research can be used to test and validate theories and hypotheses, allowing researchers to refine and develop their ideas.
• Solve practical problems: Research can be used to identify solutions to real-world problems and to inform policy and decision-making.
• Improve understanding : Research can help improve our understanding of complex phenomena and systems, such as the human body, the natural world, and social systems.
• Develop new technologies and innovations : Research can lead to the development of new technologies, products, and innovations that can improve our lives and society.
• Contribute to the development of academic fields : Research can help advance academic fields by expanding our knowledge and understanding of important topics and areas of inquiry.

Importance of Research

The importance of research lies in its ability to generate new knowledge and insights, to test existing theories and ideas, and to solve practical problems.

Some of the key reasons why research is important are:

• Advancing knowledge: Research is essential for advancing knowledge and understanding in various fields. It enables us to explore and discover new concepts, ideas, and phenomena that can contribute to scientific and technological progress.
• Solving problems : Research can help identify and solve practical problems and challenges in various domains, such as health care, agriculture, engineering, and social policy.
• Innovation : Research is a critical driver of innovation, as it enables the development of new products, services, and technologies that can improve people’s lives and contribute to economic growth.
• Evidence-based decision-making : Research provides evidence and data that can inform decision-making in various fields, such as policy-making, business strategy, and healthcare.
• Personal and professional development : Engaging in research can also contribute to personal and professional development, as it requires critical thinking, problem-solving, and communication skills.

When to use Research

Research should be used in situations where there is a need to gather new information, test existing theories, or solve problems. Some common scenarios where research is often used include:

• Scientific inquiry : Research is essential for advancing scientific knowledge and understanding, and for exploring new concepts, theories, and phenomena.
• Business and market analysis: Research is critical for businesses to gather data and insights about the market, customer preferences, and competition, to inform decision-making and strategy development.
• Social policy and public administration: Research is often used in social policy and public administration to evaluate the effectiveness of programs and policies, and to identify areas where improvements are needed.
• Healthcare: Research is essential in healthcare to develop new treatments, improve existing ones, and to understand the causes and mechanisms of diseases.
• Education : Research is critical in education to evaluate the effectiveness of teaching methods and programs, and to develop new approaches to learning.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

This paper is in the following e-collection/theme issue:

Published on 17.4.2024 in Vol 26 (2024)

Digital Interventions for Recreational Cannabis Use Among Young Adults: Systematic Review, Meta-Analysis, and Behavior Change Technique Analysis of Randomized Controlled Studies

Authors of this article:

• José Côté 1, 2, 3 , RN, PhD   ; 
• Gabrielle Chicoine 3, 4 , RN, PhD   ; 
• Billy Vinette 1, 3 , RN, MSN   ; 
• Patricia Auger 2, 3 , MSc   ; 
• Geneviève Rouleau 3, 5, 6 , RN, PhD   ; 
• Guillaume Fontaine 7, 8, 9 , RN, PhD   ; 
• Didier Jutras-Aswad 2, 10 , MSc, MD  

1 Faculty of Nursing, Université de Montréal, Montreal, QC, Canada

2 Research Centre of the Centre Hospitalier de l’Université de Montréal, Montreal, QC, Canada

3 Research Chair in Innovative Nursing Practices, Montreal, QC, Canada

4 Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada

5 Department of Nursing, Université du Québec en Outaouais, Saint-Jérôme, QC, Canada

6 Women's College Hospital Institute for Health System Solutions and Virtual Care, Women's College Hospital, Toronto, ON, Canada

7 Ingram School of Nursing, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada

8 Centre for Clinical Epidemiology, Lady Davis Institute for Medical Research, Sir Mortimer B. Davis Jewish General Hospital, Montreal, QC, Canada

9 Kirby Institute, University of New South Wales, Sydney, Australia

10 Department of Psychiatry and Addictology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada

Corresponding Author:

José Côté, RN, PhD

Research Centre of the Centre Hospitalier de l’Université de Montréal

850 Saint-Denis

Montreal, QC, H2X 0A9

Phone: 1 514 890 8000

Email: [email protected]

Background: The high prevalence of cannabis use among young adults poses substantial global health concerns due to the associated acute and long-term health and psychosocial risks. Digital modalities, including websites, digital platforms, and mobile apps, have emerged as promising tools to enhance the accessibility and availability of evidence-based interventions for young adults for cannabis use. However, existing reviews do not consider young adults specifically, combine cannabis-related outcomes with those of many other substances in their meta-analytical results, and do not solely target interventions for cannabis use.

Objective: We aimed to evaluate the effectiveness and active ingredients of digital interventions designed specifically for cannabis use among young adults living in the community.

Methods: We conducted a systematic search of 7 databases for empirical studies published between database inception and February 13, 2023, assessing the following outcomes: cannabis use (frequency, quantity, or both) and cannabis-related negative consequences. The reference lists of included studies were consulted, and forward citation searching was also conducted. We included randomized studies assessing web- or mobile-based interventions that included a comparator or control group. Studies were excluded if they targeted other substance use (eg, alcohol), did not report cannabis use separately as an outcome, did not include young adults (aged 16-35 y), had unpublished data, were delivered via teleconference through mobile phones and computers or in a hospital-based setting, or involved people with mental health disorders or substance use disorders or dependence. Data were independently extracted by 2 reviewers using a pilot-tested extraction form. Authors were contacted to clarify study details and obtain additional data. The characteristics of the included studies, study participants, digital interventions, and their comparators were summarized. Meta-analysis results were combined using a random-effects model and pooled as standardized mean differences.

Results: Of 6606 unique records, 19 (0.29%) were included (n=6710 participants). Half (9/19, 47%) of these articles reported an intervention effect on cannabis use frequency. The digital interventions included in the review were mostly web-based. A total of 184 behavior change techniques were identified across the interventions (range 5-19), and feedback on behavior was the most frequently used (17/19, 89%). Digital interventions for young adults reduced cannabis use frequency at the 3-month follow-up compared to control conditions (including passive and active controls) by −6.79 days of use in the previous month (95% CI −9.59 to −4.00; P <.001).

Conclusions: Our results indicate the potential of digital interventions to reduce cannabis use in young adults but raise important questions about what optimal exposure dose could be more effective, both in terms of intervention duration and frequency. Further high-quality research is still needed to investigate the effects of digital interventions on cannabis use among young adults.

Trial Registration: PROSPERO CRD42020196959; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=196959

Introduction

Cannabis use among young adults is recognized as a public health concern.

Young adulthood (typically the ages of 18-30 y) is a critical developmental stage characterized by a peak prevalence of substance use [ 1 , 2 ]. Worldwide, cannabis is a substance frequently used for nonmedical purposes due in part to its high availability in some regions and enhanced product variety and potency [ 3 , 4 ]. The prevalence of cannabis use (CU) among young adults is high [ 5 , 6 ], and its rates have risen in recent decades [ 7 ]. In North America and Oceania, the estimated past-year prevalence of CU is ≥25% among young adults [ 8 , 9 ].

While the vast majority of cannabis users do not experience severe problems from their use [ 4 ], the high prevalence of CU among young adults poses substantial global health concerns due to the associated acute and long-term health and psychosocial risks [ 10 , 11 ]. These include impairment of cognitive function, memory, and psychomotor skills during acute intoxication; increased engagement in behaviors with a potential for injury and fatality (eg, driving under the influence); socioeconomic problems; and diminished social functioning [ 4 , 12 - 14 ]. Importantly, an extensive body of literature reveals that subgroups engaging in higher-risk use, such as intensive or repeated use, are more prone to severe and chronic consequences, including physical ailments (eg, respiratory illness and reproductive dysfunction), mental health disorders (eg, psychosis, depression, and suicidal ideation or attempts), and the potential development of CU disorder [ 4 , 15 - 17 ].

Interventions to Reduce Public Health Impact of Young Adult CU

Given the increased prevalence of lifetime and daily CU among young adults and the potential negative impact of higher-risk CU, various prevention and intervention programs have been implemented to help users reduce or cease their CU. These programs primarily target young adults regardless of their CU status [ 2 , 18 ]. In this context, many health care organizations and international expert panels have developed evidence-based lower-risk CU guidelines to promote safer CU and intervention options to help reduce risks of adverse health outcomes from nonmedical CU [ 4 , 16 , 17 , 19 ]. Lower-risk guidance-oriented interventions for CU are based on concepts of health promotion [ 20 - 22 ] and health behavior change [ 23 - 26 ] and on other similar harm reduction interventions implemented in other areas of population health (eg, lower-risk drinking guidelines, supervised consumption sites and services, and sexual health) [ 27 , 28 ]. These interventions primarily aim to raise awareness of negative mental, physical, and social cannabis-related consequences to modify individual-level behavior-related risk factors.

Meta-analyses have shown that face-to-face prevention and treatment interventions are generally effective in reducing CU in young adults [ 18 , 29 - 32 ]. However, as the proportion of professional help seeking for CU concerns among young adults remains low (approximately 15%) [ 33 , 34 ], alternative strategies that consider the limited capacities and access-related barriers of traditional face-to-face prevention and treatment facilities are needed. Digital interventions, including websites, digital platforms, and mobile apps, have emerged as promising tools to enhance the accessibility and availability of evidence-based programs for young adult cannabis users. These interventions address barriers such as long-distance travel, concerns about confidentiality, stigma associated with seeking treatment, and the cost of traditional treatments [ 35 - 37 ]. By overcoming these barriers, digital interventions have the potential to have a stronger public health impact [ 18 , 38 ].

State of Knowledge of Digital Interventions for CU and Young Adults

The literature regarding digital interventions for substance use has grown rapidly in the past decade, as evidenced by several systematic reviews and meta-analyses of randomized controlled trial (RCT) studies on the efficacy or effectiveness of these interventions in preventing or reducing harmful substance use [ 2 , 39 - 41 ]. However, these reviews do not focus on young adults specifically. In addition, they combine CU-related outcomes with those of many other substances in their meta-analytical results. Finally, they do not target CU interventions exclusively.

In total, 4 systematic reviews and meta-analyses of digital interventions for CU among young people have reported mixed results [ 42 - 45 ]. In their systematic review (10 studies of 5 prevention and 5 treatment interventions up to 2012), Tait et al [ 44 ] concluded that digital interventions effectively reduced CU among adolescents and adults at the posttreatment time point. Olmos et al [ 43 ] reached a similar conclusion in their meta-analysis of 9 RCT studies (2 prevention and 7 treatment interventions). In their review, Hoch et al [ 42 ] reported evidence of small effects at the 3-month follow-up based on 4 RCTs of brief motivational interventions and cognitive behavioral therapy (CBT) delivered on the web. In another systematic review and meta-analysis, Beneria et al [ 45 ] found that web-based CU interventions did not significantly reduce consumption. However, these authors indicated that the programs tested varied significantly across the studies considered and that statistical heterogeneity was attributable to the inclusion of studies of programs targeting more than one substance (eg, alcohol and cannabis) and both adolescents and young adults. Beneria et al [ 45 ] recommend that future work “establish the effectiveness of the newer generation of interventions as well as the key ingredients” of effective digital interventions addressing CU by young people. This is of particular importance because behavior change interventions tend to be complex as they consist of multiple interactive components [ 46 ].

Behavior change interventions refer to “coordinated sets of activities designed to change specified behavior patterns” [ 47 ]. Their interacting active ingredients can be conceptualized as behavior change techniques (BCTs) [ 48 ]. BCTs are specific and irreducible. Each BCT has its own individual label and definition, which can be used when designing and reporting complex interventions and as a nomenclature system when coding interventions for their content [ 47 ]. The Behavior Change Technique Taxonomy version 1 (BCTTv1) [ 48 , 49 ] was developed to provide a shared, standardized terminology for characterizing complex behavior change interventions and their active ingredients. Several systematic reviews with meta-regressions that used the BCTTv1 have found interventions with certain BCTs to be more effective than those without [ 50 - 53 ]. A better understanding of the BCTs used in digital interventions for young adult cannabis users would help not only to establish the key ingredients of such interventions but also develop and evaluate effective interventions.

In the absence of any systematic review of the effectiveness and active ingredients of digital interventions designed specifically for CU among community-living young adults, we set out to achieve the following:

• conduct a comprehensive review of digital interventions for preventing, reducing, or ceasing CU among community-living young adults,
• describe the active ingredients (ie, BCTs) in these interventions from the perspective of behavior change science, and
• analyze the effectiveness of these interventions on CU outcomes.

Protocol Registration

We followed the Cochrane Handbook for Systematic Reviews of Interventions [ 54 ] in designing this systematic review and meta-analysis and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines in reporting our findings (see Multimedia Appendix 1 [ 55 ] for the complete PRISMA checklist). This review was registered in PROSPERO (CRD42020196959).

Search Strategy

The search strategy was designed by a health information specialist together with the research team and peer reviewed by another senior information specialist before execution using Peer Review of Electronic Search Strategies for systematic reviews [ 56 ]. The search strategy revolved around three concepts:

• CU (eg, “cannabis,” “marijuana,” and “hashish”)
• Digital interventions (eg, “telehealth,” “website,” “mobile applications,” and “computer”)
• Young adults (eg, “emerging adults” and “students”)

The strategy was initially implemented on March 18, 2020, and again on October 13, 2021, and February 13, 2023. The full, detailed search strategies for each database are presented in Multimedia Appendix 2 .

Information Sources

We searched 7 electronic databases of published literature: CINAHL Complete, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Embase, MEDLINE, PubMed, and PsycINFO. No publication date filters or language restrictions were applied. A combination of free-text keywords and Medical Subject Headings was tailored to the conventions of each database for optimal electronic searching. The research team also manually screened the reference lists of the included articles and the bibliographies of existing systematic reviews [ 18 , 31 , 42 - 45 ] to identify additional relevant studies (snowballing). Finally, a forward citation tracking procedure (ie, searching for articles that cited the included studies) was carried out in Google Scholar.

Inclusion Criteria

The population, intervention, comparison, outcome, and study design process is presented in Multimedia Appendix 3 . The inclusion criteria were as follows: (1) original research articles published in peer-reviewed journals; (2) use of an experimental study design (eg, RCT, cluster RCT, or pilot RCT); (3) studies evaluating the effectiveness (or efficacy) of digital interventions designed specifically to prevent, reduce, or cease CU as well as promote CU self-management or address cannabis-related harm and having CU as an outcome measure; (4) studies targeting young adults, including active and nonactive cannabis users; (5) cannabis users and nonusers not under substance use treatment used as controls in comparator, waitlist, or delayed-treatment groups offered another type of intervention (eg, pharmacotherapy or psychosocial) different from the one being investigated or participants assessed only for CU; and (6) quantitative CU outcomes (frequency and quantity) or cannabis abstinence. Given the availability of numerous CU screening and assessment tools with adequate psychometric properties and the absence of a gold standard in this regard [ 57 ], any instrument capturing aspects of CU was considered. CU outcome measures could be subjective (eg, self-reported number of CU days or joints in the previous 3 months) or objective (eg, drug screening test). CU had to be measured before the intervention (baseline) and at least once after.

Digital CU interventions were defined as web- or mobile-based interventions that included one or more activities (eg, self-directed or interactive psychoeducation or therapy, personalized feedback, peer-to-peer contact, and patient-to-expert communication) aimed at changing CU [ 58 ]. Mobile-based interventions were defined as interventions delivered via mobile phone through SMS text message, multimedia messaging service (ie, SMS text messages that include　multimedia content, such as pictures, videos, or emojis), or mobile apps, whereas web-based interventions (eg, websites and digital platforms) were defined as interventions designed to be accessed on the web (ie, the internet), mainly via computers. Interventions could include self-directed and web-based interventions with human support. We defined young adults as aged 16 to 35 years and included students and nonstudents. While young adulthood is typically defined as covering the ages of 18 to 30 years [ 59 ], we broadened the range given that the age of majority and legal age to purchase cannabis differs across countries and jurisdictions. This was also in line with the age range targeted by several digital CU interventions (college or university students or emerging adults aged 15-24 years) [ 31 , 45 ]. Given the language expertise of the research team members and the available resources, only English- and French-language articles were retained.

Exclusion Criteria

Knowledge synthesis articles, study protocols, and discussion papers or editorials were excluded, as were articles with cross-sectional, cohort, case study or report, pretest-posttest, quasi-experimental, or qualitative designs. Mixed methods designs were included only if the quantitative component was an RCT. We excluded studies if (1) use of substances other than cannabis (eg, alcohol, opioids, or stimulants) was the focus of the digital intervention (though studies that included polysubstance users were retained if CU was assessed and reported separately); (2) CU was not reported separately as an outcome or only attitudes or beliefs regarding, knowledge of, intention to reduce, or readiness or motivation to change CU was measured; and (3) the data reported were unpublished (eg, conferences and dissertations). Studies of traditional face-to-face therapy delivered via teleconference on mobile phones and computers or in a hospital-based setting and informational campaigns (eg, web-based poster presentations or pamphlets) were excluded as well. Studies with samples with a maximum age of <15 years and a minimum age of >35 years were also excluded. Finally, we excluded studies that focused exclusively on people with a mental health disorder or substance use disorder or dependence or on adolescents owing to the particular health care needs of these populations, which may differ from those of young adults [ 1 ].

Data Collection

Selection of studies.

Duplicates were removed from the literature search results in EndNote (version X9.3.3; Clarivate Analytics) using the Bramer method for deduplication of database search results for systematic reviews [ 60 ]. The remaining records were uploaded to Covidence (Veritas Health Innovation), a web-based systematic review management system. A reviewer guide was developed that included screening questions and a detailed description of each inclusion and exclusion criterion based on PICO (population, intervention, comparator, and outcome), and a calibration exercise was performed before each stage of the selection process to maximize consistency between reviewers. Titles and abstracts of studies flagged for possible inclusion were screened first by 2 independent reviewers (GC, BV, PA, and GR; 2 per article) against the eligibility criteria (stage 1). Articles deemed eligible for full-text review were then retrieved and screened for inclusion (stage 2). Full texts were assessed in detail against the eligibility criteria again by 2 reviewers independently. Disagreements between reviewers were resolved through consensus or by consulting a third reviewer.

Data Extraction Process

In total, 2 reviewers (GC, BV, PA, GR, and GF; 2 per article) independently extracted relevant data (or informal evidence) using a data extraction form developed specifically for this review and integrated into Covidence. The form was pilot-tested on 2 randomly selected studies and refined accordingly. Data pertaining to the following domains were extracted from the included studies: (1) Study characteristics included information on the first and corresponding authors, publication year, country of origin, aims and hypotheses, study period, design (including details on randomization and blinding), follow-up times, data collection methods, and types of statistical analysis. (2) Participant characteristics included study target population, participant inclusion and exclusion criteria, sex or gender, mean age, and sample sizes at each data collection time point. (3) Intervention characteristics, for which the research team developed a matrix inspired by the template for intervention description and replication 12-item checklist [ 61 ] to extract informal evidence (ie, intervention descriptions) from the included studies under the headings name of intervention, purpose, underpinning theory of design elements, treatment approach, type of technology (ie, web or mobile) and software used, delivery format (ie, self-directed, human involvement, or both), provider characteristics (if applicable), intervention duration (ie, length of treatment and number of sessions or modules), material and procedures (ie, tools or activities offered, resources provided, and psychoeducational content), tailoring, and unplanned modifications. (4) Comparator characteristics were details of the control or comparison group or groups, including nature (passive vs active), number of groups or clusters (if applicable), type and length of the intervention (if applicable), and number of participants at each data collection time point. (5) Outcome variables, including the primary outcome variable examined in this systematic review, that is, the mean difference in CU frequency before and after the intervention and between the experimental and control or comparison groups. When possible, we examined continuous variables, including CU frequency means and SDs at the baseline and follow-up time points, and standardized regression coefficients (ie, β coefficients and associated 95% CIs). The secondary outcomes examined included other CU outcome variables (eg, quantity of cannabis used and abstinence) and cannabis-related negative consequences (or problems). Details on outcome variables (ie, definition, data time points, and missing data) and measurements (ie, instruments, measurement units, and scales) were also extracted.

In addition, data on user engagement and use of the digital intervention and study attrition rates (ie, dropouts and loss to follow-up) were extracted. When articles had missing data, we contacted the corresponding authors via email (2 attempts were made over a 2-month period) to obtain missing information. Disagreements over the extracted data were limited and resolved through discussion.

Data Synthesis Methods

Descriptive synthesis.

The characteristics of the included studies, study participants, interventions, and comparators were summarized in narrative and table formats. The template for intervention description and replication 12-item checklist [ 61 ] was used to summarize and organize intervention characteristics and assess to what extent the interventions were appropriately described in the included articles. As not all studies had usable data for meta-analysis purposes and because of heterogeneity, we summarized the main findings (ie, intervention effects) of the included studies in narrative and table formats for each outcome of interest in this review.

The BCTs used in the digital interventions were identified from the descriptions of the interventions (ie, experimental groups) provided in the articles as well as any supplementary material and previously published research protocols. A BCT was defined as “an observable, replicable, and irreducible component of an intervention designed to alter or redirect causal processes that regulate behavior” [ 48 ]. The target behavior in this review was the cessation or reduction of CU by young adults. BCTs were identified and coded using the BCTTv1 [ 48 , 49 ], a taxonomy of 93 BCTs organized into 16 hierarchical thematic clusters or categories. Applying the BCTTv1 in a systematic review allows for the comparison and synthesis of evidence across studies in a structured manner. This analysis allows for the identification of the explicit mechanisms underlying the reported behavior change induced by interventions, successful or not, and, thus, avoids making implicit assumptions about what works [ 62 ].

BCT coding was performed by 2 reviewers independently—BV coded all studies, and GC and GF coded a subset of the studies. All reviewers completed web-based training on the BCTTv1, and GF is an experienced implementation scientist who had used the BCTTv1 in prior work [ 63 - 65 ]. The descriptions of the interventions in the articles were read line by line and analyzed for the clear presence of BCTs using the guidelines developed by Michie et al [ 48 ]. For each article, the BCTs identified were documented and categorized using supporting textual evidence. They were coded only once per article regardless of how many times they came up in the text. Disagreements about including a BCT were resolved through discussion. If there was uncertainty about whether a BCT was present, it was coded as absent. Excel (Microsoft Corp) was used to compare the reviewers’ independent BCT coding and generate an overall descriptive synthesis of the BCTs identified. The BCTs were summarized by study and BCT cluster.

Statistical Analysis

Meta-analyses were conducted to estimate the size of the effect of the digital interventions for young adult CU on outcomes of interest at the posttreatment and follow-up assessments compared with control or alternative intervention conditions. The outcome variables considered were (1) CU frequency and other CU outcome variables (eg, quantity of cannabis used and abstinence) at baseline and the posttreatment time point or follow-up measured using standardized instruments of self-reported CU (eg, the timeline followback [TLFB] method) [ 66 ] and (2) cannabis-related negative consequences measured using standardized instruments (eg, the Marijuana Problems Scale) [ 67 ].

Under our systematic review protocol, ≥2 studies were needed for a meta-analysis. On the basis of previous systematic reviews and meta-analyses in the field of digital CU interventions [ 31 , 42 - 45 ], we expected between-study heterogeneity regarding outcome assessment. To minimize heterogeneity, we chose to pool studies with similar outcomes of interest based on four criteria: (1) definition of outcome (eg, CU frequency, quantity consumed, and abstinence), (2) type of outcome variable (eg, days of CU in the previous 90 days, days high per week in the previous 30 days, and number of CU events in the previous month) and measure (ie, instruments or scales), (3) use of validated instruments, and (4) posttreatment or follow-up time points (eg, 2 weeks or 1 month after the baseline or 3, 6, and 12 months after the baseline).

Only articles that reported sufficient statistics to compute a valid effect size with 95% CIs were included in the meta-analyses. In the case of articles that were not independent (ie, more than one published article reporting data from the same clinical trial), only 1 was included, and it was represented only once in the meta-analysis for a given outcome variable regardless of whether the data used to compute the effect size were extracted from the original paper or a secondary analysis paper. We made sure that the independence of the studies included in the meta-analysis of each outcome was respected. In the case of studies that had more than one comparator, we used the effect size for each comparison between the intervention and control groups.

Meta-analyses were conducted only for mean differences based on the change from baseline in CU frequency at 3 months after the baseline as measured using the number of self-reported days of use in the previous month. As the true value of the estimated effect size for outcome variables might vary across different trials and samples, we used a random-effects model given that the studies retained did not have identical target populations. The random-effects model incorporates between-study variation in the study weights and estimated effect size [ 68 ]. In addition, statistical heterogeneity across studies was assessed using I 2 , which measures the proportion of heterogeneity to the total observed dispersion; 25% was considered low, 50% was considered moderate, and 75% was considered high [ 69 ]. Because only 3 studies were included in the meta-analysis [ 70 - 72 ], publication bias could not be assessed. All analyses were completed using Stata (version 18; StataCorp) [ 73 ].

Risk-of-Bias Assessment

The risk of bias (RoB) of the included RCTs was assessed using the Cochrane RoB 2 tool at the outcome level [ 74 ]. Each distinct risk domain (ie, randomization process, deviations from the intended intervention, missing outcome data, measurement of the outcome, and selection of the reported results) was assessed as “low,” “some concerns,” or “high” based on the RoB 2 criteria. In total, 2 reviewers (GC and BV) conducted the assessments independently. Disagreements were discussed, and if not resolved consensually by the 2, the matter was left for a third reviewer (GF) to settle. The assessments were summarized by risk domain and outcome and converted into figures using the RoB visualization tool robvis [ 75 ].

Search Results

The database search generated a total of 13,232 citations, of which 7822 (59.11%) were from the initial search on March 18, 2020, and 2805 (21.2%) and 2605 (19.69%) were from the updates on October 13, 2021, and February 13, 2023, respectively. Figure 1 presents the PRISMA study flow diagram [ 76 ]. Of the 6606 unique records, 6484 (98.15%) were excluded based on title and abstract screening. Full texts of the remaining 1.85% (122/6606) of the records were examined, as were those of 25 more reports found through hand searching. Of these 147 records, 128 (87.1%) were excluded after 3 rounds of full-text screening. Of these 128 records, 39 (30.5%) were excluded for not being empirical research articles (eg, research protocols). Another 28.1% (36/128) were excluded for not meeting our definition of digital CU intervention. The remaining records were excluded for reasons that occurred with a frequency of ≤14%, including young adults not being the target population and the study not meeting our study design criteria (ie, RCT, cluster RCT, or pilot RCT). Excluded studies and reasons for exclusion are listed in Multimedia Appendix 4 . Finally, 19 articles detailing the results of 19 original studies were included.

Description of Studies

Study characteristics.

Multimedia Appendix 5 [ 70 - 72 , 77 - 92 ] describes the general characteristics of the 19 included studies. The studies were published between 2010 and 2023, with 58% (11/19) published in 2018 or later. A total of 53% (10/19) of the studies were conducted in the United States [ 77 - 86 ], 11% (2/19) were conducted in Canada [ 87 , 88 ], 11% (2/19) were conducted in Australia [ 71 , 89 ], 11% (2/19) were conducted in Germany [ 72 , 90 ], 11% (2/19) were conducted in Switzerland [ 70 , 91 ], and 5% (1/19) were conducted in Sweden [ 92 ]. A total of 79% (15/19) were RCTs [ 70 - 72 , 77 , 79 , 81 - 83 , 86 - 92 ], and 21% (4/19) were pilot RCTs [ 78 , 80 , 84 , 85 ].

Participant Characteristics

The studies enrolled a total of 6710 participants—3229 (48.1%) in the experimental groups, 3358 (50%) in the control groups, and the remaining 123 (1.8%) from 1 study [ 82 ] where participant allocation to the intervention condition was not reported. Baseline sample sizes ranged from 49 [ 81 ] to 1292 [ 72 ] (mean 352.89, SD 289.50), as shown in Multimedia Appendix 5 . Participant mean ages ranged from 18.03 (SD 0.31) [ 79 ] to 35.3 (SD 12.6) years [ 88 ], and the proportion of participants who identified as female ranged from 24.7% [ 91 ] to 84.1% [ 80 ].

Of the 19 included studies, 10 (53%) targeted adults aged ≥18 years, of which 7 (70%) studies focused on adults who had engaged in past-month CU [ 70 , 71 , 80 , 84 , 85 , 90 , 91 ], 2 (20%) studies included adults who wished to reduce or cease CU [ 72 , 89 ], and 1 (10%) study focused on noncollege adults with a moderate risk associated with CU [ 88 ]. Sinadinovic et al [ 92 ] targeted young adults aged ≥16 years who had used cannabis at least once a week in the previous 6 months. The remaining 8 studies targeted college or university students (aged ≥17 y) specifically, of which 7 (88%) studies focused solely on students who reported using cannabis [ 78 , 79 , 81 - 83 , 86 , 87 ] and 1 (12%) study focused solely on students who did not report past-month CU (ie, abstainers) [ 77 ].

Intervention Characteristics

The 19 included studies assessed nine different digital interventions: (1) 5 (26%) evaluated Marijuana eCHECKUP TO GO (e-TOKE), a commercially available electronic intervention used at colleges throughout the United States and Canada [ 77 , 78 , 81 - 83 ]; (2) 2 (11%) examined the internationally known CANreduce program [ 70 , 91 ]; (3) 2 (11%) evaluated the German Quit the Shit program [ 72 , 90 ]; (4) 2 (11%) assessed a social media–delivered, physical activity–focused cannabis intervention [ 84 , 85 ]; (5) 1 (5%) investigated the Swedish Cannabishjälpen intervention [ 92 ]; (6) 1 (5%) evaluated the Australian Grassessment: Evaluate Your Use of Cannabis website program [ 89 ]; (7) 1 (5%) assessed the Canadian Ma réussite, mon choix intervention [ 87 ]; (8) 1 (5%) examined the Australian Reduce Your Use: How to Break the Cannabis Habit program [ 71 ]; and (9) 4 (21%) each evaluated a unique no-name intervention described as a personalized feedback intervention (PFI) [ 79 , 80 , 86 , 88 ]. Detailed information regarding the characteristics of all interventions as reported in each included study is provided in Multimedia Appendix 6 [ 70 - 72 , 77 - 113 ] and summarized in the following paragraphs.

In several studies (8/19, 42%), the interventions were designed to support cannabis users in reducing or ceasing their consumption [ 70 , 72 , 80 , 87 , 89 - 92 ]. In 37% (7/19) of the studies, the interventions aimed at reducing both CU and cannabis-related consequences [ 79 , 81 - 85 , 88 ]. Other interventions focused on helping college students think carefully about the decision to use cannabis [ 77 , 78 ] and on reducing either cannabis-related problems among undergraduate students [ 86 ] or symptoms associated with CU disorder in young adults [ 71 ].

In 26% (5/19) of the studies, theory was used to inform intervention design along with a clear rationale for theory use. Of these 5 articles, only 1 (20%) [ 87 ] reported using a single theory of behavior change, the theory of planned behavior [ 114 ]. A total of 21% (4/19) of the studies selected only constructs of theories (or models) for their intervention design. Of these 4 studies, 2 (50%) evaluated the same intervention [ 72 , 90 ], which focused on principles of self-regulation and self-control theory [ 93 ]; 1 (25%) [ 70 ] used the concept of adherence-focused guidance enhancement based on the supportive accountability model of guidance [ 94 ]; and 1 (25%) [ 71 ] reported that intervention design was guided by the concept of self-behavioral management.

The strategies (or approaches) used in the delivery of the digital interventions were discussed in greater detail in 84% (16/19) of the articles [ 70 - 72 , 79 - 81 , 83 - 92 ]. Many of these articles (9/19, 47%) reported using a combination of approaches based on CBT or motivational interviewing (MI) [ 70 , 71 , 79 , 83 - 85 , 90 - 92 ]. PFIs were also often mentioned as an approach to inform intervention delivery [ 7 , 71 , 79 , 86 - 88 ].

More than half (13/19, 68%) of all the digital interventions were asynchronous and based on a self-guided approach without support from a counselor or therapist. The study by Côté et al [ 87 ] evaluated the efficacy of a web-based tailored intervention focused on reinforcing a positive attitude toward and a sense of control over cannabis abstinence through psychoeducational messages delivered by a credible character in short video clips and personalized reinforcement messages. Lee et al [ 79 ] evaluated a brief, web-based personalized feedback selective intervention based on the PFI approach pioneered by Marlatt et al [ 95 ] for alcohol use prevention and on the MI approach described by Miller and Rollnick [ 96 ]. Similarly, Rooke et al [ 71 ] combined principles of MI and CBT to develop a web-based intervention delivered via web modules, which were informed by previous automated feedback interventions targeting substance use. The study by Copeland et al [ 89 ] assessed the short-term effectiveness of Grassessment: Evaluate Your Use of Cannabis, a brief web-based, self-complete intervention based on motivational enhancement therapy that included personalized feedback messages and psychoeducational material. In the studies by Buckner et al [ 80 ], Cunningham et al [ 88 ], and Walukevich-Dienst et al [ 86 ], experimental groups received a brief web-based PFI available via a computer. A total of 16% (3/19) of the studies [ 77 , 78 , 82 ] applied a program called the Marijuana eCHECKUP TO GO (e-TOKE) for Universities and Colleges, which was presented as a web-based, norm-correcting, brief preventive and intervention education program designed to prompt self-reflection on consequences and consideration of decreasing CU among students. Riggs et al [ 83 ] developed and evaluated an adapted version of e-TOKE that provided participants with university-specific personalized feedback and normative information based on protective behavioral strategies for CU [ 97 ]. Similarly, Goodness and Palfai [ 81 ] tested the efficacy of eCHECKUP TO GO-cannabis, a modified version of e-TOKE combining personalized feedback, norm correction, and a harm and frequency reduction strategy where a “booster” session was provided at 3 months to allow participants to receive repeated exposure to the intervention.

In the remaining 32% (6/19) of the studies, which examined 4 different interventions, the presence of a therapist guide was reported. The intervention evaluated by Sinadinovic et al [ 92 ] combined principles of psychoeducation, MI, and CBT organized into 13 web-based modules and a calendar involving therapist guidance, recommendations, and personal feedback. In total, 33% (2/6) of these studies evaluated a social media–delivered intervention with e-coaches that combined principles of MI and CBT and a harm reduction approach for risky CU [ 84 , 85 ]. Schaub et al [ 91 ] evaluated the efficacy of CANreduce, a web-based self-help intervention based on both MI and CBT approaches, using automated motivational and feedback emails, chat with a counselor, and web-based psychoeducational modules. Similarly, Baumgartner et al [ 70 ] investigated the effectiveness of CANreduce 2.0, a modified version of CANreduce, using semiautomated motivational and adherence-focused guidance-based email feedback with or without a personal online coach. The studies by Tossman et al [ 72 ] and Jonas et al [ 90 ] used a solution-focused approach and MI to evaluate the effectiveness of the German Quit the Shit web-based program that involves weekly feedback provided by counselors.

In addition to using different intervention strategies or approaches, the interventions were diverse in terms of the duration and frequency of the program (eg, web-based activities, sessions, or modules). Of the 12 articles that provided details in this regard, 2 (17%) on the same intervention described it as a brief 20- to 45-minute web-based program [ 77 , 78 ], 2 (17%) on 2 different interventions reported including 1 or 2 modules per week for a duration of 6 weeks [ 71 , 92 ], and 7 (58%) on 4 different interventions described them as being available over a longer period ranging from 6 weeks to 3 months [ 70 , 72 , 79 , 84 , 85 , 87 , 90 , 91 ].

Comparator Types

A total of 42% (8/19) of the studies [ 72 , 77 - 80 , 85 , 87 , 92 ] used a passive comparator only, namely, a waitlist control group ( Multimedia Appendix 5 ). A total of 26% (5/19) of the studies used an active comparator only where participants were provided with minimal general health feedback regarding recommended guidelines for sleep, exercise, and nutrition [ 81 , 82 ]; strategies for healthy stress management [ 83 ]; educational materials about risky CU [ 88 ]; or access to a website containing information about cannabis [ 71 ]. In another 21% (4/19) of the studies, which used an active comparator, participants received the same digital intervention minus a specific component: a personal web-based coach [ 70 ], extended personalized feedback [ 89 ], web-based chat counseling [ 91 ], or information on risks associated with CU [ 86 ]. A total of 21% (4/19) of the studies had more than one control group [ 70 , 84 , 90 , 91 ].

Outcome Variable Assessment and Summary of Main Findings of the Studies

The methodological characteristics and major findings of the included studies (N=19) are presented in Multimedia Appendix 7 [ 67 , 70 - 72 , 77 - 92 , 115 - 120 ] and summarized in the following sections for each outcome of interest in this review (ie, CU and cannabis-related consequences). Of the 19 studies, 11 (58%) were reported as efficacy trials [ 7 , 77 , 79 , 81 - 83 , 86 - 88 , 91 , 92 ], and 8 (42%) were reported as effectiveness trials [ 70 - 72 , 78 , 84 , 85 , 89 , 90 ].

Across all the included studies (19/19, 100%), participant attrition rates ranged from 1.6% at 1 month after the baseline [ 77 , 78 ] to 75.1% at the 3-month follow-up [ 70 ]. A total of 37% (7/19) of the studies assessed and reported results regarding user engagement [ 71 , 78 , 84 , 85 , 90 - 92 ] using different types of metrics. In one article on the Marijuana eCHECKUP TO GO (e-TOKE) web-based program [ 78 ], the authors briefly reported that participation was confirmed for 98.1% (158/161) of participants in the intervention group. In 11% (2/19) of the studies, which were on a similar social media–delivered intervention [ 84 , 85 ], user engagement was quantified by tallying the number of comments or posts and reactions (eg, likes and hearts) left by participants. In both studies [ 84 , 85 ], the intervention group, which involved a CU-related Facebook page, displayed greater interactions than the control groups, which involved a Facebook page unrelated to CU. One article [ 84 ] reported that 80% of participants in the intervention group posted at least once (range 0-60) and 50% posted at least weekly. In the other study [ 85 ], the results showed that intervention participants engaged (ie, posting or commenting or clicking reactions) on average 47.9 times each over 8 weeks. In total, 11% (2/19) of the studies [ 90 , 91 ] on 2 different web-based intervention programs, both consisting of web documentation accompanied by chat-based counseling, measured user engagement either by average duration or average number of chat sessions. Finally, 16% (3/19) of the studies [ 71 , 91 , 92 ], which involved 3 different web-based intervention programs, characterized user engagement by the mean number of web modules completed per participant. Overall, the mean number of web modules completed reported in these articles was quite similar: 3.9 out of 13 [ 92 ] and 3.2 [ 91 ] and 3.5 [ 71 ] out of 6.

Assessment of CU

As presented in Multimedia Appendix 7 , the included studies differed in terms of how they assessed CU, although all used at least one self-reported measure of frequency. Most studies (16/19, 84%) measured frequency by days of use, including days of use in the preceding week [ 91 ] or 2 [ 80 ], days of use in the previous 30 [ 70 - 72 , 78 , 84 - 86 , 88 - 90 ] or 90 days [ 79 , 81 , 82 ], and days high per week [ 83 ]. Other self-reported measures of CU frequency included (1) number of CU events in the previous month [ 87 , 90 ], (2) cannabis initiation or use in the previous month (ie, yes or no) [ 77 ], and (3) days without CU in the previous 7 days [ 92 ]. In addition to measuring CU frequency, 42% (8/19) of the studies also assessed CU via self-reported measures of quantity used, including estimated grams consumed in the previous week [ 92 ] or 30 days [ 72 , 85 , 90 ] and the number of standard-sized joints consumed in the previous 7 days [ 91 ] or the previous month [ 70 , 71 , 89 ].

Of the 19 articles included, 10 (53%) [ 70 - 72 , 80 , 84 - 86 , 89 , 90 , 92 ] reported using a validated instrument to measure CU frequency or quantity, including the TLFB instrument [ 66 ] (n=9, 90% of the studies) and the Marijuana Use Form (n=1, 10% of the studies); 1 (10%) [ 79 ] reported using CU-related questions from an adaptation of the Global Appraisal of Individual Needs–Initial instrument [ 115 ]; and 30% (3/10) [ 81 , 82 , 91 ] reported using a questionnaire accompanied by a calendar or a diary of consumption. The 19 studies also differed with regard to their follow-up time measurements for assessing CU, ranging from 2 weeks after the baseline [ 80 ] to 12 months after randomization [ 90 ], although 12 (63%) of the studies included a 3-month follow-up assessment [ 70 - 72 , 79 , 81 , 82 , 84 , 85 , 88 , 90 - 92 ].

Of all studies assessing and reporting change in CU frequency from baseline to follow-up assessments (19/19, 100%), 47% (9/19) found statistically significant differences between the experimental and control groups [ 70 - 72 , 80 , 81 , 83 , 85 , 87 , 91 ]. Importantly, 67% (6/9) of these studies showed that participants in the experimental groups exhibited greater decreases in CU frequency 3 months following the baseline assessment compared with participants in the control groups [ 70 - 72 , 81 , 85 , 91 ], 22% (2/9) of the studies showed greater decreases in CU frequency at 6 weeks after the baseline assessment [ 71 , 83 ], 22% (2/9) of the studies showed greater decreases in CU frequency at 6 months following the baseline assessment [ 81 , 85 ], 11% (1/9) of the studies showed greater decreases in CU frequency at 2 weeks after the baseline [ 80 ], and 11% (1/9) of the studies showed greater decreases in CU frequency at 2 months after treatment [ 87 ].

In the study by Baumgartner et al [ 70 ], a reduction in CU days was observed in all groups, but the authors reported that the difference was statistically significant only between the intervention group with the service team and the control group (the reduction in the intervention group with social presence was not significant). In the study by Bonar et al [ 85 ], the only statistically significant difference between the intervention and control groups at the 3- and 6-month follow-ups involved total days of cannabis vaping in the previous 30 days. Finally, in the study by Buckner et al [ 80 ], the intervention group had less CU than the control group 2 weeks after the baseline; however, this was statistically significant only for participants with moderate or high levels of social anxiety.

Assessment of Cannabis-Related Negative Consequences

A total of 53% (10/19) of the studies also assessed cannabis-related negative consequences [ 78 - 84 , 86 , 88 , 92 ]. Of these 10 articles, 8 (80%) reported using a validated self-report instrument: 4 (50%) [ 81 , 82 , 86 , 88 ] used the 19-item Marijuana Problems Scale [ 67 ], 2 (25%) [ 78 , 79 ] used the 18-item Rutgers Marijuana Problem Index [ 121 , 122 ], and 2 (25%) [ 80 , 84 ] used the Brief Marijuana Consequences Questionnaire [ 116 ]. Only 10% (1/10) of the studies [ 92 ] used a screening tool, the Cannabis Abuse Screening Test [ 117 , 118 ]. None of these 10 studies demonstrated a statistically significant difference between the intervention and control groups. Of note, Walukevich-Dienst et al [ 86 ] found that women (but not men) who received an web-based PFI with additional information on CU risks reported significantly fewer cannabis-related problems than did women in the control group at 1 month after the intervention ( B =−1.941; P =.01).

Descriptive Summary of BCTs Used in Intervention Groups

After the 19 studies included in this review were coded, a total of 184 individual BCTs targeting CU in young adults were identified. Of these 184 BCTs, 133 (72.3% ) were deemed to be present beyond a reasonable doubt, and 51 (27.7%) were deemed to be present in all probability. Multimedia Appendix 8 [ 48 , 70 - 72 , 77 - 92 ] presents all the BCTs coded for each included study summarized by individual BCT and BCT cluster.

The 184 individual BCTs coded covered 38% (35/93) of the BCTs listed in the BCTTv1 [ 48 ]. The number of individual BCTs identified per study ranged from 5 to 19, with two-thirds of the 19 studies (12/19, 63%) using ≤9 BCTs (mean 9.68). As Multimedia Appendix 8 shows, at least one BCT fell into 13 of the 16 possible BCT clusters. The most frequent clusters were feedback monitoring , natural consequences , goal planning , and comparison of outcomes .

The most frequently coded BCTs were (1) feedback on behavior (BCT 2.2; 17/19, 89% of the studies; eg, “Once a week, participants receive detailed feedback by their counselor on their entries in diary and exercises. Depending on the involvement of each participant, up to seven feedbacks are given” [ 90 ]), (2) social support (unspecified) (BCT 3.1; 15/19, 79% of the studies; eg, “The website also features [...] blogs from former cannabis users, quick assist links, and weekly automatically generated encouragement emails” [ 71 ]), and (3) pros and cons (BCT 9.2; 14/19, 74% of the studies; eg, “participants are encouraged to state their personal reasons for and against their cannabis consumption, which they can review at any time, so they may reflect on what they could gain by successfully completing the program” [ 70 ]). Other commonly identified BCTs included social comparison (BCT 6.2; 12/19, 63% of the studies) and information about social and environmental consequences (BCT 5.3; 11/19, 58% of the studies), followed by problem solving (BCT 2.1; 10/19, 53% of the studies) and information about health consequences (BCT 5.1; 10/19, 53% of the studies).

RoB Assessment

Figure 2 presents the overall assessment of risk in each domain for all the included studies, whereas Figure 3 [ 70 - 72 , 77 - 92 ] summarizes the assessment of each study at the outcome level for each domain in the Cochrane RoB 2 [ 74 ].

Figure 2 shows that, of the included studies, 93% (27/29) were rated as having a “low” RoB arising from the randomization process (ie, selection bias) and 83% (24/29) were rated as having a “low” RoB due to missing data (ie, attrition bias). For bias due to deviations from the intended intervention (ie, performance bias), 72% (21/29) were rated as having a “low” risk, and for selective reporting of results, 59% (17/29) were rated as having a “low” risk. In the remaining domain regarding bias in measurement of the outcome (ie, detection bias), 48% (14/29) of the studies were deemed to present “some concerns,” mainly owing to the outcome assessment not being blinded (eg, self-reported outcome measure of CU). Finally, 79% (15/19) of the included studies were deemed to present “some concerns” or were rated as having a “high” RoB at the outcome level ( Figure 3 [ 70 - 72 , 77 - 92 ]). The RoB assessment for CU and cannabis consequences of each included study is presented in Multimedia Appendix 9 [ 70 - 72 , 77 - 92 ].

Meta-Analysis Results

Due to several missing data points and despite contacting the authors, we were able to carry out only 1 meta-analysis of our primary outcome, CU frequency. Usable data were retrieved from only 16% (3/19) [ 70 - 72 ] of the studies included in this review. These 3 studies provided sufficient information to calculate an effect size, including mean differences based on change-from-baseline measurements and associated 95% CIs (or SE of the mean difference) and sample sizes per intervention and comparison conditions. The reasons for excluding the other 84% (16/19) of the studies included heterogeneity in outcome variables or measurements, inconsistent results, and missing data ( Multimedia Appendix 10 [ 77 - 92 ]).

Figure 4 [ 70 - 72 ] illustrates the mean differences and associated 95% CIs of 3 unique RCTs [ 70 - 72 ] that provided sufficient information to allow for the measurement of CU frequency at 3 months after the baseline relative to a comparison condition in terms of the number of self-reported days of use in the previous month using the TLFB method. Overall, the synthesized effect of digital interventions for young adult cannabis users on CU frequency, as measured using days of use in the previous month, was −6.79 (95% CI −9.59 to −4.00). This suggests that digital CU interventions had a statistically significant effect ( P <.001) on reducing CU frequency at the 3-month follow-up compared with the control conditions (both passive and active controls). The results of the meta-analysis also showed low between-study heterogeneity ( I 2 =48.3%; P =.12) across the 3 included studies.

The samples of the 3 studies included in the meta-analysis varied in size from 225 to 1292 participants (mean 697.33, SD 444.11), and the mean age ranged from 24.7 to 31.88 years (mean 26.38, SD 3.58 years). These studies involved 3 different digital interventions and used different design approaches to assess intervention effectiveness. One study assessed the effectiveness of a web-based counseling program (ie, Quit the Shit) against a waitlist control [ 72 ], another examined the effectiveness of a fully self-guided web-based treatment program for CU and related problems (ie, Reduce Your Use: How to Break the Cannabis Habit) against a control condition website consisting of basic educational information on cannabis [ 71 ], and the third used a 3-arm RCT design to investigate whether the effectiveness of a minimally guided internet-based self-help intervention (ie, CANreduce 2.0) might be enhanced by implementing adherence-focused guidance and emphasizing the social presence factor of a personal e-coach [ 70 ].

Summary of Principal Findings

The primary aim of this systematic review was to evaluate the effectiveness of digital interventions in addressing CU among community-living young adults. We included 19 randomized controlled studies representing 9 unique digital interventions aimed at preventing, reducing, or ceasing CU and evaluated the effects of 3 different digital interventions on CU. In summary, the 3 digital interventions included in the meta-analysis proved superior to control conditions in reducing the number of days of CU in the previous month at the 3-month follow-up.

Our findings are consistent with those of 2 previous meta-analyses by Olmos et al [ 43 ] and Tait et al [ 44 ] and with the findings of a recently published umbrella review of systematic reviews and meta-analyses of RCTs [ 123 ], all of which revealed a positive effect of internet- and computer-based interventions on CU. However, a recent systematic review and meta-analysis by Beneria et al [ 45 ] found that web-based CU interventions did not significantly reduce CU. Beneria et al [ 45 ] included studies with different intervention programs that targeted diverse population groups (both adolescents and young adults) and use of more than one substance (eg, alcohol and cannabis). In our systematic review, a more conservative approach was taken—we focused specifically on young adults and considered interventions targeting CU only. Although our results indicate that digital interventions hold great promise in terms of effectiveness, an important question that remains unresolved is whether there is an optimal exposure dose in terms of both duration and frequency that might be more effective. Among the studies included in this systematic review, interventions varied considerably in terms of the number of psychoeducational modules offered (from 2 to 13), time spent reviewing the material, and duration (from a single session to a 12-week spread period). Our results suggest that an intervention duration of at least 6 weeks yields better results.

Another important finding of this review is that, although almost half (9/19, 47%) of the included studies observed an intervention effect on CU frequency, none reported a statistically significant improvement in cannabis-related negative consequences, which may be considered a more distal indicator. More than half (10/19, 53%) of the included studies investigated this outcome. It seems normal to expect to find an effect on CU frequency given that reducing CU is often the primary objective of interventions and because the motivation of users’ is generally focused on changing consumption behavior. It is plausible to think that the change in behavior at the consumption level must be maintained over time before an effect on cannabis-related negative consequences can be observed. However, our results showed that, in all the included studies, cannabis-related negative consequences and change in behavior (CU frequency) were measured at the same time point, namely, 3 months after the baseline. Moreover, Grigsby et al [ 124 ] conducted a scoping review of risk and protective factors for CU and suggested that interventions to reduce negative CU consequences should prioritize multilevel methods or strategies “to attenuate the cumulative risk from a combination of psychological, contextual, and social influences.”

A secondary objective of this systematic review was to describe the active ingredients used in digital interventions for CU among young adults. The vast majority of the interventions were based on either a theory or an intervention approach derived from theories such as CBT, MI, and personalized feedback. From these theories and approaches stem behavior change strategies or techniques, commonly known as BCTs. Feedback on behavior , included in the feedback monitoring BCT cluster, was the most common BCT used in the included studies. This specific BCT appears to be a core strategy in behavior change interventions [ 125 , 126 ]. In their systematic review of remotely delivered alcohol or substance misuse interventions for adults, Howlett et al [ 53 ] found that feedback on behavior , problem solving , and goal setting were the most frequently used BCTs in the included studies. In addition, this research group noted that the most promising BCTs for alcohol misuse were avoidance/reducing exposure to cues for behavior , pros and cons , and self-monitoring of behavior, whereas 2 very promising strategies for substance misuse in general were problem solving and self-monitoring of behavior . In our systematic review, in addition to feedback on behavior , the 6 most frequently used BCTs in the included studies were social support , pros and cons , social comparison , problem solving , information about social and environmental consequences , and information about health consequences . Although pros and cons and problem solving were present in all 3 studies of digital interventions included in our meta-analysis, avoidance/reducing exposure to cues for behavior was reported in only 5% (1/19) of the articles, and feedback on behavior was more frequently used than self-monitoring of behavior. However, it should be noted that the review by Howlett et al [ 53 ] examined digital interventions for participants with alcohol or substance misuse problems, whereas in this review, we focused on interventions that targeted CU from a harm reduction perspective. In this light, avoidance/reducing exposure to cues for behavior may be a BCT better suited to populations with substance misuse problems. Lending support to this, a meta-regression by Garnett et al [ 127 ] and a Cochrane systematic review by Kaner et al [ 128 ] both found interventions that used behavior substitution and credible source to be associated with greater reduction in excessive alcohol consumption compared with interventions that used other BCTs.

Beyond the number and types of BCTs used, reflecting on the extent to which each BCT in a given intervention suits (or does not suit) the targeted determinants (ie, behavioral and environmental causes) is crucial for planning intervention programs [ 26 ]. It is important when designing digital CU interventions not merely to pick a combination of BCTs that have been associated with effectiveness. Rather, the active ingredients must fit the determinants that the interventionists seek to influence. For example, action planning would be more relevant as a BCT for young adults highly motivated and ready to take action on their CU than would pros and cons , which aims instead to bolster motivation. Given that more than half of all digital interventions are asynchronous and based on a self-guided approach and do not offer counselor or therapist support, a great deal of motivation is required to engage in intervention and behavior change. Therefore, it is essential that developers consider the needs and characteristics of the targeted population to tailor intervention strategies (ie, BCTs) for successful behavior change (eg, tailored to the participant’s stage of change). In most of the digital interventions included in this systematic review, personalization was achieved through feedback messages about CU regarding descriptive norms, motives, risks and consequences, and costs, among other things.

Despite the high number of recent studies conducted in the field of digital CU interventions, most of the included articles in our review (17/19, 89%) reported on the development and evaluation of web-based intervention programs. A new generation of health intervention modalities such as mobile apps and social media has drawn the attention of researchers in the past decade and is currently being evaluated. In this regard, the results from a recently published scoping review [ 129 ], which included 5 studies of mobile apps for nonmedical CU, suggested that these novel modes of intervention delivery demonstrated adequate feasibility and acceptability. Nevertheless, the internet remains a powerful and convenient medium for reaching young adults with digital interventions intended to support safe CU behaviors [ 123 , 130 ].

Quality of Evidence

The GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach [ 131 - 133 ] was used to assess the quality of the evidence reviewed. It was deemed to be moderate for the primary outcome of this review, that is, CU frequency in terms of days of use in the previous month (see the summary of evidence in Multimedia Appendix 11 [ 70 , 72 ]). The direction of evidence was broadly consistent—in all 3 RCT studies [ 70 - 72 ] included in the meta-analysis, participants who received digital CU interventions reduced their consumption compared with those who received no or minimal interventions. The 3 RCTs were similar in that they all involved a web-based, multicomponent intervention program aimed at reducing or ceasing CU. However, the interventions did differ or vary in terms of several characteristics, including the strategies used, content, frequency, and duration. Given the small number of studies included in the meta-analysis, we could not conclude with certainty which intervention components, if any, contributed to the effect estimate observed.

Although inconsistency, indirectness, and imprecision were not major issues in the body of evidence, we downgraded the evidence from high to moderate quality on account of RoB assessments at the outcome level. The 3 RCT studies included in the meta-analysis were rated as having “some concerns” of RoB, mainly due to lack of blinding, which significantly reduced our certainty relative to subjective outcomes (ie, self-reported measures of CU frequency). A positive feature of these digital intervention trials is that most procedures are fully automated, and so there was typically a low RoB regarding randomization procedures, allocation to different conditions, and intervention delivery. It is impossible to blind participants to these types of behavior change interventions, and although some researchers have made attempts to counter the impact of this risk, performance bias is an inescapable issue in RCT studies of this kind. Blinding of intervention providers was not an issue in the 3 RCTs included in the meta-analysis because outcome data collection was automated. However, this same automated procedure made it very difficult to ensure follow‐up. Consequently, attrition was another source of bias in these RCT studies [ 70 - 72 ]. The participants lost to follow-up likely stopped using the intervention. However, there is no way of determining whether these people would have benefited more or less than the completers if they had seen the trial through.

The 3 RCTs included in the meta-analysis relied on subjective self-reported measures of CU at baseline and follow‐up, which are subject to recall and social desirability bias. However, all 3 studies used a well-validated instrument of measurement to determine frequency of CU, the TLFB [ 66 ]. This is a widely used, subjective self-report tool for measuring frequency (or quantity) of substance use (or abstinence). It is considered a reliable measure of CU [ 134 , 135 ]. Finally, it should be pointed out that any potential bias related to self‐reported CU frequency would have affected both the intervention and control groups (particularly in cases in which control groups received cannabis‐related information), and thus, it was unlikely to account for differential intervention effects. Moreover, we found RoB due to selective reporting in some studies owing mainly to the absence of any reference to a protocol. Ultimately, these limitations may have biased the results of the meta-analysis. Consequently, future research is likely to further undermine our confidence in the effect estimate we observed and report considerably different estimates.

Strengths and Limitations

Our systematic review and meta-analysis has a number of strengths: (1) we included only randomized controlled studies to ensure that the included studies possessed a rigorous research design, (2) we focused specifically on cannabis (rather than combining multiple substances), (3) we assessed the effectiveness of 3 different digital interventions on CU frequency among community-living young adults, and (4) we performed an exhaustive synthesis and comparison of the BCTs used in the 9 digital interventions examined in the 19 studies included in our review based on the BCTTv1.

Admittedly, this systematic review and meta-analysis has limitations that should be recognized. First, although we searched a range of bibliographic databases, the review was limited to articles published in peer-reviewed journals in English or French. This may have introduced publication bias given that articles reporting positive effects are more likely to be published than those with negative or equivocal results. Consequently, the studies included in this review may have overrepresented the statistically significant effects of digital CU interventions.

Second, only a small number of studies were included in the meta-analyses because many studies did not provide adequate statistical information for calculating and synthesizing effect sizes, although significant efforts were made to contact the authors in case of missing data. Because of the small sample size used in the meta-analysis, the effect size estimates may not be highly reflective of the true effects of digital interventions on CU frequency among young adults. Furthermore, synthesizing findings across studies that evaluated different modalities of web-based intervention programs (eg, fully self-guided vs with therapist guidance) and types of intervention approaches (eg, CBT, MI, and personalized feedback) may have introduced bias in the meta-analytical results due to the heterogeneity of the included studies, although heterogeneity was controlled for using a random-effects model and our results indicated low between-study heterogeneity.

Third, we took various measures to ensure that BCT coding was carried out rigorously throughout the data extraction and analysis procedures: (1) all coders received training on how to use the BCTTv1; (2) all the included articles were read line by line so that coders became familiar with intervention descriptions before initiating BCT coding; (3) the intervention description of each included article was double coded after a pilot calibration exercise with all coders, and any disagreements regarding the presence or absence of a BCT were discussed and resolved with a third party; and (4) we contacted the article authors when necessary and possible for further details on the BCTs they used. However, incomplete reporting of intervention content is a recognized issue [ 136 ], which may have resulted in our coding BCTs incorrectly as present or absent. Reliably specifying the BCTs used in interventions allows their active ingredients to be identified, their evidence to be synthesized, and interventions to be replicated, thereby providing tangible guidance to programmers and researchers to develop more effective interventions.

Finally, although this review identified the BCTs used in digital interventions, our approach did not allow us to draw conclusions regarding their effectiveness. Coding BCTs simply as present or absent does not consider the frequency, intensity, and quality with which they were delivered. For example, it is unclear how many individuals should self‐monitor their CU. In addition, the quality of BCT implementation may be critical in digital interventions where different graphics and interface designs and the usability of the BCTs used can have considerable influence on the level of user engagement [ 137 ]. In the future, it may be necessary to develop new methods to evaluate the dosage of individual BCTs in digital health interventions and characterize their implementation quality to assess their effectiveness [ 128 , 138 ]. Despite its limitations, this review suggests that digital interventions represent a promising avenue for preventing, reducing, or ceasing CU among community-living young adults.

Conclusions

The results of this systematic review and meta-analysis lend support to the promise of digital interventions as an effective means of reducing recreational CU frequency among young adults. Despite the advent and popularity of smartphones, web-based interventions remain the most common mode of delivery for digital interventions. The active ingredients of digital interventions are varied and encompass a number of clusters of the BCTTv1, but a significant number of BCTs remain underused. Additional research is needed to further investigate the effectiveness of these interventions on CU and key outcomes at later time points. Finally, a detailed assessment of user engagement with digital interventions for CU and understanding which intervention components are the most effective remain important research gaps.

Acknowledgments

The authors would like to thank Bénédicte Nauche, Miguel Chagnon, and Paul Di Biase for their valuable support with the search strategy development, statistical analysis, and linguistic revision, respectively. This work was supported by the Ministère de la Santé et des Services sociaux du Québec as part of a broader study aimed at developing and evaluating a digital intervention for young adult cannabis users. Additional funding was provided by the Research Chair in Innovative Nursing Practices. The views and opinions expressed in this manuscript do not necessarily reflect those of these funding entities.

Data Availability

The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.

Authors' Contributions

JC contributed to conceptualization, methodology, formal analysis, writing—original draft, supervision, and funding acquisition. GC contributed to conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, visualization, and project administration. BV contributed to conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, and visualization. PA contributed to conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, visualization, and project administration. GR contributed to conceptualization, methodology, formal analysis, investigation, data curation, and writing—review and editing. GF contributed to conceptualization, methodology, formal analysis, investigation, data curation, and writing—review and editing. DJA contributed to conceptualization, methodology, formal analysis, writing—review and editing, and funding acquisition.

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

Detailed search strategies for each database.

Population, intervention, comparison, outcome, and study design strategy.

Excluded studies and reasons for exclusion.

Study and participant characteristics.

Description of intervention characteristics in the included articles.

Summary of methodological characteristics and major findings of the included studies categorized by intervention name.

Behavior change techniques (BCTs) coded in each included study summarized by individual BCT and BCT cluster.

Risk-of-bias assessment of each included study for cannabis use and cannabis consequences.

Excluded studies and reasons for exclusion from the meta-analysis.

Summary of evidence according to the Grading of Recommendations Assessment, Development, and Evaluation tool.

• Arnett JJ. The developmental context of substance use in emerging adulthood. J Drug Issues. 2005;35(2):235-254. [ CrossRef ]
• Stockings E, Hall WD, Lynskey M, Morley KI, Reavley N, Strang J, et al. Prevention, early intervention, harm reduction, and treatment of substance use in young people. Lancet Psychiatry. Mar 2016;3(3):280-296. [ CrossRef ] [ Medline ]
• ElSohly MA, Chandra S, Radwan M, Majumdar CG, Church JC. A comprehensive review of cannabis potency in the United States in the last decade. Biol Psychiatry Cogn Neurosci Neuroimaging. Jun 2021;6(6):603-606. [ CrossRef ] [ Medline ]
• Fischer B, Robinson T, Bullen C, Curran V, Jutras-Aswad D, Medina-Mora ME, et al. Lower-Risk Cannabis Use Guidelines (LRCUG) for reducing health harms from non-medical cannabis use: a comprehensive evidence and recommendations update. Int J Drug Policy. Jan 2022;99:103381. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rotermann M. What has changed since cannabis was legalized? Health Rep. Feb 19, 2020;31(2):11-20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Degenhardt L, Stockings E, Patton G, Hall WD, Lynskey M. The increasing global health priority of substance use in young people. Lancet Psychiatry. Mar 2016;3(3):251-264. [ CrossRef ] [ Medline ]
• Buckner JD, Bonn-Miller MO, Zvolensky MJ, Schmidt NB. Marijuana use motives and social anxiety among marijuana-using young adults. Addict Behav. Oct 2007;32(10):2238-2252. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Carliner H, Brown QL, Sarvet AL, Hasin DS. Cannabis use, attitudes, and legal status in the U.S.: a review. Prev Med. Nov 2017;104:13-23. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World drug report 2020. United Nations Office on Drugs and Crime. 2020. URL: https://wdr.unodc.org/wdr2020/index2020.html [accessed 2023-11-28]
• National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Population Health and Public Health Practice, Committee on the Health Effects of Marijuana: An Evidence Review and Research Agenda. The Health Effects of Cannabis and Cannabinoids: The Current State of Evidence and Recommendations for Research. Washington, DC. The National Academies Press; 2017.
• Hall WD, Patton G, Stockings E, Weier M, Lynskey M, Morley KI, et al. Why young people's substance use matters for global health. Lancet Psychiatry. Mar 2016;3(3):265-279. [ CrossRef ] [ Medline ]
• Cohen K, Weizman A, Weinstein A. Positive and negative effects of cannabis and cannabinoids on health. Clin Pharmacol Ther. May 2019;105(5):1139-1147. [ CrossRef ] [ Medline ]
• Memedovich KA, Dowsett LE, Spackman E, Noseworthy T, Clement F. The adverse health effects and harms related to marijuana use: an overview review. CMAJ Open. Aug 16, 2018;6(3):E339-E346. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Teeters JB, Armstrong NM, King SA, Hubbard SM. A randomized pilot trial of a mobile phone-based brief intervention with personalized feedback and interactive text messaging to reduce driving after cannabis use and riding with a cannabis impaired driver. J Subst Abuse Treat. Nov 2022;142:108867. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chan GC, Becker D, Butterworth P, Hines L, Coffey C, Hall W, et al. Young-adult compared to adolescent onset of regular cannabis use: a 20-year prospective cohort study of later consequences. Drug Alcohol Rev. May 2021;40(4):627-636. [ CrossRef ] [ Medline ]
• Hall W, Stjepanović D, Caulkins J, Lynskey M, Leung J, Campbell G, et al. Public health implications of legalising the production and sale of cannabis for medicinal and recreational use. Lancet. Oct 26, 2019;394(10208):1580-1590. [ CrossRef ] [ Medline ]
• The health and social effects of nonmedical cannabis use. World Health Organization. 2016. URL: https://apps.who.int/iris/handle/10665/251056 [accessed 2023-11-28]
• Boumparis N, Loheide-Niesmann L, Blankers M, Ebert DD, Korf D, Schaub MP, et al. Short- and long-term effects of digital prevention and treatment interventions for cannabis use reduction: a systematic review and meta-analysis. Drug Alcohol Depend. Jul 01, 2019;200:82-94. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jutras-Aswad D, Le Foll B, Bruneau J, Wild TC, Wood E, Fischer B. Thinking beyond legalization: the case for expanding evidence-based options for cannabis use disorder treatment in Canada. Can J Psychiatry. Feb 2019;64(2):82-87. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garnett CV, Crane D, Brown J, Kaner EF, Beyer FR, Muirhead CR, et al. Behavior change techniques used in digital behavior change interventions to reduce excessive alcohol consumption: a meta-regression. Ann Behav Med. May 18, 2018;52(6):530-543. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Glanz K, Rimer BK, Viswanath K. Health Behavior: Theory, Research, and Practice, 5th Edition. Hoboken, NJ. Jossey-Bass; Jul 2015.
• Prestwich A, Webb TL, Conner M. Using theory to develop and test interventions to promote changes in health behaviour: evidence, issues, and recommendations. Curr Opin Psychol. Oct 2015;5:1-5. [ CrossRef ]
• Webb TL, Sniehotta FF, Michie S. Using theories of behaviour change to inform interventions for addictive behaviours. Addiction. Nov 2010;105(11):1879-1892. [ CrossRef ] [ Medline ]
• Cilliers F, Schuwirth L, van der Vleuten C. Health behaviour theories: a conceptual lens to explore behaviour change. In: Cleland J, Durning SJ, editors. Researching Medical Education. Hoboken, NJ. Wiley; 2015.
• Davis R, Campbell R, Hildon Z, Hobbs L, Michie S. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev. 2015;9(3):323-344. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eldredge LK, Markham CM, Ruiter RA, Fernández ME, Kok G, Parcel GS. Planning Health Promotion Programs: An Intervention Mapping Approach, 4th Edition. Hoboken, NJ. John Wiley & Sons; Feb 2016.
• Marlatt GA, Blume AW, Parks GA. Integrating harm reduction therapy and traditional substance abuse treatment. J Psychoactive Drugs. 2001;33(1):13-21. [ CrossRef ] [ Medline ]
• Adams A, Ferguson M, Greer AM, Burmeister C, Lock K, McDougall J, et al. Guideline development in harm reduction: considerations around the meaningful involvement of people who access services. Drug Alcohol Depend Rep. Aug 12, 2022;4:100086. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Davis ML, Powers MB, Handelsman P, Medina JL, Zvolensky M, Smits JA. Behavioral therapies for treatment-seeking cannabis users: a meta-analysis of randomized controlled trials. Eval Health Prof. Mar 2015;38(1):94-114. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gates PJ, Sabioni P, Copeland J, Le Foll B, Gowing L. Psychosocial interventions for cannabis use disorder. Cochrane Database Syst Rev. May 05, 2016;2016(5):CD005336. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Halladay J, Scherer J, MacKillop J, Woock R, Petker T, Linton V, et al. Brief interventions for cannabis use in emerging adults: a systematic review, meta-analysis, and evidence map. Drug Alcohol Depend. Nov 01, 2019;204:107565. [ CrossRef ] [ Medline ]
• Imtiaz S, Roerecke M, Kurdyak P, Samokhvalov AV, Hasan OS, Rehm J. Brief interventions for cannabis use in healthcare settings: systematic review and meta-analyses of randomized trials. J Addict Med. 2020;14(1):78-88. [ CrossRef ] [ Medline ]
• Standeven LR, Scialli A, Chisolm MS, Terplan M. Trends in cannabis treatment admissions in adolescents/young adults: analysis of TEDS-A 1992 to 2016. J Addict Med. 2020;14(4):e29-e36. [ CrossRef ] [ Medline ]
• Montanari L, Guarita B, Mounteney J, Zipfel N, Simon R. Cannabis use among people entering drug treatment in europe: a growing phenomenon? Eur Addict Res. 2017;23(3):113-121. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kerridge BT, Mauro PM, Chou SP, Saha TD, Pickering RP, Fan AZ, et al. Predictors of treatment utilization and barriers to treatment utilization among individuals with lifetime cannabis use disorder in the United States. Drug Alcohol Depend. Dec 01, 2017;181:223-228. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gates P, Copeland J, Swift W, Martin G. Barriers and facilitators to cannabis treatment. Drug Alcohol Rev. May 2012;31(3):311-319. [ CrossRef ] [ Medline ]
• Hammarlund RA, Crapanzano KA, Luce L, Mulligan L, Ward KM. Review of the effects of self-stigma and perceived social stigma on the treatment-seeking decisions of individuals with drug- and alcohol-use disorders. Subst Abuse Rehabil. Nov 23, 2018;9:115-136. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bedrouni W. On the use of digital technologies to reduce the public health impacts of cannabis legalization in Canada. Can J Public Health. Dec 2018;109(5-6):748-751. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Perski O, Hébert ET, Naughton F, Hekler EB, Brown J, Businelle MS. Technology-mediated just-in-time adaptive interventions (JITAIs) to reduce harmful substance use: a systematic review. Addiction. May 2022;117(5):1220-1241. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kazemi DM, Borsari B, Levine MJ, Li S, Lamberson KA, Matta LA. A systematic review of the mHealth interventions to prevent alcohol and substance abuse. J Health Commun. May 2017;22(5):413-432. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nesvåg S, McKay JR. Feasibility and effects of digital interventions to support people in recovery from substance use disorders: systematic review. J Med Internet Res. Aug 23, 2018;20(8):e255. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hoch E, Preuss UW, Ferri M, Simon R. Digital interventions for problematic cannabis users in non-clinical settings: findings from a systematic review and meta-analysis. Eur Addict Res. 2016;22(5):233-242. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Olmos A, Tirado-Muñoz J, Farré M, Torrens M. The efficacy of computerized interventions to reduce cannabis use: a systematic review and meta-analysis. Addict Behav. Apr 2018;79:52-60. [ CrossRef ] [ Medline ]
• Tait RJ, Spijkerman R, Riper H. Internet and computer based interventions for cannabis use: a meta-analysis. Drug Alcohol Depend. Dec 01, 2013;133(2):295-304. [ CrossRef ] [ Medline ]
• Beneria A, Santesteban-Echarri O, Daigre C, Tremain H, Ramos-Quiroga JA, McGorry PD, et al. Online interventions for cannabis use among adolescents and young adults: systematic review and meta-analysis. Early Interv Psychiatry. Aug 2022;16(8):821-844. [ CrossRef ] [ Medline ]
• Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. May 2013;50(5):587-592. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Michie S, Abraham C, Eccles MP, Francis JJ, Hardeman W, Johnston M. Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implement Sci. Feb 07, 2011;6:10. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. Aug 2013;46(1):81-95. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Michie S, Johnston M, Francis J, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. Oct 2008;57(4):660-680. [ CrossRef ]
• Scott C, de Barra M, Johnston M, de Bruin M, Scott N, Matheson C, et al. Using the behaviour change technique taxonomy v1 (BCTTv1) to identify the active ingredients of pharmacist interventions to improve non-hospitalised patient health outcomes. BMJ Open. Sep 15, 2020;10(9):e036500. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dombrowski SU, Sniehotta FF, Avenell A, Johnston M, MacLennan G, Araújo-Soares V. Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: a systematic review. Health Psychol Rev. 2012;6(1):7-32. [ CrossRef ]
• Michie S, Abraham C, Whittington C, McAteer J, Gupta S. Effective techniques in healthy eating and physical activity interventions: a meta-regression. Health Psychol. Nov 2009;28(6):690-701. [ CrossRef ] [ Medline ]
• Howlett N, García-Iglesias J, Bontoft C, Breslin G, Bartington S, Freethy I, et al. A systematic review and behaviour change technique analysis of remotely delivered alcohol and/or substance misuse interventions for adults. Drug Alcohol Depend. Oct 01, 2022;239:109597. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions Version 6.4. London, UK. The Cochrane Collaboration; 2023.
• Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. Mar 29, 2021;372:n160. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. Jul 2016;75:40-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Halladay J, Petker T, Fein A, Munn C, MacKillop J. Brief interventions for cannabis use in emerging adults: protocol for a systematic review, meta-analysis, and evidence map. Syst Rev. Jul 25, 2018;7(1):106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. Apr 23, 2011;6:42. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arnett JJ. Emerging adulthood. A theory of development from the late teens through the twenties. Am Psychol. May 2000;55(5):469-480. [ Medline ]
• Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. Jul 2016;104(3):240-243. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. Mar 07, 2014;348:g1687. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Presseau J, Ivers NM, Newham JJ, Knittle K, Danko KJ, Grimshaw JM. Using a behaviour change techniques taxonomy to identify active ingredients within trials of implementation interventions for diabetes care. Implement Sci. Apr 23, 2015;10:55. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fontaine G, Cossette S, Maheu-Cadotte MA, Deschênes MF, Rouleau G, Lavallée A, et al. Effect of implementation interventions on nurses' behaviour in clinical practice: a systematic review, meta-analysis and meta-regression protocol. Syst Rev. Dec 05, 2019;8(1):305. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fontaine G, Cossette S. A theory-based adaptive E-learning program aimed at increasing intentions to provide brief behavior change counseling: randomized controlled trial. Nurse Educ Today. Dec 2021;107:105112. [ CrossRef ] [ Medline ]
• Fontaine G, Cossette S. Development and design of E_MOTIV: a theory-based adaptive e-learning program to support nurses' provision of brief behavior change counseling. Comput Inform Nurs. Mar 01, 2023;41(3):130-141. [ CrossRef ] [ Medline ]
• Sobell LC, Sobell MB. Timeline follow-back: a technique for assessing self-reported alcohol consumption. In: Litten RZ, Allen JP, editors. Measuring Alcohol Consumption. Totowa, NJ. Humana Press; 1992.
• Stephens RS, Roffman RA, Simpson EE. Treating adult marijuana dependence: a test of the relapse prevention model. J Consult Clin Psychol. 1994;62(1):92-99. [ CrossRef ]
• Harris RJ, Deeks JJ, Altman DG, Bradburn MJ, Harbord RM, Sterne JA. Metan: fixed- and random-effects meta-analysis. Stata J. 2008;8(1):3-28. [ CrossRef ]
• Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. Sep 06, 2003;327(7414):557-560. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Baumgartner C, Schaub MP, Wenger A, Malischnig D, Augsburger M, Walter M, et al. CANreduce 2.0 adherence-focused guidance for internet self-help among cannabis users: three-arm randomized controlled trial. J Med Internet Res. Apr 30, 2021;23(4):e27463. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rooke S, Copeland J, Norberg M, Hine D, McCambridge J. Effectiveness of a self-guided web-based cannabis treatment program: randomized controlled trial. J Med Internet Res. Feb 15, 2013;15(2):e26. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tossmann HP, Jonas B, Tensil MD, Lang P, Strüber E. A controlled trial of an internet-based intervention program for cannabis users. Cyberpsychol Behav Soc Netw. Nov 2011;14(11):673-679. [ CrossRef ] [ Medline ]
• StataCorp. Stata statistical software: release 18. StataCorp LLC. College Station, TX. StataCorp LLC; 2023. URL: https://www.stata.com/ [accessed 2023-11-28]
• Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. Aug 28, 2019;366:l4898. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McGuinness LA, Higgins JP. Risk-of-bias VISualization (robvis): an R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. Jan 2021;12(1):55-61. [ CrossRef ] [ Medline ]
• Haddaway NR, Page MJ, Pritchard CC, McGuinness LA. PRISMA2020: an R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Syst Rev. Mar 27, 2022;18(2):e1230. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Elliott JC, Carey KB. Correcting exaggerated marijuana use norms among college abstainers: a preliminary test of a preventive intervention. J Stud Alcohol Drugs. Nov 2012;73(6):976-980. [ CrossRef ] [ Medline ]
• Elliott JC, Carey KB, Vanable PA. A preliminary evaluation of a web-based intervention for college marijuana use. Psychol Addict Behav. Mar 2014;28(1):288-293. [ CrossRef ] [ Medline ]
• Lee CM, Neighbors C, Kilmer JR, Larimer ME. A brief, web-based personalized feedback selective intervention for college student marijuana use: a randomized clinical trial. Psychol Addict Behav. Jun 2010;24(2):265-273. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Buckner JD, Zvolensky MJ, Lewis EM. On-line personalized feedback intervention for negative affect and cannabis: a pilot randomized controlled trial. Exp Clin Psychopharmacol. Apr 2020;28(2):143-149. [ CrossRef ] [ Medline ]
• Goodness TM, Palfai TP. Electronic screening and brief intervention to reduce cannabis use and consequences among graduate students presenting to a student health center: a pilot study. Addict Behav. Jul 2020;106:106362. [ CrossRef ] [ Medline ]
• Palfai TP, Saitz R, Winter M, Brown TA, Kypri K, Goodness TM, et al. Web-based screening and brief intervention for student marijuana use in a university health center: pilot study to examine the implementation of eCHECKUP TO GO in different contexts. Addict Behav. Sep 2014;39(9):1346-1352. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Riggs NR, Conner BT, Parnes JE, Prince MA, Shillington AM, George MW. Marijuana eCHECKUPTO GO: effects of a personalized feedback plus protective behavioral strategies intervention for heavy marijuana-using college students. Drug Alcohol Depend. Sep 01, 2018;190:13-19. [ CrossRef ] [ Medline ]
• Bonar EE, Chapman L, Pagoto S, Tan CY, Duval ER, McAfee J, et al. Social media interventions addressing physical activity among emerging adults who use cannabis: a pilot trial of feasibility and acceptability. Drug Alcohol Depend. Jan 01, 2023;242:109693. [ CrossRef ] [ Medline ]
• Bonar EE, Goldstick JE, Chapman L, Bauermeister JA, Young SD, McAfee J, et al. A social media intervention for cannabis use among emerging adults: randomized controlled trial. Drug Alcohol Depend. Mar 01, 2022;232:109345. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Walukevich-Dienst K, Neighbors C, Buckner JD. Online personalized feedback intervention for cannabis-using college students reduces cannabis-related problems among women. Addict Behav. Nov 2019;98:106040. [ CrossRef ] [ Medline ]
• Côté J, Tessier S, Gagnon H, April N, Rouleau G, Chagnon M. Efficacy of a web-based tailored intervention to reduce cannabis use among young people attending adult education centers in Quebec. Telemed J E Health. Nov 2018;24(11):853-860. [ CrossRef ] [ Medline ]
• Cunningham JA, Schell C, Bertholet N, Wardell JD, Quilty LC, Agic B, et al. Online personalized feedback intervention to reduce risky cannabis use. Randomized controlled trial. Internet Interv. Nov 14, 2021;26:100484. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Copeland J, Rooke S, Rodriquez D, Norberg MM, Gibson L. Comparison of brief versus extended personalised feedback in an online intervention for cannabis users: short-term findings of a randomised trial. J Subst Abuse Treat. May 2017;76:43-48. [ CrossRef ] [ Medline ]
• Jonas B, Tensil MD, Tossmann P, Strüber E. Effects of treatment length and chat-based counseling in a web-based intervention for cannabis users: randomized factorial trial. J Med Internet Res. May 08, 2018;20(5):e166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schaub MP, Wenger A, Berg O, Beck T, Stark L, Buehler E, et al. A web-based self-help intervention with and without chat counseling to reduce cannabis use in problematic cannabis users: three-arm randomized controlled trial. J Med Internet Res. Oct 13, 2015;17(10):e232. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sinadinovic K, Johansson M, Johansson AS, Lundqvist T, Lindner P, Hermansson U. Guided web-based treatment program for reducing cannabis use: a randomized controlled trial. Addict Sci Clin Pract. Feb 18, 2020;15(1):9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kanfer FH. Implications of a self-regulation model of therapy for treatment of addictive behaviors. In: Miller WR, Heather N, editors. Treating Addictive Behaviors. Boston, MA. Springer; 1986;29-47.
• Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res. Mar 10, 2011;13(1):e30. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Marlatt GA, Baer JS, Kivlahan DR, Dimeff LA, Larimer ME, Quigley LA, et al. Screening and brief intervention for high-risk college student drinkers: results from a 2-year follow-up assessment. J Consult Clin Psychol. Aug 1998;66(4):604-615. [ CrossRef ]
• Miller WR, Rollnick S. Motivational Interviewing: Preparing People for Change. New York, NY. Guilford Press; 2002.
• Prince MA, Carey KB, Maisto SA. Protective behavioral strategies for reducing alcohol involvement: a review of the methodological issues. Addict Behav. Jul 2013;38(7):2343-2351. [ CrossRef ] [ Medline ]
• Lundqvist TN. Cognitive Dysfunctions in Chronic Cannabis Users Observed During Treatment: An Integrative Approach. Stockholm, Sweden. Almqvist & Wiksell; 1997.
• Kanter JW, Puspitasari AJ, Santos MM, Nagy GA. Behavioural activation: history, evidence and promise. Br J Psychiatry. May 2012;200(5):361-363. [ CrossRef ] [ Medline ]
• Jaffee WB, D'Zurilla TJ. Personality, problem solving, and adolescent substance use. Behav Ther. Mar 2009;40(1):93-101. [ CrossRef ] [ Medline ]
• Miller W, Rollnick S. Motivational Interviewing: Preparing People to Change Addictive Behavior. New York, NY. The Guilford Press; 1991.
• Gordon JR, Marlatt GA. Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors. 2nd edition. New York, NY. The Guilford Press; 2005.
• Platt JJ, Husband SD. An overview of problem-solving and social skills approaches in substance abuse treatment. Psychotherapy (Chic). 1993;30(2):276-283. [ FREE Full text ]
• Steinberg KL, Roffman R, Carroll K, McRee B, Babor T, Miller M. Brief counseling for marijuana dependence: a manual for treating adults. Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration, US Department of Health and Human Services. URL: https:/​/store.​samhsa.gov/​product/​brief-counseling-marijuana-dependence-manual-treating-adults/​sma15-4211 [accessed 2024-03-23]
• de Shazer S, Dolan Y. More Than Miracles: The State of the Art of Solution-Focused Brief Therapy. Oxfordshire, UK. Routledge; 2007.
• Copeland J, Swift W, Roffman R, Stephens R. A randomized controlled trial of brief cognitive-behavioral interventions for cannabis use disorder. J Subst Abuse Treat. Sep 2001;21(2):55-65. [ CrossRef ] [ Medline ]
• Linke S, McCambridge J, Khadjesari Z, Wallace P, Murray E. Development of a psychologically enhanced interactive online intervention for hazardous drinking. Alcohol Alcohol. 2008;43(6):669-674. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang ML, Waring ME, Jake-Schoffman DE, Oleski JL, Michaels Z, Goetz JM, et al. Clinic versus online social network-delivered lifestyle interventions: protocol for the get social noninferiority randomized controlled trial. JMIR Res Protoc. Dec 11, 2017;6(12):e243. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sepah SC, Jiang L, Peters AL. Translating the diabetes prevention program into an online social network: validation against CDC standards. Diabetes Educ. Jul 2014;40(4):435-443. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cunningham JA, van Mierlo T. The check your cannabis screener: a new online personalized feedback tool. Open Med Inform J. May 07, 2009;3:27-31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bertholet N, Cunningham JA, Faouzi M, Gaume J, Gmel G, Burnand B, et al. Internet-based brief intervention for young men with unhealthy alcohol use: a randomized controlled trial in a general population sample. Addiction. Nov 2015;110(11):1735-1743. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Walker DD, Roffman RA, Stephens RS, Wakana K, Berghuis J, Kim W. Motivational enhancement therapy for adolescent marijuana users: a preliminary randomized controlled trial. J Consult Clin Psychol. Jun 2006;74(3):628-632. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Miller MB, Leffingwell T, Claborn K, Meier E, Walters S, Neighbors C. Personalized feedback interventions for college alcohol misuse: an update of Walters and Neighbors (2005). Psychol Addict Behav. Dec 2013;27(4):909-920. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ajzen I. From intentions to actions: a theory of planned behavior. In: Kuhl J, Beckmann J, editors. Action Control. Berlin, Germany. Springer; 1985;11-39.
• Dennis M, Titus JC, Diamond G, Donaldson J, Godley SH, Tims FM, et al. The Cannabis Youth Treatment (CYT) experiment: rationale, study design and analysis plans. Addiction. Dec 11, 2002;97 Suppl 1(s1):16-34. [ CrossRef ] [ Medline ]
• Simons JS, Dvorak RD, Merrill JE, Read JP. Dimensions and severity of marijuana consequences: development and validation of the Marijuana Consequences Questionnaire (MACQ). Addict Behav. May 2012;37(5):613-621. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Legleye S. The Cannabis Abuse Screening Test and the DSM-5 in the general population: optimal thresholds and underlying common structure using multiple factor analysis. Int J Methods Psychiatr Res. Jun 10, 2018;27(2):e1597. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Legleye S, Karila LM, Beck F, Reynaud M. Validation of the CAST, a general population Cannabis Abuse Screening Test. J Subst Use. Jul 12, 2009;12(4):233-242. [ CrossRef ]
• Sobell LC, Sobell MB. Timeline follow back. In: Litten RZ, Allen JP, editors. Measuring Alcohol Consumption: Psychosocial and Biochemical Methods. Totowa, NJ. Humana Press; 1992;41-72.
• White HR, Labouvie EW, Papadaratsakis V. Changes in substance use during the transition to adulthood: a comparison of college students and their noncollege age peers. J Drug Issues. Aug 03, 2016;35(2):281-306. [ CrossRef ]
• White HR, Labouvie EW. Towards the assessment of adolescent problem drinking. J Stud Alcohol. Jan 1989;50(1):30-37. [ CrossRef ] [ Medline ]
• Cloutier RM, Natesan Batley P, Kearns NT, Knapp AA. A psychometric evaluation of the Marijuana Problems Index among college students: confirmatory factor analysis and measurement invariance by gender. Exp Clin Psychopharmacol. Dec 2022;30(6):907-917. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guo H, Yang H, Yuan G, Zhu Z, Zhang K, Zhang X, et al. Effectiveness of information and communication technology (ICT) for addictive behaviors: an umbrella review of systematic reviews and meta-analysis of randomized controlled trials. Comput Hum Behav. Oct 2023;147:107843. [ CrossRef ]
• Grigsby TJ, Lopez A, Albers L, Rogers CJ, Forster M. A scoping review of risk and protective factors for negative cannabis use consequences. Subst Abuse. Apr 07, 2023;17:11782218231166622. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Harkin B, Webb TL, Chang BP, Prestwich A, Conner M, Kellar I, et al. Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence. Psychol Bull. Feb 2016;142(2):198-229. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Samdal GB, Eide GE, Barth T, Williams G, Meland E. Effective behaviour change techniques for physical activity and healthy eating in overweight and obese adults; systematic review and meta-regression analyses. Int J Behav Nutr Phys Act. Mar 28, 2017;14(1):42. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garnett C, Crane D, Brown J, Kaner E, Beyer F, Muirhead C. Behavior Change Techniques Used in Digital Behavior Change Interventions to Reduce Excessive Alcohol Consumption: A Meta-regression. Ann Behav Med May 18. 2018;52(6):A. [ CrossRef ]
• Kaner EF, Beyer FR, Muirhead C, Campbell F, Pienaar ED, Bertholet N, et al. Effectiveness of brief alcohol interventions in primary care populations. Cochrane Database Syst Rev. Feb 24, 2018;2(2):CD004148. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sedrati H, Belrhiti Z, Nejjari C, Ghazal H. Evaluation of mobile health apps for non-medical cannabis use: a scoping review. Procedia Comput Sci. 2022;196:581-589. [ CrossRef ]
• Curtis BL, Ashford RD, Magnuson KI, Ryan-Pettes SR. Comparison of smartphone ownership, social media use, and willingness to use digital interventions between generation Z and millennials in the treatment of substance use: cross-sectional questionnaire study. J Med Internet Res. Apr 17, 2019;21(4):e13050. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schünemann H, Brożek J, Guyatt G, Oxman A. The GRADE Handbook. London, UK. The Cochrane Collaboration; 2013.
• Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. Apr 2011;64(4):380-382. [ CrossRef ] [ Medline ]
• Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. Apr 26, 2008;336(7650):924-926. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hjorthøj CR, Hjorthøj AR, Nordentoft M. Validity of Timeline Follow-Back for self-reported use of cannabis and other illicit substances--systematic review and meta-analysis. Addict Behav. Mar 2012;37(3):225-233. [ CrossRef ] [ Medline ]
• Robinson SM, Sobell LC, Sobell MB, Leo GI. Reliability of the Timeline Followback for cocaine, cannabis, and cigarette use. Psychol Addict Behav. Mar 2014;28(1):154-162. [ CrossRef ] [ Medline ]
• Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. May 2008;27(3):379-387. [ CrossRef ] [ Medline ]
• Garrett JJ. The Elements of User Experience: User-Centered Design for the Web and Beyond. London, UK. Pearson Education; 2010.
• Lorencatto F, West R, Bruguera C, Brose LS, Michie S. Assessing the quality of goal setting in behavioural support for smoking cessation and its association with outcomes. Ann Behav Med. Apr 24, 2016;50(2):310-318. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung, G Eysenbach; submitted 30.11.23; peer-reviewed by H Sedrati; comments to author 02.01.24; revised version received 09.01.24; accepted 08.03.24; published 17.04.24.

©José Côté, Gabrielle Chicoine, Billy Vinette, Patricia Auger, Geneviève Rouleau, Guillaume Fontaine, Didier Jutras-Aswad. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Political Typology 2017

Survey conducted June 8-18 and June 27-July 9, 2017

The Generation Gap in American Politics

Generational differences have long been a factor in U.S. politics. These divisions are now as wide as they have been in decades, with the potential to shape politics well into the future.

Political Typology Reveals Deep Fissures on the Right and Left

The partisan divide on political values grows even wider.

Gaps between Republicans and Democrats over racial discrimination, immigration and poverty assistance have widened considerably in recent years.

Partisan Shifts in Views of the Nation, but Overall Opinions Remain Negative

Republicans have become far more upbeat about the country and its future since before Donald Trump’s election victory. By contrast, Democrats have become much less positive.

Since Trump’s Election, Increased Attention to Politics – Especially Among Women

Following an election that had one of the largest gender gaps in history, women are more likely than men to say they are paying increased attention to politics.

Support for Same-Sex Marriage Grows, Even Among Groups That Had Been Skeptical

Two years after the Supreme Court decision that required states to recognize same-sex marriages nationwide, support for allowing gays and lesbians to marry legally is at its highest point in over 20 years of Pew Research Center polling on the issue.

Public Has Criticisms of Both Parties, but Democrats Lead on Empathy for Middle Class

Both political parties’ favorability ratings are more negative than positive and fewer than half say either party has high ethical standards.

Download Dataset

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.14(9); 2022 Sep

Research Question, Objectives, and Endpoints in Clinical and Oncological Research: A Comprehensive Review

Addanki purna singh.

1 Physiology, Saint James School of Medicine, The Quarter, AIA

Praveen R Shahapur

2 Microbiology, Bijapur Lingayat District Educational Association (BLDE, Deemed to be University) Shri B.M. Patil Medical College, Vijayapur, IND

Sabitha Vadakedath

3 Biochemistry, Prathima Institute of Medical Sciences, Karimnagar, IND

Vallab Ganesh Bharadwaj

4 Microbiology, Trichy Sri Ramasamy Memorial (SRM) Medical College Hospital & Research Centre, Tiruchirapalli, IND

Dr Pranay Kumar

5 Anatomy, Prathima Institute of Medical Sciences, Karimnagar , IND

Venkata BharatKumar Pinnelli

6 Biochemistry, Vydehi Institute of Medical Science & Research Center, Bangalore, IND

Vikram Godishala

7 Biotechnology, Ganapathy Degree College, Parkal, IND

Venkataramana Kandi

8 Clinical Microbiology, Prathima Institute of Medical Sciences, Karimnagar, IND

Clinical research is a systematic process of conducting research work to find solutions for human health-related problems. It is applied to understand the disease process and assist in the diagnosis, treatment, and prevention. Currently, we are experiencing global unrest caused by the coronavirus disease (COVID-19) pandemic. The novel severe acute respiratory syndrome coronavirus (SARS-CoV-2) has been responsible for the deaths of more than 50 million people worldwide. Also, it has resulted in severe morbidity among the affected population. The cause of such a huge amount of influence on human health by the pandemic was the unavailability of drugs and therapeutic interventions to treat and manage the disease. Cancer is a disease condition wherein the normal cell function is deranged, and the cells multiply in an uncontrolled manner. Based on recent reports by the World Health Organization (WHO), cancer is the second leading cause of death globally. Moreover, the rates of cancers have shown an increasing trend in the past decade. Therefore, it is essential to improve the understanding concerning clinical research to address the health concerns of humans. In this review, we comprehensively discuss critical aspects of clinical research that include the research question, research objectives, patient-reported outcome measures (PROMs), intention-to-treat and per-protocol analysis, and endpoints in clinical and oncological research.

Introduction and background

Successful clinical research can be conducted by well-trained researchers. Other essential factors of clinical research include framing a research question and relevant objectives, documenting, and recording research outcomes, and outcome measures, sample size, and research methodology including the type of randomization, among others [ 1 , 2 ].

Clinicians/physicians and surgeons are increasingly dependent on the clinical research results for improved management of patients. Therefore, researchers need to work upon a relevant research question/hypothesis and specific objectives that may potentially deliver results that can be translated into practice in the form of evidence-based medicine [ 3 ].

Essential elements that facilitate a researcher to frame a research question are in-depth knowledge of the subject and identifying possible gaps. Moreover, the feasible, interesting, novel, ethical, relevant (FINER) approach and the population of interest/target group, intervention, comparison group, outcome of interest, and time of study (PICOT) approach were previously suggested for researchers to be able to frame appropriate research questions [ 4 ].

Moreover, the research objectives should be framed by the researcher before the initiation of the study: a specific, measurable, achievable, realistic, and time-defined (SMART) approach is utilized to devise the objectives based on the research question. It is preferable to have a single primary objective whereas the secondary objectives can be multiple and may be dependent on the amount of data collected. The objectives must be simple and specific and must reflect the research question [ 5 , 6 ].

Given the evolution and the increasing requirement for emergency care, clinical researchers are advised to adopt a population, exposure, comparator, outcome (PECO)/ population, exposure, comparator, outcome (PICO) approach to construct the study objectives and carry out quantitative research. In contrast, qualitative research which is carried out to understand, explore, and examine requires the researcher to understand what and why the research is undertaken along with the roles of the researcher, research process/steps, and participants [ 7 , 8 ].

Since clinical research is envisaged in finding a solution to a health problem, choosing the appropriate endpoint requires special focus. The endpoints are the specific measures of the outcomes of an intervention and therefore they must be chosen judiciously [ 9 ]. The endpoints in a clinical trial can be single or multiple in numbers. The primary endpoints assess the major research question, and the secondary endpoints may assess alternative research questions. Moreover, there are other endpoints like surrogate endpoints, intermediate clinical endpoints, clinical outcomes, clinical outcome assessments, clinician-reported outcomes, observer-reported outcomes, patient-reported outcomes, and performance outcomes [ 10 ] (Figure ​ (Figure1 1 ).

This figure has been created by the authors

The clinical trial endpoints are essentially the indicators of the power of the interventions either to cure or control the disease progression. Due to the cost and the tedious nature of the clinical trials, it is suggested that multiple-arm trials that include more than one primary endpoint be used [ 11 ]. Integration of the primary endpoints with the patient prioritized endpoints was recently suggested especially among cardiovascular disease patients [ 12 ]. Cancer research is an increasingly evolving area because of the unavailability of therapeutic interventions for some malignancies like breast and lung cancer, among others [ 13 , 14 ]. Moreover, the drugs available for treating cancers are plagued by adverse reactions. However, since most cancer clinical trials apply overall survival as the preferred and gold standard clinical endpoint, it is difficult for the trial operators to sustain the costs associated with the long lengths of the study that follow-up patients for years to assess the clinical outcomes after interventions. In this study, we comprehensively review the essential elements of clinical research that include the research question, hypothesis, and clinical and oncological endpoints.

Research question

A research question can alternatively be called the aim of the researcher. It describes the problem that the researcher intends to solve vis-à-vis finding an answer to a question. A research question is the first step toward any kind of research process that includes both qualitative as well as quantitative research. Since the research question predicts the core of any project, it must be carefully framed. The essential elements to consider while determining a research question are feasibility, preciseness, and relevance to the real world.

A person interested in a broad subject area must first complete extensive reading of the available literature. This enables the researcher to find out the strengths, loopholes, deficiencies, and missing links that can form the basis for framing a research question. The problem to which a solution needs to be found and the potential causes/reasons for the problems help a researcher frame the research question.

The research questions should be framed in such a way that the researcher will find several possibilities as solutions to the research question rather than a simple yes or no. Among the various factors that determine the power of a research question, the most essential ones include the ability of research questions to find complex answers, focussed, and the specific nature of the question. The research questions must be answerable, debatable, and researchable [ 15 , 16 ]. Research questions differ from the type of research method selected by the researcher as shown in Figure ​ Figure2 2 .

FINER: Feasible, interesting, novel, ethical, relevant; PICOT: Population of interest/target group, intervention, comparison group, outcome of interest, time of study; PECO: Population, exposure, comparator, outcome; PICO: Population, intervention, comparator, outcome; SMART: Specific, measurable, achievable, realistic, and time defined; COVID-19: Coronavirus disease-19

Hypothesis testing

A hypothesis is an assumption by the researcher that the answers drawn with reference to the research question are either true or false. The researcher performs hypothesis testing by using appropriate statistical methods on the data collected from the research.

The hypothesis is an assumption/observation of the researcher regarding the outcome of potential research that is being conducted. There are two types of hypotheses, the null hypothesis (H0), wherein, the researcher assumes that there is no relation/causality/effect. The alternate hypothesis (HA) is when the researcher believes/assumes that there is a relationship/effect [ 17 ]. Basically, there are two types of errors while testing a hypothesis. Type I error (α) (false positive) is when the researcher rejects the null hypothesis although it is true. Type II error (β) (false negative) is when the researcher accepts the null hypothesis although it is false.

The errors in hypothesis testing occur because of bias among many other reasons in the study. Many studies set the power of the studies to essentially rule out errors. Researchers consider 5% chance (α=0.05; range: 0.01-0.10) of error in case of a type I error and up to 20% chance (β =0.20; range: 0.05-0.20) in case of type II errors [ 18 ]. The characteristics of a good hypothesis are simple and specific. Moreover, it must be decided by the researcher prior to initiating the study and while writing the study proposal/protocol [ 18 ]. 

Hypothesis testing means sample testing, wherein the information gathered after sample testing is inferred after applying statistical methods. A hypothesis may be generated in several ways that include observations, anatomical features, and other physiological facts observed by physicians [ 19 ]. Hypothesis testing also can be performed by using appropriate statistical methods. The testing of the hypothesis is done to prove the null hypothesis or otherwise use the sample data.

As a researcher, one must assume a null hypothesis, or believe that the alternate hypothesis holds good in the sample selected. After the collection of data, analysis, and interpretation, the researcher either accepts or rejects the hypothesis. Therefore, it must be noted that while a study is initiated, there is only a 50% chance of either the null hypothesis or the alternative hypothesis coming true [ 20 ].

The step-by-step process of hypothesis testing starts with an assumption, criteria for interpretation of results, analysis, and conclusion. The level of significance (95% free of type I error and 80% free of type II error) also is decided initially to ensure that the study results are replicated by the other researchers [ 21 ].

Objectives in clinical research

The most significant objective in clinical research is to find out whether the intervention attempted was successful in curing the disease/medical condition. It is important to understand the fact that research planning greatly influences the research results, and no statistical method can improve the results but a well-designed and conducted clinical research [ 22 ].

The primary objective of clinical research studies includes improvement in patient management. Most clinical research studies are aimed at discovering a new/novel drug to treat a medical condition that presently has no specific treatment, or the available drugs are not particularly effective in curing the disease.

The objectives are formed to address the five 'W's, namely who (children, women, etc.); what (the medical condition/disease/infection); why (causes of the medical condition/disease/infection); when (conditions responsible for the medical condition/disease/infection); and where (geographical aspects of the medical condition/disease/infection) as shown in Figure ​ Figure3 3 .

Clinical research can be of several types including primary research and secondary research. Also, the research can be observational (no intervention) and experimental/interventional. Clearly demarcated/framed research objectives are essential to improve the clarity, specificity, and focus of the clinical trial [ 23 ].

Patient-reported outcome measures (PROMs)

While conducting clinical research the investigators collect trial data through clinical observations, laboratory monitoring, and caregiver feedback. There are some aspects of the data like the patient-reported outcomes (PROs) that can be reported by the subject/patient him/herself either in the form of a questionnaire or through interviews. Such data collected from the patients in their words is termed PROMs. The PROMs include a global impression of the trial, the functional status and well-being, symptoms, health-related quality of life (HRQL), treatment compliance, and satisfaction. 

The questionnaire used to collect the PROs is called a PRO instrument. The data collected through this instrument is used to establish the benefit-to-risk ratio of the clinical trial drug. The PRO instruments can be designed as generic (contains a wide variety of health-related aspects and therefore can be used among different patient types), disease-specific (rheumatoid arthritis, psoriasis, etc.), dimension-specific (physical activity, cognitive levels, etc.), region/site-specific, individualized, utility measures, and summary items [ 24 ]. The patient-reported experience measure (PREMs), and the patient and public involvement programs are used to collect the patient’s feedback that invariably helps in improving the quality of healthcare facilities [ 25 ].

It is important to develop PROM tools/instruments for various diseases, especially among children as noted by a recent research report. This study suggested that a suitable PROM instrument is required to measure the status of disease (wheezing/asthma/respiratory diseases) and its control among preschool children [ 26 ].

Intention to treat and per-protocol analysis

The intention to treat (ITT) analysis is used when all the study subject’s data is analyzed including all those participants who were enrolled in the study, and those who have deviated (not signed the informed consent, discontinued from the study, not taken the trial drug as suggested). The ITT studies minimize the bias and ensure both the study and the control groups are compared.

The per-protocol (PP) analysis usually includes the data from only those subjects who have remained till the study period ended, have taken the drugs as suggested by the protocol, and was available for regular follow-up. The disadvantages of PP are potential disturbances in the balance between the study groups (randomized/placebo/control), a lower number of study participants due to the exclusion of dropouts, and non-compliant subjects. Therefore, the results from the PPA studies could be biased [ 27 ].

The randomized clinical trial (RCT) studies of superiority type use ITT analyses as against the non-inferiority and equivalence studies wherein an ITT approach may favor the study hypothesis.

According to the Committee for Proprietary Medicinal Products (CPMP), and the Consolidated Standards for reporting trials (CONSORT), both the ITT and PP analyses must be assessed to effectively interpret the results of the clinical trial including the safety and efficacy [ 28 ]. 

Endpoints in clinical research

The endpoints in clinical research determine whether the clinical trial has been successful in finding out if an intervention/drug has proven beneficial in improving the survival and quality of life of the patient. The endpoints determine the validity of the clinical trial results. There are different types of endpoints like primary endpoints, secondary endpoints, tertiary endpoints, surrogate endpoints (laboratory measurements, physical signs), and others [ 9 ].

The clinical trial endpoints could be subjective and objective in nature. The objective endpoints are survival, disease progression/remission, and the development of disease/condition. The subjective endpoints include symptoms, quality of life, and other patient-reported outcomes [ 29 , 30 ]. The significance of endpoints in clinical trials and the importance of choosing appropriate endpoints were previously reported. This study suggested that the primary endpoints help in deriving the sample size and confirm the generalizability of the results. The secondary and surrogate endpoints could be used while conducting the clinical research without ignoring the aspects of the quality of life of the subjects [ 31 ]. 

Endpoints in oncology clinical trials

Cancer clinical trials assume increased significance because the drugs developed against the cancer are intended to increase the survival of the patients. Also, anti-cancer agents are associated with several side effects. Therefore, oncology trials include several endpoints, the primary being the overall survival, and the secondary endpoints include the assessment of other outcomes that indicate the quality of life (QoL), tumor-related endpoints, and others. The disadvantages of oncology clinical trials are the cost associated with the recruitment of a greater number of subjects and long-time follow-up of the patients [ 32 ].

Although primary endpoints are considered as most significant in oncological trials, a recent report stressed the importance of surrogate markers in assessing the efficacy of anti-cancer drugs [ 33 ]. 

The endpoints in oncology clinical trials, their applications, functions, and drawbacks are summarized in Table ​ Table1 1 .

Survival endpoints in oncology clinical trials

The survival endpoint considers the time from randomization to death. This type of follow-up (daily), although difficult to do, will remove bias associated with the investigator’s interpretation. The survival studies require large sample sizes and cross-over therapies act as confounding factors for survival. The survival studies consider patient benefit over drug toxicity. 

Apart from the overall survival, oncology clinical trials use alternative ways to assess the efficacy of the drugs by using other endpoints like progression-free survival [ 34 ]. Other endpoints suggested are biomarkers, disease-free survival, objective response rate, time to progression, complete response, partial response, minor response, time to treatment failure, time to next treatment, duration of clinical benefit, objective response rate, complete response, pathological complete response, disease control rate, clinical benefit rate, milestone survival, event-free survival, and QoL [ 35 ].

Endpoints in immunological diseases and infections

Autoimmune diseases are usually chronic conditions that arise due to the immunologic responses against the self. They are usually associated with hyper-reactivity of immune cells towards the host's own cells/tissue and can cause significant morbidity and mortality among affected people.

Autoimmune diseases are generally genetic in origin, but many such diseases are attributed to other factors such as infection, food, drugs, and other substances. Frequently occurring autoimmune diseases are rheumatoid arthritis, psoriasis, systemic lupus erythematosus (SLE), ulcerative colitis, Crohn’s disease, and multiple sclerosis, among others [ 36 ].

Clinical trials with respect to the development of drugs/medicine to treat autoimmune diseases take into consideration the therapeutic efficacy of the trial drug, no risk, and only benefit to the patients. Therefore, the selection of endpoints for clinical trials in immune diseases must consider all these factors to effectively assess the pharmacological value of the trial drugs. The endpoints for autoimmune hepatitis include remission, incomplete response, treatment failure, and drug toxicity [ 37 ].

The endpoint related to the infections includes the direct measurement of the number of microorganisms. Other endpoints include the measurement of physiological aspects impaired by the infecting microbe and the measurement of immune responses against the infectious agent. The endpoints of human immunodeficiency virus (HIV) infection includes HIV-ribonucleic acid (RNA) viral load, maintenance, improvement, and decline of the cluster of differentiation 4 (CD4)+ T lymphocyte cell counts, and others.

Composite endpoints

Clinical trial for drugs is assessed based on several endpoints that establish efficacy and allow regulatory authorities to decide on approving the drugs for human use. In most instances, the clinical trials apply primary endpoints whereas in recent times the surge in candidate drugs and the necessity for life-saving drugs had ushered in the use of alternative endpoints like the composite endpoints. The composite endpoints are instrumental in reducing the trial costs, minimizing the long follow-ups, and lower subject recruitments. Since the composite endpoints combine more than one outcome during the drug trial, it enables the investigators to understand the efficacy of the drug in a short period of time [ 38 ].

While using multiple endpoints, it is important to understand that each one is as important as the other and the statistical methods must be used to confirm the overall efficacy of the trial drug. The drawbacks of applying composite endpoints in clinical trials are the complexity of the methods, low transparency, including invalid indicators, and the possibility of misleading results and conclusions [ 39 ].

Because there is no specific recommendation as to how the composite endpoints need to be selected, evaluated, and analyzed, there exists a possibility of bias. In a recent report, an index to evaluate the bias attributable to composite outcomes (BACO) was applied and suggested. The BACO index <0, 0 to <1, and >1 indicated that the composite endpoints were inverted, underestimated, and overestimated, respectively. A BACO index of 1 indicates that the composite endpoint usage resulted in unbiased results [ 40 ]. The composite endpoint in a clinical trial means the use of multiple endpoints. For the drug trials for migraine, the composite endpoints include no pain for two hours, nausea after two hours, and photosensitivity after two hours. The clinical trials with vaccines could have more than 20 endpoints. Other trials with multiple endpoints include rheumatoid arthritis (4), acne (4), sleep disorders (6), primary biliary cirrhosis (4), and glaucoma (9).

Conclusions

The clinical research design must be carefully accomplished keeping in mind the financial and time constraints. The trial must be initiated to address a specific research question, which essentially requires the research group to carry out an extensive literature search and identify the knowledge gaps. The research hypothesis needs to be carefully designed to avoid errors. The researchers should ensure the inclusion of appropriate objectives that guarantee quality outcomes and clinical benefits. It is essential to include the PROMs along with the clinician, researcher, and observer reported outcomes to assess the benefit-to-risk ratio of the investigational drug and improve the quality of healthcare facilities, among others. The safety and efficacy of a clinical trial drug should be carefully interpreted based on the results obtained from the ITT and PP analyses. Moreover, a clinical trial should incorporate specific and relevant endpoints that ensure the efficacy or otherwise of the intervention. Cancer clinical trials are even more complex because the interventions could potentially be life-saving and therefore, the selection of endpoints becomes critical as discussed briefly in this review.

The content published in Cureus is the result of clinical experience and/or research by independent individuals or organizations. Cureus is not responsible for the scientific accuracy or reliability of data or conclusions published herein. All content published within Cureus is intended only for educational, research and reference purposes. Additionally, articles published within Cureus should not be deemed a suitable substitute for the advice of a qualified health care professional. Do not disregard or avoid professional medical advice due to content published within Cureus.

The authors have declared that no competing interests exist.