multiple hypothesis generator

Hypothesis Maker Online

Looking for a hypothesis maker? This online tool for students will help you formulate a beautiful hypothesis quickly, efficiently, and for free.

Are you looking for an effective hypothesis maker online? Worry no more; try our online tool for students and formulate your hypothesis within no time.

• 🔎 How to Use the Tool?
• ⚗️ What Is a Hypothesis in Science?

👍 What Does a Good Hypothesis Mean?

• 🧭 Steps to Making a Good Hypothesis

🔗 References

📄 hypothesis maker: how to use it.

Our hypothesis maker is a simple and efficient tool you can access online for free.

If you want to create a research hypothesis quickly, you should fill out the research details in the given fields on the hypothesis generator.

Below are the fields you should complete to generate your hypothesis:

• Who or what is your research based on? For instance, the subject can be research group 1.
• What does the subject (research group 1) do?
• What does the subject affect? - This shows the predicted outcome, which is the object.
• Who or what will be compared with research group 1? (research group 2).

Once you fill the in the fields, you can click the ‘Make a hypothesis’ tab and get your results.

⚗️ What Is a Hypothesis in the Scientific Method?

A hypothesis is a statement describing an expectation or prediction of your research through observation.

It is similar to academic speculation and reasoning that discloses the outcome of your scientific test . An effective hypothesis, therefore, should be crafted carefully and with precision.

A good hypothesis should have dependent and independent variables . These variables are the elements you will test in your research method – it can be a concept, an event, or an object as long as it is observable.

You can observe the dependent variables while the independent variables keep changing during the experiment.

In a nutshell, a hypothesis directs and organizes the research methods you will use, forming a large section of research paper writing.

Hypothesis vs. Theory

A hypothesis is a realistic expectation that researchers make before any investigation. It is formulated and tested to prove whether the statement is true. A theory, on the other hand, is a factual principle supported by evidence. Thus, a theory is more fact-backed compared to a hypothesis.

Another difference is that a hypothesis is presented as a single statement , while a theory can be an assortment of things . Hypotheses are based on future possibilities toward a specific projection, but the results are uncertain. Theories are verified with undisputable results because of proper substantiation.

When it comes to data, a hypothesis relies on limited information , while a theory is established on an extensive data set tested on various conditions.

You should observe the stated assumption to prove its accuracy.

Since hypotheses have observable variables, their outcome is usually based on a specific occurrence. Conversely, theories are grounded on a general principle involving multiple experiments and research tests.

This general principle can apply to many specific cases.

The primary purpose of formulating a hypothesis is to present a tentative prediction for researchers to explore further through tests and observations. Theories, in their turn, aim to explain plausible occurrences in the form of a scientific study.

It would help to rely on several criteria to establish a good hypothesis. Below are the parameters you should use to analyze the quality of your hypothesis.

🧭 6 Steps to Making a Good Hypothesis

Writing a hypothesis becomes way simpler if you follow a tried-and-tested algorithm. Let’s explore how you can formulate a good hypothesis in a few steps:

Step #1: Ask Questions

The first step in hypothesis creation is asking real questions about the surrounding reality.

Why do things happen as they do? What are the causes of some occurrences?

Your curiosity will trigger great questions that you can use to formulate a stellar hypothesis. So, ensure you pick a research topic of interest to scrutinize the world’s phenomena, processes, and events.

Step #2: Do Initial Research

Carry out preliminary research and gather essential background information about your topic of choice.

The extent of the information you collect will depend on what you want to prove.

Your initial research can be complete with a few academic books or a simple Internet search for quick answers with relevant statistics.

Still, keep in mind that in this phase, it is too early to prove or disapprove of your hypothesis.

Step #3: Identify Your Variables

Now that you have a basic understanding of the topic, choose the dependent and independent variables.

Take note that independent variables are the ones you can’t control, so understand the limitations of your test before settling on a final hypothesis.

Step #4: Formulate Your Hypothesis

You can write your hypothesis as an ‘if – then’ expression . Presenting any hypothesis in this format is reliable since it describes the cause-and-effect you want to test.

For instance: If I study every day, then I will get good grades.

Step #5: Gather Relevant Data

Once you have identified your variables and formulated the hypothesis, you can start the experiment. Remember, the conclusion you make will be a proof or rebuttal of your initial assumption.

So, gather relevant information, whether for a simple or statistical hypothesis, because you need to back your statement.

Step #6: Record Your Findings

Finally, write down your conclusions in a research paper .

Outline in detail whether the test has proved or disproved your hypothesis.

Edit and proofread your work, using a plagiarism checker to ensure the authenticity of your text.

We hope that the above tips will be useful for you. Note that if you need to conduct business analysis, you can use the free templates we’ve prepared: SWOT , PESTLE , VRIO , SOAR , and Porter’s 5 Forces .

❓ Hypothesis Formulator FAQ

Updated: Oct 25th, 2023

• How to Write a Hypothesis in 6 Steps - Grammarly
• Forming a Good Hypothesis for Scientific Research
• The Hypothesis in Science Writing
• Scientific Method: Step 3: HYPOTHESIS - Subject Guides
• Hypothesis Template & Examples - Video & Lesson Transcript
• Free Essays
• Writing Tools
• Lit. Guides
• Donate a Paper
• Referencing Guides
• Free Textbooks
• Tongue Twisters
• Job Openings
• Expert Application
• Video Contest
• Writing Scholarship
• Discount Codes
• IvyPanda Shop
• Terms and Conditions
• Privacy Policy
• Cookies Policy
• Copyright Principles
• DMCA Request
• Service Notice

Use our hypothesis maker whenever you need to formulate a hypothesis for your study. We offer a very simple tool where you just need to provide basic info about your variables, subjects, and predicted outcomes. The rest is on us. Get a perfect hypothesis in no time!

Research Hypothesis Generator Online

• ️👍 Hypothesis Maker: the Benefits
• ️🔎 How to Use the Tool?
• ️🕵️ What Is a Research Hypothesis?
• ️⚗️ Scientific Method
• ️🔗 References

👍 Hypothesis Maker: the Benefits

Here are the key benefits of this null and alternative hypothesis generator.

🔎 Hypothesis Generator: How to Use It?

Whenever you conduct research, whether a 5-paragraph essay or a more complex assignment, you need to create a hypothesis for this study.

Clueless about how to create a good hypothesis?

No need to waste time and energy on this small portion of your writing process! You can always use our hypothesis creator to get a researchable assumption in no time.

To get a ready-made hypothesis idea, you need to:

• State the object of your study
• Specify what the object does
• Lay out the outcome of that activity
• Indicate the comparison group

Once all data is inserted into the fields, you can press the “Generate now” button and get the result from our hypothesis generator for research paper or any other academic task.

🕵️ What Is a Research Hypothesis?

A hypothesis is your assumption based on existing academic knowledge and observations of the surrounding natural world.

It also involves a healthy portion of intuition because you should arrive at an interesting, commonsense question about the phenomena or processes you observe.

The traditional formula for hypothesis generation is an “if…then” statement, reflecting its falsifiability and testability.

What do these terms mean?

• Testability means you can formulate a scientific guess and test it with data and analysis.
• Falsifiability is a related feature, allowing you to refute the hypothesis with data and show that your guess has no tangible support in real-world data.

For example, you might want to hypothesize the following:

If children are given enough free play time, their intelligence scores rise quicker.

You can test this assumption by observing and measuring two groups – children involved in much free play and those who don’t get free play time. Once the study period ends, you can measure the intelligence scores in both groups to see the difference, thus proving or disproving your hypothesis, which will be testing your hypothesis. If you find tangible differences between the two groups, your hypothesis will be proven, and if there is no difference, the hypothesis will prove false.

Null and Alternative Hypothesis

As a rule, hypotheses are presented in pairs in academic studies, as your scientific guess may be refuted or proved. Thus, you should formulate two hypotheses – a null and alternative variant of the same guess – to see which one is proved with your experiment.

The alternative hypothesis is formulated in an affirmative form, assuming a specific relationship between variables. In other words, you hypothesize that the predetermined outcome will be observed if one condition is met.

Watching films before sleep reduces the quality of sleep.

The null hypothesis is formulated in a negative form, suggesting that there is no association between the variables of your interest. For example:

Watching films before sleep doesn’t affect the quality of sleep.

⚗️ Creating a Hypothesis: the Key Steps

The development and testing of multiple hypotheses are the basis of the scientific method .

Without such inquiries, academic knowledge would never progress, and humanity would remain with a limited understanding of the natural world.

How can you contribute to the existing academic base with well-developed and rigorously planned scientific studies ? Here is an introduction to the empirical method of scientific inquiry.

Step #1: Observe the World Around You

Look around you to see what’s taking place in your academic area. If you’re a biology researcher, look into the untapped biological processes or intriguing facts that nobody has managed to explain before you.

What’s surprising or unusual in your observations? How can you approach this area of interest?

That’s the starting point of an academic journey to new knowledge.

Step #2: Ask Questions

Now that you've found a subject of interest, it's time to generate scientific research questions .

A question can be called scientific if it is well-defined, focuses on measurable dimensions, and is largely testable.

Some hints for a scientific question are:

• What effect does X produce on Y?
• What happens if the intensity of X’s impact reduces or rises?
• What is the primary cause of X?
• How is X related to Y in this group of people?
• How effective is X in the field of C?

As you can see, X is the independent variable , and Y is the dependent variable.

This principle of hypothesis formulation is vital for cases when you want to illustrate or measure the strength of one variable's effect on the other.

Step #3: Generate a Research Hypothesis

After asking the scientific question, you can hypothesize what your answer to it can be.

You don't have any data yet to answer the question confidently, but you can assume what effect you will observe during an empirical investigation.

For example, suppose your background research shows that protein consumption boosts muscle growth.

In that case, you can hypothesize that a sample group consuming much protein after physical training will exhibit better muscle growth dynamics compared to those who don’t eat protein. This way, you’re making a scientific guess based on your prior knowledge of the subject and your intuition.

Step #4: Hold an Experiment

With a hypothesis at hand, you can proceed to the empirical study for its testing. As a rule, you should have a clearly formulated methodology for proving or disproving your hypothesis before you create it. Otherwise, how can you know that it is testable? An effective hypothesis usually contains all data about the research context and the population of interest.

For example:

Marijuana consumption among U. S. college students reduces their motivation and academic achievement.

• The study sample here is college students.
• The dependent variable is motivation and academic achievement, which you can measure with any validated scale (e.g., Intrinsic Motivation Inventory).
• The inclusion criterion for the study's experimental group is marijuana use.
• The control group might be a group of marijuana non-users from the same population.
• A viable research methodology is to ask both groups to fill out the survey and compare the results.

Step #5: Analyze Your Findings

Once the study is over and you have the collected dataset, it's time to analyze the findings.

The methodology should also delineate the criteria for proving or disproving the hypothesis.

Using the previous section's example, your hypothesis is proven if the experimental group reveals lower motivational scores and has a lower GPA . If both groups' motivation and GPA scores aren't statistically different, your hypothesis is false.

Step #6: Formulate Your Conclusion

Using your study's hypothesis and outcomes, you can now generate a conclusion . If the alternative hypothesis is proven, you can conclude that marijuana use hinders students' achievement and motivation. If the null hypothesis is validated, you should report no identified relationship between low academic achievement and weed use.

Thank you for reading this article! Note that if you need to conduct a business analysis, you can try our free tools: SWOT , VRIO , SOAR , PESTEL , and Porter’s Five Forces .

❓ Research Hypothesis Generator FAQ

❓ what is a research hypothesis.

A hypothesis is a guess or assumption you make by looking at the available data from the natural world. You assume a specific relationship between variables or phenomena and formulate that supposition for further testing with experimentation and analysis.

❓ How to write a hypothesis?

To compose an effective hypothesis, you need to look at your research question and formulate a couple of ways to answer it. The available scientific data can guide you to assume your study's outcome. Thus, the hypothesis is a guess of how your research question will be answered by the end of your research.

❓ What is the difference between prediction and hypothesis?

A prediction is your forecast about the outcome of some activities or experimentation. It is a guess of what will happen if you perform some actions with a specific object or person. A hypothesis is a more in-depth inquiry into the way things are related. It is more about explaining specific mechanisms and relationships.

❓ What makes a good hypothesis?

A strong hypothesis should indicate the dependent and independent variables, specifying the relationship you assume between them. You can also strengthen your hypothesis by indicating a specific population group, an intervention period, and the context in which you'll hold the study.

🔗 References

• What is and How to Write a Good Hypothesis in Research?
• Research questions, hypotheses and objectives - PMC - NCBI
• Developing the research hypothesis - PubMed
• Alternative Hypothesis - SAGE Research Methods
• Alternative Hypothesis Guide: Definition, Types and Examples

Hypothesis Generator

Generate hypotheses for your research.

• Academic Research: Generate hypotheses for your thesis, dissertation, or any academic paper.
• Data Analysis: Create hypotheses to guide your data exploration and analysis.
• Market Research: Develop hypotheses to understand market trends and consumer behavior.
• Product Development: Formulate hypotheses to guide your product testing and development process.
• Scientific Research: Generate hypotheses for your experiments or observational studies.

New & Trending Tools

Ai journalist, text style transfer, one-page report writer.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Strong Hypothesis | Steps & Examples

How to Write a Strong Hypothesis | Steps & Examples

Published on May 6, 2022 by Shona McCombes . Revised on November 20, 2023.

A hypothesis is a statement that can be tested by scientific research. If you want to test a relationship between two or more variables, you need to write hypotheses before you start your experiment or data collection .

Example: Hypothesis

Daily apple consumption leads to fewer doctor’s visits.

Table of contents

What is a hypothesis, developing a hypothesis (with example), hypothesis examples, other interesting articles, frequently asked questions about writing hypotheses.

A hypothesis states your predictions about what your research will find. It is a tentative answer to your research question that has not yet been tested. For some research projects, you might have to write several hypotheses that address different aspects of your research question.

A hypothesis is not just a guess – it should be based on existing theories and knowledge. It also has to be testable, which means you can support or refute it through scientific research methods (such as experiments, observations and statistical analysis of data).

Variables in hypotheses

Hypotheses propose a relationship between two or more types of variables .

• An independent variable is something the researcher changes or controls.
• A dependent variable is something the researcher observes and measures.

If there are any control variables , extraneous variables , or confounding variables , be sure to jot those down as you go to minimize the chances that research bias  will affect your results.

In this example, the independent variable is exposure to the sun – the assumed cause . The dependent variable is the level of happiness – the assumed effect .

Prevent plagiarism. Run a free check.

Step 1. ask a question.

Writing a hypothesis begins with a research question that you want to answer. The question should be focused, specific, and researchable within the constraints of your project.

Step 2. Do some preliminary research

Your initial answer to the question should be based on what is already known about the topic. Look for theories and previous studies to help you form educated assumptions about what your research will find.

At this stage, you might construct a conceptual framework to ensure that you’re embarking on a relevant topic . This can also help you identify which variables you will study and what you think the relationships are between them. Sometimes, you’ll have to operationalize more complex constructs.

Step 3. Formulate your hypothesis

Now you should have some idea of what you expect to find. Write your initial answer to the question in a clear, concise sentence.

4. Refine your hypothesis

You need to make sure your hypothesis is specific and testable. There are various ways of phrasing a hypothesis, but all the terms you use should have clear definitions, and the hypothesis should contain:

• The relevant variables
• The specific group being studied
• The predicted outcome of the experiment or analysis

5. Phrase your hypothesis in three ways

To identify the variables, you can write a simple prediction in  if…then form. The first part of the sentence states the independent variable and the second part states the dependent variable.

In academic research, hypotheses are more commonly phrased in terms of correlations or effects, where you directly state the predicted relationship between variables.

If you are comparing two groups, the hypothesis can state what difference you expect to find between them.

6. Write a null hypothesis

If your research involves statistical hypothesis testing , you will also have to write a null hypothesis . The null hypothesis is the default position that there is no association between the variables. The null hypothesis is written as H 0 , while the alternative hypothesis is H 1 or H a .

• H 0 : The number of lectures attended by first-year students has no effect on their final exam scores.
• H 1 : The number of lectures attended by first-year students has a positive effect on their final exam scores.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A hypothesis is not just a guess — it should be based on existing theories and knowledge. It also has to be testable, which means you can support or refute it through scientific research methods (such as experiments, observations and statistical analysis of data).

Null and alternative hypotheses are used in statistical hypothesis testing . The null hypothesis of a test always predicts no effect or no relationship between variables, while the alternative hypothesis states your research prediction of an effect or relationship.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). How to Write a Strong Hypothesis | Steps & Examples. Scribbr. Retrieved March 30, 2024, from https://www.scribbr.com/methodology/hypothesis/

Is this article helpful?

Shona McCombes

Other students also liked, construct validity | definition, types, & examples, what is a conceptual framework | tips & examples, operationalization | a guide with examples, pros & cons, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Discover Your Solutions

Making a better world; one solution at a time..

• Ultimate Guide
• Get Organized
• Know Your Problem
• Know Yourself

Generate Hypotheses

• Cross the Finish Line
• Analyze Your Problem
• Create Solutions
• Evaluate Solutions
• All-In-One Solvers
• Other Resources
• Publications
• Detailed Information
• Introduction
• Download Center
• News & PR

Not a member yet?

Click here to 

Creating Your Solutions

In this section, you create solutions and test them.

Creating Hypothetical Solutions

First, create potential solutions using the Hypothesis Generator tool.

Now, use the Multiple Hypothesis Generator to develop a large set of possible solutions.

Testing Solutions

Next, create a matrix to test and rank your solutions.

For ranking really complex solutions, you may want to use a free computer program called 'Analysis of Completing Hypotheses (ACH).

Potential Problem Analysis

Finally, analyze your solutions for potential problems.

Go to the next section - 'Cross the Finish Line'.

Copyright © 2012-15 Discover Your Solutions LLC. All Rights Reserved. Privacy Policy | Terms of Use

Online Hypothesis Generator

Add the required information into the fields below to build a list of well-formulated hypotheses.

• If patients follow medical prescriptions, then their condition will improve.
• If patients follow medical prescriptions, then their condition will show better results.
• If patients follow medical prescriptions, then their condition will show better results than those who do not follow medical prescriptions.
• H0 (null hypothesis) - Attending most lectures by first-year students has no effect on their exam scores.
• H1 (alternative hypothesis) - Attending most lectures by first-year students has a positive effect on their exam scores.

* Hint - choose either null or alternative hypothesis

⭐️ Hypothesis Creator: the Benefits

• 🔎 How to Use the Tool?
• 🤔 What Is a Hypothesis?
• 👣 Steps to Generating a Hypothesis
• 🔍 References

🔎 Hypothesis Generator: How to Use It?

The generation of a workable hypothesis is not an easy task for many students. You need to research widely, understand the gaps in your study area, and comprehend the method of hypothesis formulation to the dot. Lucky for you, we have a handy hypothesis generator that takes hours of tedious work out of your study process.

To use our hypothesis generator, you’ll need to do the following:

• Indicate your experimental group (people, phenomena, event)
• Stipulate what it does
• Add the effect that the subject’s activities produce
• Specify the comparison group

Once you put all this data into our online hypothesis generator, click on the “Generate hypothesis” tab and enjoy instant results. The tool will come up with a well-formulated hypothesis in seconds.

🤔 What Is a Research Hypothesis?

A hypothesis is a claim or statement you make about the assumed relationship between the dependent and independent variables you're planning to test. It is formulated at the beginning of your study to show the direction you will take in the analysis of your subject of interest.

The hypothesis works in tandem with your research purpose and research question , delineating your entire perspective.

For example, if you focus on the quality of palliative care in the USA , your perspective may be as follows.

This way, your hypothesis serves as a tentative answer to your research question, which you aim to prove or disprove with scientific data, statistics, and analysis.

Hypothesis Types

In most scholarly studies, you’ll be required to write hypotheses in pairs – as a null and alternative hypothesis :

• The alternative hypothesis assumes a statistically significant relationship between the identified variables. Thus, if you find that relationship in the analysis process, you can consider the alternative hypothesis proven.
• A null hypothesis is the opposite; it assumes that there is no relationship between the variables. Thus, if you find no statistically significant association, the null hypothesis is considered proven.

A handy example is as follows:

You are researching the impact of sugar intake on child obesity . So, based on your data, you can either find that the number of sugar spoons a day directly impacts obesity or that the sugar intake is not associated with obesity in your sample. The hypotheses for this study would be as follows:

ALTERNATIVE

There is a relationship between the number of sugar spoons consumed daily and obesity in U.S. preschoolers.

There is no relationship between the number of sugar spoons consumed daily and obesity in U.S. preschoolers.

Besides, hypotheses can be directional and non-directional by type:

• A directional hypothesis assumes a cause-and-effect relationship between variables, clearly designating the assumed difference in study groups or parameters.
• A non-directional hypothesis , in turn, only assumes a relationship or difference without a clear estimate of its direction.

NON-DIRECTIONAL

Students in high school and college perform differently on critical thinking tests.

DIRECTIONAL

College students perform better on critical thinking tests that high-school students.

👣 How to Make a Hypothesis in Research

Now let’s cover the algorithm of hypothesis generation to make this process simple and manageable for you.

Step #1: Formulate Your Research Question

The first step is to create a research question . Study the topic of interest and clarify what aspect you're fascinated about, wishing to learn more about the hidden connections, effects, and relationships.

Step #2: Research the Topic

Next, you should conduct some research to test your assumption and see whether there’s enough published evidence to back up your point. You should find credible sources that discuss the concepts you’ve singled out for the study and delineate a relationship between them. Once you identify a reasonable body of research, it’s time to go on.

Step #3: Make an Assumption

With some scholarly data, you should now be better positioned to make a researchable assumption.

For instance, if you find out that many scholars associate heavy social media use with a feeling of loneliness, you can hypothesize that the hours spent on social networks will directly correlate with perceived loneliness intensity.

Step #4: Improve Your Hypothesis

Now that you have a hypothesis, it’s time to refine it by adding context and population specifics. Who will you study? What social network will you focus on? In this example, you can focus on the student sample’s use of Instagram .

Step #5: Try Different Phrasing

The final step is the proper presentation of your hypothesis. You can try several variants, focusing on the variables, correlations , or groups you compare.

For instance, you can say that students spending 3+ hours on Instagram every day are lonelier than their peers. Otherwise, you can hypothesize that heavy social media use leads to elevated feelings of loneliness.

👀 Null Hypothesis Examples

If you’re unsure about how to generate great hypotheses, get some inspiration from the list of examples formulated by our writing pros.

Thank you for reading this article! If you’re planning to analyze business issues, try our free templates: PEST , PESTEL , SWOT , SOAR , VRIO , and Five Forces .

❓ Hypothesis Generator FAQ

❓ what does hypothesis mean.

A hypothesis in an essay or a larger research assignment is your claim or prediction of the relationship you assume between the identified dependent and independent variables. You share an assumption that you’re going to test with research and data analysis in the later sections of your paper.

❓ How to create a hypothesis?

The first step to formulating a good hypothesis is to ask a question about your subject of interest and understand what effects it may experience from external sources or how it changes over time. You can identify differences between groups and inquire into the nature of those distinctions. In any way, you need to voice some assumption that you’ll further test with data; that assumption will be your hypothesis for a study.

❓ What is a null and alternative hypothesis?

You need to formulate a null and alternative hypothesis if you plan to test some relationship between variables with statistical instruments. For example, you might compare a group of students on an emotional intelligence scale to determine whether first-year students are less emotionally competent than graduates. In this case, your alternative hypothesis would state that they are, and a null hypothesis would say that there is no difference between student groups.

❓ What does it mean to reject the null hypothesis?

A null hypothesis assumes that there is no difference between groups or that the dependent variables don't have any sizable impact on the independent variable. If your null hypothesis gets rejected, it means that your alternative hypothesis has been proved, showing that there is a tangible difference or relationship between your variables.

🔗 References

• How to Write a Hypothesis in 6 Steps - Grammarly
• The Hypothesis in Science Writing
• Hypothesis Definition & Examples - Simply Psychology
• Hypothesis Examples: Different Types in Science and Research
• Forming a Good Hypothesis for Scientific Research

MultipleTesting.com: A tool for life science researchers for multiple hypothesis testing correction

Affiliations.

• 1 Department of Bioinformatics, Semmelweis University, Budapest, Hungary.
• 2 Research Centre for Natural Sciences, Cancer Biomarker Research Group, Institute of Enzymology, Budapest, Hungary.
• 3 A5 Genetics Ltd, Und, Hungary.
• 4 2nd Department of Pediatrics, Semmelweis University, Budapest, Hungary.
• PMID: 34106935
• PMCID: PMC8189492
• DOI: 10.1371/journal.pone.0245824

Scientists from nearly all disciplines face the problem of simultaneously evaluating many hypotheses. Conducting multiple comparisons increases the likelihood that a non-negligible proportion of associations will be false positives, clouding real discoveries. Drawing valid conclusions require taking into account the number of performed statistical tests and adjusting the statistical confidence measures. Several strategies exist to overcome the problem of multiple hypothesis testing. We aim to summarize critical statistical concepts and widely used correction approaches while also draw attention to frequently misinterpreted notions of statistical inference. We provide a step-by-step description of each multiple-testing correction method with clear examples and present an easy-to-follow guide for selecting the most suitable correction technique. To facilitate multiple-testing corrections, we developed a fully automated solution not requiring programming skills or the use of a command line. Our registration free online tool is available at www.multipletesting.com and compiles the five most frequently used adjustment tools, including the Bonferroni, the Holm (step-down), the Hochberg (step-up) corrections, allows to calculate False Discovery Rates (FDR) and q-values. The current summary provides a much needed practical synthesis of basic statistical concepts regarding multiple hypothesis testing in a comprehensible language with well-illustrated examples. The web tool will fill the gap for life science researchers by providing a user-friendly substitute for command-line alternatives.

Publication types

• Research Support, Non-U.S. Gov't
• Biological Science Disciplines / methods*
• Data Interpretation, Statistical
• Models, Theoretical*
• Statistics as Topic*

Grants and funding

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• R Soc Open Sci
• v.7(6); 2020 Jun

A modern method of multiple working hypotheses to improve inference in ecology

Scott w. yanco.

1 Department of Integrative Biology, University of Colorado Denver, Denver, CO, USA

Andrew McDevitt

Clive n. trueman.

2 Ocean and Earth Science, University of Southampton, National Oceanography Centre, Southampton, UK

Laurel Hartley

Michael b. wunder, associated data.

No data are included in this paper. All code generating worked examples are included in electronic supplementary material, 1 and 2, and the checkyourself package used in those examples is available on github: https://github.com/syanco/checkyourself and has been archived within the Zenodo repository https://doi.org/10.5281/zenodo.3743038 .

Science provides a method to learn about the relationships between observed patterns and the processes that generate them. However, inference can be confounded when an observed pattern cannot be clearly and wholly attributed to a hypothesized process. Over-reliance on traditional single-hypothesis methods (i.e. null hypothesis significance testing) has resulted in replication crises in several disciplines, and ecology exhibits features common to these fields (e.g. low-power study designs, questionable research practices, etc.). Considering multiple working hypotheses in combination with pre-data collection modelling can be an effective means to mitigate many of these problems. We present a framework for explicitly modelling systems in which relevant processes are commonly omitted, overlooked or not considered and provide a formal workflow for a pre-data collection analysis of multiple candidate hypotheses. We advocate for and suggest ways that pre-data collection modelling can be combined with consideration of multiple working hypotheses to improve the efficiency and accuracy of research in ecology.

1. Replication crises and inferential frameworks

The ultimate goal of science is to learn about the relationships between observable patterns in the world around us and the processes that generate those patterns. Most commonly, scientists identify and/or quantify the links between process and pattern by hypothesizing the existence of a particular relationship between the two and using data as evidence for or against that hypothesis. However, inference may be unreliable if the scientist does not consider all potentially relevant processes. For example, inference is confounded when unconsidered hypotheses produce the same observed pattern as the stated hypothesis. Similarly, inference is muddled when hypotheses overlook additional variance-inflating processes, effectively rendering the link between process and pattern indiscernible. In either case, researchers who do not carefully guard against such pitfalls may make inferences that are either too strong or too weak.

Recently, several scientific disciplines have experienced ‘replication crises' (e.g. cancer biology [ 1 ] and psychology [ 2 ] among others [ 3 ]). Many factors have probably contributed to replication crises: publication bias [ 4 , 5 ], hypothesizing after results are known [ 6 ], p -hacking [ 5 , 7 ] and data fabrication [ 8 ] to name a few. In addition to these factors, irreproducibility has also been driven by an over-reliance on null hypothesis significance testing (NHST; [ 1 , 9 – 12 ]). The limitations, misuse and outright abuse of NHST are myriad and, by now, well known (see, for example, [ 5 , 12 – 15 ]). NHST produces erroneous inference both because it is frequently misinterpreted by researchers [ 12 , 14 , 16 – 18 ] and because it is prone to manipulation [ 5 , 14 ].

One potentially underappreciated limitation of NHST is that it does not produce evidential support for hypotheses, instead providing only weak evidence of incongruence between observed data and a null hypothesis [ 12 ]. The ubiquitous p -value quantifies only the probability of hypothetical future data resulting in some summary statistic that would be less consistent with summary statistics computed from data generated by the null hypothesis. If that probability is sufficiently low (e.g. p < 0.05), the researcher ‘rejects' the null hypothesis as having been unlikely to generate the observed data (as in [ 19 ]). Often, ‘rejection of the null' leads (illogically) to acceptance of whatever was proposed as the alternative to that strawman; the alternative hypothesis is accepted without any positive inferential support [ 14 ]. Furthermore, the NHST framework considers only a single hypothesis. Indeed, the complement to the null hypothesis comprises a set of alternative hypotheses. In other words, a ‘significant' significance test indicates that data like ours are improbable given a single null hypothesis [ 12 , 14 ]––it produces no information about the infinite number of possible alternative hypotheses [ 20 ]. Imagine the potential for error when an automatically accepted alternative hypothesis is not uniquely distinguishable from some other hypothesis the researcher never considered.

Here, we describe methods for considering multiple hypotheses by advocating for the implementation of multi-hypothesis modelling prior to data collection. Akin to in silico experimentation, design phase modelling helps to identify a plausible set of candidate hypotheses and determine which of the set might lead to any of several different observable patterns [ 21 , 22 ]. Below, we detail the nature of problematic sets of hypotheses and draw on the oft-invoked ‘method of multiple working hypotheses' [ 23 ] as a partial solution. This method has been repeatedly invoked as an important component of good scientific practice (e.g. [ 24 – 26 ]). In this paper, we propose a workflow invoking the method of multiple working hypotheses in the context of pre-data collection modelling. Our workflow applies recommended practices in theoretical modelling to the problem of design phase modelling with particular emphasis on the consideration of multiple hypotheses. The practical recommendations in our approach are intended to facilitate wider adoption of multiple hypothesis methods, guard against inferential errors to which multi-hypothesis methods are still prone and provide a formal framework for such analyses. This combination of multi-hypothesis inference and pre-data collection modelling represents a powerful alternative incarnation of the scientific method geared towards stronger inference that is less susceptible to errors arising from unconsidered processes.

Specifically, we outline five steps for vetting hypotheses. These steps can be repeated iteratively until the proposed mechanisms and observation patterns adequately map to one another ( figure 1 ). The steps are:

• 1. specify candidate hypotheses;
• 2. write a model for each hypothesis;
• 3. generate sampling distributions of simulated data from each hypothesis;
• 4. quantify the variance within and overlap between sampling distributions; and
• 5. revise hypotheses as necessary and repeat steps 1–4.

Conceptual flow chart of hypothesis vetting process. Researchers first specify a set of candidate hypotheses to consider before writing them as formal models. Formal models are checked for internal coherence and revised, if necessary. Sampling distributions of simulated response variables are generated from each candidate hypothesis which can then be compared to one another for evidence of degeneracy or noisiness. If no such problems exist, the researcher proceeds with data-based inference. Alternatively, the researcher revises the set of candidate hypotheses and begins the hypothesis vetting anew.

2. The effects of unconsidered alternative hypotheses

Scientific inference and, in particular, inference using NHST assumes that processes are uniquely identifiable from the observable patterns they generate ( figure 2 ). That is, they depend on the statistical concept of identifiability. Model parametrizations are identifiable if and only if distinct parametrizations lead to different probability distribution functions [ 27 ].

Heuristic relationships between processes and observable patterns that drive inferential outcomes. Boxes linked by arrows represent individual hypotheses that can or cannot be parsed based on observed patterns. Density plots show the examples of sampling distributions arising from each hypothesis. Processes detectable from patterns: for a hypothesis to be testable, the response patterns must reliably, quantifiably and uniquely correspond to the hypothesized mechanisms. Note how each process is uniquely linked to a distinct pattern with little or no overlap between sampling distributions. Degenerate relationship: multiple mechanisms degenerating to an indistinguishable response pattern. Each unique process leads to the same observation pattern, and sampling distributions are almost completely overlapping. Noisy relationship: a single mechanism does not reliably produce a concordant response pattern. A single hypothesized process leads to a widely varying response pattern. High variance and/or multi-modal sampling distribution makes estimation difficult or impossible.

Muddled inference (i.e. non-identifiability) manifests in two ways: (i) degenerate relationship: multiple processes produce indistinguishable patterns, or (ii) noisy relationship: processes do not reliably produce a single identifiable pattern ( figure 2 ; [ 28 , 29 ]).

2.1. Degenerate relationship

Hypotheses with degenerate relationships between pattern and process are not testable––a fundamental requirement to differentiate hypotheses. In degenerate cases, a single observed pattern could have been produced by more than one process ( figure 2 ; [ 28 , 29 ]). Thus, no single process can be uniquely implicated by the observation. Degeneracy may occur because unconsidered deterministic or stochastic processes modify the resultant pattern. At its heart, this phenomenon arises due to model misspecification wherein two or more models (hypotheses), as specified by the researchers, produce indistinguishable response patterns [ 29 ]. In this situation, no observation can serve as evidence of any unique process because multiple processes could have produced the same pattern.

2.2. Noisy relationship

Noisy relationships are those wherein a single mechanism produces multiple and varied response patterns potentially due to unrecognized or unconsidered mechanisms ( figure 2 ). Too much variance leads to low predictive power and imprecise estimates of model parameters [ 27 ]. Like the degenerate relationship problem, this also results in the same muddled inference. Noisy relationships between patterns and processes commonly arise from observation or measurement errors, or from a mis-specified model.

3. The method of multiple working hypotheses revisited again

While inferential failures leading to replication crises have garnered much recent attention [ 15 ], they are hardly new. Cohen [ 30 ] pointed out flaws in NHST over 25 years ago––and in so doing reminded readers that Bakan [ 31 ] made similar arguments over 30 years prior to that. In 1964, Platt described ‘strong inference' which grounded much of what Ioannidis [ 9 ] demonstrated over 40 years later. In fact, as early as 1890, Thomas Chamberlin described the ‘method of multiple working hypotheses' and it has since been repeatedly advocated as a way to mitigate the risk of omitting potentially relevant processes from inference [ 23 – 25 , 32 ].

Chamberlin [ 23 ] advocated that, to avoid foreseeable inferential errors, researchers should explicitly consider multiple working hypotheses from the outset. The method is intended to reduce cognitive biases which cause researchers to only collect evidence for favoured hypotheses. Additionally, Chamberlin points out that single-hypothesis frameworks fail to adequately account for complex systems wherein multiple processes may play causal roles—as is common in ecology [ 23 , 25 ]. Using this method, a researcher ‘competes' evidence about as many hypotheses as are plausible rather than simply considering the evidence against a strawman hypothesis (as in NHST).

Despite at least 130 years of advocacy for multi-hypothesis approaches, consideration of multiple hypotheses in ecology continues to be rare [ 33 ]. For example, Betini et al . [ 26 ] found that only 21% of a sample of recently published papers in ecology and evolution considered multiple hypotheses. Yet, the systems investigated in these fields are precisely those which stand to benefit from multi-hypothesis approaches (i.e. those involving multiple interacting causal factors; [ 34 ]).

Observable patterns arising from myriad interacting variance-generating processes is the norm in ecology. Such complex causal structures are prone to both the degenerate and noisy relationship problems [ 34 ]. Consider just a few examples chosen from sub-disciplines within ecology: Boeklen et al . [ 35 ] identified at least 44, hierarchically organized, factors that influence emergent patterns of tissue stable isotopes used in trophic ecology studies. Several authors have observed sufficient variance in species distributions to produce absurd or impossible model fits [ 36 ]. For example, Fourcade et al . [ 37 ] demonstrated that rasterized paintings projected onto landscapes provided comparable or better fitting models for species distributions than real environmental variables (see also box 1 and electronic supplementary material, 1). Finally, Nathan et al . [ 40 ] showed that animal movements emerge from an interaction between the organism's motility, capacity to navigate, internal state and external environmental setting––each component of which may themselves entail multiple interacting variables (see electronic supplementary material, 2). These are examples of fields wherein identifying mechanistic drivers via observed patterns is challenging because of the multifaceted nature of the problems at hand—a ubiquitous scenario in ecology. As such, establishing the identifiability of the set of plausible hypotheses should be regarded as the default first step towards reliable inference in ecology.

Vetting hypotheses about what drives species distributions.

Species distribution models (SDMs) seek to explain the spatial distribution and abundance of organisms as a function of some environmental variable(s). However, these models often overfit datasets with the complex combinations of environmental variables while failing to provide useful predictive power resulting in occasionally impossible parameter estimates or model selections [ 36 ]. For example, Fourcade et al . [ 37 ] demonstrated that rasterized paintings projected onto the landscape provided comparable or better fitting models for species distributions than real environmental variables ( figure 3 ). This suggests that SDMs may not be considering the full range of potential mechanistic drivers of species distributions (e.g. conspecific attraction, neutral distributions, density dependence, etc.)

Reproduced from [ 37 ] ‘Workflow used in analyses: 20 pseudo-predictors were created from the projection of paintings on the Western Palaearctic geographical space (examples: top: John Singer Sargent, Blonde Model, bottom: Zhang Daqian, Spring dawns upon the colourful hills) and were used to compute species distribution models (SDMs) after principal components analysis (PCA). A set of 20 true environmental variables (climate and topography) was also used to compute SDMs for the same species. Both types of models were evaluated using area under the receiver operating curve (AUC) and true skill statistics (TSS). The SDMs presented at the bottom show the example of a species ( Candidula unifasciata , a land snail species) for which the SDM computed with pseudo-predictors led to better evaluation metrics (here computed by randomly splitting occurrences into training and testing datasets) than that computed with real environmental variables (suitability increases from blue to red). AUCp = AUC for model computed with painting-derived pseudo-predictors; AUCe = AUC for model computed with real environmental variables; TSSp = TSS for model computed with painting-derived pseudo-predictors; TSSe = TSS for model computed with real environmental variables.'

We used a simple individual-based simulation model (more details in electronic supplementary material, 1) of animals settling a landscape to consider multiple competing hypotheses about processes that give rise to species distributions. Specifically, we vetted three competing hypotheses about how a population of animals may settle a patchy landscape:

• 1. Null model . Individuals settle the landscape randomly with no influence of habitat or neighbours.
• 2. Habitat preference (HP) model . Individuals settle the landscape preferring ‘Habitat A' over ‘Habitat B'.
• 3. Conspecific attraction (CA) model . Individuals settle the landscape preferring to settle near already-settled locations.

Noisy hypotheses

In order to examine the variances produced by each model, compare variances between models and examine how variance relates to parametrization, we calculated and plotted the range for each sampling distribution produced by the 11 model parametrization combinations ( figure 4 ).

Whisker plot of sampling distribution ranges for each parametrization of each hypothesis used to detect noisy hypotheses. Wider whiskers indicate lower precision in parameter estimation and potential evidence of a noisy relationship.

The models containing the strongest conspecific attraction produced the highest variances. As conspecific attraction gets weaker, the values and variance become comparable to the null model. There is also clear structure in the values estimated by the habitat preference models: we observed a higher proportion of ‘Habitat A' selected by models containing stronger habitat preference. Variance was relatively constant between models suggesting that parameter estimation under this hypothesis would be similarly accurate regardless of the magnitude of the parameter estimate itself.

Hypothesis degeneration

To compare sampling distributions to each other to search for degenerate relationships, we calculated the unidirectional pairwise overlap between all sampling distributions. Each overlap was unidirectional, since different model parametrizations produced unequal variances––the overlap between any two sampling distributions was asymmetric. We combined all unidirectional pairwise comparisons into heatmaps to assess patterns of overlap in parameter combinations; each unidirectional pairwise proportion of overlap represents the conditional probability of one hypothesis generating response data capable of being produced by another hypothesis ( figure 5 ).

Heatmap of sampling distribution overlap. Panels clockwise from top-left: p(HP|RAND) shows the proportion of habitat preference model simulations that overlapped the range of null models for each parametrization; p(HP|CA) shows the proportion of habitat preference model simulations that overlapped the range of conspecific attraction models for each parametrization; p(CA|RAND) shows the proportion of conspecific attraction model simulations that overlapped the range of null models for each parametrization; p(RAND|CA) shows the proportion of null model simulations that overlapped the range of conspecific attraction models for each parametrization; p(CA|HP)shows the proportion of conspecific attraction model simulations that overlapped the range of habitat preference models for each parametrization; p(RAND|HP) shows the proportion of null model simulations that overlapped the range of habitat preference models for each parametrization.

We observed clear structures in the degeneracy of certain model parametrization combinations. For example, the proportion of habitat preference models overlapped by conspecific attraction models was very high for models with low strength of preference and/or strong conspecific attraction. Conversely, the proportion of conspecific attraction models that overlapped habitat preference models was generally low except for models with very strong habitat preference and strong conspecific attraction ( figure 5 ).

Revising hypotheses

Given both the large variance generated for the null model and the high amount of overlap in sampling distributions between several model parametrization combinations, it is reasonable to assume that a researcher in this situation would seek to refine their proposed study. There are myriad options for such revision and in a ‘real-world' examination this would rest on the judgement and system-specific knowledge of the researcher as well as the specific aims of the study. We offer a few potential revisions here to illustrate the types of changes that could be made but in no way suggest that these revisions are exhaustive or appropriate to the system.

By including spatial measures as part of the observed response pattern, models that produced degenerate response patterns may now be parsed. For example, many of the models that hypothesized conspecific attraction exhibited strong spatial clustering, probably resulting from the strong influence of the initially settled location ( figure 6 ).

Examples of spatial distributions of settled agents in two model iterations. Left: A strong conspecific attraction parametrization. Note the very strong spatial clustering. Right: Parametrized for habitat preference, the model generates a more diffuse spatial pattern. While both these models produced substantially overlapping sampling distributions, spatial metrics could be used to parse hypotheses.

Manipulative experimentation could also parse convergent hypotheses. For example, decoy experiments have been used as a test of conspecific attraction (e.g. [ 38 ]). Alternatively, habitat manipulation could also parse degenerate hypotheses (e.g. [ 39 ]).

Addressing model parametrizations exhibiting high levels of variance may be more difficult. Because the simulation model assumes no observation error, additional processes or poorly constrained processes are the likely culprits. Indeed, we can see that the spatial distribution in the conspecific attraction model is actually a combination of two separate processes: (i) the initial individual settles randomly and (ii) subsequent individuals settle based on the conspecific attraction decisions rules.

Compounding the effects of underlying complexity, problematic inferential practices may be common in ecology. For example, Fraser et al . [ 41 ] found that questionable research practices were widespread––observing rates comparable to fields whose replication crises are well established. In fact, recent, large-scale studies have shown that early, low-power findings in some sub-fields apparently do not replicate (e.g. [ 42 ]). In combination, these facts make clear that ecology must improve its inferential toolbox.

Frameworks that support data-based inference between multiple hypotheses are well established (e.g. information theoretic approaches to multi-model inference, [ 13 , 33 , 43 – 47 ]), and have even been explicitly linked to Chamberlin's method [ 25 ]. Yet, while some sub-disciplines within ecology have seen wider adoption of these tools [ 25 ], clearly they remain underused [ 26 , 33 ]. More importantly, a posteriori multi-hypothesis methods cannot disentangle hypotheses that are confounded a priori (i.e. those that are structurally non-identifiable). As such, employing Chamberlin's method at the earliest stages of research may improve inference. Therefore, pre-data collection modelling is an essential first step in considering multiple hypotheses.

4. Pre-data collection modelling enables the method of multiple working hypotheses

Models constructed prior to data collection can provide insights allowing researchers to quantify and, ultimately, to increase clarity and transparency about hypotheses [ 22 , 48 , 49 ]. These models are essentially in silico experiments which simulate response variables using predefined parameters taking on biologically relevant values. Specifying a model forces the researcher to explicitly consider the nature of linkages between the process(es) under investigation and the pattern(s) observed [ 49 ]. By using biologically defined parameters, the simulated pattern is clearly understood because the structural components of the model explicitly represent biological links between process and pattern [ 22 ]. Comparing simulated responses across multiple alternative hypotheses allows a researcher to quantify the identifiability of each candidate model.

This step, though formally distinct, is analogous to a power analysis wherein researchers use pre-data collection models to ensure that the proposed sample will be sufficient to answer the question at hand. Whereas a power analysis assesses the sufficiency of sample sizes (given some assumed effect size), our framework assesses the identifiability of each hypothesis. Both analyses are ways to ensure, at the outset, that a proposed study is even theoretically capable of producing an answer.

Modelling in this context embraces the method of multiple hypotheses: researchers consider not only a favoured hypothesis but also alternative formulations. This uncovers situations wherein multiple processes might produce observable patterns that are indistinguishable from one another. Of course, engaging in this process does not guarantee that all possible processes will be identified. In fact, there always remains the possibility of a plausible hypothesis a researcher has yet to consider. Betini et al . [ 26 ] describe typical cognitive barriers that prevent researchers from articulating a complete set of multiple working hypotheses (e.g. lack of creativity, lack of time or incentives to expend the effort, lack of practice with or comfort with brainstorming alternative hypotheses, etc.). The workflow we propose below does not, by itself, overcome those cognitive challenges but we believe it provides a facilitating framework.

5. A workflow for vetting multiple working hypotheses

We term the process of modelling multiple hypotheses in the design phase ‘hypothesis vetting'. The outcome of hypothesis vetting is to determine whether each candidate hypothesis is uniquely identifiable. Hypothesis vetting is carried out by formally defining variance-generating process(es) and the pattern(s) they produce for each competing hypothesis. In simple systems, this can be accomplished using an analytically tractable equation or set of equations. In more complex systems, this process may require numerical approaches or researchers might instead employ algorithmic simulation models (particularly stochastic models and/or agent-based models).

Each hypothesis is modelled as a unique combination of processes (i.e. variables) and/or a unique combination of linkages between processes and patterns (specific parametrizations). Subsequently, the comparisons of simulation outputs quantify the degree to which patterns can uniquely identify hypothesized process(es). Importantly, this approach provides inference only about the ability to differentiate between simulation models (as in [ 22 ]), and not about the validity of any specific model itself.

Below we describe each step in the workflow; box 1 contains a stylized example of implementing this workflow (R code, using the checkyourself package, for this example and another is contained in electronic supplementary material, 1 and 2). The provided example is drawn from spatial ecology, but the workflow could (and should) be extended/applied across the diverse sub-disciplines in ecology. For a non-spatial example, we also refer readers to Vagle and McCain [ 50 ] who demonstrated a priori degeneracy between competing hypotheses about the mechanisms underlying primary productivity–diversity relationships.

5.1. Step 1: specify candidate hypotheses

To vet hypotheses, a researcher first conceives of the set of candidate hypotheses. Importantly, the researcher ought not only specify their favoured hypotheses but should specify as many additional plausible hypotheses as possible [ 23 ]. The set of hypotheses should consider both alternative combinations of processes and alternative linkages between these processes and resultant patterns (i.e. parametrizations). Further, observation error and study design elements can (and often should) be included as components of hypotheses since they influence the pattern that is ultimately observed.

The complexity of ecological systems has led to some criticism of multi-hypothesis approaches in the field. For example, Simberloff [ 51 ] criticized Platt's method [ 24 ] on the basis that ‘strong inference' is incompatible with ecological processes typically involving multiple, non-mutually exclusive additive or interacting causative factors. This argument rests on the notion that multi-causal systems cannot be subjected to Popperian falsification (as in [ 52 ]) because no sufficient model can be written for the hypothesis; as the falsification process proceeds, the ecologist is ultimately left with a set of inseparable, and therefore, unfalsifiable causal factors which all have relevance [ 51 ]. Several authors correctly point out that alternative epistemological frameworks accommodate this complexity by estimating probabilistic support for hypotheses, rather than seeking to accept the hypothesis which is complementary to the set of falsified hypotheses [ 53 – 55 ]. When generating hypotheses, researchers should consider the inferential framework (e.g. hypothetico-deductive or inductive/probabilistic) to which their hypotheses will ultimately be subjected. Will a ‘crucial experiment' [ 24 ] be possible or should probabilistic support be evaluated across a set of models (e.g. [ 45 , 56 ])?

Platt [ 24 ] has several fine suggestions for conceiving of competing hypotheses, in the context of ‘strong inference': dedicated time/effort to the task; using logic trees to describe the system; and modularizing the processes. Betini et al . [ 26 ] also discuss potential barriers to hypothesis generation and suggest several new approaches to overcome those barriers. Burnham and Anderson [ 45 ] also provide much useful guidance for the generation of competing hypotheses under an inductive framework (information theory). We would also add that the entire workflow is iterative, and the construction, implementation and analysis of models may also help to reveal additional hypotheses (e.g. see electronic supplementary material, 2).

5.2. Step 2: write a formal model for each hypothesis

In this step, the researcher converts the conceptual models to formal models. There is considerable flexibility in the type of model promulgated here. Researchers could generate fully mechanistic mathematical models or phenomenological statistical models, fully deterministic models or stochastic simulations. The chosen model type should account for the complexity of the underlying processes, the relevant level of biological organization under study and the nature of the available data (i.e. do the data allow for direct observation of mechanistic processes?; [ 57 ]). Key questions to guide the development of hypotheses should include: what data can actually be collected and does the model match those data? What level(s) of biological organization is relevant to the question and does the model match that hierarchy? Should the hypothesis directly model all relevant processual steps or do latent variables need to be included? Is enough known about the subject to specify a truly process-based model or should statistical links between phenomena be simulated without direct mechanistic components? Is the process deterministic or stochastic?

Seemingly straightforward, this step can be surprisingly complex. Verbal models do not always have obvious mathematical or computer code analogues and creative solutions may be required to translate conceptual models to formal ones. Keep in mind that each version of a model and each parametrization of a version requires its own specification and subsequent analysis, so it pays to think clearly and succinctly about identifying the set of models in step 1.

This step is also the point to consider the logical plausibility of a hypothesis. By formally translating a hypothesis into a model, one is immediately confronted with the logical structure of that hypothesis [ 22 ]. At this step, illogical hypotheses reveal themselves and can be corrected or removed from the set of working hypotheses. Note that logical consistency is not equivalent to ‘truth'; it is an indication that the hypothesis/model is internally coherent. For example, the intermediate disturbance hypothesis (IDH; [ 58 , 59 ]), as originally stated, contained internally incoherent elements such that the premises of the model did not support the predictions [ 60 ]. By specifying the IDH as a mathematical population model, Fox [ 60 ] showed that intermediate disturbance frequencies do not, in fact, predict ‘hump-shaped' diversity curves. Interestingly, both Fox [ 60 ] and Sheil and Burslem [ 61 ] point out that modern competition–colonization trade-off theory (e.g. [ 62 ]) rescues the IDH from logical implausibility, exemplifying the model plausibility check for which we advocate here.

5.3. Step 3: generate sampling distributions

Because many models in ecology may contain at least some stochastic components [ 56 ], the simulated patterns can vary across iterations (where an iteration is ‘running' the model once to generate a single simulated pattern). Therefore, a single iteration is insufficient to compare one candidate hypothesis to another––how could we know if the difference between patterns is due to ‘real' differences between models/hypotheses or to inter-iteration stochasticity? Just as data-based inference is centred on estimated sampling distributions, hypothesis vetting is centred on sampling distributions derived from multiple iterations of a simulation model (or the direct calculation of a sampling distribution using a closed-form model). With sampling distributions for a set of working hypotheses in hand, a researcher can identify degenerate and/or noisy relationships among the multiple working hypotheses by comparing the sampling distributions.

A sampling distribution is required for all hypotheses under consideration, including any/all parametrization(s) thereof. Thus, for a model containing a single free parameter that may assume a range of values, the researcher must generate a sampling distribution for all such parameter values (or at least a bracketed range of parametrizations). Therefore, careful articulation of plausible parametrizations is recommended, because densely sampling the parameter space (the set of possible values a parameter could take) quickly increases computational burden.

5.4. Step 4: quantify overlap between sampling distributions

In this step, simulated sampling distributions of response variables are examined for evidence of degeneracy between or noisiness within hypotheses. This process resembles inference performed with data but in this case the inference is between simulated data from modelled hypotheses. This kind of inference can help differentiate the relative identifiability of hypotheses but does not provide support for or against any one hypothesis itself.

To detect a degenerate relationship, we want to quantify the extent to which the output of one simulated hypothesis could also have been generated by any of the others. Simple comparisons of the proportion of a sampling distribution overlapping some plausibly bracketed range of another model's sampling distribution provides a first order estimate. For example, if the sampling distribution generated by Process A is entirely contained within the sampling distribution generated by Process B, then no observation of a pattern consistent with Process A could ever rule out Process B. Note that this calculation is conceptually equivalent to the familiar p -value from the null hypothesis testing framework but can be used to compare the probability of any model output conditional upon any other model (including, but not limited to, a simulated null). To detect a noisy relationship, we simply quantify the variance within a sampling distribution relative to the magnitude of the estimate. Box 1 and electronic supplementary material, 1 and 2 contain examples of quantifying and visualizing both convergent and noisy relationships.

Classifying hypotheses as degenerate or noisy requires context-specific judgement by the researcher. No pre-prescribed degree of overlap between two hypotheses is automatically ‘too much', nor is there a standard upper limit for variance. Instead, the researcher must decide if the precision with which parameters may be estimated or hypotheses may be parsed is sufficient for the purposes of answering the question at hand. This judgement requires system-specific knowledge and sober consideration of the ultimate inferential aims (see [ 63 ]). Key questions to consider include: what level of precision is required for the estimated parameter? This will depend on e.g. expected effects sizes, intended uses of the research output and the scale at which the ecological process unfolds. What probability of error in hypothesis selection/rejection is acceptable—is this work exploratory or confirmatory? Exploratory work may be more forgiving of a moderate probability of error whereas confirmatory work may be incompatible with all but a very low probability of error.

5.5. Step 5: revise hypotheses and repeat vetting procedure

If the results of the previous step indicate that hypotheses are degenerate or noisy, the researcher must consider whether they can be adequately revised while remaining biologically relevant. Degenerate hypotheses can be replaced by alternative hypotheses, including revisions that more explicitly address problematic confounding issues arising from measurement methods (e.g. studies where detection of an event is imperfect). In other words, researchers might think carefully about sources of variance not included in the hypotheses that would help parse the observable patterns. Modelling allows a researcher to quantify the degree of degeneracy in a set of hypotheses and to test alternative measures or analyses that lead to identifiability.

When the noisy relationship problem is encountered, pattern variance unrelated to the mechanism is often the culprit. In such cases, researchers can consider approaches to either reduce observation error or to better define or constrain the hypothesis about the relationship between process and pattern. Reducing variance from observation or measurement error can be straightforward: improve measurement techniques by design or integrate models for observation error into the analysis. For example, the spatial resolution of modern global positioning system tracking devices is orders of magnitude more precise than, for example, banding data or intrinsic geographic markers such as tissue stable isotopes [ 64 ]; occupancy modelling represents a widely used incorporation of error variance into estimates of species distributions [ 65 ]. Reducing variance from unconstrained hypotheses requires refining proposed models for the underlying processes. This may be a matter of reducing the stochastic complexity of model structure, or of adding deterministic processes (increasing the complexity) to the hypothesis. Very simple models often sacrifice predictive specificity in seeking broad generality [ 66 , 67 ]. For example, neutral theory of species coexistence [ 68 ] predicts highly variable sampling distributions of community compositions. However, increasing model complexity to include niche stabilizing forces [ 69 ] improves the model's predictive specificity [ 70 ].

It might also be helpful to reconsider the response variable or the study design. For example, extending the response variable from univariate to multivariate might help to specify confounding covariance. Alternatively, manipulative experimentation may help to parse hypotheses (see box 1 for examples of both). If neither modification to the response variable nor manipulative experimentation is likely to solve the problem, it may be necessary to fully revise or reconsider the hypothesis itself. This makes sense when the simulated noisiness or degeneracy results from a model misspecification (i.e. cases where the processes should have been uniquely detectable from pattern but were not). This is often also an indication that some processes have been omitted from the candidate hypothesis set. If hypotheses are revised in any way, either via modification to response variables, proposing experimentation, or revising hypotheses entirely, the hypothesis vetting process is then repeated until a workable set of hypotheses is identified at which point data-based inference proceeds.

6. Conclusion

Pre-data collection modelling of multiple hypotheses should be considered the default mode for scientific investigations. Both NHST and multi-model approaches are susceptible to inferential errors when alternative hypotheses are a priori non-identifiable and never formally considered. Adopting a multi-hypothesis approach to data-based inference is a necessary but insufficient first step. Ecologists ought to also consider in the abstract (prior to collecting data) whether proposed hypotheses are even theoretically uniquely identifiable. We have outlined a simple framework for determining the identifiability of hypotheses a priori by invoking Chamberlin's 130-year-old method of multiple working hypotheses. We hope that wider adoption of this approach will lead to more robust inference in ecology.

Supplementary Material

Acknowledgements.

The authors are grateful to Elizabeth Hobson for helpful comments on an early version.

No human or animal subjects were included in this study. Therefore, no ethical approvals were required.

Data accessibility

Authors' contributions.

S.W.Y., A.M., L.H. and M.B.W. conceived of the original idea; S.W.Y. and C.N.T. substantially revised the structure and aims of the paper. S.W.Y. wrote the initial draft and all authors contributed critically to revisions. All authors gave final approval for publication.

Competing interests

The authors declare no competing interests.

S.W.Y. and A.M. were supported by teaching assistantships from the University of Colorado Denver. Collaboration between S.W.Y. and C.N.T. was supported by the Fritz Knopf Fellowship.