in qualitative research internal validity

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

Internal Validity in Research | Definition, Threats, & Examples

Internal Validity in Research | Definition, Threats & Examples

Published on May 1, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Internal validity is the extent to which you can be confident that a cause-and-effect relationship established in a study cannot be explained by other factors.

Why internal validity matters, how to check whether your study has internal validity, trade-off between internal and external validity, threats to internal validity and how to counter them, other interesting articles, frequently asked questions about internal validity.

Internal validity makes the conclusions of a causal relationship credible and trustworthy. Without high internal validity, an experiment cannot demonstrate a causal link between two variables.

Once they arrive at the laboratory, the treatment group participants are given a cup of coffee to drink, while control group participants are given water. You also give both groups memory tests. After analyzing the results, you find that the treatment group performed better than the control group on the memory test.

For your conclusion to be valid, you need to be able to rule out other explanations (including control , extraneous , and confounding variables) for the results.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

There are three necessary conditions for internal validity. All three conditions must occur to experimentally establish causality between an independent variable A (your treatment variable) and dependent variable B (your response variable).

Your treatment and response variables change together.
Your treatment precedes changes in your response variables
No confounding or extraneous factors can explain the results of your study.

In the research example above, only two out of the three conditions have been met.

Drinking coffee and memory performance increased together.
Drinking coffee happened before the memory test.
The time of day of the sessions is an extraneous factor that can equally explain the results of the study.

Because you assigned participants to groups based on the schedule, the groups were different at the start of the study. Any differences in memory performance may be due to a difference in the time of day. Therefore, you cannot say for certain whether the time of day or drinking a cup of coffee improved memory performance.

That means your study has low internal validity, and you cannot deduce a causal relationship between drinking coffee and memory performance.

External validity is the extent to which you can generalize the findings of a study to other measures, settings or groups. In other words, can you apply the findings of your study to a broader context?

There is an inherent trade-off between internal and external validity ; the more you control extraneous factors in your study, the less you can generalize your findings to a broader context.

Threats to internal validity are important to recognize and counter in a research design for a robust study. Different threats can apply to single-group and multi-group studies.

Single-group studies

How to counter threats in single-group studies.

Altering the experimental design can counter several threats to internal validity in single-group studies.

Adding a comparable control group counters threats to single-group studies. If comparable control and treatment groups each face the same threats, the outcomes of the study won’t be affected by them.
A large sample size counters testing, because results would be more sensitive to any variability in the outcomes and less likely to suffer from sampling bias .
Using filler-tasks or questionnaires to hide the purpose of study also counters testing threats and demand characteristics .

Multi-group studies

How to counter threats in multi-group studies.

Altering the experimental design can counter several threats to internal validity in multi-group studies.

Random assignment of participants to groups counters selection bias and regression to the mean by making groups comparable at the start of the study.
Blinding participants to the aim of the study counters the effects of social interaction.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Normal distribution
Degrees of freedom
Null hypothesis
Discourse analysis
Control groups
Mixed methods research
Non-probability sampling
Quantitative research
Ecological validity

Research bias

Rosenthal effect
Implicit bias
Cognitive bias
Selection bias
Negativity bias
Status quo bias

I nternal validity is the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables .

External validity is the extent to which your results can be generalized to other contexts.

The validity of your experiment depends on your experimental design .

There are eight threats to internal validity : history, maturation, instrumentation, testing, selection bias , regression to the mean, social interaction and attrition .

Attrition bias is a threat to internal validity . In experiments, differential rates of attrition between treatment and control groups can skew results.

This bias can affect the relationship between your independent and dependent variables . It can make variables appear to be correlated when they are not, or vice versa .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). Internal Validity in Research | Definition, Threats & Examples. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/methodology/internal-validity/

Is this article helpful?

Pritha Bhandari

Other students also liked, external validity | definition, types, threats & examples, guide to experimental design | overview, steps, & examples, correlation vs. causation | difference, designs & examples, what is your plagiarism score.

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
Write for Us
BMJ Journals More You are viewing from: Google Indexer

Helen Noble 1 ,
Joanna Smith 2
1 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
2 School of Human and Health Sciences, University of Huddersfield , Huddersfield , UK
Correspondence to Dr Helen Noble School of Nursing and Midwifery, Queens's University Belfast, Medical Biology Centre, 97 Lisburn Rd, Belfast BT9 7BL, UK; helen.noble{at}qub.ac.uk

https://doi.org/10.1136/eb-2015-102054

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Evaluating the quality of research is essential if findings are to be utilised in practice and incorporated into care delivery. In a previous article we explored ‘bias’ across research designs and outlined strategies to minimise bias. 1 The aim of this article is to further outline rigour, or the integrity in which a study is conducted, and ensure the credibility of findings in relation to qualitative research. Concepts such as reliability, validity and generalisability typically associated with quantitative research and alternative terminology will be compared in relation to their application to qualitative research. In addition, some of the strategies adopted by qualitative researchers to enhance the credibility of their research are outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research. 2–4 In the broadest context these terms are applicable, with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures. 4 However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alterative frameworks for establishing rigour are appropriate. 3 Lincoln and Guba 5 offer alternative criteria for demonstrating rigour within qualitative research namely truth value, consistency and neutrality and applicability. Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

View inline

Terminology and criteria used to evaluate the credibility of research findings

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Unlike quantitative researchers, who apply statistical methods for establishing validity and reliability of research findings, qualitative researchers aim to design and incorporate methodological strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

Accounting for personal biases which may have influenced findings; 6

Acknowledging biases in sampling and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis; 3

Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent; 3 , 4

Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented; 6 , 7

Including rich and thick verbatim descriptions of participants’ accounts to support findings; 7

Demonstrating clarity in terms of thought processes during data analysis and subsequent interpretations 3 ;

Engaging with other researchers to reduce research bias; 3

Respondent validation: includes inviting participants to comment on the interview transcript and whether the final themes and concepts created adequately reflect the phenomena being investigated; 4

Data triangulation, 3 , 4 whereby different methods and perspectives help produce a more comprehensive set of findings. 8 , 9

Table 2 provides some specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the impact of being a family carer to patients with stage 5 chronic kidney disease managed without dialysis. 10

Strategies for enhancing the credibility of qualitative research

In summary, it is imperative that all qualitative researchers incorporate strategies to enhance the credibility of a study during research design and implementation. Although there is no universally accepted terminology and criteria used to evaluate qualitative research, we have briefly outlined some of the strategies that can enhance the credibility of study findings.

Sandelowski M
Lincoln YS ,
Barrett M ,
Mayan M , et al
Greenhalgh T
Lingard L ,

Twitter Follow Joanna Smith at @josmith175 and Helen Noble at @helnoble

Competing interests None.

Read the full text or download the PDF:

Bipolar Disorder
Therapy Center
When To See a Therapist
Types of Therapy
Best Online Therapy
Best Couples Therapy
Best Family Therapy
Managing Stress
Sleep and Dreaming
Understanding Emotions
Self-Improvement
Healthy Relationships
Student Resources
Personality Types
Guided Meditations
Verywell Mind Insights
2023 Verywell Mind 25
Mental Health in the Classroom
Editorial Process
Meet Our Review Board
Crisis Support

Internal Validity vs. External Validity in Research

Both help determine how meaningful the results of the study are

Arlin Cuncic, MA, is the author of The Anxiety Workbook and founder of the website About Social Anxiety. She has a Master's degree in clinical psychology.

in qualitative research internal validity

Rachel Goldman, PhD FTOS, is a licensed psychologist, clinical assistant professor, speaker, wellness expert specializing in eating behaviors, stress management, and health behavior change.

Verywell / Bailey Mariner

Internal Validity
External Validity

Internal validity is a measure of how well a study is conducted (its structure) and how accurately its results reflect the studied group.

External validity relates to how applicable the findings are in the real world. These two concepts help researchers gauge if the results of a research study are trustworthy and meaningful.

Conclusions are warranted

Controls extraneous variables

Eliminates alternative explanations

Focus on accuracy and strong research methods

Findings can be generalized

Outcomes apply to practical situations

Results apply to the world at large

Results can be translated into another context

What Is Internal Validity in Research?

Internal validity is the extent to which a research study establishes a trustworthy cause-and-effect relationship. This type of validity depends largely on the study's procedures and how rigorously it is performed.

Internal validity is important because once established, it makes it possible to eliminate alternative explanations for a finding. If you implement a smoking cessation program, for instance, internal validity ensures that any improvement in the subjects is due to the treatment administered and not something else.

Internal validity is not a "yes or no" concept. Instead, we consider how confident we can be with study findings based on whether the research avoids traps that may make those findings questionable. The less chance there is for "confounding," the higher the internal validity and the more confident we can be.

Confounding refers to uncontrollable variables that come into play and can confuse the outcome of a study, making us unsure of whether we can trust that we have identified the cause-and-effect relationship.

In short, you can only be confident that a study is internally valid if you can rule out alternative explanations for the findings. Three criteria are required to assume cause and effect in a research study:

The cause preceded the effect in terms of time.
The cause and effect vary together.
There are no other likely explanations for the relationship observed.

Factors That Improve Internal Validity

To ensure the internal validity of a study, you want to consider aspects of the research design that will increase the likelihood that you can reject alternative hypotheses. Many factors can improve internal validity in research, including:

Blinding : Participants—and sometimes researchers—are unaware of what intervention they are receiving (such as using a placebo on some subjects in a medication study) to avoid having this knowledge bias their perceptions and behaviors, thus impacting the study's outcome
Experimental manipulation : Manipulating an independent variable in a study (for instance, giving smokers a cessation program) instead of just observing an association without conducting any intervention (examining the relationship between exercise and smoking behavior)
Random selection : Choosing participants at random or in a manner in which they are representative of the population that you wish to study
Randomization or random assignment : Randomly assigning participants to treatment and control groups, ensuring that there is no systematic bias between the research groups
Strict study protocol : Following specific procedures during the study so as not to introduce any unintended effects; for example, doing things differently with one group of study participants than you do with another group

Internal Validity Threats

Just as there are many ways to ensure internal validity, there is also a list of potential threats that should be considered when planning a study.

Attrition : Participants dropping out or leaving a study, which means that the results are based on a biased sample of only the people who did not choose to leave (and possibly who all have something in common, such as higher motivation)
Confounding : A situation in which changes in an outcome variable can be thought to have resulted from some type of outside variable not measured or manipulated in the study
Diffusion : This refers to the results of one group transferring to another through the groups interacting and talking with or observing one another; this can also lead to another issue called resentful demoralization, in which a control group tries less hard because they feel resentful over the group that they are in
Experimenter bias : An experimenter behaving in a different way with different groups in a study, which can impact the results (and is eliminated through blinding)
Historical events : May influence the outcome of studies that occur over a period of time, such as a change in the political leader or a natural disaster that occurs, influencing how study participants feel and act
Instrumentation : This involves "priming" participants in a study in certain ways with the measures used, causing them to react in a way that is different than they would have otherwise reacted
Maturation : The impact of time as a variable in a study; for example, if a study takes place over a period of time in which it is possible that participants naturally change in some way (i.e., they grew older or became tired), it may be impossible to rule out whether effects seen in the study were simply due to the impact of time
Statistical regression : The natural effect of participants at extreme ends of a measure falling in a certain direction due to the passage of time rather than being a direct effect of an intervention
Testing : Repeatedly testing participants using the same measures influences outcomes; for example, if you give someone the same test three times, it is likely that they will do better as they learn the test or become used to the testing process, causing them to answer differently

What Is External Validity in Research?

External validity refers to how well the outcome of a research study can be expected to apply to other settings. This is important because, if external validity is established, it means that the findings can be generalizable to similar individuals or populations.

External validity affirmatively answers the question: Do the findings apply to similar people, settings, situations, and time periods?

Population validity and ecological validity are two types of external validity. Population validity refers to whether you can generalize the research outcomes to other populations or groups. Ecological validity refers to whether a study's findings can be generalized to additional situations or settings.

Another term called transferability refers to whether results transfer to situations with similar characteristics. Transferability relates to external validity and refers to a qualitative research design.

Factors That Improve External Validity

If you want to improve the external validity of your study, there are many ways to achieve this goal. Factors that can enhance external validity include:

Field experiments : Conducting a study outside the laboratory, in a natural setting
Inclusion and exclusion criteria : Setting criteria as to who can be involved in the research, ensuring that the population being studied is clearly defined
Psychological realism : Making sure participants experience the events of the study as being real by telling them a "cover story," or a different story about the aim of the study so they don't behave differently than they would in real life based on knowing what to expect or knowing the study's goal
Replication : Conducting the study again with different samples or in different settings to see if you get the same results; when many studies have been conducted on the same topic, a meta-analysis can also be used to determine if the effect of an independent variable can be replicated, therefore making it more reliable
Reprocessing or calibration : Using statistical methods to adjust for external validity issues, such as reweighting groups if a study had uneven groups for a particular characteristic (such as age)

External Validity Threats

External validity is threatened when a study does not take into account the interaction of variables in the real world. Threats to external validity include:

Pre- and post-test effects : When the pre- or post-test is in some way related to the effect seen in the study, such that the cause-and-effect relationship disappears without these added tests
Sample features : When some feature of the sample used was responsible for the effect (or partially responsible), leading to limited generalizability of the findings
Selection bias : Also considered a threat to internal validity, selection bias describes differences between groups in a study that may relate to the independent variable—like motivation or willingness to take part in the study, or specific demographics of individuals being more likely to take part in an online survey
Situational factors : Factors such as the time of day of the study, its location, noise, researcher characteristics, and the number of measures used may affect the generalizability of findings

While rigorous research methods can ensure internal validity, external validity may be limited by these methods.

Internal Validity vs. External Validity

Internal validity and external validity are two research concepts that share a few similarities while also having several differences.

Similarities

One of the similarities between internal validity and external validity is that both factors should be considered when designing a study. This is because both have implications in terms of whether the results of a study have meaning.

Both internal validity and external validity are not "either/or" concepts. Therefore, you always need to decide to what degree a study performs in terms of each type of validity.

Each of these concepts is also typically reported in research articles published in scholarly journals . This is so that other researchers can evaluate the study and make decisions about whether the results are useful and valid.

Differences

The essential difference between internal validity and external validity is that internal validity refers to the structure of a study (and its variables) while external validity refers to the universality of the results. But there are further differences between the two as well.

For instance, internal validity focuses on showing a difference that is due to the independent variable alone. Conversely, external validity results can be translated to the world at large.

Internal validity and external validity aren't mutually exclusive. You can have a study with good internal validity but be overall irrelevant to the real world. You could also conduct a field study that is highly relevant to the real world but doesn't have trustworthy results in terms of knowing what variables caused the outcomes.

Examples of Validity

Perhaps the best way to understand internal validity and external validity is with examples.

Internal Validity Example

An example of a study with good internal validity would be if a researcher hypothesizes that using a particular mindfulness app will reduce negative mood. To test this hypothesis, the researcher randomly assigns a sample of participants to one of two groups: those who will use the app over a defined period and those who engage in a control task.

The researcher ensures that there is no systematic bias in how participants are assigned to the groups. They do this by blinding the research assistants so they don't know which groups the subjects are in during the experiment.

A strict study protocol is also used to outline the procedures of the study. Potential confounding variables are measured along with mood , such as the participants' socioeconomic status, gender, age, and other factors. If participants drop out of the study, their characteristics are examined to make sure there is no systematic bias in terms of who stays in.

External Validity Example

An example of a study with good external validity would be if, in the above example, the participants used the mindfulness app at home rather than in the laboratory. This shows that results appear in a real-world setting.

To further ensure external validity, the researcher clearly defines the population of interest and chooses a representative sample . They might also replicate the study's results using different technological devices.

A Word From Verywell

Setting up an experiment so that it has both sound internal validity and external validity involves being mindful from the start about factors that can influence each aspect of your research.

It's best to spend extra time designing a structurally sound study that has far-reaching implications rather than to quickly rush through the design phase only to discover problems later on. Only when both internal validity and external validity are high can strong conclusions be made about your results.

San Jose State University. Internal and external validity .

Michael RS. Threats to internal & external validity: Y520 strategies for educational inquiry .

Pahus L, Burgel PR, Roche N, Paillasseur JL, Chanez P. Randomized controlled trials of pharmacological treatments to prevent COPD exacerbations: applicability to real-life patients . BMC Pulm Med . 2019;19(1):127. doi:10.1186/s12890-019-0882-y

By Arlin Cuncic, MA Arlin Cuncic, MA, is the author of The Anxiety Workbook and founder of the website About Social Anxiety. She has a Master's degree in clinical psychology.

Buy Me a Coffee

Home » Internal Validity – Threats, Examples and Guide

Internal Validity – Threats, Examples and Guide

Table of Contents

Internal Validity

Definition:

Internal validity refers to the extent to which a research study accurately establishes a cause-and-effect relationship between the independent variable (s) and the dependent variable (s) being investigated. It assesses whether the observed changes in the dependent variable(s) are actually caused by the manipulation of the independent variable(s) rather than other extraneous factors.

How to Increase Internal Validity

To enhance internal validity, researchers need to carefully design and conduct their studies. Here are some considerations for improving internal validity:

Random Assignment: Use random assignment to allocate participants to different groups in experimental studies. Random assignment helps ensure that the groups are comparable, minimizing the influence of individual differences on the results.
Control Group: Include a control group in experimental studies. This group should be similar to the experimental group but not exposed to the treatment or intervention being tested. The control group helps establish a baseline against which the effects of the treatment can be compared.
Control Extraneous Variables: Identify and control for extraneous variables that could potentially influence the relationship being studied. This can be achieved through techniques like matching participants, using homogeneous samples, or statistically controlling for the variables.
Standardized Procedures: Use standardized procedures and protocols across all participants and conditions. This helps ensure consistency in the administration of the study, reducing the potential for systematic biases.
Counterbalancing: In studies with multiple conditions or treatment sequences, employ counterbalancing techniques. This involves systematically varying the order of conditions or treatments across participants to eliminate any potential order effects.
Minimize Experimenter Bias: Take steps to minimize experimenter bias or expectancy effects. These biases can inadvertently influence the behavior of participants or the interpretation of results. Using blind or double-blind procedures, where the experimenter is unaware of the conditions or group assignments, can help mitigate these biases.
Use Reliable and Valid Measures: Ensure that the measures used in the study are reliable and valid. Reliable measures yield consistent results, while valid measures accurately assess the construct being measured.
Pilot Testing: Conduct pilot testing before the main study to refine the study design and procedures. Pilot testing helps identify potential issues, such as unclear instructions or unforeseen confounds, and allows for necessary adjustments to enhance internal validity.
Sample Size: Increase the sample size to improve statistical power and reduce the likelihood of random variation influencing the results. Adequate sample sizes increase the generalizability and reliability of the findings.
Researcher Bias: Researchers need to be aware of their own biases and take steps to minimize their impact on the study. This can be done through careful experimental design, blind data collection and analysis, and the use of standardized protocols.

Threats To Internal Validity

Several threats can undermine internal validity and compromise the validity of research findings. Here are some common threats to internal validity:

Events or circumstances that occur during the course of a study and affect the outcome, making it difficult to attribute the results solely to the treatment or intervention being studied.

Changes that naturally occur in participants over time, such as physical or psychological development, which can influence the results independently of the treatment or intervention.

Testing Effects

The act of being tested or measured on a particular variable in an initial assessment may influence participants’ subsequent responses. This effect can arise due to familiarity with the test or increased sensitization to the topic being studied.

Instrumentation

Changes or inconsistencies in the measurement tools or procedures used across different stages or conditions of the study. If the measurement methods are not standardized or if there are variations in the administration of tests, it can lead to measurement errors and threaten internal validity.

Selection Bias

When there are systematic differences between the characteristics of individuals selected for different groups or conditions in a study. If participants are not randomly assigned to groups or conditions, the results may be influenced by pre-existing differences rather than the treatment itself.

Attrition or Dropout

The loss of participants from a study over time can introduce bias if those who drop out differ systematically from those who remain. The characteristics of participants who drop out may affect the outcomes and compromise internal validity.

Regression to The Mean

The tendency for extreme scores on a variable to move closer to the average on subsequent measurements. If participants are selected based on extreme scores, their scores are likely to regress toward the mean in subsequent measurements, leading to erroneous conclusions about the effectiveness of a treatment.

Diffusion of Treatment

When participants in one group of a study receive knowledge or benefits from participants in another group, it can dilute the treatment effect and compromise internal validity. This can occur through communication or sharing of information among participants.

Demand Characteristics

Cues or expectations within a study that may influence participants to respond in a certain way or guess the purpose of the research. Participants may modify their behavior to align with perceived expectations, leading to biased results.

Experimenter Bias

Biases or expectations on the part of the researchers that may unintentionally influence the study’s outcomes. Researchers’ behavior, interactions, or inadvertent cues can impact participants’ responses, introducing bias and threatening internal validity.

Types of Internal Validity

There are several types of internal validity that researchers consider when designing and conducting studies. Here are some common types of internal validity:

Construct validity

Refers to the extent to which the operational definitions of the variables used in the study accurately represent the theoretical concepts they are intended to measure. It ensures that the measurements or manipulations used in the study accurately reflect the intended constructs.

Statistical Conclusion Validity

Relates to the degree to which the statistical analysis accurately reflects the relationships between variables. It involves ensuring that the appropriate statistical tests are used, the data is analyzed correctly, and the reported findings are reliable.

Internal Validity of Causal Inferences

Focuses on establishing a cause-and-effect relationship between the independent variable (treatment or intervention) and the dependent variable (outcome or response variable). It involves eliminating alternative explanations or confounding factors that could account for the observed relationship.

Temporal Precedence

Ensures that the cause (independent variable) precedes the effect (dependent variable) in time. It establishes the temporal sequence necessary for making causal claims.

Covariation

Refers to the presence of a relationship or association between the independent variable and the dependent variable. It ensures that changes in the independent variable are accompanied by corresponding changes in the dependent variable.

Elimination of Confounding Variables

Involves controlling for and minimizing the influence of extraneous variables that could affect the relationship between the independent and dependent variables. It helps isolate the true effect of the independent variable on the dependent variable.

Selection bias Control

Ensures that the process of assigning participants to different groups or conditions (randomization) is unbiased. Random assignment helps create equivalent groups, reducing the influence of participant characteristics on the dependent variable.

Controlling for Testing Effects

Involves minimizing the impact of repeated testing or measurement on participants’ responses. Counterbalancing, using control groups, or employing appropriate time intervals between assessments can help control for testing effects.

Controlling for Experimenter Effects

Aims to minimize the influence of the experimenter on participants’ responses. Blinding, using standardized protocols, or automating data collection processes can reduce the potential for experimenter bias.

Replication

Conducting the study multiple times with different samples or settings to verify the consistency and generalizability of the findings. Replication enhances internal validity by ensuring that the observed effects are not due to chance or specific characteristics of the study sample.

Internal Validity Examples

Here are some real-time examples that illustrate internal validity:

Drug Trial: A pharmaceutical company conducts a clinical trial to test the effectiveness of a new medication for treating a specific disease. The study uses a randomized controlled design, where participants are randomly assigned to receive either the medication or a placebo. The internal validity is high because the random assignment helps ensure that any observed differences between the groups can be attributed to the medication rather than other factors.

Education Intervention: A researcher investigates the impact of a new teaching method on student performance in mathematics. The researcher selects two comparable groups of students from the same school and randomly assigns one group to receive the new teaching method while the other group continues with the traditional method. By controlling for factors such as the school environment and student characteristics, the study enhances internal validity by isolating the effects of the teaching method.

Psychological Experiment: A psychologist conducts an experiment to examine the relationship between sleep deprivation and cognitive performance. Participants are randomly assigned to either a sleep-deprived group or a control group. The internal validity is strengthened by manipulating the independent variable (amount of sleep) and controlling for other variables that could influence cognitive performance, such as age, gender, and prior sleep habits.

Quasi-Experimental Study: A researcher investigates the impact of a new traffic law on accident rates in a specific city. Since random assignment is not feasible, the researcher selects two similar neighborhoods: one where the law is implemented and another where it is not. By comparing accident rates before and after the law’s implementation in both areas, the study attempts to establish a causal relationship while acknowledging potential confounding variables, such as driver behavior or road conditions.

Workplace Training Program: An organization introduces a new training program aimed at improving employee productivity. To assess the effectiveness of the program, the company implements a pre-post design where performance metrics are measured before and after the training. By tracking changes in productivity within the same group of employees, the study attempts to attribute any improvements to the training program while controlling for individual differences.

Applications of Internal Validity

Internal validity is a crucial concept in research design and is applicable across various fields of study. Here are some applications of internal validity:

Experimental Research

Internal validity is particularly important in experimental research, where researchers manipulate independent variables to determine their effects on dependent variables. By ensuring strong internal validity, researchers can confidently attribute any observed changes in the dependent variable to the manipulation of the independent variable, establishing a cause-and-effect relationship.

Quasi-experimental Research

Quasi-experimental studies aim to establish causal relationships but lack random assignment to groups. Internal validity becomes crucial in such designs to minimize alternative explanations for the observed effects. Careful selection and control of potential confounding variables help strengthen internal validity in quasi-experimental research.

Observational Studies

While observational studies may not involve experimental manipulation, internal validity is still relevant. Researchers need to identify and control for confounding variables to establish a relationship between variables of interest and rule out alternative explanations for observed associations.

Program Evaluation

Internal validity is essential in evaluating the effectiveness of interventions, programs, or policies. By designing rigorous evaluation studies with strong internal validity, researchers can determine whether the observed outcomes can be attributed to the specific intervention or program being evaluated.

Clinical Trials

Internal validity is critical in clinical trials to determine the effectiveness of new treatments or therapies. Well-designed randomized controlled trials (RCTs) with strong internal validity can provide reliable evidence on the efficacy of interventions and guide clinical decision-making.

Longitudinal Studies

Longitudinal studies track participants over an extended period to examine changes and establish causal relationships. Maintaining internal validity throughout the study helps ensure that observed changes in the dependent variable(s) are indeed caused by the independent variable(s) under investigation and not other factors.

Psychology and Social Sciences

Internal validity is pertinent in psychological and social science research. Researchers aim to understand human behavior and social phenomena, and establishing strong internal validity allows them to draw accurate conclusions about the causal relationships between variables.

Advantages of Internal Validity

Internal validity is essential in research for several reasons. Here are some of the advantages of having high internal validity in a study:

Causal Inference: Internal validity allows researchers to make valid causal inferences. When a study has high internal validity, it establishes a cause-and-effect relationship between the independent variable (treatment or intervention) and the dependent variable (outcome). This provides confidence that changes in the dependent variable are genuinely due to the manipulation of the independent variable.
Elimination of Confounding Factors: High internal validity helps eliminate or control confounding factors that could influence the relationship being studied. By systematically accounting for potential confounds, researchers can attribute the observed effects to the intended independent variable rather than extraneous variables.
Accuracy of Measurements: Internal validity ensures accurate and reliable measurements. Researchers employ rigorous methods to measure variables, reducing measurement errors and increasing the validity and precision of the data collected.
Replicability and Generalizability: Studies with high internal validity are more likely to yield consistent results when replicated by other researchers. This is important for the advancement of scientific knowledge, as replication strengthens the validity of findings and allows for the generalizability of results across different populations and settings.
Intervention Effectiveness: High internal validity helps determine the effectiveness of interventions or treatments. By controlling for confounding factors and utilizing robust research designs, researchers can accurately assess whether an intervention produces the desired outcomes or effects.
Enhanced Decision-making: Studies with high internal validity provide a solid basis for decision-making. Policymakers, practitioners, and professionals can rely on research with high internal validity to make informed decisions about the implementation of interventions or treatments in real-world settings.
Validity of Theory Development: Internal validity contributes to the development and refinement of theories. By establishing strong cause-and-effect relationships, researchers can build and test theories, enhancing our understanding of underlying mechanisms and contributing to theoretical advancements.
Scientific Credibility: Research with high internal validity enhances the overall credibility of the scientific field. Studies that prioritize internal validity uphold the rigorous standards of scientific inquiry and contribute to the accumulation of reliable knowledge.

Limitations of Internal Validity

While internal validity is crucial for research, it is important to recognize its limitations. Here are some limitations or considerations associated with internal validity:

Artificial Experimental Settings: Research studies with high internal validity often take place in controlled laboratory settings. While this allows for rigorous control over variables, it may limit the generalizability of the findings to real-world settings. The controlled environment may not fully capture the complexity and variability of natural settings, potentially affecting the external validity of the study.
Demand Characteristics and Experimenter Effects: Participants in a study may behave differently due to demand characteristics or their awareness of being in a research setting. They might alter their behavior to align with their perceptions of the expected or desired responses, which can introduce bias and compromise internal validity. Similarly, experimenter effects, such as unintentional cues or biases conveyed by the researcher, can influence participant responses and affect internal validity.
Selection Bias: The process of selecting participants for a study may introduce biases and limit the generalizability of the findings. For example, if participants are not randomly selected or if they self-select into the study, the sample may not represent the larger population, impacting both internal and external validity.
Reactive or Interactive Effects: Participants’ awareness of being observed or their exposure to the experimental manipulation may elicit reactive or interactive effects. These effects can influence their behavior, leading to artificial responses that may not be representative of their natural behavior in real-world situations.
Limited Sample Characteristics: The characteristics of the sample used in a study can affect internal validity. If the sample is not diverse or representative of the population of interest, it can limit the generalizability of the findings. Additionally, small sample sizes may reduce statistical power and increase the likelihood of chance findings.
Time-related Factors: Internal validity can be influenced by factors related to the timing of the study. For example, the immediate effects observed in a short-term study may not reflect the long-term effects of an intervention. Additionally, history or maturation effects occurring during the course of the study may confound the relationship being studied.
Exclusion of Complex Variables: To establish internal validity, researchers often simplify the research design by focusing on a limited number of variables. While this allows for controlled experimentation, it may neglect the complex interactions and multiple factors that exist in real-world situations. This limitation can impact the ecological validity and external validity of the findings.
Publication Bias: Publication bias occurs when studies with significant or positive results are more likely to be published, while studies with null or negative results remain unpublished or overlooked. This bias can distort the body of evidence and compromise the overall internal validity of the research field.

Alo see Validity

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

Validity – Types, Examples and Guide

Alternate Forms Reliability – Methods, Examples...

Construct Validity – Types, Threats and Examples

Reliability Vs Validity

Internal Consistency Reliability – Methods...

Split-Half Reliability – Methods, Examples and...

Chapter 6. Reflexivity

Introduction.

Related to epistemological issues of how we know anything about the social world, qualitative researchers understand that we the researchers can never be truly neutral or outside the study we are conducting. As observers, we see things that make sense to us and may entirely miss what is either too obvious to note or too different to comprehend. As interviewers, as much as we would like to ask questions neutrally and remain in the background, interviews are a form of conversation, and the persons we interview are responding to us . Therefore, it is important to reflect upon our social positions and the knowledges and expectations we bring to our work and to work through any blind spots that we may have. This chapter discusses the concept of reflexivity and its importance for conducting reliable qualitative research.

Reflexivity: What It Is and Why It Is Important

Remember our discussion in epistemology ? Qualitative researchers tend to question assertions of absolute fact or reality, unmediated through subject positions and subject knowledge. There are limits to what we know because we are part of the social worlds we inhabit. To use the terminology of standpoint theorists, we have a standpoint from which we observe the world just as much as anyone else. In this, we too are the blind men, and the world is our elephant. None of us are omniscient or neutral observers. Because of this epistemological standpoint, qualitative researchers value the ability to reflect upon and think hard about our own effects on our research. We call this reflexivity. Reflexivity “generally involves the self-examination of how research findings were produced, and, particularly, the role of the researcher in their construction” ( Heaton 2004:104 ).

There are many aspects of being reflexive. First, there is the simple fact that we are human beings with the limitations that come with that condition. We have likes and dislikes, biases, blind spots, preferences, and so on. If we do not take these into account, they can prevent us from being the best researcher we can be. Being reflective means, first and foremost, trying as best as possible to bracket out elements of our own character and understanding that get in the way. It is important to note that bias (in this context, at least) is not inherently wrong. It just is. Unavoidable. But by noting it, we can minimize its impact or, in some cases, help explain more clearly what it is we see or why it is that we are asking the questions we are asking. For example, I might want to communicate to my audience that I grew up poor and that I have a lot of sympathy and concern for first-generation college students as a result. This “bias” of mine motivates me to do the work I do, even as I try to ensure that it does not blind me to things I find out in the course of my research. [1]

A second aspect of being reflexive is being aware that you yourself are part of the research when you are conducting qualitative research. This is particularly true when conducting interviews, observing interactions, or participating in activities. You have a body, and it will be “read” by those in the field. You will be perceived as an insider or an outsider, as a friend or foe, as empathetic or hostile. Some of this will be wrong. People will prejudge you based on the color of your skin, your presented gender, the accent of your language. People will classify you based on the clothes you wear, and they will be more open to you if you remind them of a friendly aunt or uncle and more reserved if you remind them of someone they don’t like. This is all natural and inevitable. Your research will suffer if you do not take this into account, if you do not reflect upon how you are being read and how this might be influencing what people tell you or what they are willing to do in front of you. The flip side of this problem is that your particular body and presence will open some doors barred to other researchers. Finding sites and contexts where your presented self is a benefit rather than a burden is an important part of your individual research career. Be honest with yourself about this, and you will be more successful as a qualitative researcher. Learn to leverage yourself in your research.

The third aspect of being reflexive is related to how we communicate our work to others. Being honest with our position, as I am about my own social background and its potential impact on what I study or about how I leveraged my own position to get people to open up to me, helps our audiences evaluate what we have found. Maybe I haven’t entirely eliminated my biases or weaknesses, but by telling my audience who I am and where I potentially stand, they can take account of those biases and weaknesses in their reading of my findings. Letting them know that I wore pink when talking with older men because that made them more likely to be kind to me (a strategy acknowledged by Posselt [ 2016 ]) helps them understand the interview context. In other words, my research becomes more reliable when my own social position and the strategies I used are communicated.

Some people think being reflective is just another form of narcissistic navel-gazing. “The study is not about you!” they might cry. True, to some degree—but that also misses the point. All studies on the social world are inevitably about us as well because we are part of that social world. It is actually more dangerous to pretend that we are neutral observers, outside what we are observing. Pierre Bourdieu makes this point several times, and I think it is worth quoting him here: “The idea of a neutral science is fiction, an interested fiction which enables its authors to present a version of the dominant representation of the social world, naturalized and euphemized into a particularly misrecognizable and symbolically, therefore, particularly effective form, and to call it scientific” (quoted in Lemert 1981:278 ).

Bourdieu ( 1984 ) argues that reflective analysis is “not an epistemological scruple” but rather “an indispensable pre-condition of scientific knowledge of the object” ( 92 ). It would be narcissistic to present findings without reflection, as that would give much more weight to any findings or insights that emerge than is due.

The critics are right about one thing, however. Putting oneself at the center of the research is also inappropriate. [2] The focus should be on what is being researched, and the reflexivity is there to advance the study, not to push it aside. This issue has emerged at times when researchers from dominant social positions reflect upon their social locations vis-à-vis study participants from marginalized locations. A researcher who studies how low-income women of color experience unemployment might need to address her White, upper-class, fully employed social location, but not at the cost of crowding out the stories, lived experiences, and understandings of the women she has interviewed. This can sometimes be a delicate balance, and not everyone will agree that a person has walked it correctly.

Examples of Reflexivity in Practice

Most qualitative researchers include a positionality statement in any “methods section” of their publications. This allows readers to understand the location of the researcher, which is often helpful for gauging reliability . Many journals now require brief positionality statements as well. Here are a few examples of such statements.

The first is from an ethnographic study of elite golfers. Ceron-Anaya ( 2017 ) writes about his class, race, and gender and how these aspects of his identity and social location affected his interactions with research participants:

My own class origins, situated near the intersection between the middle and the lower-middle class, hindered cooperation in some cases. For example, the amiable interaction with one club member changed toward the end of the interview when he realized that I commonly moved about in the city by public transportation (which is a strong class indicator). He was not rude but stopped elaborating on the answers as he had been doing up to that point.…Bodily confidence is a privilege of the privileged. My subordinate position, vis-à-vis golfers, was ameliorated by my possession of cultural capital, objectified in my status of researcher/student in a western university. However, my cultural capital dwindled in its value at the invisible but firm boundary between the upper-middle and the upper class. The few contacts I made with members of the upper class produced no connections with other members of the same group, illustrating how the research process is also inserted in the symbolic and material dynamics that shape the field. ( 288 )

What did you learn from Ceron-Anaya’s reflection? If he hadn’t told you about his background, would this have made a difference in reading about elite golfers? Would the findings be different had Ceron-Anaya driven up to the club in a limousine? Is it helpful to know he came by bus?

The second example is from a study on first-generation college students. Hinz ( 2016 ) discusses both differences and similarities between herself and those she interviewed and how both could have affected the study:

I endeavored to avoid researcher bias by allowing the data to speak for itself, but my own habitus as a White, female, middle-class second-generation college student with a few years of association with Selective State [elite university] may have influenced my interpretation. Being a Selective State student at the time of the interviews provided a familiarity with the environment in which the participants were living, and an ease of communication facilitated by a shared institutional culture. And yet, not being a first-gen myself, it seemed as if I were standing on the periphery of their experience, looking in. ( 289–290 )

Note that Hinz cannot change who she is, nor should she. Being aware (reflective) that she may “stand on the periphery” of the experience of those she interviews has probably helped her listen more closely rather than assume she understands what is really going on. Do you find her more reliable given this?

These statements can be quite long, especially when found in methodological appendixes in books rather than short statements in articles. This last lengthy example comes from my own work. I try to place myself, explaining the motivations for the research I conducted at small liberal arts colleges:

I began this project out of a deep curiosity about how college graduates today were faring in an increasingly debt-ridden and unequal labor market. I was working at a small liberal arts college when I began thinking about this project and was poised to take a job at another one. During my interview for the new job, I was told that I was a good fit, because I had attended Barnard College, so I knew what the point of a liberal arts college was. I did. A small liberal arts college was a magical place. You could study anything you wanted, for no reason at all, simply for the love of it. And people would like you for it. You were surrounded by readers, by people who liked to dress up in costume and recite Shakespeare, by people who would talk deep into the night about the meaning of life or whether “beauty” existed out there, in nature, or was simply a projection of our own circumstances. My own experience at Barnard had been somewhat like that. I studied Ancient Greek and Latin, wrote an undergraduate thesis on the legal standing of Vestal Virgins in Ancient Rome, and took frequent subway rides to the Cloisters, the medieval annex of the Metropolitan Museum of Art, where I sketched the courtyard and stared at unicorn tapestries. But I also worked full-time, as a waitress at a series of hectic and demanding restaurants around the city, as a security guard for the dorm, as a babysitter for some pretty privileged professors who lived in doorman buildings along Riverside Park, and at the library (the best job by far). I also constantly worried I would not be able to finish my degree, as every year I was unsure how I would come up with the money to pay for costs of college above and beyond the tuition (which, happily, was covered by the college given my family’s low income). Indeed, the primary reason I studied the Classics was because all the books were freely available in the library. There are no modern textbooks—you just find a copy of the Iliad. There are a lot of those in a city like New York. Due to my fears, I pushed to graduate one year early, taking a degree in “Ancient Studies” instead of “Classics,” which could have led on to graduate training. From there, I went to law school, which seemed like a safe choice. I do not remember ever having a conversation with anyone about how to find a job or what kinds of job one could do with a degree in Ancient Studies. I had little to no social networks, as I had spent my time studying and working. And I was very lucky, because I graduated with almost zero debt. For years, until that job interview, I hadn’t really thought my Barnard experience had been that great or unusual. But now it was directly helping me get a job, about fifteen years after graduation. And it probably had made me a better person, whatever that means. Had I graduated with debt, however, I am not so sure that it would have been worth it. Was it, on balance, a real opportunity and benefit for poor students like me? Even now? I had a hunch of what I might find if I looked: small liberal arts colleges were unique places of opportunity for low-income first-generation working-class students who somehow managed to find and get in to one of them (no easy task). I thought that, because of their ethos, their smallness, the fact that one could not hide from professors, these colleges would do a fair job equalizing opportunities and experiences for all their students. I wanted to tell this story. But that is not the story that I found, or not entirely. While everyone benefits from the kind of education a small liberal arts college can offer, because students begin and continue so differently burdened and privileged, the advantages of the already-advantaged are amplified, potentially increasing rather than decreasing initial inequalities. That is not really a surprising story, but it is an important one to tell and to remember. Education doesn’t reduce inequality. Going to a good college doesn’t level the playing field for low-income, first-generation, working-class students. But perhaps it can help them write a book about that. ( Hurst 2019:259–261 )

What do you think? Did you learn something about the author that would help you, as a reader, understand the reasons and context for the study? Would you trust the researcher? If you said yes, why?

How to Do It

How does one become a reflective researcher? Practice! Nearly every great qualitative researcher maintains a reflexive journal (there are exceptions that prove the rule), a type of diary where they record their thinking on the research process itself. This might include writing about the research design (chapter 2), plotting out strategies for sample selection (chapter 6), or talking through what one believes can be known (chapter 3). During analysis, this journal is a place to record ideas and insights and pose questions for further reflection or follow-up studies. This journal should be highly personal. It is a place to record fears, concerns, and hopes as well. Why are you studying what you are studying? What is really motivating you? Being clear with yourself and being able to put it down in words are invaluable to the research process.

Today, there are many blogs out there on writing reflective journals, with helpful suggestions and examples. Although you may want to take a look at some of these, the form of your own journal will probably be unique. This is you, the researcher, on the page. Each of us looks different. Use the journal to interrogate your decisions and clarify your intent. If you find something during the study of note, you might want to ask yourself what led you to note that. Why do you think this “thing” is a “thing”? What about your own position, background, or researcher status that makes you take note? And asking yourself this question might lead you to think about what you did not notice. Other questions to ask yourself include the following: How do I know “that thing” I noted? So what? What does it mean? What are the implications? Who cares about this and why? Remember that doing qualitative research well is recursive , meaning that we may begin with a research design, but the steps of doing the research often loop back to the beginning. By keeping a reflective journal, you allow yourself to circle back to the beginning, to make changes to the study to keep it in line with what you are really interested in knowing.

One might also consider designing research that includes multiple investigators, particularly those who may not share your preconceptions about the study. For example, if you are studying conservative students on campus, and you yourself thoroughly identify as liberal, you might want to pair up with a researcher interested in the topic who grew up in a conservative household. If you are studying racial regimes, consider creating a racially diverse team of researchers. Or you might include in your research design a component of participatory research wherein members of the community of interest become coresearchers. Even if you can’t form a research team, you can reach out to others for feedback as you move along. Doing research can be a lonely enterprise, so finding people who will listen to you and nudge you to clarify your thinking where necessary or move you to consider an aspect you have missed is invaluable.

Finally, make it a regular part of your practice to write a paragraph reporting your perspectives, positions, values, and beliefs and how these may have influenced the research. This paragraph may be included in publications upon request.

Internal Validity

Being reflexive can help ensure that our studies are internally valid. All research must be valid to be helpful. We say a study’s findings are externally valid when they are equally true of other times, places, people. Quantitative researchers often spend a lot of time grappling with external validity , as they are often trying to demonstrate that their sample is representative of a larger population. Although we do not do that in qualitative research, we do sometimes make claims that the processes and mechanisms we uncover here, in this particular setting, are likely to be equally active in that setting over there, although there may be (will be!) contextual differences as well. Internal validity is more peculiar to qualitative research. Is your finding an accurate representation of what you are studying? Are you describing the people you are observing or interviewing as they really are? This is internal validity , and you should be able to see how this connects with the requirement of reflexivity. To the extent that you leave unexamined your own biases or preconceptions, you will fail at accurately representing those people and processes you study. Remember that “bias” here is not a moral failing in the way we commonly use bias in the nonresearch world but an inevitable product of our being social beings who inhabit social worlds, with all the various complexities surrounding that. Because of things that have happened to you, certain things (concepts, quotes, activities) might jump out at you as being particularly important. Being reflexive allows you to take a step back and grapple with the larger picture, reflecting on why you might be seeing X (which is present) but also missing Y (which is also present). It also allows you to consider what effect/impact your presence has on what you are observing or being told and to make any adjustments necessary to minimize your impact or, at the very least, to be aware of these effects and talk about them in any descriptions or presentations you make. There are other ways of ensuring internal validity (e.g., member checking , triangulation ), but being reflective is an essential component.

Advanced: Bourdieu on Reflexivity

One researcher who really tackled the issue of reflexivity was Pierre Bourdieu. [3] Known in the US primarily as a theorist, Bourdieu was a very capable and thorough researcher, who employed a variety of methods in his wide-ranging studies. Originally trained as an anthropologist, he became uncomfortable with the unreflective “outsider perspective” he was taught to follow. How was he supposed to observe and write about the various customs and rules of the people he was studying if he did not take into account his own supposedly neutral position in the observations? And even more interestingly, how could he write about customs and rules as if they were lifted from but outside of the understandings and practice of the people following them? When you say “God bless you” to someone who sneezes, are you really following a social custom that requires the prevention of illness through some performative verbal ritual of protection, or are you saying words out of reflex and habit? Bourdieu wondered what it meant that anthropologists were so ready to attribute meaning to actions that, to those performing them, were probably unconsidered. This caused him to ponder those deep epistemological questions about the possibilities of knowledge, particularly what we can know and truly understand about others. Throughout the following decades, as he developed his theories about the social world out of the deep and various studies he engaged in, he thought about the relationship between the researcher and the researched. He came to several conclusions about this relationship.

First, he argued that researchers needed to be reflective about their position vis-à-vis the object of study. The very fact that there is a subject and an object needs to be accounted for. Too often, he argued, the researcher forgets that part of the relationship, bracketing out the researcher entirely, as if what is being observed or studied exists entirely independently of the study. This can lead to false reports, as in the case where a blind man grasps the trunk of the elephant and claims the elephant is cylindrical, not having recognized how his own limitations of sight reduced the elephant to only one of its parts.

As mentioned previously, Bourdieu ( 1984 ) argued that “reflective analysis of the tools of analysis is not an epistemological scruple but an indispensable precondition of scientific knowledge of the object” ( 92 ). It is not that researchers are inherently biased—they are—but rather that the relationship between researcher and researched is an unnatural one that needs to be accounted for in the analysis. True and total objectivity is impossible, as researchers are human subjects themselves, called to research what interests them (or what interests their supervisors) and also inhabiting the social world. The solution to this problem is to be reflective and to account for these aspects in the analysis itself. Here is how Bourdieu explains this charge:

To adopt the viewpoint of REFLEXIVITY is not to renounce objectivity but to question the privilege of the knowing subject, which the antigenetic vision arbitrarily frees, as purely noetic, from the labor of objectification. To adopt this viewpoint is to strive to account for the empirical “subject” in the very terms of the objectivity constructed by the scientific subject (notably by situating it in a determined place in social space-time) and thereby to give oneself awareness and (possible) mastery of the constraints which may be exercised on the scientific subject via all the ties which attach it to the empirical “subject,” to its interests, motives, assumptions, beliefs, its doxa, and which it must break in order to constitute itself . ( 1996:207 ; emphases added)

Reflexivity, for Bourdieu, was a trained state of mind for the researcher, essential for proper knowledge production. Let’s use a story from Hans Christian Andersen to illustrate this point. If you remember this story from your childhood, it goes something like this: Two con artists show up in a town in which its chief monarch spends a lot of money on expensive clothes and splashy displays. They sense an opportunity to make some money out of this situation and pretend they are talented weavers from afar. They tell the vain emperor that they can make the most magnificent clothes anyone has ever seen (or not seen, as the case may be!). Because what they really do is “pretend” to weave and sew and hand the emperor thin air, which they then help him to put on in an elaborate joke. They tell him that only the very stupid and lowborn will be unable to see the magnificent clothes. Embarrassed that he can’t see them either, he pretends he can. Everyone pretends they can see clothes, when really the emperor walks around in his bare nakedness. As he parades through town, people redden and bow their heads, but no one says a thing. That is, until one child looks at the naked emperor and starts to laugh. His laughter breaks the spell, and everyone realizes the “naked truth.”

Now let us add a new thread to this story. The boy did not laugh. Years go by, and the emperor continues to wear his new clothes. At the start of every day, his aides carefully drape the “new clothes” around his naked body. Decades go by, and this is all “normal.” People don’t even see a naked emperor but a fully robed leader of the free world. A researcher, raised in this milieu, visits the palace to observe court habits. She observes the aides draping the emperor. She describes the care they take in doing so. She nowhere reports that the clothes are nonexistent because she herself has been trained to see them . She thus misses a very important fact—that there are no clothes at all! Note that it is not her individual “biases” that are getting in the way but her unreflective acceptance of the reality she inhabits that binds her to report things less accurately than she might.

In his later years, Bourdieu turned his attention to science itself and argued that the promise of modern science required reflectivity among scientists. We need to develop our reflexivity as we develop other muscles, through constant practice. Bourdieu ( 2004 ) urged researchers “to convert reflexivity into a disposition, constitutive of their scientific habitus, a reflexivity reflex , capable of acting not ex poste , on the opus operatum , but a priori , on the modus operandi ” ( 89 ). In other words, we need to build into our research design an appreciation of the relationship between researcher and researched.

To do science properly is to be reflective, to be aware of the social waters in which one swims and to turn one’s researching gaze on oneself and one’s researcher position as well as on the object of the research. Above all, doing science properly requires one to acknowledge science as a social process. We are not omniscient gods, lurking above the humans we observe and talk to. We are human too.

Request consultation

Do you need support in running a pricing or product study? We can help you with agile consumer research and conjoint analysis.

Looking for an online survey platform?

Conjointly offers a great survey tool with multiple question types, randomisation blocks, and multilingual support. The Basic tier is always free.

Research Methods Knowledge Base

Navigating the Knowledge Base
Foundations
Construct Validity
Reliability
Levels of Measurement
Survey Research
Scaling in Measurement
The Qualitative Debate
Qualitative Data
Qualitative Approaches
Qualitative Methods

Qualitative Validity

Unobtrusive Measures
Research Design
Table of Contents

Fully-functional online survey tool with various question types, logic, randomisation, and reporting for unlimited number of surveys.

Completely free for academics and students .

Depending on their philosophical perspectives , some qualitative researchers reject the framework of validity that is commonly accepted in more quantitative research in the social sciences. They reject the basic realist assumption that there is a reality external to our perception of it. Consequently, it doesn’t make sense to be concerned with the “truth” or “falsity” of an observation with respect to an external reality (which is a primary concern of validity). These qualitative researchers argue for different standards for judging the quality of research.

For instance, Guba and Lincoln proposed four criteria for judging the soundness of qualitative research and explicitly offered these as an alternative to more traditional quantitatively-oriented criteria. They felt that their four criteria better reflected the underlying assumptions involved in much qualitative research. Their proposed criteria and the “analogous” quantitative criteria are listed in the table.

Credibility

The credibility criteria involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in the research. Since from this perspective, the purpose of qualitative research is to describe or understand the phenomena of interest from the participant’s eyes, the participants are the only ones who can legitimately judge the credibility of the results.

Transferability

Transferability refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings. From a qualitative perspective transferability is primarily the responsibility of the one doing the generalizing. The qualitative researcher can enhance transferability by doing a thorough job of describing the research context and the assumptions that were central to the research. The person who wishes to “transfer” the results to a different context is then responsible for making the judgment of how sensible the transfer is.

Dependability

The traditional quantitative view of reliability is based on the assumption of replicability or repeatability. Essentially it is concerned with whether we would obtain the same results if we could observe the same thing twice. But we can’t actually measure the same thing twice – by definition if we are measuring twice, we are measuring two different things. In order to estimate reliability, quantitative researchers construct various hypothetical notions (e.g., true score theory ) to try to get around this fact.

The idea of dependability, on the other hand, emphasizes the need for the researcher to account for the ever-changing context within which research occurs. The research is responsible for describing the changes that occur in the setting and how these changes affected the way the research approached the study.

Confirmability

Qualitative research tends to assume that each researcher brings a unique perspective to the study. Confirmability refers to the degree to which the results could be confirmed or corroborated by others. There are a number of strategies for enhancing confirmability. The researcher can document the procedures for checking and rechecking the data throughout the study. Another researcher can take a “devil’s advocate” role with respect to the results, and this process can be documented. The researcher can actively search for and describe and negative instances that contradict prior observations. And, after he study, one can conduct a data audit that examines the data collection and analysis procedures and makes judgements about the potential for bias or distortion.

There has been considerable debate among methodologists about the value and legitimacy of this alternative set of standards for judging qualitative research. On the one hand, many quantitative researchers see the alternative criteria as just a relabeling of the very successful quantitative criteria in order to accrue greater legitimacy for qualitative research. They suggest that a correct reading of the quantitative criteria would show that they are not limited to quantitative research alone and can be applied equally well to qualitative data. They argue that the alternative criteria represent a different philosophical perspective that is subjectivist rather than realist in nature. They claim that research inherently assumes that there is some reality that is being observed and can be observed with greater or less accuracy or validity. if you don’t make this assumption, they would contend, you simply are not engaged in research (although that doesn’t mean that what you are doing is not valuable or useful).

Perhaps there is some legitimacy to this counter argument. Certainly a broad reading of the traditional quantitative criteria might make them appropriate to the qualitative realm as well. But historically the traditional quantitative criteria have been described almost exclusively in terms of quantitative research. No one has yet done a thorough job of translating how the same criteria might apply in qualitative research contexts. For instance, the discussions of external validity have been dominated by the idea of statistical sampling as the basis for generalizing. And, considerations of reliability have traditionally been inextricably linked to the notion of true score theory.

But qualitative researchers do have a point about the irrelevance of traditional quantitative criteria. How could we judge the external validity of a qualitative study that does not use formalized sampling methods? And, how can we judge the reliability of qualitative data when there is no mechanism for estimating the true score? No one has adequately explained how the operational procedures used to assess validity and reliability in quantitative research can be translated into legitimate corresponding operations for qualitative research.

While alternative criteria may not in the end be necessary (and I personally hope that more work is done on broadening the “traditional” criteria so that they legitimately apply across the entire spectrum of research approaches), and they certainly can be confusing for students and newcomers to this discussion, these alternatives do serve to remind us that qualitative research cannot easily be considered only an extension of the quantitative paradigm into the realm of nonnumeric data.

Cookie Consent

Conjointly uses essential cookies to make our site work. We also use additional cookies in order to understand the usage of the site, gather audience analytics, and for remarketing purposes.

For more information on Conjointly's use of cookies, please read our Cookie Policy .

Which one are you?

I am new to conjointly, i am already using conjointly.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
J Bras Pneumol
v.44(3); May-Jun 2018

Internal and external validity: can you apply research study results to your patients?

Cecilia maria patino.

1 . Methods in Epidemiologic, Clinical, and Operations Research-MECOR-program, American Thoracic Society/Asociación Latinoamericana del Tórax, Montevideo, Uruguay.

2 . Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.

Juliana Carvalho Ferreira

3 . Divisão de Pneumologia, Instituto do Coração, Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo (SP) Brasil.

CLINICAL SCENARIO

In a multicenter study in France, investigators conducted a randomized controlled trial to test the effect of prone vs. supine positioning ventilation on mortality among patients with early, severe ARDS. They showed that prolonged prone-positioning ventilation decreased 28-day mortality [hazard ratio (HR) = 0.39; 95% CI: 0.25-0.63]. 1

STUDY VALIDITY

The validity of a research study refers to how well the results among the study participants represent true findings among similar individuals outside the study. This concept of validity applies to all types of clinical studies, including those about prevalence, associations, interventions, and diagnosis. The validity of a research study includes two domains: internal and external validity.

Internal validity is defined as the extent to which the observed results represent the truth in the population we are studying and, thus, are not due to methodological errors. In our example, if the authors can support that the study has internal validity, they can conclude that prone positioning reduces mortality among patients with severe ARDS. The internal validity of a study can be threatened by many factors, including errors in measurement or in the selection of participants in the study, and researchers should think about and avoid these errors.

Once the internal validity of the study is established, the researcher can proceed to make a judgment regarding its external validity by asking whether the study results apply to similar patients in a different setting or not ( Figure 1 ). In the example, we would want to evaluate if the results of the clinical trial apply to ARDS patients in other ICUs. If the patients have early, severe ARDS, probably yes, but the study results may not apply to patients with mild ARDS . External validity refers to the extent to which the results of a study are generalizable to patients in our daily practice, especially for the population that the sample is thought to represent.

An external file that holds a picture, illustration, etc.
Object name is 1806-3713-jbpneu-44-03-00183-gf1.jpg

Lack of internal validity implies that the results of the study deviate from the truth, and, therefore, we cannot draw any conclusions; hence, if the results of a trial are not internally valid, external validity is irrelevant. 2 Lack of external validity implies that the results of the trial may not apply to patients who differ from the study population and, consequently, could lead to low adoption of the treatment tested in the trial by other clinicians.

INCREASING VALIDITY OF RESEARCH STUDIES

To increase internal validity, investigators should ensure careful study planning and adequate quality control and implementation strategies-including adequate recruitment strategies, data collection, data analysis, and sample size. External validity can be increased by using broad inclusion criteria that result in a study population that more closely resembles real-life patients, and, in the case of clinical trials, by choosing interventions that are feasible to apply. 2

Home page for the journal Education Policy Analysis Archives

Research, interrupted: Addressing practical and methodological challenges under turbulent conditions

Megan Kuhfeld NWEA
Lou Mariano RAND Corporation

The COVID-19 pandemic caused tremendous upheaval in schooling. In addition to devasting effects on students, these disruptions had consequences for researchers conducting studies on education programs and policies. Given the likelihood of future large-scale disruptions, it is important for researchers to plan resilient studies and think critically about adaptations when such turbulence arises. In this article, we utilize qualitative analysis of interviews with research study leaders to illuminate practical and methodological challenges, as well as promising practices that arose during the pandemic period. We find that researchers made pivots to address practical challenges and protect the feasibility of their studies. We also find that researchers took precautions, where possible, to understand and bolster internal validity. However, these pivots frequently surfaced additional threats to construct and external validity. We conclude with recommendations for future studies conducted in times of prolonged unplanned school closures or other large-scale disruptions.

Author Biographies

Susan bush-mecenas, rand corporation.

Susan Bush-Mecenas is the policy researcher at the RAND Corporation. Her research examines the implementation of PreK–12 education policies and practices, with attention to the interaction of school systems, their organizational and institutional context, and educators and school leaders as agents of change.

Jonathan Schweig, RAND Corporation

Jonathan D. Schweig is a senior social scientist at RAND. His research focuses on the measurement of teaching, and how safe and supportive learning environments can nurture positive adolescent development.

Megan Kuhfeld, NWEA

Megan Kuhfeld is the Director of Growth Modeling and Analytics at NWEA, a division of HMH. Her research seeks to understand students’ academic and social-emotional learning trajectories and the school and neighborhood influences that promote optimal growth

Lou Mariano, RAND Corporation

Louis T. Mariano is a senior statistician at RAND. His research focuses on evaluation of the efficacy of education policy programs and reforms.

Melissa Diliberti, RAND Corporation

Melissa Diliberti is an assistant policy researcher at RAND and a Ph.D. candidate at the Pardee RAND Graduate School. Her research uses nationally representative surveys of educators to investigate how the COVID-19 pandemic has changed the U.S. education system.

How to Cite

Endnote/Zotero/Mendeley (RIS)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License .

Most read articles by the same author(s)

Susan Bush-Mecenas, David Montes de Oca, Julie Marsh, Heather Hough, “Seeing the whole elephant”: Changing mindsets and empowering stakeholders to meaningfully manage accountability and improvement , Education Policy Analysis Archives: Vol. 26 (2018)

Education Policy Analysis Archives/Archivos Analíticos de Políticas Educativas/Arquivos Analíticos de Políticas Educativas (EPAA/AAPE; ISSN 1068-2341 ) is a peer-reviewed, open-access, international, multilingual, and multidisciplinary journal designed for researchers, practitioners, policy makers, and development analysts concerned with education policies.

Português (Brasil)
Español (España)

Submit a Manuscript or Revision

Recent special issues.

Youth and Adult Education, Literacies and Decoloniality

Global Policy Mobilities in Federal Education Systems

Teacher Subjectivities in Latin America

More information about the publishing system, Platform and Workflow by OJS/PKP.

The Federal Register

The daily journal of the united states government, request access.

Due to aggressive automated scraping of FederalRegister.gov and eCFR.gov, programmatic access to these sites is limited to access to our extensive developer APIs.

If you are human user receiving this message, we can add your IP address to a set of IPs that can access FederalRegister.gov & eCFR.gov; complete the CAPTCHA (bot test) below and click "Request Access". This process will be necessary for each IP address you wish to access the site from, requests are valid for approximately one quarter (three months) after which the process may need to be repeated.

An official website of the United States government.

If you want to request a wider IP range, first request access for your current IP, and then use the "Site Feedback" button found in the lower left-hand side to make the request.

IMAGES

How to establish the validity and reliability of qualitative research?
What is internal validity in research: Definition, tips & examples
PPT
11 Internal Validity Examples (2024)
Research Review: 7
Study validity and reliability

VIDEO

Internal Validity l Threats of Internal validity
Definitions of research terms used in psychology
BSN
Threats to Internal Validity in Experimental Research
Threats to Internal Validity in Experimental Research
Threats to Internal Validity in Experimental Research

COMMENTS

Internal Validity in Research
Internal validity makes the conclusions of a causal relationship credible and trustworthy. Without high internal validity, an experiment cannot demonstrate a causal link between two variables. Research example. You want to test the hypothesis that drinking a cup of coffee improves memory. You schedule an equal number of college-aged ...
Validity, reliability, and generalizability in qualitative research
From a realism standpoint, Porter then proposes multiple and open approaches for validity in qualitative research that incorporate parallel perspectives[43,44] and diversification of meanings. Any work of qualitative research, when read by the readers, is always a two-way interactive process, such that validity and quality has to be judged by ...
Validity in Qualitative Research: A Processual Approach
In this way, a processual view of validity changes the importance of the latter. The need for processual validity comes from the symbiosis itself or the co-participation of the researcher in qualitative research. In this, the researcher engages him- or herself and is involved in and by the research.
A Review of the Quality Indicators of Rigor in Qualitative Research
Abstract. Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework ...
Validity in Qualitative Evaluation: Linking Purposes, Paradigms, and
However, the increased importance given to qualitative information in the evidence-based paradigm in health care and social policy requires a more precise conceptualization of validity criteria that goes beyond just academic reflection. After all, one can argue that policy verdicts that are based on qualitative information must be legitimized by valid research, just as quantitative effect ...
PDF VALIDITY IN QUALITATIVE RESEARCH
Generalization: Internal generalization, which is the type of interest to qualitative researchers, "refers to the generalizability of a conclusion within the setting or group studies" (Maxwell, 1996, p. 97). It is the qualitative analog of the quantitative statistical conclusion validity. Validity Tests 1.
Quantitative and Qualitative Strategies to Strengthen Internal Validity
In this paper, we review factors that introduce biases in RCTs and we propose quantitative and qualitative strategies for colleting relevant data to strengthen internal validity. The factors are related to participants' reactions to randomization, attrition, treatment perceptions, and implementation of the intervention.
Issues of validity and reliability in qualitative research
Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research.2-4 In the broadest context these terms are applicable, with validity referring to the integrity and ...
Redefining Qualitative Methods: Believability in the Fifth Moment
Unlike traditional qualitative researchers, who emphasized internal validity in the initial design, constructivists and naturalists emphasized internal validity both prior to and during the research. To ensure internal validity, the researchers employed numerous validity checks throughout the course of the research ( Creswell & Miller, 2000 ...
Internal, External, and Ecological Validity in Research Design, Conduct
Internal validity examines whether the study design, conduct, and analysis answer the research questions without bias. External validity examines whether the study findings can be generalized to other contexts. Ecological validity examines, specifically, whether the study findings can be generalized to real-life settings; thus ecological ...
Internal Validity vs. External Validity in Research
Differences. The essential difference between internal validity and external validity is that internal validity refers to the structure of a study (and its variables) while external validity refers to the universality of the results. But there are further differences between the two as well. For instance, internal validity focuses on showing a ...
Rigor or Reliability and Validity in Qualitative Research: P ...
nts the concept of rigor in qualitative research using a phenomenological study as an exemplar to further illustrate the process. Elaborating on epistemological and theoretical conceptualizations by Lincoln and Guba, strategies congruent with qualitative perspective for ensuring validity to establish the credibility of the study are described. A synthesis of the historical development of ...
(PDF) Validity and Reliability in Qualitative Research
The criterion of validity and reliability in qualitative research can be achieved by presenting evaluations related to concepts in the context of trustworthiness such as credibility, accuracy of ...
Validity in Qualitative Research
Validity in qualitative research can also be checked by a technique known as respondent validation. This technique involves testing initial results with participants to see if they still ring true. Although the research has been interpreted and condensed, participants should still recognize the results as authentic and, at this stage, may even ...
Understanding Reliability and Validity in Qualitative Research
Kirk and Miller (1986) identify three types of reliability referred to in quantitative research, which relate to: (1) the degree to which a measurement, given repeatedly, remains the same (2) the stability of a measurement over time; and (3) the similarity of measurements within. a given time period (pp. 41-42).
International Journal of Qualitative Methods Validity in Qualitative
Abstract. This article provides a discussion on the question of validity in qualitative evaluation. Although validity in qualitative inquiry has been widely reflected upon in the methodological literature (and is still often subject of debate), the link with evaluation research is underexplored. Elaborating on epistemological and theoretical ...
Internal Validity
Internal Validity. Definition: Internal validity refers to the extent to which a research study accurately establishes a cause-and-effect relationship between the independent variable(s) and the dependent variable(s) being investigated. It assesses whether the observed changes in the dependent variable(s) are actually caused by the manipulation of the independent variable(s) rather than other ...
Levels of Evidence, Quality Assessment, and Risk of Bias: Evaluating
The level of evidence framework for assessing internal validity assumes that internal validity can be determined based on the study design alone, and thus makes the strongest assumptions. Risk of bias assessments involve an evaluation of the potential for bias in the context of a specific study, and thus involve the least assumptions about ...
Chapter 6. Reflexivity
Internal Validity. Being reflexive can help ensure that our studies are internally valid. All research must be valid to be helpful. We say a study's findings are externally valid when they are equally true of other times, places, people. ... Because qualitative research often requires interaction with live humans, failing to take into account ...
Quality in qualitative research: Through the lens of validity
Issues of trustworthiness in qualitative leisure research, often demonstrated through particular techniques of reliability and/or validity, is often either nonexistent, unsubstantial, or unexplained.
Processual Validity in Qualitative Research in Healthcare
However, a major concern regarding qualitative research is the great variety of epistemic, philosophical, and ontological aspects involved. 13,14,18 Recent evidence suggests that some structured ways of dealing with the diverse quality dimensions of qualitative research, such as internal and external validity, reliability, objectivity ...
Qualitative Validity
Qualitative Validity. Depending on their philosophical perspectives, some qualitative researchers reject the framework of validity that is commonly accepted in more quantitative research in the social sciences. They reject the basic realist assumption that there is a reality external to our perception of it. Consequently, it doesn't make sense to be concerned with the "truth" or ...
Internal and external validity: can you apply research study results to
The validity of a research study includes two domains: internal and external validity. Internal validity is defined as the extent to which the observed results represent the truth in the population we are studying and, thus, are not due to methodological errors. In our example, if the authors can support that the study has internal validity ...
Research, interrupted: Addressing practical and methodological
In this article, we utilize qualitative analysis of interviews with research study leaders to illuminate practical and methodological challenges, as well as promising practices that arose during the pandemic period. We find that researchers made pivots to address practical challenges and protect the feasibility of their studies.
Federal Register :: Proposed Collection; 60-Day Comment Request
The proposed information collection provides a means to garner qualitative customer and stakeholder feedback in an efficient, timely manner, in accordance with the Administration's commitment to improving service delivery. By qualitative feedback we mean information that provides useful insights on perceptions and opinions.

Have a language expert improve your writing

Internal Validity in Research | Definition, Threats & Examples

Table of contents

Here's why students love Scribbr's proofreading services

Single-group studies

Multi-group studies

Cite this Scribbr article

Is this article helpful?

Pritha Bhandari

Log in using your username and password

You are here

Statistics from Altmetric.com

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Read the full text or download the PDF:

Internal Validity vs. External Validity in Research

What Is Internal Validity in Research?

Factors That Improve Internal Validity

Internal Validity Threats

What Is External Validity in Research?

Factors That Improve External Validity

External Validity Threats

Internal Validity vs. External Validity

Similarities

Differences

Examples of Validity

Internal Validity Example

External Validity Example

A Word From Verywell

Internal Validity – Threats, Examples and Guide

Internal Validity

How to Increase Internal Validity

Threats To Internal Validity

Testing Effects

Instrumentation

Selection Bias

Attrition or Dropout

Regression to The Mean

Diffusion of Treatment

Demand Characteristics

Experimenter Bias

Types of Internal Validity

Construct validity

Statistical Conclusion Validity

Internal Validity of Causal Inferences

Temporal Precedence

Covariation

Elimination of Confounding Variables

Selection bias Control

Controlling for Testing Effects

Controlling for Experimenter Effects

Replication

Internal Validity Examples

Applications of Internal Validity

Advantages of Internal Validity

Limitations of Internal Validity

About the author

Muhammad Hassan

You may also like

Validity – Types, Examples and Guide

Alternate Forms Reliability – Methods, Examples...

Construct Validity – Types, Threats and Examples

Reliability Vs Validity

Internal Consistency Reliability – Methods...

Split-Half Reliability – Methods, Examples and...

Chapter 6. Reflexivity

Reflexivity: What It Is and Why It Is Important

Examples of Reflexivity in Practice

How to Do It

Internal Validity

Advanced: Bourdieu on Reflexivity

Further Readings

Popular searches

Request consultation

Looking for an online survey platform?

Research Methods Knowledge Base

Qualitative Validity

Credibility

Transferability

Dependability