empirical and literature review difference

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 2 Chapter 3: What is Empirical Literature & Where can it be Found?

In Module 1, you read about the problem of pseudoscience. Here, we revisit the issue in addressing how to locate and assess scientific or empirical literature . In this chapter you will read about:

• distinguishing between what IS and IS NOT empirical literature
• how and where to locate empirical literature for understanding diverse populations, social work problems, and social phenomena.

Probably the most important take-home lesson from this chapter is that one source is not sufficient to being well-informed on a topic. It is important to locate multiple sources of information and to critically appraise the points of convergence and divergence in the information acquired from different sources. This is especially true in emerging and poorly understood topics, as well as in answering complex questions.

What Is Empirical Literature

Social workers often need to locate valid, reliable information concerning the dimensions of a population group or subgroup, a social work problem, or social phenomenon. They might also seek information about the way specific problems or resources are distributed among the populations encountered in professional practice. Or, social workers might be interested in finding out about the way that certain people experience an event or phenomenon. Empirical literature resources may provide answers to many of these types of social work questions. In addition, resources containing data regarding social indicators may also prove helpful. Social indicators are the “facts and figures” statistics that describe the social, economic, and psychological factors that have an impact on the well-being of a community or other population group.The United Nations (UN) and the World Health Organization (WHO) are examples of organizations that monitor social indicators at a global level: dimensions of population trends (size, composition, growth/loss), health status (physical, mental, behavioral, life expectancy, maternal and infant mortality, fertility/child-bearing, and diseases like HIV/AIDS), housing and quality of sanitation (water supply, waste disposal), education and literacy, and work/income/unemployment/economics, for example.

Three characteristics stand out in empirical literature compared to other types of information available on a topic of interest: systematic observation and methodology, objectivity, and transparency/replicability/reproducibility. Let’s look a little more closely at these three features.

Systematic Observation and Methodology. The hallmark of empiricism is “repeated or reinforced observation of the facts or phenomena” (Holosko, 2006, p. 6). In empirical literature, established research methodologies and procedures are systematically applied to answer the questions of interest.

Objectivity. Gathering “facts,” whatever they may be, drives the search for empirical evidence (Holosko, 2006). Authors of empirical literature are expected to report the facts as observed, whether or not these facts support the investigators’ original hypotheses. Research integrity demands that the information be provided in an objective manner, reducing sources of investigator bias to the greatest possible extent.

Transparency and Replicability/Reproducibility.   Empirical literature is reported in such a manner that other investigators understand precisely what was done and what was found in a particular research study—to the extent that they could replicate the study to determine whether the findings are reproduced when repeated. The outcomes of an original and replication study may differ, but a reader could easily interpret the methods and procedures leading to each study’s findings.

What is NOT Empirical Literature

By now, it is probably obvious to you that literature based on “evidence” that is not developed in a systematic, objective, transparent manner is not empirical literature. On one hand, non-empirical types of professional literature may have great significance to social workers. For example, social work scholars may produce articles that are clearly identified as describing a new intervention or program without evaluative evidence, critiquing a policy or practice, or offering a tentative, untested theory about a phenomenon. These resources are useful in educating ourselves about possible issues or concerns. But, even if they are informed by evidence, they are not empirical literature. Here is a list of several sources of information that do not meet the standard of being called empirical literature:

• your course instructor’s lectures
• political statements
• advertisements
• newspapers & magazines (journalism)
• television news reports & analyses (journalism)
• many websites, Facebook postings, Twitter tweets, and blog postings
• the introductory literature review in an empirical article

You may be surprised to see the last two included in this list. Like the other sources of information listed, these sources also might lead you to look for evidence. But, they are not themselves sources of evidence. They may summarize existing evidence, but in the process of summarizing (like your instructor’s lectures), information is transformed, modified, reduced, condensed, and otherwise manipulated in such a manner that you may not see the entire, objective story. These are called secondary sources, as opposed to the original, primary source of evidence. In relying solely on secondary sources, you sacrifice your own critical appraisal and thinking about the original work—you are “buying” someone else’s interpretation and opinion about the original work, rather than developing your own interpretation and opinion. What if they got it wrong? How would you know if you did not examine the primary source for yourself? Consider the following as an example of “getting it wrong” being perpetuated.

Example: Bullying and School Shootings . One result of the heavily publicized April 1999 school shooting incident at Columbine High School (Colorado), was a heavy emphasis placed on bullying as a causal factor in these incidents (Mears, Moon, & Thielo, 2017), “creating a powerful master narrative about school shootings” (Raitanen, Sandberg, & Oksanen, 2017, p. 3). Naturally, with an identified cause, a great deal of effort was devoted to anti-bullying campaigns and interventions for enhancing resilience among youth who experience bullying.  However important these strategies might be for promoting positive mental health, preventing poor mental health, and possibly preventing suicide among school-aged children and youth, it is a mistaken belief that this can prevent school shootings (Mears, Moon, & Thielo, 2017). Many times the accounts of the perpetrators having been bullied come from potentially inaccurate third-party accounts, rather than the perpetrators themselves; bullying was not involved in all instances of school shooting; a perpetrator’s perception of being bullied/persecuted are not necessarily accurate; many who experience severe bullying do not perpetrate these incidents; bullies are the least targeted shooting victims; perpetrators of the shooting incidents were often bullying others; and, bullying is only one of many important factors associated with perpetrating such an incident (Ioannou, Hammond, & Simpson, 2015; Mears, Moon, & Thielo, 2017; Newman &Fox, 2009; Raitanen, Sandberg, & Oksanen, 2017). While mass media reports deliver bullying as a means of explaining the inexplicable, the reality is not so simple: “The connection between bullying and school shootings is elusive” (Langman, 2014), and “the relationship between bullying and school shooting is, at best, tenuous” (Mears, Moon, & Thielo, 2017, p. 940). The point is, when a narrative becomes this publicly accepted, it is difficult to sort out truth and reality without going back to original sources of information and evidence.

What May or May Not Be Empirical Literature: Literature Reviews

Investigators typically engage in a review of existing literature as they develop their own research studies. The review informs them about where knowledge gaps exist, methods previously employed by other scholars, limitations of prior work, and previous scholars’ recommendations for directing future research. These reviews may appear as a published article, without new study data being reported (see Fields, Anderson, & Dabelko-Schoeny, 2014 for example). Or, the literature review may appear in the introduction to their own empirical study report. These literature reviews are not considered to be empirical evidence sources themselves, although they may be based on empirical evidence sources. One reason is that the authors of a literature review may or may not have engaged in a systematic search process, identifying a full, rich, multi-sided pool of evidence reports.

There is, however, a type of review that applies systematic methods and is, therefore, considered to be more strongly rooted in evidence: the systematic review .

Systematic review of literature. A systematic reviewis a type of literature report where established methods have been systematically applied, objectively, in locating and synthesizing a body of literature. The systematic review report is characterized by a great deal of transparency about the methods used and the decisions made in the review process, and are replicable. Thus, it meets the criteria for empirical literature: systematic observation and methodology, objectivity, and transparency/reproducibility. We will work a great deal more with systematic reviews in the second course, SWK 3402, since they are important tools for understanding interventions. They are somewhat less common, but not unheard of, in helping us understand diverse populations, social work problems, and social phenomena.

Locating Empirical Evidence

Social workers have available a wide array of tools and resources for locating empirical evidence in the literature. These can be organized into four general categories.

Journal Articles. A number of professional journals publish articles where investigators report on the results of their empirical studies. However, it is important to know how to distinguish between empirical and non-empirical manuscripts in these journals. A key indicator, though not the only one, involves a peer review process . Many professional journals require that manuscripts undergo a process of peer review before they are accepted for publication. This means that the authors’ work is shared with scholars who provide feedback to the journal editor as to the quality of the submitted manuscript. The editor then makes a decision based on the reviewers’ feedback:

• Accept as is
• Accept with minor revisions
• Request that a revision be resubmitted (no assurance of acceptance)

When a “revise and resubmit” decision is made, the piece will go back through the review process to determine if it is now acceptable for publication and that all of the reviewers’ concerns have been adequately addressed. Editors may also reject a manuscript because it is a poor fit for the journal, based on its mission and audience, rather than sending it for review consideration.

Indicators of journal relevance. Various journals are not equally relevant to every type of question being asked of the literature. Journals may overlap to a great extent in terms of the topics they might cover; in other words, a topic might appear in multiple different journals, depending on how the topic was being addressed. For example, articles that might help answer a question about the relationship between community poverty and violence exposure might appear in several different journals, some with a focus on poverty, others with a focus on violence, and still others on community development or public health. Journal titles are sometimes a good starting point but may not give a broad enough picture of what they cover in their contents.

In focusing a literature search, it also helps to review a journal’s mission and target audience. For example, at least four different journals focus specifically on poverty:

• Journal of Children & Poverty
• Journal of Poverty
• Journal of Poverty and Social Justice
• Poverty & Public Policy

Let’s look at an example using the Journal of Poverty and Social Justice . Information about this journal is located on the journal’s webpage: http://policy.bristoluniversitypress.co.uk/journals/journal-of-poverty-and-social-justice . In the section headed “About the Journal” you can see that it is an internationally focused research journal, and that it addresses social justice issues in addition to poverty alone. The research articles are peer-reviewed (there appear to be non-empirical discussions published, as well). These descriptions about a journal are almost always available, sometimes listed as “scope” or “mission.” These descriptions also indicate the sponsorship of the journal—sponsorship may be institutional (a particular university or agency, such as Smith College Studies in Social Work ), a professional organization, such as the Council on Social Work Education (CSWE) or the National Association of Social Work (NASW), or a publishing company (e.g., Taylor & Frances, Wiley, or Sage).

Indicators of journal caliber.  Despite engaging in a peer review process, not all journals are equally rigorous. Some journals have very high rejection rates, meaning that many submitted manuscripts are rejected; others have fairly high acceptance rates, meaning that relatively few manuscripts are rejected. This is not necessarily the best indicator of quality, however, since newer journals may not be sufficiently familiar to authors with high quality manuscripts and some journals are very specific in terms of what they publish. Another index that is sometimes used is the journal’s impact factor . Impact factor is a quantitative number indicative of how often articles published in the journal are cited in the reference list of other journal articles—the statistic is calculated as the number of times on average each article published in a particular year were cited divided by the number of articles published (the number that could be cited). For example, the impact factor for the Journal of Poverty and Social Justice in our list above was 0.70 in 2017, and for the Journal of Poverty was 0.30. These are relatively low figures compared to a journal like the New England Journal of Medicine with an impact factor of 59.56! This means that articles published in that journal were, on average, cited more than 59 times in the next year or two.

Impact factors are not necessarily the best indicator of caliber, however, since many strong journals are geared toward practitioners rather than scholars, so they are less likely to be cited by other scholars but may have a large impact on a large readership. This may be the case for a journal like the one titled Social Work, the official journal of the National Association of Social Workers. It is distributed free to all members: over 120,000 practitioners, educators, and students of social work world-wide. The journal has a recent impact factor of.790. The journals with social work relevant content have impact factors in the range of 1.0 to 3.0 according to Scimago Journal & Country Rank (SJR), particularly when they are interdisciplinary journals (for example, Child Development , Journal of Marriage and Family , Child Abuse and Neglect , Child Maltreatmen t, Social Service Review , and British Journal of Social Work ). Once upon a time, a reader could locate different indexes comparing the “quality” of social work-related journals. However, the concept of “quality” is difficult to systematically define. These indexes have mostly been replaced by impact ratings, which are not necessarily the best, most robust indicators on which to rely in assessing journal quality. For example, new journals addressing cutting edge topics have not been around long enough to have been evaluated using this particular tool, and it takes a few years for articles to begin to be cited in other, later publications.

Beware of pseudo-, illegitimate, misleading, deceptive, and suspicious journals . Another side effect of living in the Age of Information is that almost anyone can circulate almost anything and call it whatever they wish. This goes for “journal” publications, as well. With the advent of open-access publishing in recent years (electronic resources available without subscription), we have seen an explosion of what are called predatory or junk journals . These are publications calling themselves journals, often with titles very similar to legitimate publications and often with fake editorial boards. These “publications” lack the integrity of legitimate journals. This caution is reminiscent of the discussions earlier in the course about pseudoscience and “snake oil” sales. The predatory nature of many apparent information dissemination outlets has to do with how scientists and scholars may be fooled into submitting their work, often paying to have their work peer-reviewed and published. There exists a “thriving black-market economy of publishing scams,” and at least two “journal blacklists” exist to help identify and avoid these scam journals (Anderson, 2017).

This issue is important to information consumers, because it creates a challenge in terms of identifying legitimate sources and publications. The challenge is particularly important to address when information from on-line, open-access journals is being considered. Open-access is not necessarily a poor choice—legitimate scientists may pay sizeable fees to legitimate publishers to make their work freely available and accessible as open-access resources. On-line access is also not necessarily a poor choice—legitimate publishers often make articles available on-line to provide timely access to the content, especially when publishing the article in hard copy will be delayed by months or even a year or more. On the other hand, stating that a journal engages in a peer-review process is no guarantee of quality—this claim may or may not be truthful. Pseudo- and junk journals may engage in some quality control practices, but may lack attention to important quality control processes, such as managing conflict of interest, reviewing content for objectivity or quality of the research conducted, or otherwise failing to adhere to industry standards (Laine & Winker, 2017).

One resource designed to assist with the process of deciphering legitimacy is the Directory of Open Access Journals (DOAJ). The DOAJ is not a comprehensive listing of all possible legitimate open-access journals, and does not guarantee quality, but it does help identify legitimate sources of information that are openly accessible and meet basic legitimacy criteria. It also is about open-access journals, not the many journals published in hard copy.

An additional caution: Search for article corrections. Despite all of the careful manuscript review and editing, sometimes an error appears in a published article. Most journals have a practice of publishing corrections in future issues. When you locate an article, it is helpful to also search for updates. Here is an example where data presented in an article’s original tables were erroneous, and a correction appeared in a later issue.

• Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2017). A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 12(8): e0181722. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5558917/
• Marchant, A., Hawton, K., Stewart A., Montgomery, P., Singaravelu, V., Lloyd, K., Purdy, N., Daine, K., & John, A. (2018).Correction—A systematic review of the relationship between internet use, self-harm and suicidal behaviour in young people: The good, the bad and the unknown. PLoS One, 13(3): e0193937.  http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193937

Search Tools. In this age of information, it is all too easy to find items—the problem lies in sifting, sorting, and managing the vast numbers of items that can be found. For example, a simple Google® search for the topic “community poverty and violence” resulted in about 15,600,000 results! As a means of simplifying the process of searching for journal articles on a specific topic, a variety of helpful tools have emerged. One type of search tool has previously applied a filtering process for you: abstracting and indexing databases . These resources provide the user with the results of a search to which records have already passed through one or more filters. For example, PsycINFO is managed by the American Psychological Association and is devoted to peer-reviewed literature in behavioral science. It contains almost 4.5 million records and is growing every month. However, it may not be available to users who are not affiliated with a university library. Conducting a basic search for our topic of “community poverty and violence” in PsychINFO returned 1,119 articles. Still a large number, but far more manageable. Additional filters can be applied, such as limiting the range in publication dates, selecting only peer reviewed items, limiting the language of the published piece (English only, for example), and specified types of documents (either chapters, dissertations, or journal articles only, for example). Adding the filters for English, peer-reviewed journal articles published between 2010 and 2017 resulted in 346 documents being identified.

Just as was the case with journals, not all abstracting and indexing databases are equivalent. There may be overlap between them, but none is guaranteed to identify all relevant pieces of literature. Here are some examples to consider, depending on the nature of the questions asked of the literature:

• Academic Search Complete—multidisciplinary index of 9,300 peer-reviewed journals
• AgeLine—multidisciplinary index of aging-related content for over 600 journals
• Campbell Collaboration—systematic reviews in education, crime and justice, social welfare, international development
• Google Scholar—broad search tool for scholarly literature across many disciplines
• MEDLINE/ PubMed—National Library of medicine, access to over 15 million citations
• Oxford Bibliographies—annotated bibliographies, each is discipline specific (e.g., psychology, childhood studies, criminology, social work, sociology)
• PsycINFO/PsycLIT—international literature on material relevant to psychology and related disciplines
• SocINDEX—publications in sociology
• Social Sciences Abstracts—multiple disciplines
• Social Work Abstracts—many areas of social work are covered
• Web of Science—a “meta” search tool that searches other search tools, multiple disciplines

Placing our search for information about “community violence and poverty” into the Social Work Abstracts tool with no additional filters resulted in a manageable 54-item list. Finally, abstracting and indexing databases are another way to determine journal legitimacy: if a journal is indexed in a one of these systems, it is likely a legitimate journal. However, the converse is not necessarily true: if a journal is not indexed does not mean it is an illegitimate or pseudo-journal.

Government Sources. A great deal of information is gathered, analyzed, and disseminated by various governmental branches at the international, national, state, regional, county, and city level. Searching websites that end in.gov is one way to identify this type of information, often presented in articles, news briefs, and statistical reports. These government sources gather information in two ways: they fund external investigations through grants and contracts and they conduct research internally, through their own investigators. Here are some examples to consider, depending on the nature of the topic for which information is sought:

• Agency for Healthcare Research and Quality (AHRQ) at https://www.ahrq.gov/
• Bureau of Justice Statistics (BJS) at https://www.bjs.gov/
• Census Bureau at https://www.census.gov
• Morbidity and Mortality Weekly Report of the CDC (MMWR-CDC) at https://www.cdc.gov/mmwr/index.html
• Child Welfare Information Gateway at https://www.childwelfare.gov
• Children’s Bureau/Administration for Children & Families at https://www.acf.hhs.gov
• Forum on Child and Family Statistics at https://www.childstats.gov
• National Institutes of Health (NIH) at https://www.nih.gov , including (not limited to):
• National Institute on Aging (NIA at https://www.nia.nih.gov
• National Institute on Alcohol Abuse and Alcoholism (NIAAA) at https://www.niaaa.nih.gov
• National Institute of Child Health and Human Development (NICHD) at https://www.nichd.nih.gov
• National Institute on Drug Abuse (NIDA) at https://www.nida.nih.gov
• National Institute of Environmental Health Sciences at https://www.niehs.nih.gov
• National Institute of Mental Health (NIMH) at https://www.nimh.nih.gov
• National Institute on Minority Health and Health Disparities at https://www.nimhd.nih.gov
• National Institute of Justice (NIJ) at https://www.nij.gov
• Substance Abuse and Mental Health Services Administration (SAMHSA) at https://www.samhsa.gov/
• United States Agency for International Development at https://usaid.gov

Each state and many counties or cities have similar data sources and analysis reports available, such as Ohio Department of Health at https://www.odh.ohio.gov/healthstats/dataandstats.aspx and Franklin County at https://statisticalatlas.com/county/Ohio/Franklin-County/Overview . Data are available from international/global resources (e.g., United Nations and World Health Organization), as well.

Other Sources. The Health and Medicine Division (HMD) of the National Academies—previously the Institute of Medicine (IOM)—is a nonprofit institution that aims to provide government and private sector policy and other decision makers with objective analysis and advice for making informed health decisions. For example, in 2018 they produced reports on topics in substance use and mental health concerning the intersection of opioid use disorder and infectious disease,  the legal implications of emerging neurotechnologies, and a global agenda concerning the identification and prevention of violence (see http://www.nationalacademies.org/hmd/Global/Topics/Substance-Abuse-Mental-Health.aspx ). The exciting aspect of this resource is that it addresses many topics that are current concerns because they are hoping to help inform emerging policy. The caution to consider with this resource is the evidence is often still emerging, as well.

Numerous “think tank” organizations exist, each with a specific mission. For example, the Rand Corporation is a nonprofit organization offering research and analysis to address global issues since 1948. The institution’s mission is to help improve policy and decision making “to help individuals, families, and communities throughout the world be safer and more secure, healthier and more prosperous,” addressing issues of energy, education, health care, justice, the environment, international affairs, and national security (https://www.rand.org/about/history.html). And, for example, the Robert Woods Johnson Foundation is a philanthropic organization supporting research and research dissemination concerning health issues facing the United States. The foundation works to build a culture of health across systems of care (not only medical care) and communities (https://www.rwjf.org).

While many of these have a great deal of helpful evidence to share, they also may have a strong political bias. Objectivity is often lacking in what information these organizations provide: they provide evidence to support certain points of view. That is their purpose—to provide ideas on specific problems, many of which have a political component. Think tanks “are constantly researching solutions to a variety of the world’s problems, and arguing, advocating, and lobbying for policy changes at local, state, and federal levels” (quoted from https://thebestschools.org/features/most-influential-think-tanks/ ). Helpful information about what this one source identified as the 50 most influential U.S. think tanks includes identifying each think tank’s political orientation. For example, The Heritage Foundation is identified as conservative, whereas Human Rights Watch is identified as liberal.

While not the same as think tanks, many mission-driven organizations also sponsor or report on research, as well. For example, the National Association for Children of Alcoholics (NACOA) in the United States is a registered nonprofit organization. Its mission, along with other partnering organizations, private-sector groups, and federal agencies, is to promote policy and program development in research, prevention and treatment to provide information to, for, and about children of alcoholics (of all ages). Based on this mission, the organization supports knowledge development and information gathering on the topic and disseminates information that serves the needs of this population. While this is a worthwhile mission, there is no guarantee that the information meets the criteria for evidence with which we have been working. Evidence reported by think tank and mission-driven sources must be utilized with a great deal of caution and critical analysis!

In many instances an empirical report has not appeared in the published literature, but in the form of a technical or final report to the agency or program providing the funding for the research that was conducted. One such example is presented by a team of investigators funded by the National Institute of Justice to evaluate a program for training professionals to collect strong forensic evidence in instances of sexual assault (Patterson, Resko, Pierce-Weeks, & Campbell, 2014): https://www.ncjrs.gov/pdffiles1/nij/grants/247081.pdf . Investigators may serve in the capacity of consultant to agencies, programs, or institutions, and provide empirical evidence to inform activities and planning. One such example is presented by Maguire-Jack (2014) as a report to a state’s child maltreatment prevention board: https://preventionboard.wi.gov/Documents/InvestmentInPreventionPrograming_Final.pdf .

When Direct Answers to Questions Cannot Be Found. Sometimes social workers are interested in finding answers to complex questions or questions related to an emerging, not-yet-understood topic. This does not mean giving up on empirical literature. Instead, it requires a bit of creativity in approaching the literature. A Venn diagram might help explain this process. Consider a scenario where a social worker wishes to locate literature to answer a question concerning issues of intersectionality. Intersectionality is a social justice term applied to situations where multiple categorizations or classifications come together to create overlapping, interconnected, or multiplied disadvantage. For example, women with a substance use disorder and who have been incarcerated face a triple threat in terms of successful treatment for a substance use disorder: intersectionality exists between being a woman, having a substance use disorder, and having been in jail or prison. After searching the literature, little or no empirical evidence might have been located on this specific triple-threat topic. Instead, the social worker will need to seek literature on each of the threats individually, and possibly will find literature on pairs of topics (see Figure 3-1). There exists some literature about women’s outcomes for treatment of a substance use disorder (a), some literature about women during and following incarceration (b), and some literature about substance use disorders and incarceration (c). Despite not having a direct line on the center of the intersecting spheres of literature (d), the social worker can develop at least a partial picture based on the overlapping literatures.

Figure 3-1. Venn diagram of intersecting literature sets.

Take a moment to complete the following activity. For each statement about empirical literature, decide if it is true or false.

Social Work 3401 Coursebook Copyright © by Dr. Audrey Begun is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

Literature Review vs Systematic Review

• Literature Review vs. Systematic Review
• Primary vs. Secondary Sources
• Databases and Articles
• Specific Journal or Article

Subject Guide

Definitions

It’s common to confuse systematic and literature reviews because both are used to provide a summary of the existent literature or research on a specific topic. Regardless of this commonality, both types of review vary significantly. The following table provides a detailed explanation as well as the differences between systematic and literature reviews. 

Kysh, Lynn (2013): Difference between a systematic review and a literature review. [figshare]. Available at:  http://dx.doi.org/10.6084/m9.figshare.766364

• << Previous: Home
• Next: Primary vs. Secondary Sources >>
• Last Updated: Dec 15, 2023 10:19 AM
• URL: https://libguides.sjsu.edu/LitRevVSSysRev

• Meriam Library

SWRK 330 - Social Work Research Methods

• Literature Reviews and Empirical Research
• Databases and Search Tips
• Article Citations
• Scholarly Journal Evaulation
• Statistical Sources
• Books and eBooks

What is a Literature Review?

Empirical research.

• Annotated Bibliographies

A literature review  summarizes and discusses previous publications  on a topic.

It should also:

explore past research and its strengths and weaknesses.

be used to validate the target and methods you have chosen for your proposed research.

consist of books and scholarly journals that provide research examples of populations or settings similar to your own, as well as community resources to document the need for your proposed research.

The literature review does not present new  primary  scholarship. 

be completed in the correct citation format requested by your professor  (see the  C itations Tab)

Access Purdue  OWL's Social Work Literature Review Guidelines here .  

Empirical Research  is  research  that is based on experimentation or observation, i.e. Evidence. Such  research  is often conducted to answer a specific question or to test a hypothesis (educated guess).

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."  Ask yourself: Could I recreate this study and test these results?

These are some key features to look for when identifying empirical research.

NOTE:  Not all of these features will be in every empirical research article, some may be excluded, use this only as a guide.

• Statement of methodology
• Research questions are clear and measurable
• Individuals, group, subjects which are being studied are identified/defined
• Data is presented regarding the findings
• Controls or instruments such as surveys or tests were conducted
• There is a literature review
• There is discussion of the results included
• Citations/references are included

See also Empirical Research Guide

• << Previous: Citations
• Next: Annotated Bibliographies >>
• Last Updated: Feb 6, 2024 8:38 AM
• URL: https://libguides.csuchico.edu/SWRK330

Meriam Library | CSU, Chico

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Korean J Anesthesiol
• v.71(2); 2018 Apr

Introduction to systematic review and meta-analysis

1 Department of Anesthesiology and Pain Medicine, Inje University Seoul Paik Hospital, Seoul, Korea

2 Department of Anesthesiology and Pain Medicine, Chung-Ang University College of Medicine, Seoul, Korea

Systematic reviews and meta-analyses present results by combining and analyzing data from different studies conducted on similar research topics. In recent years, systematic reviews and meta-analyses have been actively performed in various fields including anesthesiology. These research methods are powerful tools that can overcome the difficulties in performing large-scale randomized controlled trials. However, the inclusion of studies with any biases or improperly assessed quality of evidence in systematic reviews and meta-analyses could yield misleading results. Therefore, various guidelines have been suggested for conducting systematic reviews and meta-analyses to help standardize them and improve their quality. Nonetheless, accepting the conclusions of many studies without understanding the meta-analysis can be dangerous. Therefore, this article provides an easy introduction to clinicians on performing and understanding meta-analyses.

Introduction

A systematic review collects all possible studies related to a given topic and design, and reviews and analyzes their results [ 1 ]. During the systematic review process, the quality of studies is evaluated, and a statistical meta-analysis of the study results is conducted on the basis of their quality. A meta-analysis is a valid, objective, and scientific method of analyzing and combining different results. Usually, in order to obtain more reliable results, a meta-analysis is mainly conducted on randomized controlled trials (RCTs), which have a high level of evidence [ 2 ] ( Fig. 1 ). Since 1999, various papers have presented guidelines for reporting meta-analyses of RCTs. Following the Quality of Reporting of Meta-analyses (QUORUM) statement [ 3 ], and the appearance of registers such as Cochrane Library’s Methodology Register, a large number of systematic literature reviews have been registered. In 2009, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [ 4 ] was published, and it greatly helped standardize and improve the quality of systematic reviews and meta-analyses [ 5 ].

Levels of evidence.

In anesthesiology, the importance of systematic reviews and meta-analyses has been highlighted, and they provide diagnostic and therapeutic value to various areas, including not only perioperative management but also intensive care and outpatient anesthesia [6–13]. Systematic reviews and meta-analyses include various topics, such as comparing various treatments of postoperative nausea and vomiting [ 14 , 15 ], comparing general anesthesia and regional anesthesia [ 16 – 18 ], comparing airway maintenance devices [ 8 , 19 ], comparing various methods of postoperative pain control (e.g., patient-controlled analgesia pumps, nerve block, or analgesics) [ 20 – 23 ], comparing the precision of various monitoring instruments [ 7 ], and meta-analysis of dose-response in various drugs [ 12 ].

Thus, literature reviews and meta-analyses are being conducted in diverse medical fields, and the aim of highlighting their importance is to help better extract accurate, good quality data from the flood of data being produced. However, a lack of understanding about systematic reviews and meta-analyses can lead to incorrect outcomes being derived from the review and analysis processes. If readers indiscriminately accept the results of the many meta-analyses that are published, incorrect data may be obtained. Therefore, in this review, we aim to describe the contents and methods used in systematic reviews and meta-analyses in a way that is easy to understand for future authors and readers of systematic review and meta-analysis.

Study Planning

It is easy to confuse systematic reviews and meta-analyses. A systematic review is an objective, reproducible method to find answers to a certain research question, by collecting all available studies related to that question and reviewing and analyzing their results. A meta-analysis differs from a systematic review in that it uses statistical methods on estimates from two or more different studies to form a pooled estimate [ 1 ]. Following a systematic review, if it is not possible to form a pooled estimate, it can be published as is without progressing to a meta-analysis; however, if it is possible to form a pooled estimate from the extracted data, a meta-analysis can be attempted. Systematic reviews and meta-analyses usually proceed according to the flowchart presented in Fig. 2 . We explain each of the stages below.

Flowchart illustrating a systematic review.

Formulating research questions

A systematic review attempts to gather all available empirical research by using clearly defined, systematic methods to obtain answers to a specific question. A meta-analysis is the statistical process of analyzing and combining results from several similar studies. Here, the definition of the word “similar” is not made clear, but when selecting a topic for the meta-analysis, it is essential to ensure that the different studies present data that can be combined. If the studies contain data on the same topic that can be combined, a meta-analysis can even be performed using data from only two studies. However, study selection via a systematic review is a precondition for performing a meta-analysis, and it is important to clearly define the Population, Intervention, Comparison, Outcomes (PICO) parameters that are central to evidence-based research. In addition, selection of the research topic is based on logical evidence, and it is important to select a topic that is familiar to readers without clearly confirmed the evidence [ 24 ].

Protocols and registration

In systematic reviews, prior registration of a detailed research plan is very important. In order to make the research process transparent, primary/secondary outcomes and methods are set in advance, and in the event of changes to the method, other researchers and readers are informed when, how, and why. Many studies are registered with an organization like PROSPERO ( http://www.crd.york.ac.uk/PROSPERO/ ), and the registration number is recorded when reporting the study, in order to share the protocol at the time of planning.

Defining inclusion and exclusion criteria

Information is included on the study design, patient characteristics, publication status (published or unpublished), language used, and research period. If there is a discrepancy between the number of patients included in the study and the number of patients included in the analysis, this needs to be clearly explained while describing the patient characteristics, to avoid confusing the reader.

Literature search and study selection

In order to secure proper basis for evidence-based research, it is essential to perform a broad search that includes as many studies as possible that meet the inclusion and exclusion criteria. Typically, the three bibliographic databases Medline, Embase, and Cochrane Central Register of Controlled Trials (CENTRAL) are used. In domestic studies, the Korean databases KoreaMed, KMBASE, and RISS4U may be included. Effort is required to identify not only published studies but also abstracts, ongoing studies, and studies awaiting publication. Among the studies retrieved in the search, the researchers remove duplicate studies, select studies that meet the inclusion/exclusion criteria based on the abstracts, and then make the final selection of studies based on their full text. In order to maintain transparency and objectivity throughout this process, study selection is conducted independently by at least two investigators. When there is a inconsistency in opinions, intervention is required via debate or by a third reviewer. The methods for this process also need to be planned in advance. It is essential to ensure the reproducibility of the literature selection process [ 25 ].

Quality of evidence

However, well planned the systematic review or meta-analysis is, if the quality of evidence in the studies is low, the quality of the meta-analysis decreases and incorrect results can be obtained [ 26 ]. Even when using randomized studies with a high quality of evidence, evaluating the quality of evidence precisely helps determine the strength of recommendations in the meta-analysis. One method of evaluating the quality of evidence in non-randomized studies is the Newcastle-Ottawa Scale, provided by the Ottawa Hospital Research Institute 1) . However, we are mostly focusing on meta-analyses that use randomized studies.

If the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) system ( http://www.gradeworkinggroup.org/ ) is used, the quality of evidence is evaluated on the basis of the study limitations, inaccuracies, incompleteness of outcome data, indirectness of evidence, and risk of publication bias, and this is used to determine the strength of recommendations [ 27 ]. As shown in Table 1 , the study limitations are evaluated using the “risk of bias” method proposed by Cochrane 2) . This method classifies bias in randomized studies as “low,” “high,” or “unclear” on the basis of the presence or absence of six processes (random sequence generation, allocation concealment, blinding participants or investigators, incomplete outcome data, selective reporting, and other biases) [ 28 ].

The Cochrane Collaboration’s Tool for Assessing the Risk of Bias [ 28 ]

Data extraction

Two different investigators extract data based on the objectives and form of the study; thereafter, the extracted data are reviewed. Since the size and format of each variable are different, the size and format of the outcomes are also different, and slight changes may be required when combining the data [ 29 ]. If there are differences in the size and format of the outcome variables that cause difficulties combining the data, such as the use of different evaluation instruments or different evaluation timepoints, the analysis may be limited to a systematic review. The investigators resolve differences of opinion by debate, and if they fail to reach a consensus, a third-reviewer is consulted.

Data Analysis

The aim of a meta-analysis is to derive a conclusion with increased power and accuracy than what could not be able to achieve in individual studies. Therefore, before analysis, it is crucial to evaluate the direction of effect, size of effect, homogeneity of effects among studies, and strength of evidence [ 30 ]. Thereafter, the data are reviewed qualitatively and quantitatively. If it is determined that the different research outcomes cannot be combined, all the results and characteristics of the individual studies are displayed in a table or in a descriptive form; this is referred to as a qualitative review. A meta-analysis is a quantitative review, in which the clinical effectiveness is evaluated by calculating the weighted pooled estimate for the interventions in at least two separate studies.

The pooled estimate is the outcome of the meta-analysis, and is typically explained using a forest plot ( Figs. 3 and ​ and4). 4 ). The black squares in the forest plot are the odds ratios (ORs) and 95% confidence intervals in each study. The area of the squares represents the weight reflected in the meta-analysis. The black diamond represents the OR and 95% confidence interval calculated across all the included studies. The bold vertical line represents a lack of therapeutic effect (OR = 1); if the confidence interval includes OR = 1, it means no significant difference was found between the treatment and control groups.

Forest plot analyzed by two different models using the same data. (A) Fixed-effect model. (B) Random-effect model. The figure depicts individual trials as filled squares with the relative sample size and the solid line as the 95% confidence interval of the difference. The diamond shape indicates the pooled estimate and uncertainty for the combined effect. The vertical line indicates the treatment group shows no effect (OR = 1). Moreover, if the confidence interval includes 1, then the result shows no evidence of difference between the treatment and control groups.

Forest plot representing homogeneous data.

Dichotomous variables and continuous variables

In data analysis, outcome variables can be considered broadly in terms of dichotomous variables and continuous variables. When combining data from continuous variables, the mean difference (MD) and standardized mean difference (SMD) are used ( Table 2 ).

Summary of Meta-analysis Methods Available in RevMan [ 28 ]

The MD is the absolute difference in mean values between the groups, and the SMD is the mean difference between groups divided by the standard deviation. When results are presented in the same units, the MD can be used, but when results are presented in different units, the SMD should be used. When the MD is used, the combined units must be shown. A value of “0” for the MD or SMD indicates that the effects of the new treatment method and the existing treatment method are the same. A value lower than “0” means the new treatment method is less effective than the existing method, and a value greater than “0” means the new treatment is more effective than the existing method.

When combining data for dichotomous variables, the OR, risk ratio (RR), or risk difference (RD) can be used. The RR and RD can be used for RCTs, quasi-experimental studies, or cohort studies, and the OR can be used for other case-control studies or cross-sectional studies. However, because the OR is difficult to interpret, using the RR and RD, if possible, is recommended. If the outcome variable is a dichotomous variable, it can be presented as the number needed to treat (NNT), which is the minimum number of patients who need to be treated in the intervention group, compared to the control group, for a given event to occur in at least one patient. Based on Table 3 , in an RCT, if x is the probability of the event occurring in the control group and y is the probability of the event occurring in the intervention group, then x = c/(c + d), y = a/(a + b), and the absolute risk reduction (ARR) = x − y. NNT can be obtained as the reciprocal, 1/ARR.

Calculation of the Number Needed to Treat in the Dichotomous table

Fixed-effect models and random-effect models

In order to analyze effect size, two types of models can be used: a fixed-effect model or a random-effect model. A fixed-effect model assumes that the effect of treatment is the same, and that variation between results in different studies is due to random error. Thus, a fixed-effect model can be used when the studies are considered to have the same design and methodology, or when the variability in results within a study is small, and the variance is thought to be due to random error. Three common methods are used for weighted estimation in a fixed-effect model: 1) inverse variance-weighted estimation 3) , 2) Mantel-Haenszel estimation 4) , and 3) Peto estimation 5) .

A random-effect model assumes heterogeneity between the studies being combined, and these models are used when the studies are assumed different, even if a heterogeneity test does not show a significant result. Unlike a fixed-effect model, a random-effect model assumes that the size of the effect of treatment differs among studies. Thus, differences in variation among studies are thought to be due to not only random error but also between-study variability in results. Therefore, weight does not decrease greatly for studies with a small number of patients. Among methods for weighted estimation in a random-effect model, the DerSimonian and Laird method 6) is mostly used for dichotomous variables, as the simplest method, while inverse variance-weighted estimation is used for continuous variables, as with fixed-effect models. These four methods are all used in Review Manager software (The Cochrane Collaboration, UK), and are described in a study by Deeks et al. [ 31 ] ( Table 2 ). However, when the number of studies included in the analysis is less than 10, the Hartung-Knapp-Sidik-Jonkman method 7) can better reduce the risk of type 1 error than does the DerSimonian and Laird method [ 32 ].

Fig. 3 shows the results of analyzing outcome data using a fixed-effect model (A) and a random-effect model (B). As shown in Fig. 3 , while the results from large studies are weighted more heavily in the fixed-effect model, studies are given relatively similar weights irrespective of study size in the random-effect model. Although identical data were being analyzed, as shown in Fig. 3 , the significant result in the fixed-effect model was no longer significant in the random-effect model. One representative example of the small study effect in a random-effect model is the meta-analysis by Li et al. [ 33 ]. In a large-scale study, intravenous injection of magnesium was unrelated to acute myocardial infarction, but in the random-effect model, which included numerous small studies, the small study effect resulted in an association being found between intravenous injection of magnesium and myocardial infarction. This small study effect can be controlled for by using a sensitivity analysis, which is performed to examine the contribution of each of the included studies to the final meta-analysis result. In particular, when heterogeneity is suspected in the study methods or results, by changing certain data or analytical methods, this method makes it possible to verify whether the changes affect the robustness of the results, and to examine the causes of such effects [ 34 ].

Heterogeneity

Homogeneity test is a method whether the degree of heterogeneity is greater than would be expected to occur naturally when the effect size calculated from several studies is higher than the sampling error. This makes it possible to test whether the effect size calculated from several studies is the same. Three types of homogeneity tests can be used: 1) forest plot, 2) Cochrane’s Q test (chi-squared), and 3) Higgins I 2 statistics. In the forest plot, as shown in Fig. 4 , greater overlap between the confidence intervals indicates greater homogeneity. For the Q statistic, when the P value of the chi-squared test, calculated from the forest plot in Fig. 4 , is less than 0.1, it is considered to show statistical heterogeneity and a random-effect can be used. Finally, I 2 can be used [ 35 ].

I 2 , calculated as shown above, returns a value between 0 and 100%. A value less than 25% is considered to show strong homogeneity, a value of 50% is average, and a value greater than 75% indicates strong heterogeneity.

Even when the data cannot be shown to be homogeneous, a fixed-effect model can be used, ignoring the heterogeneity, and all the study results can be presented individually, without combining them. However, in many cases, a random-effect model is applied, as described above, and a subgroup analysis or meta-regression analysis is performed to explain the heterogeneity. In a subgroup analysis, the data are divided into subgroups that are expected to be homogeneous, and these subgroups are analyzed. This needs to be planned in the predetermined protocol before starting the meta-analysis. A meta-regression analysis is similar to a normal regression analysis, except that the heterogeneity between studies is modeled. This process involves performing a regression analysis of the pooled estimate for covariance at the study level, and so it is usually not considered when the number of studies is less than 10. Here, univariate and multivariate regression analyses can both be considered.

Publication bias

Publication bias is the most common type of reporting bias in meta-analyses. This refers to the distortion of meta-analysis outcomes due to the higher likelihood of publication of statistically significant studies rather than non-significant studies. In order to test the presence or absence of publication bias, first, a funnel plot can be used ( Fig. 5 ). Studies are plotted on a scatter plot with effect size on the x-axis and precision or total sample size on the y-axis. If the points form an upside-down funnel shape, with a broad base that narrows towards the top of the plot, this indicates the absence of a publication bias ( Fig. 5A ) [ 29 , 36 ]. On the other hand, if the plot shows an asymmetric shape, with no points on one side of the graph, then publication bias can be suspected ( Fig. 5B ). Second, to test publication bias statistically, Begg and Mazumdar’s rank correlation test 8) [ 37 ] or Egger’s test 9) [ 29 ] can be used. If publication bias is detected, the trim-and-fill method 10) can be used to correct the bias [ 38 ]. Fig. 6 displays results that show publication bias in Egger’s test, which has then been corrected using the trim-and-fill method using Comprehensive Meta-Analysis software (Biostat, USA).

Funnel plot showing the effect size on the x-axis and sample size on the y-axis as a scatter plot. (A) Funnel plot without publication bias. The individual plots are broader at the bottom and narrower at the top. (B) Funnel plot with publication bias. The individual plots are located asymmetrically.

Funnel plot adjusted using the trim-and-fill method. White circles: comparisons included. Black circles: inputted comparisons using the trim-and-fill method. White diamond: pooled observed log risk ratio. Black diamond: pooled inputted log risk ratio.

Result Presentation

When reporting the results of a systematic review or meta-analysis, the analytical content and methods should be described in detail. First, a flowchart is displayed with the literature search and selection process according to the inclusion/exclusion criteria. Second, a table is shown with the characteristics of the included studies. A table should also be included with information related to the quality of evidence, such as GRADE ( Table 4 ). Third, the results of data analysis are shown in a forest plot and funnel plot. Fourth, if the results use dichotomous data, the NNT values can be reported, as described above.

The GRADE Evidence Quality for Each Outcome

N: number of studies, ROB: risk of bias, PON: postoperative nausea, POV: postoperative vomiting, PONV: postoperative nausea and vomiting, CI: confidence interval, RR: risk ratio, AR: absolute risk.

When Review Manager software (The Cochrane Collaboration, UK) is used for the analysis, two types of P values are given. The first is the P value from the z-test, which tests the null hypothesis that the intervention has no effect. The second P value is from the chi-squared test, which tests the null hypothesis for a lack of heterogeneity. The statistical result for the intervention effect, which is generally considered the most important result in meta-analyses, is the z-test P value.

A common mistake when reporting results is, given a z-test P value greater than 0.05, to say there was “no statistical significance” or “no difference.” When evaluating statistical significance in a meta-analysis, a P value lower than 0.05 can be explained as “a significant difference in the effects of the two treatment methods.” However, the P value may appear non-significant whether or not there is a difference between the two treatment methods. In such a situation, it is better to announce “there was no strong evidence for an effect,” and to present the P value and confidence intervals. Another common mistake is to think that a smaller P value is indicative of a more significant effect. In meta-analyses of large-scale studies, the P value is more greatly affected by the number of studies and patients included, rather than by the significance of the results; therefore, care should be taken when interpreting the results of a meta-analysis.

When performing a systematic literature review or meta-analysis, if the quality of studies is not properly evaluated or if proper methodology is not strictly applied, the results can be biased and the outcomes can be incorrect. However, when systematic reviews and meta-analyses are properly implemented, they can yield powerful results that could usually only be achieved using large-scale RCTs, which are difficult to perform in individual studies. As our understanding of evidence-based medicine increases and its importance is better appreciated, the number of systematic reviews and meta-analyses will keep increasing. However, indiscriminate acceptance of the results of all these meta-analyses can be dangerous, and hence, we recommend that their results be received critically on the basis of a more accurate understanding.

1) http://www.ohri.ca .

2) http://methods.cochrane.org/bias/assessing-risk-bias-included-studies .

3) The inverse variance-weighted estimation method is useful if the number of studies is small with large sample sizes.

4) The Mantel-Haenszel estimation method is useful if the number of studies is large with small sample sizes.

5) The Peto estimation method is useful if the event rate is low or one of the two groups shows zero incidence.

6) The most popular and simplest statistical method used in Review Manager and Comprehensive Meta-analysis software.

7) Alternative random-effect model meta-analysis that has more adequate error rates than does the common DerSimonian and Laird method, especially when the number of studies is small. However, even with the Hartung-Knapp-Sidik-Jonkman method, when there are less than five studies with very unequal sizes, extra caution is needed.

8) The Begg and Mazumdar rank correlation test uses the correlation between the ranks of effect sizes and the ranks of their variances [ 37 ].

9) The degree of funnel plot asymmetry as measured by the intercept from the regression of standard normal deviates against precision [ 29 ].

10) If there are more small studies on one side, we expect the suppression of studies on the other side. Trimming yields the adjusted effect size and reduces the variance of the effects by adding the original studies back into the analysis as a mirror image of each study.

• 66 Ogoja Road, Abakaliki, Ebonyi State 23480 NG.
• Sun - Fri 24Hours Saturday CLOSED
• support [@] writersking.com
• +23480-6075-5653 Hot Line

• Data Collection/Analysis
• Hire Proposal Writers
• Hire Essay Writers
• Hire Paper Writers
• Proofreading Services
• Thesis/Dissertation Writers
• Virtual Supervisor
• Turnitin Checker
• Book Chapter Writer
• Hire Business Writing Services
• Hire Blog Writers
• Writers King TV
• Proposal Sample
• Chapter 1-3 Sample
• Term Paper Sample
• Report Assignment Sample
• Course work Sample
• Payment Options
• Privacy Policy
• Terms of Service/Use
• Business Guide
• Academic Writing Guide
• General Writing Guide
• Research News
• Writing Paper Samples
• February 19, 2022
• Posted by: Chimnecherem Eke
• Category: Academic Writing Guide

Differences between Empirical Review and Literature Review

As a researcher, you might wonder about the difference between an Empirical Review and Literature Review. If you are going to write an essay or article, you must first figure out what your topic is. The subject determines the flow of the writing, the material provided, and every other element.

Content Outline

Get your seat closer; let’s get to what you need to know about Empirical Review and Literature Review and their notable differences.

Students often accost empirical studies and literature reviews while preparing a research paper . It’s critical to know the difference between the two in order to craft a solid piece of writing. Both articles are responsible for presenting the facts, but their strategies differ.

Students are often perplexed when asked what empirical research is. The following is an explanation of the differences between a systematic review and a literature review:

Definition of Empirical review -Empirical Review and Literature Review

An empirical literature review, also known as a systematic literature review, analyzes previous empirical studies in order to provide an answer to a specific research topic.

Rather than drawing information from theories or beliefs, empirical research relies on observations and measurements to arrive at conclusions. To address specific research inquiries, it could involve making a list of people, behaviours, or events that are being researched.

In some cases, reviews of studies that involve experiments are used in empirical reviews to generate findings based on experience that may be seen directly or indirectly. Most of the time, the analysis entails quantifying the data and drawing conclusions.

The goal is to provide data that can be quantified using established scientific methods . Research reviewers thoroughly examine all findings of other authors before drawing any conclusions in an essay or paper.

As a result of carefully planned and monitored observations, the experiment is carried out, and the resulting conclusion is rigorously monitored. In contrast to other types of literature reviews, the focus here is on the most recent results of the experimental studies as it is now being conducted. A hypothesis may also be a forecast of a previously presented theory based on prior material.

Empirical evidence refers to data gathered via testing or observation in this context. These data are collected and analyzed by scientists.

For example, an empirical review could involve reviewing the study of another researcher on a group of listeners exposed to upbeat music or a work on learning and improvisation that examines other studies on work that theorizes about the educational value of improvisational principles and practices, such as Viola Spolin and Keith Johnstone ‘s writings in which they present their beliefs, impressions, ideas, and theories about those.

Defining Literature Review -Empirical Review and Literature Review

The literature review , instead of an empirical review, necessitates reading several related studies. Other theoretical sources are used to compile the facts and information included in this piece. The accumulation of all literary works may lead to new deductions. Information and hypotheses, on the other hand, have already been developed.

To generate cohesive findings, a literature review compiles all necessary data. No new theories can be developed since no experimental work can be done.

There is an important function for the literature review in uncovering defining, and clarifying key ideas that will be utilized throughout the empirical parts of the paper argues.

A well-written review article may shed light on the current state of knowledge, explain apparent inconsistencies, pinpoint areas in need of more study, and even help to forge a consensus where none previously existed. A well-written review may also assist you in your professional life. Reviews aid in recognition and advancement due to their high citation frequency.

Selecting the type of review to conduct -Empirical Review and Literature Review

College students are often required to write several papers as part of their studies. When a student does a literature review, he or she attempts to use the written word to support or refute an idea or hypothesis. He or she may test a theory or try to find an answer to a specific issue based on already-known information.

An empirical review is a piece of writing based on a study that was done purely for the purpose of publishing it. Calibrated instruments are used to conduct the experiment in a scientifically controlled way.

In this age of AI, research has become more interesting as you can use AI for your research study to make reviewing literature easier (Thesis, Dissertation, Research paper, among others). AI tools like SciSpace literature review can help you compare, contrast, and analyze research papers more efficiently.

It would be best to start writing as soon as you finish the experiment. During the study, observations should be recorded methodically. This aids in developing coherence, which is more easily understood even in the future. Additionally, starting the writing process early allows for more time for revision and results in higher-quality work.

Because experiments may take some time to produce the desired results, this is especially important for empirical studies. Leaving the writing to the last minute and beginning it when the deadline is nearing will just add to the stress and complexity of the process. This interferes with the job and lowers the quality. As a result, staying on top of your job helps your paper and your personal life grow.

Almost every research article includes a review of the literature. An empirical study must first be established inside an accepted theory before they can publish their findings. In other words, before we get into our methodology and research questions, we’ll go through what’s been done previously and how the variables we want to investigate fit into the theories and frameworks of our research field.

An empirical literature review, also known as a systematic literature review, analyzes previous empirical studies in order to provide an answer to a specific research topic. Randomized controlled trials are the most common kind of empirical study.

Both of these tasks are similar in that they need to review previous work on the topic. The empirical literature review, on the other hand, seeks to address a particular empirical issue by analyzing data. The theoretical literature review serves primarily to place your research within a broader framework. A theoretical review will be included in systematic empirical reviews to help researchers understand why a specific research topic is worth investigating.

Thank you for your time, we hope you got value reading Differences between Empirical Review and Literature Review?

Frequently Asked Questions

What is an example of an empirical review.

An empirical review is an academic review that focuses on summarizing and analyzing the findings of empirical research studies. Empirical research involves collecting and analysing primary data through experiments, surveys, observations, or any other data collection method(s).

An example of an empirical review would be a critical examination of several recent scientific studies that have investigated the effectiveness of a specific drug in treating a particular medical condition. The empirical review would include the study objectives, a summary of the research methods, key findings, and a critical evaluation of the research design, data collection, and statistical analysis methods employed in those studies.

What is the purpose of a literature review in empirical research studies?

A literature review in empirical research studies provides a comprehensive and systematic overview of existing published research and scholarly works related to the topic under investigation. This serves several important purposes:

Contextualization : It helps researchers place their study within the context of existing knowledge and identify gaps or areas where further research is needed.

Identification of Theoretical Frameworks : A literature review aids in identifying relevant theories and concepts that can inform the research design and data analysis in empirical studies.

Methodological Guidance: It offers insights into prior studies’ methodologies and research methods, helping researchers make informed decisions about their own data collection and analysis techniques.

Hypothesis Development: It can assist in formulating research hypotheses and research questions based on the existing literature.

Validation of Research Question: By reviewing the literature, researchers can ensure that their research questions or hypotheses have not been adequately addressed in prior studies.

Identification of Variables: It helps identify key variables and factors examined in previous research, which can guide the selection of variables in the new study.

In summary, a literature review in empirical research serves as the foundation upon which a new empirical study is built, helping researchers gain a deep understanding of the existing knowledge in their field, refine their research questions, and make informed decisions about the research design and methods they will employ.

Some quotations on code mixing and code switching base on empirical review.

Drop your comment, question or suggestion for the post improvement Cancel reply

Find Us Today

Writers King LTD,  Akachukwu Plaza, 

66 Ogoja Road Abakaliki,  Ebonyi State, 

480101 Nigeria

Phone: 0806-075-5653

• Website: https://writersking.com/
• Email: support {@} writersking.com
• +2348060755653

Quick Links

Writing guide.

• UTEP Library
• UTEP Library Research Guides

PSYC 3101: General Experimental Methods

Empirical articles vs. review articles.

• Use Library Resources from Anywhere Off-Campus
• Search Tips
• Article Anatomy
• Use RefWorks This link opens in a new window
• University Writing Center
• What is a Peer-Reviewed Article Anyway?

Empirical Articles

Review Articles

What is a Peer Reviewed Article?

Peer review is a process that many, but not all, journals use. Article manuscripts submitted to peer-reviewed journals are not automatically accepted and published.

In peer review, a panel of experts in the given field review the manuscript to determine aspects such as the quality of research, appropriateness for the journal, and relevance to the field. One of three decisions is made: accept, reject, or revise based on commentary from reviewers.

The process of peer review is thought to help ensure that high quality articles appear in journals.

Another term for peer-reviewed is  refereed . Peer-reviewed journals may also be called  scholarly.

Remember that magazines, Internet sources, and books are not the same as peer-reviewed journals.

In psychology, articles that report on original/new research studies may be referred to as primary sources or empirical .

• Common sections in a research/empirical article include introduction, literature review, methods/process, data, results, discussion, conclusion / suggestions for further study, and references.
• If the article is not divided into sections, it does not automatically mean it is not an empirical article.
• You cannot assume that an article is empirical just because it is divided into sections.
• In the methods section [which may be called something similar], or otherwise usually toward the beginning or middle of the article [if it does not have sections]: The authors will describe how they actively conducted new or original research -- such as an experiment or survey. Examples of what would likely be explained: How they identified participants, that they received Institutional Review Board (IRB) approval, control vs. experimental groups, and so forth about their research.

Articles that either interpret or analyze empirical articles are considered  review articles.  Such articles are often referred to as  secondary  sources  or  secondary research.

• An entire article that is purely a literature review [usually a review of select other articles considered to be the best support for a research question/topic]
• Systematic review
• Meta-analysis / meta-analyses
• Meta-synthesis / meta-syntheses
• If an article is a review article, it is likely [but not always] to have the words literature review, systematic review, integrative review, meta-analysis, meta-synthesis, or other mention of review in the title.
• The main indicator of a review article is if authors are just interpreting, analyzing, and/or comparing the results of empirical articles. So, in comparison to an empirical article, the authors of a review article do not describe an experiment or survey they conducted.
• If there is a methods section, it will usually describe how the authors searched for other articles [which databases they searched, what search terms they used] and decided the criteria for articles to include and exclude as part of their review. Again, they will not be describing how they conducted new or original research, such as an experiment or survey.
• In a literature review, for example, the authors' might point out what they believe to be the most pertinent/applicable research articles.
• << Previous: Use Library Resources from Anywhere Off-Campus
• Next: Databases >>
• Last Updated: Feb 20, 2024 9:46 AM
• URL: https://libguides.utep.edu/psyc3101

MSU Libraries

• Need help? Ask a Librarian

Research Methods Resources - Kinesiology: Empirical vs Review Articles

• Finding Journal Articles
• Searching Indexes and Databases
• How to Locate the Full Text of an Article, once you have found the Citation
• Not sure if your Journal Article is considered Academic/Scholarly or Peer-reviewed?
• Obtaining Materials not Owned by MSU
• Connecting to Library Resources from Off Campus
• Empirical vs Review Articles
• Grants & Fundraising
• General Information
• Literature Review
• Citation Mapping
• SAGE Research Methods

Empirical Journal Articles

Empirical Article - (Original Research) Based on experience and observation, rather than systematic logic. (according to MedicineNet.com)

The articles contain original research (such as scientific experiments, surveys and research studies) A list of references or sources is provided at the end of each article An editorial board, composed of experts in the field, reviews articles to decide whether they should be accepted; this is also known as "refereed," "peer-reviewed," "professional," "scholarly", or "academic". Uses a specialized vocabulary for that field.

Below are two websites that explain Empirical articles and research:

Empirical Research: How to Recognize and Locate (Penn State University) - Empirical Research PDF

Review Journal Articles

Review Article

An article that summarizes the progress or current state in some particular subject, area, or topic.

How to write a "Review Article?" (National Library of Medicine) - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/

Subject Guide

• << Previous: Connecting to Library Resources from Off Campus
• Next: Grants & Fundraising >>
• Last Updated: May 10, 2022 5:57 PM
• URL: https://libguides.lib.msu.edu/c.php?g=96527

PSYC 200 Lab in Experimental Methods (Atlanta)

• Find Research Articles

Empirical vs. Review Articles

How to recognize empirical journal articles, scholarly vs. non-scholarly sources.

• Cite Sources
• Find Tests & Measures
• Find Research Methods Materials
• Post-Library Lab Activity on Finding Tests and Measures
• Find Books, E-books, and Films

Psychology Librarian

Know the difference between empirical and review articles.

Empirical article An empirical (research) article reports methods and findings of an original research study conducted by the authors of the article.  

Literature Review article A review article or "literature review" discusses past research studies on a given topic.

Definition of an empirical study:  An empirical research article reports the results of a study that uses data derived from actual observation or experimentation. Empirical research articles are examples of primary research.

Parts of a standard empirical research article:  (articles will not necessary use the exact terms listed below.)

• Abstract  ... A paragraph length description of what the study includes.
• Introduction ...Includes a statement of the hypotheses for the research and a review of other research on the topic.
• Who are participants
• Design of the study
• What the participants did
• What measures were used
• Results ...Describes the outcomes of the measures of the study.
• Discussion ...Contains the interpretations and implications of the study.
• References ...Contains citation information on the material cited in the report. (also called bibliography or works cited)

Characteristics of an Empirical Article:

• Empirical articles will include charts, graphs, or statistical analysis.
• Empirical research articles are usually substantial, maybe from 8-30 pages long.
• There is always a bibliography found at the end of the article.

Type of publications that publish empirical studies:

• Empirical research articles are published in scholarly or academic journals
• These journals are also called “peer-reviewed,” or “refereed” publications.

Examples of such publications include:

• Computers in Human Behavior
• Journal of Educational Psychology

Examples of databases that contain empirical research:  (selected list only)

• Web of Science

This page is adapted from the Sociology Research Guide: Identify Empirical Articles page at Cal State Fullerton Pollak Library.

Know the difference between scholarly and non-scholarly articles.

"Scholarly" journal = "Peer-Reviewed" journal = "Refereed" journal

When researching your topic, you may come across many different types of sources and articles. When evaluating these sources, it is important to think about: 

• Who is the author? 
• Who is the audience or why was this written? 
• Where was this published? 
• Is this relevant to your research? 
• When was this written? Has it been updated? 
• Are there any citations? Who do they cite?  

Helpful Links and Guides

Here are helpful links and guides to check out for more information on scholarly sources: 

• This database contains data on different types of serials and can be used to determine whether a periodical is peer-reviewed or not:  Ulrich's Periodicals Directory  
• The UC Berkeley Library published this useful guide on evaluating resources, including the differences between scholarly and popular sources, as well as how to find primary sources:  UC Berkeley's Evaluating Resources LibGuide
• << Previous: Quick Poll
• Next: Cite Sources >>
• Last Updated: Feb 14, 2024 3:32 PM
• URL: https://guides.libraries.emory.edu/main/psyc200

Identifying and Locating Empirical Articles

• Types of Scholarly Articles

Empirical Articles vs Review Articles

• Locating Empirical Articles

Empirical articles are written to share the results of original research. Their authors will share their findings, including results, data, and ideas for future research. This will allow other researchers to learn more and conduct further studies.

Review articles are written to compare and discuss the results of multiple articles. They may be structured similarly to original research articles, but they are synthesizing what others have written about, rather than reporting on their own research.

Watch the video below (~1:30 min) for more information.

• << Previous: Types of Scholarly Articles
• Next: Locating Empirical Articles >>
• Last Updated: Sep 29, 2023 2:51 PM
• URL: https://libguides.ucmerced.edu/identifying-locating-empirical-articles

Colorado College

Research guides.

Tutt Library Research Guides

• Find Articles
• Search Tips
• Empirical v. Theoretical
• Credibility
• Position Papers
• Curricular Education Databases
• Statistics & Data
• Streaming Films & DVDs
• Print Books
• Professional Development & Careers
• Citation Guide This link opens in a new window
• ED 101 Information

Empirical or Theoretical?

Empirical: Based on data gathered by original experiments or observations.

Theoretical: Analyzes and makes connections between empirical studies to define or advance a theoretical position.

Videos on Finding Empirical Articles

Where Can I Find Empirically-Based Education Articles?

The most direct route is to search PsycInfo, linked above.

This will take you to the Advanced Search, where you can type in your key words at the top. Then scroll down through all the limiting options to the Methodology menu. Select Empirical Study. 

In other databases without the Methodology limiter, such as Education Source , try keywords like empirical , study , and research .

How Can I Tell if an Article is Empirical?

Check for these components:

• Peer-reviewed
• Charts, graphs, tables, and/or statistical analyses
• More than 5 pages
• Sections with names like: Abstract, Introduction, Literature Review, Method, Data, Analysis, Results, Discussion, References

Look for visual cues of data collection and analysis:

• << Previous: Search Tips
• Next: Credibility >>
• Last Updated: Mar 27, 2024 10:27 AM
• URL: https://coloradocollege.libguides.com/education

Ohio State nav bar

The Ohio State University

• BuckeyeLink
• Find People
• Search Ohio State

Literature Review, Theoretical Review or Conceptual Review?

I was researching and refreshing what I learned about literature reviews and research methodology in the 6625 ESLTECH class and came across this post in ResearchGate:

While  I was interested in finding out how to conduct a lit review,  and what methodologies fit with my “Begin With the End in mind” framework, I remembered that I was actually conducting a conceptual analysis of the Hackathon through two instruments:  the opportunistic interviews and my observations.  Applying Backward Design as research methodology has been used in the biological sciences.  It is called Backward Design for Education Research (BDER) and basically it instructs how to apply teaching-as-research (based on the Scholarship of Teaching and Learning(SoTL) model) to see what works in “providing evidence-based pedagogy and a deeper understanding of causal mechanisms for the broader education community.” (Jenkins, Bailey, Kummer & Weber, 2017). My approach was  more like “Backward Market Research,”  which consists of eight steps that ultimately resemble Wiggins and McTighe’s Backward Design for curriculum development. The key to backward market research lies in identifying the desired outcome (i.e., what data would answer the question you are asking) before embarking on the project …” (Jenkins, Bailey, Kummer & Weber, 2017). My question is broad  – what insights come out of the Hackathon – and the results are formative – data reports generate more questions and identify the need to code certain responses,  or to add a new question next year (as with the multitude of comments about Judging from 2017) – but a body or knowledge is being formed.

• Undergraduate
• Short Courses
• Online Courses
• Masters/Postgraduate
• Postdoctoral
• Study in Africa
• Study in Asia
• Study in Australia
• Study in Europe
• Study in USA
• Fellowships
• Internships
• Volunteering
• OD Live Series
• Young Person of the Month
• Success Stories
• General Tips
• ODIC 2023 Application
• ODIC Judges
• ODIC 2022 Winners
• ODIC 2021 Winners
• ODIC 2019 Winners
• ODIC 2018 Winners
• ODIC 2017 Winners
• ODIC 2016 Winners
• Australia and Oceania

Empirical Study vs. Literature Review

When writing an essay or an article, it is imperative to identify its theme. The flow of the written content, the information that is presented and all other aspects are based on the topic. When writing the research paper, students often accost empirical studies and literature review. To write a legit paper, it is essential to understand the difference between the two concepts. Both papers account for showing the facts, however, they vary in the approach. If there are difficulties with writing essays, a student can always turn to  write my essays services. They will help you write a paper on almost any subject and on any topic. A question what is empirical research often confuses those who learn. Systematic review vs literature review description is shown below:

1. Legit meaning of empirical research

Empirical research derives conclusions based on experience, which can be directly visualized or indirectly observed with the help of experiments. Normally, the analysis is performed by quantifying the results and the inference is drawn. The idea is to present the findings that are quantifiably observable through calibrated scientific applications. Paper review writers investigate every result before writing any conclusion in a paper or an essay. The experiment is carried out under a planned or controlled observation and the inference is strictly monitored. In contrast to the existing literature, emphasis is on the current findings of the experiment under operation. The previous literature is used to make a hypothesis, which can also be a prediction of a previously proposed theory.

2. How to work on the writing

The results are recorded, and the conclusions are drawn from the same. It is a frequent occurrence that science students are not adept for writing scientific papers. There are multiple services available online that help students with their assignments. The learners can easily avail the services online and get the paper written in accordance with their specific requirements. Services like Edusson or EssayServiceScanner are genuine ones. Students need to login and specify their requirements so that an expert assist with developing the paper. However, before paying a writer, it is better to go through the reviews about the platform. Ensure that it is not a scam. It is also recommended to review the article before you make any submissions.

3. Literature review

In contrast to the empirical study, it is essential to review different researches while working on the literature review. Such an article contains facts and information derived from other theoretical sources. New inferences may be drawn from the compilation of all literature pieces. However, the information and hypothesis are already existent. The literature review is a compilation of all the relevant data in order to produce coherent results. No experimental work may be conducted and the results come out of already existing theories in a systematic manner.

4. Deciding the type of article

College students often have to write different papers. A literature review is when a student attempts to prove or disprove a conceived point or an analysis through literature. By means of existing facts, he or she can challenge a hypothesis or attempt to answer a particular question. The article based on an experiment that was conducted solely to write the paper is an empirical review . The experiment is carried out by means of calibrated instruments in a controlled manner.

It is better to begin writing while conducting the experiment. Observations should be written down systematically during the research. This helps bring out coherence, which is easier to comprehend even later. Furthermore, beginning the writing process earlier provides one with enough time for the review and also produces better work. This is particularly necessary for the empirical papers as the experiments may take time to deliver the desired results. Leaving the writing behind and starting to write it when the deadline is approaching may make the process stressful enough and may lead to confusion. This also hampers with the work leading to decreased levels of quality. Hence, working in a timely way helps in developing the paper and your life on the whole.

For more information, visit OD Blog .

Opportunity Desk is the one stop place for all global opportunities. Connect with us on Facebook, Twitter and Instagram - @opportunitydesk

Related Posts

20 hot jobs for young professionals worldwide., digital classrooms: shaping the future of learning, social impact: making a difference while making a living.

Type above and press Enter to search. Press Esc to cancel.