critical and interpretive literature review

Which review is that? A guide to review types.

• Which review is that?
• Review Comparison Chart
• Decision Tool
• Critical Review
• Integrative Review
• Narrative Review
• State of the Art Review
• Narrative Summary
• Systematic Review
• Meta-analysis
• Comparative Effectiveness Review
• Diagnostic Systematic Review
• Network Meta-analysis
• Prognostic Review
• Psychometric Review
• Review of Economic Evaluations
• Systematic Review of Epidemiology Studies
• Living Systematic Reviews
• Umbrella Review
• Review of Reviews
• Rapid Review
• Rapid Evidence Assessment
• Rapid Realist Review
• Qualitative Evidence Synthesis
• Qualitative Interpretive Meta-synthesis
• Qualitative Meta-synthesis
• Qualitative Research Synthesis
• Framework Synthesis - Best-fit Framework Synthesis
• Meta-aggregation
• Meta-ethnography
• Meta-interpretation
• Meta-narrative Review
• Meta-summary
• Thematic Synthesis
• Mixed Methods Synthesis
• Narrative Synthesis
• Bayesian Meta-analysis
• EPPI-Centre Review

Critical Interpretive Synthesis

• Realist Synthesis - Realist Review
• Scoping Review
• Mapping Review
• Systematised Review
• Concept Synthesis
• Expert Opinion - Policy Review
• Technology Assessment Review
• Methodological Review
• Systematic Search and Review

"Critical interpretive synthesis involves an iterative approach to refining the research question and searching and selecting from the literature (using theoretical sampling) and defining and applying codes and categories. It also has a particular approach to appraising quality, using relevance – i.e. likely contribution to theory development – rather than methodological characteristics as a means of determining the 'quality' of individual papers" (Barnett-Page et al, 2009).

Further Reading/Resources

Chalmiers, M. A., Karaki, F., Muriki, M., Mody, S., Chen, A., & de Bocanegra, H. T. (2021). Refugee women's experiences with contraceptive care after resettlement in high-income countries: A Critical Interpretive Synthesis. Contraception . Full Text

References Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. BMC medical research methodology , 9 (1), 1-11. Full Text

• << Previous: EPPI-Centre Review
• Next: Realist Synthesis - Realist Review >>
• Last Updated: Mar 5, 2024 1:14 PM
• URL: https://unimelb.libguides.com/whichreview

• Technical advance
• Open access
• Published: 26 July 2006

Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups

• Mary Dixon-Woods 1 ,
• Debbie Cavers 2 ,
• Shona Agarwal 3 ,
• Ellen Annandale 4 ,
• Antony Arthur 5 ,
• Janet Harvey 6 ,
• Ron Hsu 1 ,
• Savita Katbamna 7 ,
• Richard Olsen 7 ,
• Lucy Smith 1 ,
• Richard Riley 1 &
• Alex J Sutton 1  

BMC Medical Research Methodology volume  6 , Article number:  35 ( 2006 ) Cite this article

105k Accesses

1057 Citations

49 Altmetric

Metrics details

Conventional systematic review techniques have limitations when the aim of a review is to construct a critical analysis of a complex body of literature. This article offers a reflexive account of an attempt to conduct an interpretive review of the literature on access to healthcare by vulnerable groups in the UK

This project involved the development and use of the method of Critical Interpretive Synthesis (CIS). This approach is sensitised to the processes of conventional systematic review methodology and draws on recent advances in methods for interpretive synthesis.

Many analyses of equity of access have rested on measures of utilisation of health services, but these are problematic both methodologically and conceptually. A more useful means of understanding access is offered by the synthetic construct of candidacy. Candidacy describes how people's eligibility for healthcare is determined between themselves and health services. It is a continually negotiated property of individuals, subject to multiple influences arising both from people and their social contexts and from macro-level influences on allocation of resources and configuration of services. Health services are continually constituting and seeking to define the appropriate objects of medical attention and intervention, while at the same time people are engaged in constituting and defining what they understand to be the appropriate objects of medical attention and intervention. Access represents a dynamic interplay between these simultaneous, iterative and mutually reinforcing processes. By attending to how vulnerabilities arise in relation to candidacy, the phenomenon of access can be better understood, and more appropriate recommendations made for policy, practice and future research.

By innovating with existing methods for interpretive synthesis, it was possible to produce not only new methods for conducting what we have termed critical interpretive synthesis, but also a new theoretical conceptualisation of access to healthcare. This theoretical account of access is distinct from models already extant in the literature, and is the result of combining diverse constructs and evidence into a coherent whole. Both the method and the model should be evaluated in other contexts.

Peer Review reports

Like many areas of healthcare practice and policy, the literature on access to healthcare is large, diverse, and complex. It includes empirical work using both qualitative and quantitative methods; editorial comment and theoretical work; case studies; evaluative, epidemiological, trial, descriptive, sociological, psychological, management, and economics papers, as well as policy documents and political statements. "Access" itself has not been consistently defined or operationalised across the field. There are substantial adjunct literatures, including those on quality in healthcare, priority-setting, and patient satisfaction. A review of the area would be of most benefit if it were to produce a "mid-range" theoretical account of the evidence and existing theory that is neither so abstract that it lacks empirical applicability nor so specific that its explanatory scope is limited.

In this paper, we suggest that conventional systematic review methodology is ill-suited to the challenges that conducting such a review would pose, and describe the development of a new form of review which we term "Critical Interpretive Synthesis" (CIS). This approach draws is sensitised to the range of issues involved in conducting reviews that conventional systematic review methodology has identified, but draws on a distinctive tradition of qualitative inquiry, including recent interpretive approaches to review [ 1 ]. We suggest that that using CIS to synthesise a diverse body of evidence enables the generation of theory with strong explanatory power. We illustrate this briefly using an example based on synthesis of the literature on access to healthcare in the UK by socio-economically disadvantaged people.

Aggregative and interpretive reviews

Conventional systematic review developed as a specific methodology for searching for, appraising, and synthesising findings of primary studies [ 2 ]. It offers a way of systematising, rationalising, and making more explicit the processes of review, and has demonstrated considerable benefits in synthesising certain forms of evidence where the aim is to test theories, perhaps especially about "what works". It is more limited when the aim, as here, is to include many different forms of evidence with the aim of generating theory [ 3 ]. Conventional systematic review methods are thus better suited to the production of aggregative rather than interpretive syntheses.

This distinction between aggregative and interpretive syntheses, noted by Noblit and Hare in their ground-breaking book on meta-ethnography[ 1 ] allows a useful (though necessarily crude) categorisation of two principal approaches to conducting reviews [ 4 ]. Aggregative reviews are concerned with assembling and pooling data, may use techniques such as meta-analysis, and require a basic comparability between phenomena so that the data can be aggregated for analysis. Their defining characteristics are a focus on summarising data , and an assumption that the concepts (or variables) under which those data are to be summarised are largely secure and well specified. Key concepts are defined at an early stage in the review and form the categories under which the data from empirical studies are to be summarised.

Interpretive reviews, by contrast, see the essential tasks of synthesis as involving both induction and interpretation. Their primary concern is with the development of concepts and theories that integrate those concepts. An interpretive review will therefore avoid specifying concepts in advance of the synthesis. The interpretive analysis that yields the synthesis is conceptual in process and output. The product of the synthesis is not aggregations of data, but theory grounded in the studies included in the review. Although there is a tendency at present to conduct interpretive synthesis only of qualitative studies, it should in principle be possible and indeed desirable to conduct interpretive syntheses of all forms of evidence, since theory-building need not be based only on one form of evidence. Indeed, Glaser and Strauss [ 5 ] in their seminal text, included an (often forgotten) chapter on the use of quantitative data for theory-building.

Recent years have seen the emergence of a range of methods that draw on a more interpretive tradition, but these also have limitations when attempting a synthesis of a large and complex body of evidence. In general, the use to date of interpretive approaches to synthesis has been confined to the synthesis of qualitative research only [ 6 – 8 ]. Meta-ethnography, an approach in which there has been recent significant activity and innovation, has similarly been used solely to synthesise qualitative studies, and has typically been used only with small samples [ 9 – 11 ]. Few approaches have attempted to apply an interpretive approach to the whole corpus of evidence (regardless of study type) included in a review, and few have treated the literature they examine as itself an object of scrutiny, for example by questioning the ways in which the literature constructs its problematics, the nature of the assumptions on the literature draw, or what has influenced proposed solutions.

In this paper we offer a reflexive account of our attempt to conduct an interpretive synthesis of all types of evidence relevant to access to National Health Service (NHS) healthcare in the UK by potentially vulnerable groups. These groups had been defined at the outset by the funders of the project (the UK Department of Health Service Delivery and Organisation R&D Programme) as children, older people, members of minority ethnicities, men/women, and socio-economically disadvantaged people. We explain in particular our development of Critical Interpretive Synthesis as a method for conducting this review.

Formulating the review question

Conventional systematic review methodology [ 12 , 13 ] emphasises the need for review questions to be precisely formulated. A tightly focused research question allows the parameters of the review to be identified and the study selection criteria to be defined in advance, and in turn limits the amount of evidence required to address the review question.

This strategy is successful where the phenomenon of interest, the populations, interventions, and outcomes are all well specified – i.e. if the aim of the review is aggregative . For our project, it was neither possible nor desirable to specify in advance the precise review question, a priori definitions, or categories under which the data could be summarised, since one of its aims was to allow the definition of the phenomenon of access to emerge from our analysis of the literature [ 14 ]. This is not to say that we did not have a review question, only that it was not a specific hypothesis. Instead it was, as Greenhalgh and colleagues [ 15 ] describe, tentative, fuzzy and contested at the outset of the project. It did include a focus on equity and on how access, particularly for potentially vulnerable groups, can best be understood in the NHS, a health care system that is, unlike most in the world, free at the point of use.

The approach we used to further specify the review question was highly iterative, modifying the question in response to search results and findings from retrieved items. It treated, as Eakin and Mykhalovskiy [ 16 ] suggest, the question as a compass rather than an anchor, and as something that would not finally be settled until the end of the review. In the process of refining the question, we benefited from the multidisciplinary nature of our review team: this allowed a range of perspectives to be incorporated into the process, something that was also helpful and important in other elements of the review.

Searching the literature

A defining characteristic of conventional systematic review methodology is its use of explicit searching strategies, and its requirement that reviewers be able to give a clear account of how they searched for relevant evidence, such that the search methods can be reproduced. [ 2 ] Searching normally involves a range of strategies, but relies heavily on electronic bibliographic databases.

We piloted the use of a highly structured search strategy using protocol-driven searches across a range of electronic databases but, like Greenhalgh and Peacock [ 17 ] found that was this unsatisfactory. In particular, it risked missing relevant materials by failing to pick up papers that, while not ostensibly about "access", were nonetheless important to the aim of the review. We then developed a more organic process that fitted better with the emergent and exploratory nature of the review questions. This combined a number of strategies, including searching of electronic databases; searching websites; reference chaining; and contacts with experts. Crucially, we also used expertise within the team to identify relevant literature from adjacent fields not immediately or obviously relevant to the question of "access".

However, searching generated thousands of potentially relevant items – at one stage over 100,000 records. A literature of this size would clearly be unmanageable, and well exceed the capacity of the review team. We therefore redefined the aim of the searching phase. Rather than aiming for comprehensive identification and inclusion of all relevant literature, as would be required under conventional systematic review methodology, we saw the purpose of the searching phase as identifying potentially relevant papers to provide a sampling frame. Our sampling frame eventually totalled approximately 1,200 records.

Conventional systematic review methodology limits the number of papers to be included in a review by having tightly specified inclusion criteria for papers. Effectively, this strategy constructs the field to be known as having specific boundaries, defined as research that has specifically addressed the review question, used particular study designs and fulfilled the procedural requirements for the proper execution of these. Interpretive reviews might construct the field to be known rather differently, seeing the boundaries as more diffuse and ill-defined, as potentially overlapping with other fields, and as shifting as the review progresses. Nonetheless, there is a need to limit the number of papers to be included in an interpretive synthesis not least for practical reasons, including the time available. Sampling is also warranted theoretically, in that the focus in interpretive synthesis is on the development of concepts and theory rather than on exhaustive summary of all data. A number of authors [ 18 – 20 ] suggest drawing on the sampling techniques of primary qualitative research, including principles of theoretical sampling and theoretical saturation, when conducting a synthesis of qualitative literature.

For purposes of our synthesis, we used purposive sampling initially to select papers that were clearly concerned with aspects of access to healthcare, partly informed by an earlier scoping study [ 21 ] and later used theoretical sampling to add, test and elaborate the emerging analysis. Sampling therefore involved a constant dialectic process conducted concurrently with theory generation.

Determination of quality

Conventional systematic review methodology uses assessment of study quality in a number of ways. First, as indicated above, studies included in a review may be limited to particular study designs, often using a "hierarchy of evidence" approach that sees some designs (e.g. randomized controlled trials) as being more robust than others (e.g. case-control studies). Second, it is usual to devise broad inclusion criteria – for example adequate randomisation for RCTs – and to exclude studies that fail to meet these. Third, an appraisal of included studies, perhaps using a structured quality checklist, may be undertaken to allow sensitivity analyses aimed at assessing the effects of weaker papers.

Using this approach when confronted with a complex literature, including qualitative research, poses several challenges. No hierarchy of study designs exists for qualitative research. How or whether to appraise papers for inclusion in an interpretive reviews has received a great deal of attention, but there is little sign of an emergent consensus [ 22 ]. Some argue that formal appraisals of quality may not be necessary, and some argue that there is a risk of discounting important studies for the sake of "surface mistakes" [ 23 ]. Others propose that weak papers should be excluded from the review altogether, and several published syntheses of qualitative research have indeed used quality criteria to make decisions about excluding papers. [ 10 , 24 ]

We aimed to prioritise papers that appeared to be relevant, rather than particular study types or papers that met particular methodological standards. We might therefore be said to be prioritising "signal" (likely relevance) over "noise" (the inverse of methodological quality) [ 25 ]. We felt it important, for purposes of an interpretive review, that a low threshold be applied to maximise the inclusion and contribution of a wide variety of papers at the level of concepts . We therefore took a two-pronged approach to quality. First, we decided that only papers that were deemed to be fatally flawed would be excluded. Second, once in the review, the synthesis itself crucially involved judgements and interpretations of credibility and contribution, as we discuss later.

To identify fatally flawed papers, we used the criteria in Table 1 , adapted from those proposed (at the time of our review) by the National Health Service (NHS) National Electronic Library for Health for the evaluation of qualitative research, to inform judgements on the quality of the papers. These criteria were used for assessing all empirical papers (but not those classified as 'reviews') regardless of study type. The final judgement about inclusion of the review rested both on an assessment of relevance as well as on the assessment of the quality of the individual papers. Decisions about relevance and quality were recorded, and a small sample of decisions about relevance and quality was reviewed. In the event, very few papers – approximately 20 – were excluded on grounds of being "fatally flawed", because even weak papers were often judged to have potentially high relevance. The value of deferring judgements of credibility and contribution until the synthesis became increasingly evident.

Most fundamentally, as the review progressed, we became increasingly convinced that the assumption that all studies deemed to have satisfactorily fulfilled criteria of execution and reporting can contribute equally to a synthesis is flawed. As we discuss further below, one of the distinctive characteristics of a critical interpretive synthesis is its emphasis not only on summary of data reported in the literature but also on a more fundamental critique, which may involve questioning taken-for-granted assumptions.

Data extraction

A data-extraction pro-forma was initially devised to assist in systematically identifying characteristics of research participants, methods of data collection, methods of data analysis and major findings of each paper. For both qualitative and quantitative papers, this involved extracting the titles of the categories and sub-categories using the terms used in the paper itself and a summary of the relevant material. Practically, however, it proved impossible to conduct this form of data extraction on all documents included in the review, including very large documents. We therefore summarised some documents more informally, for example using highlighter pen. More generally, the value of formal data extraction for purposes of this type of study will require further evaluation.

Conducting an interpretive synthesis

We had intended, at the outset of this project, to use meta-ethnography, a method for interpretive synthesis where there is currently an active programme of methodological research, [ 9 – 11 ] as our approach to synthesis. However, this had previously only been used to synthesise qualitative studies. Our experiences of working with a large sample of papers using multiple methods led us to refine and respecify some of the concepts and techniques of meta-ethnography in order to enable synthesis of a very large and methodologically diverse literature. Eventually we had made so many amendments and additions to the original methodology that we felt it was more appropriate, helpful and informative to deem it a new methodology with its own title and processes. It is this approach which we term critical interpretive synthesis (CIS). It is important to emphasise, however, that CIS is an approach to review and is not solely a method for synthesis.

Meta-ethnography, as originally proposed [ 1 ], involves three major strategies.:

1. Reciprocal translational analysis (RTA). The key metaphors, themes, or concepts in each study report are identified. An attempt is then made to translate the concepts into each other. Judgements about the ability of the concept of one study to capture concepts from others are based on attributes of the themes themselves, and the concept that is "most adequate" is chosen.

2. Refutational synthesis. Contradictions between the study reports are characterised, and an attempt made to explain them.

3. Lines-of-argument synthesis (LOA) involves building a general interpretation grounded in the findings of the separate studies. The themes or categories that are most powerful in representing the entire dataset are identified by constant comparisons between individual accounts.

Reciprocal translational analysis

Reciprocal translational analysis involves translating findings of one paper into another by systematically comparing findings from each study, using techniques such as maps. [ 9 ] We encountered considerable methodological and practical problems in trying to apply RTA across a large set of papers, in part because of the kinds of iterations we were conducting in refining the sample. These meant that there were difficulties in identifying a stable "set" of papers on which an RTA could be conducted. RTA appears to be most suitable for a well-defined, relatively small (fewer than 50) and complete set of papers, because substitution or deletion of papers causes problems with both identifying index concepts and showing which concepts from other papers translate into these. A further problem is that, when confronted with a very large and diverse literature such as ours, RTA tends to provide only a summary in terms that have already been used in the literature. Although this may be a useful strategy as a stage on the way to a more interpretive synthesis, its value may be more limited than is the case for smaller samples of qualitative study reports where its benefits have been more evident.

Before our review, RTA had previously only been used for synthesising interpretive research, not a large and diverse body of literature, so this may be one reason why it was unsuccessful for our purposes. It is important to distinguish between the doubtful value of RTA in our synthesis (particularly because of the size and diversity of the literature), and the doubtful use of RTA in general. The diversity of the literature would also have prevented us from undertaking an aggregative synthesis using meta-analysis, but this clearly could not be read as a criticism of meta-analysis itself, but of its limitations when applying it to a diverse literature.

Lines of argument synthesis

Recent work [ 9 – 11 ] has innovated in the methodology of lines-of-argument (LOA) synthesis originally proposed by Noblit and Hare by building on Schutz's [ 26 ] notions of "orders" of constructs Schutz used the idea of "first order construct" to refer to the everyday understandings of ordinary people and "second order construct" to refer to the constructs of the social sciences. The explanations and theories used by authors in primary study reports could therefore be seen as second order interpretations. This recent work uses LOA synthesis to develop what are referred to as "third order" interpretations, which build on the explanations and interpretations of the constituent studies, and are simultaneously consistent with the original results while extending beyond them. Our experiences have led us to respecify some of this approach.

We suggest that the appropriate way of conceptualising the output of an LOA synthesis is as a synthesising argument . This argument integrates evidence from across the studies in the review into a coherent theoretical framework comprising a network of constructs and the relationships between them. Its function is to provide more insightful, formalised, and generalisable ways of understanding a phenomenon. A synthesising argument can be generated through detailed analysis of the evidence included in a review, analogous to the analysis undertaken in primary qualitative research. It may require the generation of what we call synthetic constructs , which are the result of a transformation of the underlying evidence into a new conceptual form. Synthetic constructs are grounded in the evidence, but result from an interpretation of the whole of that evidence, and allow the possibility of several disparate aspects of a phenomenon being unified in a more useful and explanatory way.

What we have called a "synthetic construct" might also be seen as a "third order construct". We suggest that the term "synthetic construct" is a more useful term because it is more explicit, and also because we emphasise that a synthesising argument need not consist solely of synthetic constructs. Instead, synthesising arguments may explicitly link not only synthetic constructs, but also second order constructs already reported in the literature. In effect, therefore, our approach does not make this precise distinction between second and third order constructs.

Refutational syntheses

We further suggest that what Noblit and Hare [ 1 ] call "refutational syntheses" are best conducted as part of the analysis that produces the synthesising argument Few published meta-ethnographies have in fact reported a separate refutational synthesis. It is, we suggest, more productive instead to adopt a critical and reflexive approach to the literature, including consideration of contradictions and flaws in evidence and theory.

An important element of producing a synthesising argument is the need, when conducting the analysis, to consider and reflect on the credibility of the evidence, to make critical judgements about how it contributes to the development of the synthesising argument, and to root the synthesising argument appropriately in critique of existing evidence. Clearly, credibility depends on the quality of the research, its currency, and the robustness of its theoretical base. But more generally, a critical interpretive synthesis is critical in the broader sense of critique rather than this more limited sense of critical appraisal, in which each study is judged against the standards of its type. Critique may involve identification of the research traditions or meta-narratives that have guided particular fields of research [ 27 ] as well as critical analysis of particular forms of discourses. Its aim is therefore to treat the literature as warranting critical scrutiny in its own right.

Conducting the analysis

Our analysis of the evidence, in order to produce a synthesising argument, was similar to that undertaken in primary qualitative research. We began with detailed inspection of the papers, gradually identifying recurring themes and developing a critique. We then generated themes that helped to explain the phenomena being described in the literature, constantly comparing the theoretical structures we were developing against the data in the papers, and attempting to specify the categories of our analysis and the relationships between them. To facilitate the process of identifying patterns, themes, and categories across the large volumes of text-based data in our study, we used QSR N5 software. However, it is important to note that, as with any qualitative analysis, full transparency is not possible because of the creative, interpretive processes involved. Nonetheless, the large multidisciplinary team involved in the review, and the continual dialogue made necessary by this, helped to introduce "checks and balances" that guarded against framing of the analysis according to a single perspective.

A key feature of this process that distinguishes it from some other current approaches to interpretive synthesis (and indeed of much primary qualitative research) was its aim of being critical : its questioning of the ways in which the literature had constructed the problematics of access, the nature of the assumptions on which it drew, and what has influenced its choice of proposed solutions. Our critique of the literature was thus dynamic, recursive and reflexive, and, rather than being a stage in which individual papers are excluded or weighted, it formed a key part of the synthesis, informing the sampling and selection of material and playing a key role in theory generation.

Findings: access to healthcare by socio-economically disadvantaged people

Our critical interpretive synthesis of the literature on access to healthcare by socio-economically disadvantaged people in the UK included 119 papers. Early analytic categories were tentative and contingent, but gradually became firmed up and more highly specified as our analysis continued. Our synthesis involved a critique of the tendency to use measures of utilisation as a means of assessing the extent to which access to healthcare is equitable. It further involved the generation of a synthesising argument that has the synthetic construct of candidacy at its core. For space reasons, we can report here only a brief illustrative summary.

Critique of utilisation as a measure of access

Much of the evidence on whether access to healthcare in the UK is equitable has relied on measuring utilisation of health services. This approach measures the units of healthcare (consultations, procedures, etc) that people have actually consumed. The literature suggests that different groups have identifiable patterns of use of services, but the significance of these is often difficult to interpret. General practice (GP) consultation rates among socio-economically disadvantaged people have generally been found to be higher [ 28 , 29 ] though some recent work has suggested that social class variables are generally insignificant in explaining health service use [ 30 ] Studies that have attempted to adjust for need, usually on the basis of estimates of morbidity, have generally suggested that the apparent excess of GP consultation can be explained by higher need [ 31 ].

Our critique of the literature suggests that utilisation is a generally unhelpful measure of equity of access. Not only do the logistical and practical problems of conducting utilisation studies pose substantial threats to validity and reliability, these studies are problematic for other reasons. They rely on a largely untested set of normative (i.e. ideas about how the world ought to be) and somewhat questionable assumptions about the "correct" level of utilisation, and on a difficult-to-measure (or conceputalise) estimates of "need". They often invoke normative assumptions about need relative to some apparently privileged though often ill-defined reference group (such as "affluent" people), and therefore risk failing to identify problems in access for that reference group. Misleadingly reassuring results may be produced that indicate that "need" and use or receipt are proportionate. We argue that utilisation, or, more appropriately, receipt of healthcare is the outcome of many different complex processes, which all need to be recognised if access is to be properly understood.

Our analysis suggested that a focus instead on candidacy , a synthetic construct that we generated during the course of our analysis, would demonstrate the vulnerabilities associated with socio-economic disadvantage, emphasise the highly dynamic, multi-dimensional and contingent character of access, and allow a more insightful interpretation of the evidence on receipt of healthcare.

Our synthesising argument around access to healthcare by socio-economically disadvantaged people is organised around a set of central concepts and, in particular, the core synthetic category of "candidacy". Candidacy functions as a synthetic construct because it is the product of the transformation of the evidence into a new conceptual form. It is distinct from earlier uses of the term "candidacy", including its use in the lay epidemiology of heart disease [ 32 ].

We have defined candidacy as follows: candidacy describes the ways in which people's eligibility for medical attention and intervention is jointly negotiated between individuals and health services. Our synthesising argument runs as follows: candidacy is a dynamic and contingent process, constantly being defined and redefined through interactions between individuals and professionals, including how "cases" are constructed. Accomplishing access to healthcare requires considerable work on the part of users, and the amount, difficulty, and complexity of that work may operate as barriers to receipt of care. The social patterning of perceptions of health and health services, and a lack of alignment between the priorities and competencies of disadvantaged people and the organisation of health services, conspire to create vulnerabilities. Candidacy is managed in the context of operating conditions that are influenced by individuals, the setting and environment in which care takes place, situated activity, the dynamics of face-to-face activity, and aspects of self (such as gender), the typifications staff use in categorising people and diseases, availability of economic and other resources such as time, local pressures, and policy imperatives.

Identification of candidacy

How people recognise their symptoms as needing medical attention or intervention is clearly key to understanding how they assert a claim to candidacy. Our analysis suggests that people in more deprived circumstances are likely to manage health and to recognise candidacy as a series of crises. There is significant evidence of lower use of preventive services among more deprived groups, [ 33 , 34 ] as well as evidence of higher use of accident and emergency facilities, emergency admissions and out-of-hours use [ 35 , 35 , 37 , 38 ]. Among more deprived groups, there is a tendency to seek help in response to specific events that are seen as warranting candidacy. "Warning signs" may be downgraded in importance by socio-economically disadvantaged populations because of a lack of a positive conceptualisation of health, [ 39 , 40 ] the normalisation of symptoms within deprived communities [ 41 – 43 ], and fear of being "blamed" by health professionals [ 44 ].

Using services requires considerable work on the part of people. First, people must be aware of the services on offer, and there has been persistent concern that more deprived people may lack awareness of some services [ 45 , 46 ]. Second, using health services requires the mobilisation of a range of practical resources that may be variably available in the population. A key practical resource that impacts on the ability to seek care for the socio-economically disadvantaged, for example, is transport [ 44 , 47 , 48 ]. Other practical resources that may impact on the ability of disadvantaged groups to negotiate health services include more rigid patterns of working life [ 47 ]. Goddard and Smith [ 49 ] summarise evidence suggesting that those from more deprived social groups face financial costs of attending health services which, though not sufficient to dissuade them from using services when they are ill (i.e. in response to a specific "event"), act as a barrier to attending "optional" services related to health promotion and health prevention

The permeability of services

Patterns of use of health services reflect issues in the organisation of services as much as they reflect a tendency to manage health as a series of crises on the part of disadvantaged people. We generated the synthetic construct of "permeability" to refer to the ease with which people can use services. Porous services require few qualifications of candidacy to use them, and may require the mobilisation of relatively fewer resources. Such services might include Accident and Emergency departments. Services that are less permeable demand qualifications (such as a referral), and also demand a higher degree of cultural alignment between themselves and their users, particularly in respect of the extent to which people feel comfortable with the organisational values of the service. Such services might include out-patients clinics in hospitals.

Services that are less permeable tend to have high levels of default by socio-economically disadvantaged people [ 50 – 53 ]. Appointments systems, for example, are a threat to permeability by socio-economically disadvantaged people because they require resources and competencies (including stable addresses, being able to read, and being able to present in particular places at particular times [ 33 , 50 , 54 ] In addition, the extent to which people feel alienated from the cultural values of health services and their satisfaction with services have important implications for which services they choose to use [ 41 , 55 ].

Appearances at health services

Appearing at health services involves people in asserting a claim to candidacy for medical attention or intervention. Whatever the nature of the claim, making it clearly involves work that requires a set of competencies, including the ability to formulate and articulate the issue for which help is being sought, and the ability to present credibly. More deprived people are at risk in these situations: they may be less used to or less able to provide coherent abstracted explanations of need, and may feel intimidated by their social distance from health professionals. Sword [ 56 ] points out that people with low incomes may feel alienated by the power relations that often characterise encounters with professionals. Dixon et al [ 57 ] and, in the US, Cooper and Roter [ 58 ] suggest that middle class people may be more adept at using their "voice" to demand better and extensive services: they may be more articulate, more confident, and more persistent, while people from lower class backgrounds are less verbally active. Somerset et al [ 59 ] report that in making referral decisions, patients' social status and their ability to articulate verbally act as background (and unexpressed) influences that affect the likelihood of referral.

Adjudications

Once a patient has asserted their candidacy by presenting to health services, the professional judgements made about that candidacy strongly influence subsequent access to attention and interventions. We generated the synthetic construct of "adjudication" to refer to the judgements and decisions made by professionals which allow or inhibit continued progression of candidacy. May et al's [ 60 ] analysis suggests doctors' practices are often exercised through a repertoire of routine judgements about the possibilities presented by individual patients and the routinely available means of solving these. These typifications are, we suggest, strongly influenced by local conditions, including the operating conditions in which practitioners work and sensitivity to resource constraints. Candidacy of socially disadvantaged people appears to be at risk of being judged to be less eligible, at least for some types of interventions, although the evidence that this happens is not particularly strong.

Our analysis suggests that it is likely that professionals' perceptions of patients who are likely to "do well" as a result of interventions may disadvantage people in more deprived circumstances. As Hughes and Griffiths [ 61 ] identify, clinical decisions may rest on often implicit social criteria about which patients "ought" to receive care People in disadvantaged groups are more likely to smoke, to be overweight and to have co-morbidities, and professional perceptions of the cultural and health capital required to convert a unit of health provision into a given unit of health gain may function as barriers to healthcare [ 34 ]. In addition, perceptions of social "deservingness" may play a role [ 61 , 62 ]. Goddard and Smith [ 49 ] summarise evidence suggesting that independent of the severity of the disease, some GPs are more likely to refer the economically active and those with dependants. Clearly, there is potential for socially disadvantaged people to be disfavoured in such decisions.

Offers and resistance

Much of the work on utilisation of healthcare explicitly or implicitly assumes that non-utilisation is a direct reflection of non-offer. However, this type of normative analysis fails to acknowledge that people may choose to refuse offers. There is some evidence of patterns of resistance to offers. Referral implies that a GP has identified particular features of candidacy and is seeking to match those to a service that deals with that form of candidacy, but patients can resist being referred [ 42 , 63 ] and can resist offers of medication [ 64 , 65 ].

Operating conditions and the local production of candidacy

A small body of recent research has identified what might be called local influences on the production of candidacy, and in our analysis these are hugely important. These are the contingent and locally specific influences on interactions between practitioners and patients, which may be emergent over time through repeated encounters. Crucial to the local production of candidacy is the perceived or actual availability and suitability of resources to address that candidacy [ 60 , 63 ].

Demands from health policy-makers and managers for syntheses of evidence that are useful, rigorous and relevant are fuelling interest in the development of methods that can allow the integration of diverse types of evidence [ 66 ]. With the diversity of techniques for evidence synthesis now beginning to appear, those using existing, 'new' or evolving techniques need to produce critical reflexive accounts of their experiences of using the methods [ 3 ]. Our experience of conducting a review of access to healthcare, where there is a large, amorphous and complex body of literature, and a need to assemble the findings into a form that is useful in informing policy and that is empirically and theoretically grounded [ 67 ], has led us to propose a new method – Critical Interpretive Synthesis – which is sensitised to the kinds of processes involved in conventional systematic review while drawing on a distinctively qualitative tradition of inquiry.

Conventional systematic review methodology is well-suited to aggregative syntheses, where what is required is a summary of the findings of the literature under a set of categories which are largely pre-specified, secure, and well-defined. It has been important in drawing attention to the weaknesses of informal reviews, including perceived failures in their procedural specification and the possibility that the (thus) undisciplined reviewer might be chaotic or negligent in identifying the relevant evidence, or might construct idiosyncratic theories and marshall the evidence in support of these. It has thus revealed some of the pitfalls of informal literature review. Conventional systematic review methodology has demonstrated considerable benefits in synthesising certain forms of evidence where the aim is to test theories (in the form of hypotheses), perhaps especially about "what works". However, this approach is limited when the aim, confronted with a complex body of evidence, is to generate theory [ 15 , 27 ].

Current methods for conducting an interpretive synthesis of the literature, (such as meta-ethnography) are also limited, in part because application of many interpretive methods for synthesis has remained confined to studies reporting qualitative research. Realist synthesis [ 68 ], which does include diverse forms of evidence, is oriented towards theory evaluation, in particular by focusing on theories of change. Methods for including qualitative and quantitative evidence in systematic reviews developed by the EPPI Centre at the Institute of Education, London, have involved refinements and extensions of conventional systematic review methodology [ 6 – 8 ], and have limited their application of interpretive techniques to synthesis of qualitative evidence.

More generally, many current approaches fail to be sufficiently critical , in the sense of offering a critique. There is rarely an attempt to reconceptualise the phenomenon of interest, to provide a more sweeping critique of the ways in which the literature in the area have chosen to represent it, or to question the epistemological and normative assumptions of the literature. With notable exceptions such as the recent approach of meta-narrative analysis [ 15 ], critique of papers in current approaches to review tends to be limited to appraisal of the methodological specificities of the individual papers.

Conducting an interpretive review of the literature on access to healthcare by vulnerable groups in the UK therefore required methodological innovation that would be alert to the issues raised by systematic review methodology but also move beyond both its limitations and those of other current interpretive methods. The methods for review that we developed in this project (Table 2 ) built on conventional systematic review methodology in their sensitivity to the need for attentiveness to a range of methodological processes. Crucially, in doing so, we drew explicitly on traditions of qualitative research inquiry, and in particular on the principles of grounded theory [ 5 ].

In addition to its explicit orientation towards theory generation, perhaps what most distinguishes CIS from conventional systematic review methods is its rejection of a "stage" approach to review. Processes of question formulation, searching, selection, data extraction, critique and synthesis are characterised as iterative, interactive, dynamic and recursive rather than as fixed procedures to be accomplished in a pre-defined sequence. CIS recognises the need for flexibility in the conduct of review, and future work would need to assess how far formal methods of critical appraisal and data extraction will be essential elements of the method. Our experience suggests that while attention to scientific quality is required, more generally the emphasis should be on critique rather than critical appraisal, and an ongoing critical orientation to the material examined and to emerging theoretical ideas. Formal data extraction may also be an unnecessarily constraining and burdensome process.

CIS emphasises the need for theoretical categories to be generated from the available evidence and for those categories to be submitted to rigorous scrutiny as the review progresses. Further, it emphasises a need for constant reflexivity to inform the emerging theoretical notions, and guides the sampling of articles. Although CIS demands attention to flaws in study design, execution and reporting in our judgements of the quality of individual papers, its critical approach goes beyond standard approaches. Thus, in our review, some methodologically weak papers were important in terms of their theoretical contribution, or in terms of demonstrating the breadth of evidence considered in the construction of particular categories, or in terms of providing a more comprehensive summary of the evidence, while a single strong paper might be pivotal in the development of the synthesis. Hughes and Griffiths' paper on micro-rationing of healthcare [ 61 ], for example, was a key paper in helping to generate the construct of candidacy that later came to unify the themes of our analysis. The critical interpretation in our analysis focused on how a synthesising argument could be fashioned from the available evidence, given the quality of the evidence and the kinds of critiques that could be offered of the theory and assumptions that lay behind particular approaches. In treating the literature as an object of scrutiny in its own right, CIS problematises the literature in ways that are quite distinctive from most current approaches to literature reviewing.

Access to healthcare

The CIS approaches we adopted deferred final definition of the phenomenon of access and the appropriate ways of conceptualising it until our analysis was complete. Our critique of the current literature focused on the inadequacies of studies of utilisation as a guide to explaining inequities in health care. The conceptual model of access that we developed emphasises candidacy as the core organising construct, and recasts access as highly dynamic and contingent, and subject to constant negotiation.

In this conceptual model of access to healthcare, health services are continually constituting and seeking to define the appropriate objects of medical attention and intervention, while at the same time people are engaged in constituting and defining what they understand to be the appropriate objects of medical attention and intervention. Candidacy describes how people's eligibility for healthcare is determined between themselves and health services. Candidacy is a continually negotiated property of individuals, subject to multiple influences arising both from people and their social contexts and from macro-level influences on allocation of resources and configuration of services. "Access" represents a dynamic interplay between these simultaneous, iterative and mutually reinforcing processes. By attending to how vulnerabilities arise in relation to candidacy, the phenomenon of access can be much better understood, and more appropriate recommendations made for policy, practice and future research. Although our review focused on the UK, we suggest that the construct of candidacy is transferable, and has useful explanatory value in other contexts.

In addition to the core construct of candidacy, our analysis required the production of a number of other linked synthetic constructs – constructs generated through an attempt to summarise and integrate diverse concepts and data – including "adjudications" and "offers". It was also possible to link existing "second order" constructs, for example relating to help-seeking as the identification of candidacy by patients, into the synthesising argument, and making these work as synthesising constructs. We feel that this approach allows maximum benefit to be gained from previous analyses as well as the new synthesis.

Reflections on the method

Clearly, questions can be raised about the validity and credibility of the CIS analysis we have presented here. Conventional systematic review methodology sets great store by the reproducibility of its protocols and findings. It would certainly have been possible to produce an account of the evidence that was more reproducible. For example, we could have used the evidence to produce a thematic summary that stuck largely to the terms and concepts used in the evidence itself. However, we felt it important that we produced an interpretation of the evidence that could produce new insights and fresh ways of understanding the phenomenon of access, and that the "critical voice" of our interpretation was maintained throughout the analysis. Simply to have produced a thematic summary of what the literature was saying would have run the risk of accepting that the accounts offered in the evidence-base were the only valid way of understanding the phenomenon of access to healthcare by vulnerable groups. We therefore make no claim to reproducibility, but wish to address some possible concerns. First, it could be argued that a different team using the same set of papers would have produced a different theoretical model. However, the same would be true for qualitative researchers working with primary qualitative data, who accept that other possible interpretations might be given to, say, the same set of transcripts. Clearly, the production of a synthesizing argument, as an interpretive process, produces one privileged reading of the evidence, and, as the product of an authorial voice, it cannot be defended as an inherently reproducible process or product. We would suggest, however, that our analysis can be defended on the grounds that it is demonstrably grounded in the evidence; that it is plausible; that it offers insights that are consistent with the available evidence; and that it can generate testable hypotheses and empirically valuable questions for future research.

Second, subjecting a question to continual review and refinement, as we did, may make it more difficult for those conducting critical interpretive reviews to demonstrate, as required by conventional systematic review methodology, the "transparency", comprehensiveness, and reproducibility of search strategies. This dilemma between the "answerable" question and the "meaningful" question has received little attention, but it underpins key tensions between the two ends of the academic/pragmatic systematic review spectrum. On balance, faced with a large and amorphous body of evidence in an area such as access to healthcare, and given the aims of an interpretive synthesis, we feel that our decision not to limit the focus of the review at the outset, and our subsequent sampling strategies, were well justified. Our decision not to commit to a particular view of what access might be and how it should be assessed at the outset of the project was critical to our subsequent development of a more satisfactory understanding of access.

Third, it could be argued that we have synthesized too small a sample of the available papers, or that the processes used to select the papers are not transparent. We recognize that we have analyzed and synthesized only a fraction of all relevant papers in the area of access to healthcare by vulnerable groups. However, a common strategy in conventional systematic review is to limit the study types to be included; this strategy also might result in only a proportion of the potentially relevant literature being synthesised. While we have described our methods for sampling as purposive, it is possible that another team using the same approach could have come up with a different sample, because, particularly in the later stages of our review, our sampling was highly intuitive and guided by the emerging theory.

The final version of the conceptual model of access to healthcare that we eventually developed did not emerge until quite late in the review process, and much of the later sampling was directed at testing and purposively challenging the theory as we began to develop it. Again, such forms of searching and sampling do not lend themselves easily to reproducibility or indeed auditability. Testing whether the interpretations change in response to different findings will be an important focus for future research, which will also need to evaluate whether apparently disconfirming evidence is the result of methodological flaws or poses a genuine challenge to theory.

Conducting interpretive reviews in challenging areas where there is a large body of diverse evidence demands an approach that can draw on the strengths of conventional systematic review methodology and on the recent advances in methods for interpretive synthesis. We have termed the approach we developed to this review "critical interpretive synthesis". We believe that this methodology offers the potential for insight, vividness, illumination, and reconceptualisation of research questions, particularly in challenging areas such as access to healthcare, and look forward to further evaluations of its application.

Noblit G, Hare R: Meta-ethnography: synthesising qualitative studies. 1988, Newbury Park, CA: Sage

Chapter   Google Scholar  

Egger M, Davey Smith G, Phillips A: Meta-analysis: principles and procedures. BMJ. 1997, 315: 1533-1537.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B: How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative Research. 2006, 6: 27-44. 10.1177/1468794106058867.

Article   Google Scholar  

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005, 10: 45-53. 10.1258/1355819052801804.

Article   PubMed   Google Scholar  

Glaser B, Strauss A: The discovery of grounded theory: strategies for qualitative research. 1967, New York: Aldine de Gruyter

Google Scholar  

Shepherd J, Harden A, Rees R, Brunton G, Garcia J, Oliver S, Oakley A: Young people and healthy eating: a systematic review of research on barriers and facilitators. Health Educ Res. 2006, 21: 239-257. 10.1093/her/cyh060.

Article   CAS   PubMed   Google Scholar  

Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people's views: an example from public health research. J Epidemiol Community Health. 2004, 58: 794-800. 10.1136/jech.2003.014829.

Article   PubMed   PubMed Central   Google Scholar  

Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh J: Integrating qualitative research with trials in systematic reviews. BMJ. 2004, 328: 1010-1012. 10.1136/bmj.328.7446.1010.

Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R: Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005, 61: 133-155. 10.1016/j.socscimed.2004.11.063.

Campbell R, Pound P, Pope C, et al: Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003, 56: 671-684. 10.1016/S0277-9536(02)00064-3.

Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesise qualitative research: a worked example. J Health Serv Res Policy. 2002, 7: 209-215. 10.1258/135581902320432732.

NHS Centre for Reviews and Dissemination: Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews. 2001, York: NHS CRD

Alderson P, Green S, Higgins JP: Cochrane Reviewers' Handbook 4.2. 2004, The Cochrane Library, 2

Jensen L, Allen M: Meta-synthesis of qualitative findings. Qual Health Res. 1996, 6: 553-560.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R: Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005, 61: 417-430. 10.1016/j.socscimed.2004.12.001.

Eakin JM, Mykhalovskiy E: Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract. 2003, 9: 187-194. 10.1046/j.1365-2753.2003.00392.x.

Greenhalgh T, Peacock R: Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005, 331: 1064-1065. 10.1136/bmj.38636.593461.68.

Schreiber RCD, Stern P: Qualitative meta-analysis. In: Completing a qualitative project: details and dialogue. Edited by: Morse JM. 1997, Thousand Oaks, CA: Sage

Booth A: Cochrane or cock-eyed? How shoul we conduct systematic reviews of qualitative research?. Proceedings of the Qualitative Evidence-based Practice Conference, Taking a Critical Stance. 2001, Coventry University, May 14 2001

Paterson B, Thorne S, C C, Jillings C: Meta-study of qualitative health research. 2001, Thousand Oaks: Sage

Gulliford M, Hughes D, Figeroa-Munoz J, Hudson M, Connell P, Morgan M, Beech R, Gibson B, Arumugam C, Mohiddin A, Sedgwick J: Access to health care: Report of a scoping exercise for NHS Service Delivery and Organisation R&D (NCCSDO). National Co-ordinating Centre for NHS Service Delivery and Organisation Research. 2001

Dixon-Woods M, Shaw RL, Agarwal S, Smith JA: The problem of appraising qualitative research. Qual Saf Health Care. 2004, 13: 223-225. 10.1136/qshc.2003.008714.

Sandelowski M, Docherty S, Emden C: Focus on qualitative methods. Qualitative meta-synthesis: issues and techniques. Res Nurs Health. 1997, 20: 365-371. 10.1002/(SICI)1098-240X(199708)20:4<365::AID-NUR9>3.0.CO;2-E.

Paterson B, Thorne S, Dewis M: Adapting to and managing diabetes. Image. 1998, 30: 57-62.

CAS   Google Scholar  

Edwards A, Elwyn G, Hood K, Rollnick S: Judging the 'weighting of evidence' in systematic reviews: introducing rigour into the qualitative overview stage by assessing Signal and Noise. J Eval Clin Pract. 2000, 6: 177-184. 10.1046/j.1365-2753.2000.00212.x.

Schutz A: Collected Papers. 1962, The Hague: Martinus Nijhoff, 1:

Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R: A systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery. 2004, London: NHS SDO Programme

Carr-Hill R, Rice N, Roland M: Socioeconomic determinants of rates of consultation in general practice based on fourth national morbidity survey of general practices. BMJ. 1996, 312: 1008-1012.

Carlisle R, Johnstone S: The relationship between census-derived socio-economic variables and general practice consultation rates in three town centre practices. Br J Gen Pract. 1998, 48: 1675-1678.

CAS   PubMed   PubMed Central   Google Scholar  

Sutton M, Gravelle H, Leyland A, Morris S, Muirhead M, Noble M, Parkin D, Windmeijer F: Small area study of supply and demand determinants of healthcare resource use and estimation of relative needs for primary care trusts in England. English Department of Health/Advisory Committee on Resource Allocation. 2002

Wyke S, Hunt K, Walker J, Wilson P: Frequent attendance, socio-economic status and burden of ill health: an investigation in the West of Scotland. Eur J Gen Pract. 2003, 9: 48-

Davison C, Davey Smith G, Frankel S: Lay epidemiology and the prevention paradox: the implications of coronary candidacy for health education. Sociol Health Illn. 1991, 13: 1-19. 10.1111/1467-9566.ep11340301.

Martin R, Sterne J: Social and economic variation in general practice consultation rates amongst men aged 16–39. Health Statistics Quarterly. 2001, 09: 29-36.

Watt G: The inverse care law today. Lancet. 2002, 360: 252-254. 10.1016/S0140-6736(02)09466-7.

Majeed F, Cook D, Hilton S, Poloniecki J, Hagen A: Annual night visiting rates in 129 general practices in one family health services authority: Association with patient and general practice characteristics. Br J Gen Pract. 1995, 45: 531-535.

Reid F, Cook D, Majeed A: Explaining variation in hospital admission rates between general practices: cross sectional study. BMJ. 1999, 319: 98-103.

Blatchford O, Capewell S, Murray S, Blatchford M: Emergency medical admissions in Glasgow: general practices vary despite adjustment for age, sex, and deprivation. Br J Gen Pract. 1999, 49: 551-554.

Carlisle R, Groom L, Avery A, Boot D, Earwicker S: Relation of out-of-hours activity by general practice and accident and emergency services with deprivation in Nottingham: longitudinal survey. BMJ. 1998, 316: 520-523.

Cornwell J: Hard Earned Lives. 1984, London: Tavistock

Calnan M: Health and Illness: the lay perspective. Tavistock. 1987

Richards H, Reid M, Watt C: Socioeconomic variations in responses to chest pain: qualitative study. BMJ. 2002, 324: 1308-1310. 10.1136/bmj.324.7349.1308.

Gardner K, Chapple A: Barriers to referral in patients with angina: qualitative study. BMJ. 1999, 319: 418-421.

Hunt K, Emslie C, Watt G: Lay constructions of a family history of heart disease: potential for misunderstandings in the clinical encounter?. Lancet. 2001, 357: 1168-1171. 10.1016/S0140-6736(00)04334-8.

Tod A, Read C, Lacey A, Abbott J: Barriers to uptake of services for coronary heart disease: qualitative study. BMJ. 2001, 323: 214-217. 10.1136/bmj.323.7306.214.

Anderson E, Petersen S, Wailoo M: Health concerns and needs of traveller families. Health Visitor. 1997, 70: 148-150.

House of Commons Health Committee: Inequalities in access to maternity services. Eighth report of session 2002–2003. 2003, London: Stationery Office

Field K, Briggs D: Socio-economic and locational determinants of accessibility and utilization of primary health-care. Health Soc Care Community. 2001, 9: 294-308. 10.1046/j.0966-0410.2001.00303.x.

Shipman C, Payne F, Dale J, Jessopp L: Patient-perceived benefits of and barriers to using out-of-hours primary care centres. Fam Pract. 2001, 18: 149-155. 10.1093/fampra/18.2.149.

Goddard M, Smith P: Equity of access to health care. 1998, York: University of York

George A, Rubin G: Non-attendance in general practice: A systematic review and its implications for access to primary health care. Fam Pract. 2003, 20: 178-184. 10.1093/fampra/20.2.178.

McClure R, Newell S, Edwards S: Patient characteristics affecting attendance at general outpatient clinics. Arch Dis Child. 1996, 74: 121-125.

Macleod U, Ross S, Twelves C, George W, Gillis C, Watt G: Primary and secondary care management of women with early breast cancer from affluent and deprived areas: retrospective review of hospital and general practice records. BMJ. 2000, 320: 1442-1445. 10.1136/bmj.320.7247.1442.

Sharp D, Hamilton W: Non-attendance at general practices and outpatient clinics. BMJ. 2002, 323: 1081-1082. 10.1136/bmj.323.7321.1081.

van Cleemput P: Health care needs of travellers. Arch Dis Child. 2000, 82: 32-37. 10.1136/adc.82.1.32.

Rogers A, Hassell K, Nicolas G: Demanding patients? Analysing the use of primary care. 1999, Buckingham: Open University Press

Sword W: A socio-ecological approach to understanding barriers to prenatal care for women of low income. J Adv Nurs. 1999, 29: 1170-1177. 10.1046/j.1365-2648.1999.00986.x.

Dixon A, Le Grand J, Henderson J, Murray R, Poteliakhoff E: Is the NHS equitable? A review of the evidence. London. LSE Health and Social Care. 2003

Cooper L, Roter D: Patient-provider communication: the effect of race and ethnicity on process and outcomes of healthcare. Unequal treatment: confronting racial and ethnic disparities in health care. Edited by: Smedley BD, Stith AY, Nelson AR. 2003, Washington DC: National Academies Press

Somerset M, Faulkner A, Shaw A, Dunn L, Sharp D: Obstacles on the path to a primary-care led National Health Service: complexities of outpatient care. Soc Sci Med. 1999, 48: 213-225. 10.1016/S0277-9536(98)00338-4.

May C, Allison G, Chapple A, Chew-Graham C, Dixon C, Gask L, Graham R, Rogers A, Roland M: Framing the doctor-patient relationship in chronic illness: a comparative study of general practitioners' accounts. Sociol Health Illn. 2004, 26: 135-158. 10.1111/j.1467-9566.2004.00384.x.

Hughes D, Griffiths L: Ruling in and Ruling out: two approaches to the micro-rationing of health care. Soc Sci Med. 1997, 44: 589-599. 10.1016/S0277-9536(96)00207-9.

Dodier N, Camus A: Openness and specialisation: dealing with patients in a hospital emergency service. Sociol Health Illn. 1998, 20: 413-444. 10.1111/1467-9566.00109.

Fuat A, Hungin A, Murphy J: Barriers to accurate diagnosis and effective management of heart failure in primary care: qualitative study. BMJ. 2003, 326: 196-200. 10.1136/bmj.326.7382.196.

Britten N, Stevenson F, Gafaranga J, Barry C, Bradley C: The expression of aversion to medicines in general practice consultations. Soc Sci Med. 2004, 59: 1495-1503. 10.1016/j.socscimed.2004.01.019.

Scahfheutle E, Hassell K, Noyce P, Weiss M: Access to medicines: cost as an influence on the views and behaviour of patients. Health Soc Care Community. 2002, 10: 187-195. 10.1046/j.1365-2524.2002.00356.x.

Sheldon TA: Making evidence synthesis more useful for management and policy-making. J Health Serv Res Policy. 2005, 10 (Suppl 1): 1-5. 10.1258/1355819054308521.

Dixon-Woods M, Fitzpatrick R, Roberts K: Including qualitative research in systematic reviews: problems and opportunities. J Eval Clin Pract. 2001, 7: 125-133. 10.1046/j.1365-2753.2001.00257.x.

Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005, 10 (Suppl 1): 21-34. 10.1258/1355819054308530.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/6/35/prepub

Download references

Acknowledgements

This project was funded by the NHS Service Delivery and Organisation (SDO) Research Programme.

Author information

Authors and affiliations.

Department of Health Sciences, University of Leicester, 22–28 Princess Road West, Leicester, LE1 6TP, UK

Mary Dixon-Woods, Ron Hsu, Lucy Smith, Richard Riley & Alex J Sutton

Division of Oncology/General Practice, University of Edinburgh, Edinburgh Centre for Neuro-Oncology, Western General Hospital, Crewe Road South, Edinburgh, EH4 2XU, UK

Debbie Cavers

Department of Health Sciences, University of Leicester, Leicester General Hospital, Leicester, LE5 4PW, UK

Shona Agarwal

Department of Sociology, University of Leicester, Leicester, LE1 7RH, UK

Ellen Annandale

School of Nursing, University of Nottingham, Queens Medical Centre, Nottingham, NG7 2HA, UK

Antony Arthur

Centre for Research in Social Policy, Loughborough University, Leicestershire, LE11 3TU, UK

Janet Harvey

Nuffield Research Unit, Department of Health Sciences, University of Leicester, 22–28 Princess Road West, Leicester, LE1 6TP, UK

Savita Katbamna & Richard Olsen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mary Dixon-Woods .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors' contributions

MDW designed the project, led and supervised its execution, and drafted the manuscript. EA, AA, JH, RH, SK, RO, LS, RR and AJS participated in the design of the study. All authors engaged in searching, screening, sampling, data extraction, and critical appraisal/critique activities, and contributed to the thematic analysis. DC and SA managed the searching, maintained the databases and coded material using N6 software. All authors contributed to the draft of the manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Dixon-Woods, M., Cavers, D., Agarwal, S. et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol 6 , 35 (2006). https://doi.org/10.1186/1471-2288-6-35

Download citation

Received : 19 April 2006

Accepted : 26 July 2006

Published : 26 July 2006

DOI : https://doi.org/10.1186/1471-2288-6-35

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• National Health Service
• Review Question
• Disadvantaged People
• Order Construct
• Diverse Literature

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Reviewing Literature in Bioethics Research: Increasing Rigour in Non-Systematic Reviews

• PMID: 25655982
• DOI: 10.1111/bioe.12149

The recent interest in systematic review methods in bioethics has highlighted the need for greater transparency in all literature review processes undertaken in bioethics projects. In this article, I articulate features of a good bioethics literature review that does not aim to be systematic, but rather to capture and analyse the key ideas relevant to a research question. I call this a critical interpretive literature review. I begin by sketching and comparing three different types of literature review conducted in bioethics scholarship. Then, drawing on Dixon-Wood's concept of critical interpretive synthesis, I put forward six features of a good critical interpretive literature review in bioethics: answering a research question, capturing the key ideas relevant to the research question, analysing the literature as a whole, generating theory, not excluding papers based on rigid quality assessment criteria, and reporting the search strategy.

Keywords: critical interpretive synthesis; literature review; methodology; systematic reviews.

© 2015 John Wiley & Sons Ltd.

Publication types

• Research Support, Non-U.S. Gov't
• Evidence-Based Medicine
• Research Design*
• Research Report / standards*
• Review Literature as Topic*

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• AIMS Public Health
• v.3(1); 2016

What Synthesis Methodology Should I Use? A Review and Analysis of Approaches to Research Synthesis

Kara schick-makaroff.

1 Faculty of Nursing, University of Alberta, Edmonton, AB, Canada

Marjorie MacDonald

2 School of Nursing, University of Victoria, Victoria, BC, Canada

Marilyn Plummer

3 College of Nursing, Camosun College, Victoria, BC, Canada

Judy Burgess

4 Student Services, University Health Services, Victoria, BC, Canada

Wendy Neander

Associated data, additional file 1.

When we began this process, we were doctoral students and a faculty member in a research methods course. As students, we were facing a review of the literature for our dissertations. We encountered several different ways of conducting a review but were unable to locate any resources that synthesized all of the various synthesis methodologies. Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use ‘research synthesis’ as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.

We conducted an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis. We searched five databases, reviewed websites of key organizations, hand-searched several journals, and examined relevant texts from the reference lists of the documents we had already obtained.

We identified four broad categories of research synthesis methodology including conventional, quantitative, qualitative, and emerging syntheses. Each of the broad categories was compared to the others on the following: key characteristics, purpose, method, product, context, underlying assumptions, unit of analysis, strengths and limitations, and when to use each approach.

Conclusions

The current state of research synthesis reflects significant advancements in emerging synthesis studies that integrate diverse data types and sources. New approaches to research synthesis provide a much broader range of review alternatives available to health and social science students and researchers.

1. Introduction

Since the turn of the century, public health emergencies have been identified worldwide, particularly related to infectious diseases. For example, the Severe Acute Respiratory Syndrome (SARS) epidemic in Canada in 2002-2003, the recent Ebola epidemic in Africa, and the ongoing HIV/AIDs pandemic are global health concerns. There have also been dramatic increases in the prevalence of chronic diseases around the world [1] – [3] . These epidemiological challenges have raised concerns about the ability of health systems worldwide to address these crises. As a result, public health systems reform has been initiated in a number of countries. In Canada, as in other countries, the role of evidence to support public health reform and improve population health has been given high priority. Yet, there continues to be a significant gap between the production of evidence through research and its application in practice [4] – [5] . One strategy to address this gap has been the development of new research synthesis methodologies to deal with the time-sensitive and wide ranging evidence needs of policy makers and practitioners in all areas of health care, including public health.

As doctoral nursing students facing a review of the literature for our dissertations, and as a faculty member teaching a research methods course, we encountered several ways of conducting a research synthesis but found no comprehensive resources that discussed, compared, and contrasted various synthesis methodologies on their purposes, processes, strengths and limitations. To complicate matters, writers use terms interchangeably or use different terms to mean the same thing, and the literature is often contradictory about various approaches. Some texts [6] , [7] – [9] did provide a preliminary understanding about how research synthesis had been taken up in nursing, but these did not meet our requirements. Thus, in this article we address the need for a comprehensive overview of research synthesis methodologies to guide public health, health care, and social science researchers and practitioners.

Research synthesis is relatively new in public health but has a long history in other fields dating back to the late 1800s. Research synthesis, a research process in its own right [10] , has become more prominent in the wake of the evidence-based movement of the 1990s. Research syntheses have found their advocates and detractors in all disciplines, with challenges to the processes of systematic review and meta-analysis, in particular, being raised by critics of evidence-based healthcare [11] – [13] .

Our purpose was to conduct an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis [14] – [15] . We synthesize and critique the main approaches to research synthesis that are relevant for public health, health care, and social scientists. Research synthesis is the overarching term we use to describe approaches to combining, aggregating, integrating, and synthesizing primary research findings. Each synthesis methodology draws on different types of findings depending on the purpose and product of the chosen synthesis (see Additional File 1 ).

3. Method of Review

Based on our current knowledge of the literature, we identified these approaches to include in our review: systematic review, meta-analysis, qualitative meta-synthesis, meta-narrative synthesis, scoping review, rapid review, realist synthesis, concept analysis, literature review, and integrative review. Our first step was to divide the synthesis types among the research team. Each member did a preliminary search to identify key texts. The team then met to develop search terms and a framework to guide the review.

Over the period of 2008 to 2012 we extensively searched the literature, updating our search at several time points, not restricting our search by date. The dates of texts reviewed range from 1967 to 2015. We used the terms above combined with the term “method* (e.g., “realist synthesis” and “method*) in the database Health Source: Academic Edition (includes Medline and CINAHL). This search yielded very few texts on some methodologies and many on others. We realized that many documents on research synthesis had not been picked up in the search. Therefore, we also searched Google Scholar, PubMed, ERIC, and Social Science Index, as well as the websites of key organizations such as the Joanna Briggs Institute, the University of York Centre for Evidence-Based Nursing, and the Cochrane Collaboration database. We hand searched several nursing, social science, public health and health policy journals. Finally, we traced relevant documents from the references in obtained texts.

We included works that met the following inclusion criteria: (1) published in English; (2) discussed the history of research synthesis; (3) explicitly described the approach and specific methods; or (4) identified issues, challenges, strengths and limitations of the particular methodology. We excluded research reports that resulted from the use of particular synthesis methodologies unless they also included criteria 2, 3, or 4 above.

Based on our search, we identified additional types of research synthesis (e.g., meta-interpretation, best evidence synthesis, critical interpretive synthesis, meta-summary, grounded formal theory). Still, we missed some important developments in meta-analysis, for example, identified by the journal's reviewers that have now been discussed briefly in the paper. The final set of 197 texts included in our review comprised theoretical, empirical, and conceptual papers, books, editorials and commentaries, and policy documents.

In our preliminary review of key texts, the team inductively developed a framework of the important elements of each method for comparison. In the next phase, each text was read carefully, and data for these elements were extracted into a table for comparison on the points of: key characteristics, purpose, methods, and product; see Additional File 1 ). Once the data were grouped and extracted, we synthesized across categories based on the following additional points of comparison: complexity of the process, degree of systematization, consideration of context, underlying assumptions, unit of analysis, and when to use each approach. In our results, we discuss our comparison of the various synthesis approaches on the elements above. Drawing only on documents for the review, ethics approval was not required.

We identified four broad categories of research synthesis methodology: Conventional, quantitative, qualitative, and emerging syntheses. From our dataset of 197 texts, we had 14 texts on conventional synthesis, 64 on quantitative synthesis, 78 on qualitative synthesis, and 41 on emerging syntheses. Table 1 provides an overview of the four types of research synthesis, definitions, types of data used, products, and examples of the methodology.

Although we group these types of synthesis into four broad categories on the basis of similarities, each type within a category has unique characteristics, which may differ from the overall group similarities. Each could be explored in greater depth to tease out their unique characteristics, but detailed comparison is beyond the scope of this article.

Additional File 1 presents one or more selected types of synthesis that represent the broad category but is not an exhaustive presentation of all types within each category. It provides more depth for specific examples from each category of synthesis on the characteristics, purpose, methods, and products than is found in Table 1 .

4.1. Key Characteristics

4.1.1. what is it.

Here we draw on two types of categorization. First, we utilize Dixon Woods et al.'s [49] classification of research syntheses as being either integrative or interpretive . (Please note that integrative syntheses are not the same as an integrative review as defined in Additional File 1 .) Second, we use Popay's [80] enhancement and epistemological models .

The defining characteristics of integrative syntheses are that they involve summarizing the data achieved by pooling data [49] . Integrative syntheses include systematic reviews, meta-analyses, as well as scoping and rapid reviews because each of these focus on summarizing data. They also define concepts from the outset (although this may not always be true in scoping or rapid reviews) and deal with a well-specified phenomenon of interest.

Interpretive syntheses are primarily concerned with the development of concepts and theories that integrate concepts [49] . The analysis in interpretive synthesis is conceptual both in process and outcome, and “the product is not aggregations of data, but theory” [49] , [p.12]. Interpretive syntheses involve induction and interpretation, and are primarily conceptual in process and outcome. Examples include integrative reviews, some systematic reviews, all of the qualitative syntheses, meta-narrative, realist and critical interpretive syntheses. Of note, both quantitative and qualitative studies can be either integrative or interpretive

The second categorization, enhancement versus epistemological , applies to those approaches that use multiple data types and sources [80] . Popay's [80] classification reflects the ways that qualitative data are valued in relation to quantitative data.

In the enhancement model , qualitative data adds something to quantitative analysis. The enhancement model is reflected in systematic reviews and meta-analyses that use some qualitative data to enhance interpretation and explanation. It may also be reflected in some rapid reviews that draw on quantitative data but use some qualitative data.

The epistemological model assumes that quantitative and qualitative data are equal and each has something unique to contribute. All of the other review approaches, except pure quantitative or qualitative syntheses, reflect the epistemological model because they value all data types equally but see them as contributing different understandings.

4.1.2. Data type

By and large, the quantitative approaches (quantitative systematic review and meta-analysis) have typically used purely quantitative data (i.e., expressed in numeric form). More recently, both Cochrane [81] and Campbell [82] collaborations are grappling with the need to, and the process of, integrating qualitative research into a systematic review. The qualitative approaches use qualitative data (i.e., expressed in words). All of the emerging synthesis types, as well as the conventional integrative review, incorporate qualitative and quantitative study designs and data.

4.1.3. Research question

Four types of research questions direct inquiry across the different types of syntheses. The first is a well-developed research question that gives direction to the synthesis (e.g., meta-analysis, systematic review, meta-study, concept analysis, rapid review, realist synthesis). The second begins as a broad general question that evolves and becomes more refined over the course of the synthesis (e.g., meta-ethnography, scoping review, meta-narrative, critical interpretive synthesis). In the third type, the synthesis begins with a phenomenon of interest and the question emerges in the analytic process (e.g., grounded formal theory). Lastly, there is no clear question, but rather a general review purpose (e.g., integrative review). Thus, the requirement for a well-defined question cuts across at least three of the synthesis types (e.g., quantitative, qualitative, and emerging).

4.1.4. Quality appraisal

This is a contested issue within and between the four synthesis categories. There are strong proponents of quality appraisal in the quantitative traditions of systematic review and meta-analysis based on the need for strong studies that will not jeopardize validity of the overall findings. Nonetheless, there is no consensus on pre-defined criteria; many scales exist that vary dramatically in composition. This has methodological implications for the credibility of findings [83] .

Specific methodologies from the conventional, qualitative, and emerging categories support quality appraisal but do so with caveats. In conventional integrative reviews appraisal is recommended, but depends on the sampling frame used in the study [18] . In meta-study, appraisal criteria are explicit but quality criteria are used in different ways depending on the specific requirements of the inquiry [54] . Among the emerging syntheses, meta-narrative review developers support appraisal of a study based on criteria from the research tradition of the primary study [67] , [84] – [85] . Realist synthesis similarly supports the use of high quality evidence, but appraisal checklists are viewed with scepticism and evidence is judged based on relevance to the research question and whether a credible inference may be drawn [69] . Like realist, critical interpretive syntheses do not judge quality using standardized appraisal instruments. They will exclude fatally flawed studies, but there is no consensus on what ‘fatally flawed’ means [49] , [71] . Appraisal is based on relevance to the inquiry, not rigor of the study.

There is no agreement on quality appraisal among qualitative meta-ethnographers with some supporting and others refuting the need for appraisal. [60] , [62] . Opponents of quality appraisal are found among authors of qualitative (grounded formal theory and concept analysis) and emerging syntheses (scoping and rapid reviews) because quality is not deemed relevant to the intention of the synthesis; the studies being reviewed are not effectiveness studies where quality is extremely important. These qualitative synthesis are often reviews of theoretical developments where the concept itself is what is important, or reviews that provide quotations from the raw data so readers can make their own judgements about the relevance and utility of the data. For example, in formal grounded theory, the purpose of theory generation and authenticity of data used to generate the theory is not as important as the conceptual category. Inaccuracies may be corrected in other ways, such as using the constant comparative method, which facilitates development of theoretical concepts that are repeatedly found in the data [86] – [87] . For pragmatic reasons, evidence is not assessed in rapid and scoping reviews, in part to produce a timely product. The issue of quality appraisal is unresolved across the terrain of research synthesis and we consider this further in our discussion.

4.2. Purpose

All research syntheses share a common purpose -- to summarize, synthesize, or integrate research findings from diverse studies. This helps readers stay abreast of the burgeoning literature in a field. Our discussion here is at the level of the four categories of synthesis. Beginning with conventional literature syntheses, the overall purpose is to attend to mature topics for the purpose of re-conceptualization or to new topics requiring preliminary conceptualization [14] . Such syntheses may be helpful to consider contradictory evidence, map shifting trends in the study of a phenomenon, and describe the emergence of research in diverse fields [14] . The purpose here is to set the stage for a study by identifying what has been done, gaps in the literature, important research questions, or to develop a conceptual framework to guide data collection and analysis.

The purpose of quantitative systematic reviews is to combine, aggregate, or integrate empirical research to be able to generalize from a group of studies and determine the limits of generalization [27] . The focus of quantitative systematic reviews has been primarily on aggregating the results of studies evaluating the effectiveness of interventions using experimental, quasi-experimental, and more recently, observational designs. Systematic reviews can be done with or without quantitative meta-analysis but a meta-analysis always takes place within the context of a systematic review. Researchers must consider the review's purpose and the nature of their data in undertaking a quantitative synthesis; this will assist in determining the approach.

The purpose of qualitative syntheses is broadly to synthesize complex health experiences, practices, or concepts arising in healthcare environments. There may be various purposes depending on the qualitative methodology. For example, in hermeneutic studies the aim may be holistic explanation or understanding of a phenomenon [42] , which is deepened by integrating the findings from multiple studies. In grounded formal theory, the aim is to produce a conceptual framework or theory expected to be applicable beyond the original study. Although not able to generalize from qualitative research in the statistical sense [88] , qualitative researchers usually do want to say something about the applicability of their synthesis to other settings or phenomena. This notion of ‘theoretical generalization’ has been referred to as ‘transferability’ [89] – [90] and is an important criterion of rigour in qualitative research. It applies equally to the products of a qualitative synthesis in which the synthesis of multiple studies on the same phenomenon strengthens the ability to draw transferable conclusions.

The overarching purpose of emerging syntheses is challenging the more traditional types of syntheses, in part by using data from both quantitative and qualitative studies with diverse designs for analysis. Beyond this, however, each emerging synthesis methodology has a unique purpose. In meta-narrative review, the purpose is to identify different research traditions in the area, synthesize a complex and diverse body of research. Critical interpretive synthesis shares this characteristic. Although a distinctive approach, critical interpretive synthesis utilizes a modification of the analytic strategies of meta-ethnography [61] (e.g., reciprocal translational analysis, refutational synthesis, and lines of argument synthesis) but goes beyond the use of these to bring a critical perspective to bear in challenging the normative or epistemological assumptions in the primary literature [72] – [73] . The unique purpose of a realist synthesis is to amalgamate complex empirical evidence and theoretical understandings within a diverse body of literature to uncover the operative mechanisms and contexts that affect the outcomes of social interventions. In a scoping review, the intention is to find key concepts, examine the range of research in an area, and identify gaps in the literature. The purpose of a rapid review is comparable to that of a scoping review, but done quickly to meet the time-sensitive information needs of policy makers.

4.3. Method

4.3.1. degree of systematization.

There are varying degrees of systematization across the categories of research synthesis. The most systematized are quantitative systematic reviews and meta-analyses. There are clear processes in each with judgments to be made at each step, although there are no agreed upon guidelines for this. The process is inherently subjective despite attempts to develop objective and systematic processes [91] – [92] . Mullen and Ramirez [27] suggest that there is often a false sense of rigour implied by the terms ‘systematic review’ and ‘meta-analysis’ because of their clearly defined procedures.

In comparison with some types of qualitative synthesis, concept analysis is quite procedural. Qualitative meta-synthesis also has defined procedures and is systematic, yet perhaps less so than concept analysis. Qualitative meta-synthesis starts in an unsystematic way but becomes more systematic as it unfolds. Procedures and frameworks exist for some of the emerging types of synthesis [e.g., [50] , [63] , [71] , [93] ] but are not linear, have considerable flexibility, and are often messy with emergent processes [85] . Conventional literature reviews tend not to be as systematic as the other three types. In fact, the lack of systematization in conventional literature synthesis was the reason for the development of more systematic quantitative [17] , [20] and qualitative [45] – [46] , [61] approaches. Some authors in the field [18] have clarified processes for integrative reviews making them more systematic and rigorous, but most conventional syntheses remain relatively unsystematic in comparison with other types.

4.3.2. Complexity of the process

Some synthesis processes are considerably more complex than others. Methodologies with clearly defined steps are arguably less complex than the more flexible and emergent ones. We know that any study encounters challenges and it is rare that a pre-determined research protocol can be followed exactly as intended. Not even the rigorous methods associated with Cochrane [81] systematic reviews and meta-analyses are always implemented exactly as intended. Even when dealing with numbers rather than words, interpretation is always part of the process. Our collective experience suggests that new methodologies (e.g., meta-narrative synthesis and realist synthesis) that integrate different data types and methods are more complex than conventional reviews or the rapid and scoping reviews.

4.4. Product

The products of research syntheses usually take three distinct formats (see Table 1 and Additional File 1 for further details). The first representation is in tables, charts, graphical displays, diagrams and maps as seen in integrative, scoping and rapid reviews, meta-analyses, and critical interpretive syntheses. The second type of synthesis product is the use of mathematical scores. Summary statements of effectiveness are mathematically displayed in meta-analyses (as an effect size), systematic reviews, and rapid reviews (statistical significance).

The third synthesis product may be a theory or theoretical framework. A mid-range theory can be produced from formal grounded theory, meta-study, meta-ethnography, and realist synthesis. Theoretical/conceptual frameworks or conceptual maps may be created in meta-narrative and critical interpretive syntheses, and integrative reviews. Concepts for use within theories are produced in concept analysis. While these three product types span the categories of research synthesis, narrative description and summary is used to present the products resulting from all methodologies.

4.5. Consideration of context

There are diverse ways that context is considered in the four broad categories of synthesis. Context may be considered to the extent that it features within primary studies for the purpose of the review. Context may also be understood as an integral aspect of both the phenomenon under study and the synthesis methodology (e.g., realist synthesis). Quantitative systematic reviews and meta-analyses have typically been conducted on studies using experimental and quasi-experimental designs and more recently observational studies, which control for contextual features to allow for understanding of the ‘true’ effect of the intervention [94] .

More recently, systematic reviews have included covariates or mediating variables (i.e., contextual factors) to help explain variability in the results across studies [27] . Context, however, is usually handled in the narrative discussion of findings rather than in the synthesis itself. This lack of attention to context has been one criticism leveled against systematic reviews and meta-analyses, which restrict the types of research designs that are considered [e.g., [95] ].

When conventional literature reviews incorporate studies that deal with context, there is a place for considering contextual influences on the intervention or phenomenon. Reviews of quantitative experimental studies tend to be devoid of contextual considerations since the original studies are similarly devoid, but context might figure prominently in a literature review that incorporates both quantitative and qualitative studies.

Qualitative syntheses have been conducted on the contextual features of a particular phenomenon [33] . Paterson et al. [54] advise researchers to attend to how context may have influenced the findings of particular primary studies. In qualitative analysis, contextual features may form categories by which the data can be compared and contrasted to facilitate interpretation. Because qualitative research is often conducted to understand a phenomenon as a whole, context may be a focus, although this varies with the qualitative methodology. At the same time, the findings in a qualitative synthesis are abstracted from the original reports and taken to a higher level of conceptualization, thus removing them from the original context.

Meta-narrative synthesis [67] , [84] , because it draws on diverse research traditions and methodologies, may incorporate context into the analysis and findings. There is not, however, an explicit step in the process that directs the analyst to consider context. Generally, the research question guiding the synthesis is an important factor in whether context will be a focus.

More recent iterations of concept analysis [47] , [96] – [97] explicitly consider context reflecting the assumption that a concept's meaning is determined by its context. Morse [47] points out, however, that Wilson's [98] approach to concept analysis, and those based on Wilson [e.g., [45] ], identify attributes that are devoid of context, while Rodgers' [96] , [99] evolutionary method considers context (e.g., antecedents, consequences, and relationships to other concepts) in concept development.

Realist synthesis [69] considers context as integral to the study. It draws on a critical realist logic of inquiry grounded in the work of Bhaskar [100] , who argues that empirical co-occurrence of events is insufficient for inferring causation. One must identify generative mechanisms whose properties are causal and, depending on the situation, may nor may not be activated [94] . Context interacts with program/intervention elements and thus cannot be differentiated from the phenomenon [69] . This approach synthesizes evidence on generative mechanisms and analyzes contextual features that activate them; the result feeds back into the context. The focus is on what works, for whom, under what conditions, why and how [68] .

4.6. Underlying Philosophical and Theoretical Assumptions

When we began our review, we ‘assumed’ that the assumptions underlying synthesis methodologies would be a distinguishing characteristic of synthesis types, and that we could compare the various types on their assumptions, explicit or implicit. We found, however, that many authors did not explicate the underlying assumptions of their methodologies, and it was difficult to infer them. Kirkevold [101] has argued that integrative reviews need to be carried out from an explicit philosophical or theoretical perspective. We argue this should be true for all types of synthesis.

Authors of some emerging synthesis approaches have been very explicit about their assumptions and philosophical underpinnings. An implicit assumption of most emerging synthesis methodologies is that quantitative systematic reviews and meta-analyses have limited utility in some fields [e.g., in public health – [13] , [102] ] and for some kinds of review questions like those about feasibility and appropriateness versus effectiveness [103] – [104] . They also assume that ontologically and epistemologically, both kinds of data can be combined. This is a significant debate in the literature because it is about the commensurability of overarching paradigms [105] but this is beyond the scope of this review.

Realist synthesis is philosophically grounded in critical realism or, as noted above, a realist logic of inquiry [93] , [99] , [106] – [107] . Key assumptions regarding the nature of interventions that inform critical realism have been described above in the section on context. See Pawson et al. [106] for more information on critical realism, the philosophical basis of realist synthesis.

Meta-narrative synthesis is explicitly rooted in a constructivist philosophy of science [108] in which knowledge is socially constructed rather than discovered, and what we take to be ‘truth’ is a matter of perspective. Reality has a pluralistic and plastic character, and there is no pre-existing ‘real world’ independent of human construction and language [109] . See Greenhalgh et al. [67] , [85] and Greenhalgh & Wong [97] for more discussion of the constructivist basis of meta-narrative synthesis.

In the case of purely quantitative or qualitative syntheses, it may be an easier matter to uncover unstated assumptions because they are likely to be shared with those of the primary studies in the genre. For example, grounded formal theory shares the philosophical and theoretical underpinnings of grounded theory, rooted in the theoretical perspective of symbolic interactionism [110] – [111] and the philosophy of pragmatism [87] , [112] – [114] .

As with meta-narrative synthesis, meta-study developers identify constructivism as their interpretive philosophical foundation [54] , [88] . Epistemologically, constructivism focuses on how people construct and re-construct knowledge about a specific phenomenon, and has three main assumptions: (1) reality is seen as multiple, at times even incompatible with the phenomenon under consideration; (2) just as primary researchers construct interpretations from participants' data, meta-study researchers also construct understandings about the primary researchers' original findings. Thus, meta-synthesis is a construction of a construction, or a meta-construction; and (3) all constructions are shaped by the historical, social and ideological context in which they originated [54] . The key message here is that reports of any synthesis would benefit from an explicit identification of the underlying philosophical perspectives to facilitate a better understanding of the results, how they were derived, and how they are being interpreted.

4.7. Unit of Analysis

The unit of analysis for each category of review is generally distinct. For the emerging synthesis approaches, the unit of analysis is specific to the intention. In meta-narrative synthesis it is the storyline in diverse research traditions; in rapid review or scoping review, it depends on the focus but could be a concept; and in realist synthesis, it is the theories rather than programs that are the units of analysis. The elements of theory that are important in the analysis are mechanisms of action, the context, and the outcome [107] .

For qualitative synthesis, the units of analysis are generally themes, concepts or theories, although in meta-study, the units of analysis can be research findings (“meta-data-analysis”), research methods (“meta-method”) or philosophical/theoretical perspectives (“meta-theory”) [54] . In quantitative synthesis, the units of analysis range from specific statistics for systematic reviews to effect size of the intervention for meta-analysis. More recently, some systematic reviews focus on theories [115] – [116] , therefore it depends on the research question. Similarly, within conventional literature synthesis the units of analysis also depend on the research purpose, focus and question as well as on the type of research methods incorporated into the review. What is important in all research syntheses, however, is that the unit of analysis needs to be made explicit. Unfortunately, this is not always the case.

4.8. Strengths and Limitations

In this section, we discuss the overarching strengths and limitations of synthesis methodologies as a whole and then highlight strengths and weaknesses across each of our four categories of synthesis.

4.8.1. Strengths of Research Syntheses in General

With the vast proliferation of research reports and the increased ease of retrieval, research synthesis has become more accessible providing a way of looking broadly at the current state of research. The availability of syntheses helps researchers, practitioners, and policy makers keep up with the burgeoning literature in their fields without which evidence-informed policy or practice would be difficult. Syntheses explain variation and difference in the data helping us identify the relevance for our own situations; they identify gaps in the literature leading to new research questions and study designs. They help us to know when to replicate a study and when to avoid excessively duplicating research. Syntheses can inform policy and practice in a way that well-designed single studies cannot; they provide building blocks for theory that helps us to understand and explain our phenomena of interest.

4.8.2. Limitations of Research Syntheses in General

The process of selecting, combining, integrating, and synthesizing across diverse study designs and data types can be complex and potentially rife with bias, even with those methodologies that have clearly defined steps. Just because a rigorous and standardized approach has been used does not mean that implicit judgements will not influence the interpretations and choices made at different stages.

In all types of synthesis, the quantity of data can be considerable, requiring difficult decisions about scope, which may affect relevance. The quantity of available data also has implications for the size of the research team. Few reviews these days can be done independently, in particular because decisions about inclusion and exclusion may require the involvement of more than one person to ensure reliability.

For all types of synthesis, it is likely that in areas with large, amorphous, and diverse bodies of literature, even the most sophisticated search strategies will not turn up all the relevant and important texts. This may be more important in some synthesis methodologies than in others, but the omission of key documents can influence the results of all syntheses. This issue can be addressed, at least in part, by including a library scientist on the research team as required by some funding agencies. Even then, it is possible to miss key texts. In this review, for example, because none of us are trained in or conduct meta-analyses, we were not even aware that we had missed some new developments in this field such as meta-regression [117] – [118] , network meta-analysis [119] – [121] , and the use of individual patient data in meta-analyses [122] – [123] .

One limitation of systematic reviews and meta-analyses is that they rapidly go out of date. We thought this might be true for all types of synthesis, although we wondered if those that produce theory might not be somewhat more enduring. We have not answered this question but it is open for debate. For all types of synthesis, the analytic skills and the time required are considerable so it is clear that training is important before embarking on a review, and some types of review may not be appropriate for students or busy practitioners.

Finally, the quality of reporting in primary studies of all genres is variable so it is sometimes difficult to identify aspects of the study essential for the synthesis, or to determine whether the study meets quality criteria. There may be flaws in the original study, or journal page limitations may necessitate omitting important details. Reporting standards have been developed for some types of reviews (e.g., systematic review, meta-analysis, meta-narrative synthesis, realist synthesis); but there are no agreed upon standards for qualitative reviews. This is an important area for development in advancing the science of research synthesis.

4.8.3. Strengths and Limitations of the Four Synthesis Types

The conventional literature review and now the increasingly common integrative review remain important and accessible approaches for students, practitioners, and experienced researchers who want to summarize literature in an area but do not have the expertise to use one of the more complex methodologies. Carefully executed, such reviews are very useful for synthesizing literature in preparation for research grants and practice projects. They can determine the state of knowledge in an area and identify important gaps in the literature to provide a clear rationale or theoretical framework for a study [14] , [18] . There is a demand, however, for more rigour, with more attention to developing comprehensive search strategies and more systematic approaches to combining, integrating, and synthesizing the findings.

Generally, conventional reviews include diverse study designs and data types that facilitate comprehensiveness, which may be a strength on the one hand, but can also present challenges on the other. The complexity inherent in combining results from studies with diverse methodologies can result in bias and inaccuracies. The absence of clear guidelines about how to synthesize across diverse study types and data [18] has been a challenge for novice reviewers.

Quantitative systematic reviews and meta-analyses have been important in launching the field of evidence-based healthcare. They provide a systematic, orderly and auditable process for conducting a review and drawing conclusions [25] . They are arguably the most powerful approaches to understanding the effectiveness of healthcare interventions, especially when intervention studies on the same topic show very different results. When areas of research are dogged by controversy [25] or when study results go against strongly held beliefs, such approaches can reduce the uncertainty and bring strong evidence to bear on the controversy.

Despite their strengths, they also have limitations. Systematic reviews and meta-analyses do not provide a way of including complex literature comprising various types of evidence including qualitative studies, theoretical work, and epidemiological studies. Only certain types of design are considered and qualitative data are used in a limited way. This exclusion limits what can be learned in a topic area.

Meta-analyses are often not possible because of wide variability in study design, population, and interventions so they may have a narrow range of utility. New developments in meta-analysis, however, can be used to address some of these limitations. Network meta-analysis is used to explore relative efficacy of multiple interventions, even those that have never been compared in more conventional pairwise meta-analyses [121] , allowing for improved clinical decision making [120] . The limitation is that network meta-analysis has only been used in medical/clinical applications [119] and not in public health. It has not yet been widely accepted and many methodological challenges remain [120] – [121] . Meta-regression is another development that combines meta-analytic and linear regression principles to address the fact that heterogeneity of results may compromise a meta-analysis [117] – [118] . The disadvantage is that many clinicians are unfamiliar with it and may incorrectly interpret results [117] .

Some have accused meta-analysis of combining apples and oranges [124] raising questions in the field about their meaningfulness [25] , [28] . More recently, the use of individual rather than aggregate data has been useful in facilitating greater comparability among studies [122] . In fact, Tomas et al. [123] argue that meta-analysis using individual data is now the gold standard although access to the raw data from other studies may be a challenge to obtain.

The usefulness of systematic reviews in synthesizing complex health and social interventions has also been challenged [102] . It is often difficult to synthesize their findings because such studies are “epistemologically diverse and methodologically complex” [ [69] , p.21]. Rigid inclusion/exclusion criteria may allow only experimental or quasi-experimental designs into consideration resulting in lost information that may well be useful to policy makers for tailoring an intervention to the context or understanding its acceptance by recipients.

Qualitative syntheses may be the type of review most fraught with controversy and challenge, while also bringing distinct strengths to the enterprise. Although these methodologies provide a comprehensive and systematic review approach, they do not generally provide definitive statements about intervention effectiveness. They do, however, address important questions about the development of theoretical concepts, patient experiences, acceptability of interventions, and an understanding about why interventions might work.

Most qualitative syntheses aim to produce a theoretically generalizable mid-range theory that explains variation across studies. This makes them more useful than single primary studies, which may not be applicable beyond the immediate setting or population. All provide a contextual richness that enhances relevance and understanding. Another benefit of some types of qualitative synthesis (e.g., grounded formal theory) is that the concept of saturation provides a sound rationale for limiting the number of texts to be included thus making reviews potentially more manageable. This contrasts with the requirements of systematic reviews and meta-analyses that require an exhaustive search.

Qualitative researchers debate about whether the findings of ontologically and epistemological diverse qualitative studies can actually be combined or synthesized [125] because methodological diversity raises many challenges for synthesizing findings. The products of different types of qualitative syntheses range from theory and conceptual frameworks, to themes and rich descriptive narratives. Can one combine the findings from a phenomenological study with the theory produced in a grounded theory study? Many argue yes, but many also argue no.

Emerging synthesis methodologies were developed to address some limitations inherent in other types of synthesis but also have their own issues. Because each type is so unique, it is difficult to identify overarching strengths of the entire category. An important strength, however, is that these newer forms of synthesis provide a systematic and rigorous approach to synthesizing a diverse literature base in a topic area that includes a range of data types such as: both quantitative and qualitative studies, theoretical work, case studies, evaluations, epidemiological studies, trials, and policy documents. More than conventional literature reviews and systematic reviews, these approaches provide explicit guidance on analytic methods for integrating different types of data. The assumption is that all forms of data have something to contribute to knowledge and theory in a topic area. All have a defined but flexible process in recognition that the methods may need to shift as knowledge develops through the process.

Many emerging synthesis types are helpful to policy makers and practitioners because they are usually involved as team members in the process to define the research questions, and interpret and disseminate the findings. In fact, engagement of stakeholders is built into the procedures of the methods. This is true for rapid reviews, meta-narrative syntheses, and realist syntheses. It is less likely to be the case for critical interpretive syntheses.

Another strength of some approaches (realist and meta-narrative syntheses) is that quality and publication standards have been developed to guide researchers, reviewers, and funders in judging the quality of the products [108] , [126] – [127] . Training materials and online communities of practice have also been developed to guide users of realist and meta-narrative review methods [107] , [128] . A unique strength of critical interpretive synthesis is that it takes a critical perspective on the process that may help reconceptualize the data in a way not considered by the primary researchers [72] .

There are also challenges of these new approaches. The methods are new and there may be few published applications by researchers other than the developers of the methods, so new users often struggle with the application. The newness of the approaches means that there may not be mentors available to guide those unfamiliar with the methods. This is changing, however, and the number of applications in the literature is growing with publications by new users helping to develop the science of synthesis [e.g., [129] ]. However, the evolving nature of the approaches and their developmental stage present challenges for novice researchers.

4.9. When to Use Each Approach

Choosing an appropriate approach to synthesis will depend on the question you are asking, the purpose of the review, and the outcome or product you want to achieve. In Additional File 1 , we discuss each of these to provide guidance to readers on making a choice about review type. If researchers want to know whether a particular type of intervention is effective in achieving its intended outcomes, then they might choose a quantitative systemic review with or without meta-analysis, possibly buttressed with qualitative studies to provide depth and explanation of the results. Alternately, if the concern is about whether an intervention is effective with different populations under diverse conditions in varying contexts, then a realist synthesis might be the most appropriate.

If researchers' concern is to develop theory, they might consider qualitative syntheses or some of the emerging syntheses that produce theory (e.g., critical interpretive synthesis, realist review, grounded formal theory, qualitative meta-synthesis). If the aim is to track the development and evolution of concepts, theories or ideas, or to determine how an issue or question is addressed across diverse research traditions, then meta-narrative synthesis would be most appropriate.

When the purpose is to review the literature in advance of undertaking a new project, particularly by graduate students, then perhaps an integrative review would be appropriate. Such efforts contribute towards the expansion of theory, identify gaps in the research, establish the rationale for studying particular phenomena, and provide a framework for interpreting results in ways that might be useful for influencing policy and practice.

For researchers keen to bring new insights, interpretations, and critical re-conceptualizations to a body of research, then qualitative or critical interpretive syntheses will provide an inductive product that may offer new understandings or challenges to the status quo. These can inform future theory development, or provide guidance for policy and practice.

5. Discussion

What is the current state of science regarding research synthesis? Public health, health care, and social science researchers or clinicians have previously used all four categories of research synthesis, and all offer a suitable array of approaches for inquiries. New developments in systematic reviews and meta-analysis are providing ways of addressing methodological challenges [117] – [123] . There has also been significant advancement in emerging synthesis methodologies and they are quickly gaining popularity. Qualitative meta-synthesis is still evolving, particularly given how new it is within the terrain of research synthesis. In the midst of this evolution, outstanding issues persist such as grappling with: the quantity of data, quality appraisal, and integration with knowledge translation. These topics have not been thoroughly addressed and need further debate.

5.1. Quantity of Data

We raise the question of whether it is possible or desirable to find all available studies for a synthesis that has this requirement (e.g., meta-analysis, systematic review, scoping, meta-narrative synthesis [25] , [27] , [63] , [67] , [84] – [85] ). Is the synthesis of all available studies a realistic goal in light of the burgeoning literature? And how can this be sustained in the future, particularly as the emerging methodologies continue to develop and as the internet facilitates endless access? There has been surprisingly little discussion on this topic and the answers will have far-reaching implications for searching, sampling, and team formation.

Researchers and graduate students can no longer rely on their own independent literature search. They will likely need to ask librarians for assistance as they navigate multiple sources of literature and learn new search strategies. Although teams now collaborate with library scientists, syntheses are limited in that researchers must make decisions on the boundaries of the review, in turn influencing the study's significance. The size of a team may also be pragmatically determined to manage the search, extraction, and synthesis of the burgeoning data. There is no single answer to our question about the possibility or necessity of finding all available articles for a review. Multiple strategies that are situation specific are likely to be needed.

5.2. Quality Appraisal

While the issue of quality appraisal has received much attention in the synthesis literature, scholars are far from resolution. There may be no agreement about appraisal criteria in a given tradition. For example, the debate rages over the appropriateness of quality appraisal in qualitative synthesis where there are over 100 different sets of criteria and many do not overlap [49] . These differences may reflect disciplinary and methodological orientations, but diverse quality appraisal criteria may privilege particular types of research [49] . The decision to appraise is often grounded in ontological and epistemological assumptions. Nonetheless, diversity within and between categories of synthesis is likely to continue unless debate on the topic of quality appraisal continues and evolves toward consensus.

5.3. Integration with Knowledge Translation

If research syntheses are to make a difference to practice and ultimately to improve health outcomes, then we need to do a better job of knowledge translation. In the Canadian Institutes of Health Research (CIHR) definition of knowledge translation (KT), research or knowledge synthesis is an integral component [130] . Yet, with few exceptions [131] – [132] , very little of the research synthesis literature even mentions the relationship of synthesis to KT nor does it discuss strategies to facilitate the integration of synthesis findings into policy and practice. The exception is in the emerging synthesis methodologies, some of which (e.g., realist and meta-narrative syntheses, scoping reviews) explicitly involve stakeholders or knowledge users. The argument is that engaging them in this way increases the likelihood that the knowledge generated will be translated into policy and practice. We suggest that a more explicit engagement with knowledge users in all types of synthesis would benefit the uptake of the research findings.

Research synthesis neither makes research more applicable to practice nor ensures implementation. Focus must now turn seriously towards translation of synthesis findings into knowledge products that are useful for health care practitioners in multiple areas of practice and develop appropriate strategies to facilitate their use. The burgeoning field of knowledge translation has, to some extent, taken up this challenge; however, the research-practice gap continues to plague us [133] – [134] . It is a particular problem for qualitative syntheses [131] . Although such syntheses have an important place in evidence-informed practice, little effort has gone into the challenge of translating the findings into useful products to guide practice [131] .

5.4. Limitations

Our study took longer than would normally be expected for an integrative review. Each of us were primarily involved in our own dissertations or teaching/research positions, and so this study was conducted ‘off the sides of our desks.’ A limitation was that we searched the literature over the course of 4 years (from 2008–2012), necessitating multiple search updates. Further, we did not do a comprehensive search of the literature after 2012, thus the more recent synthesis literature was not systematically explored. We did, however, perform limited database searches from 2012–2015 to keep abreast of the latest methodological developments. Although we missed some new approaches to meta-analysis in our search, we did not find any new features of the synthesis methodologies covered in our review that would change the analysis or findings of this article. Lastly, we struggled with the labels used for the broad categories of research synthesis methodology because of our hesitancy to reinforce the divide between quantitative and qualitative approaches. However, it was very difficult to find alternative language that represented the types of data used in these methodologies. Despite our hesitancy in creating such an obvious divide, we were left with the challenge of trying to find a way of characterizing these broad types of syntheses.

6. Conclusion

Our findings offer methodological clarity for those wishing to learn about the broad terrain of research synthesis. We believe that our review makes transparent the issues and considerations in choosing from among the four broad categories of research synthesis. In summary, research synthesis has taken its place as a form of research in its own right. The methodological terrain has deep historical roots reaching back over the past 200 years, yet research synthesis remains relatively new to public health, health care, and social sciences in general. This is rapidly changing. New developments in systematic reviews and meta-analysis, and the emergence of new synthesis methodologies provide a vast array of options to review the literature for diverse purposes. New approaches to research synthesis and new analytic methods within existing approaches provide a much broader range of review alternatives for public health, health care, and social science students and researchers.

Acknowledgments

KSM is an assistant professor in the Faculty of Nursing at the University of Alberta. Her work on this article was largely conducted as a Postdoctoral Fellow, funded by KRESCENT (Kidney Research Scientist Core Education and National Training Program, reference #KRES110011R1) and the Faculty of Nursing at the University of Alberta.

MM's work on this study over the period of 2008-2014 was supported by a Canadian Institutes of Health Research Applied Public Health Research Chair Award (grant #92365).

We thank Rachel Spanier who provided support with reference formatting.

List of Abbreviations (in Additional File 1 )

Conflict of interest: The authors declare that they have no conflicts of interest in this article.

Authors' contributions: KSM co-designed the study, collected data, analyzed the data, drafted/revised the manuscript, and managed the project.

MP contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

JB contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

WN contributed to searching the literature, developing the analytic framework, and extracting data for the Additional File.

All authors read and approved the final manuscript.

Additional Files: Additional File 1 – Selected Types of Research Synthesis

This Additional File is our dataset created to organize, analyze and critique the literature that we synthesized in our integrative review. Our results were created based on analysis of this Additional File.

Critical Appraisal Tools

Jbi’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers..

These tools have been revised. Recently published articles detail the revision.

"Assessing the risk of bias of quantitative analytical studies: introducing the vision for critical appraisal within JBI systematic reviews"

"revising the jbi quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process".

End to end support for developing systematic reviews

Analytical Cross Sectional Studies  

Checklist for analytical cross sectional studies, how to cite, associated publication(s), case control studies  , checklist for case control studies, case reports  , checklist for case reports, case series  , checklist for case series.

Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A, Stephenson M, Aromataris E. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evidence Synthesis. 2020;18(10):2127-2133

Methodological quality of case series studies: an introduction to the JBI critical appraisal tool

Cohort studies  , checklist for cohort studies, diagnostic test accuracy studies  , checklist for diagnostic test accuracy studies.

Campbell JM, Klugar M, Ding S, Carmody DP, Hakonsen SJ, Jadotte YT, White S, Munn Z. Chapter 9: Diagnostic test accuracy systematic reviews. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020

JBI Manual for Evidence Synthesis

Chapter 9: Diagnostic test accuracy systematic reviews

Economic Evaluations  

Checklist for economic evaluations, prevalence studies  , checklist for prevalence studies.

Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Chapter 5: Systematic reviews of prevalence and incidence. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020

Chapter 5: Systematic reviews of prevalence and incidence

Qualitative Research  

Checklist for qualitative research.

Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc. 2015;13(3):179–187

Chapter 2: Systematic reviews of qualitative evidence

Qualitative research synthesis

Methodological guidance for systematic reviewers utilizing meta-aggregation

Quasi-Experimental Studies  

Checklist for quasi-experimental studies, randomized controlled trials  , randomized controlled trials.

Barker TH, Stone JC, Sears K, Klugar M, Tufanaru C, Leonardi-Bee J, Aromataris E, Munn Z. The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evidence Synthesis. 2023;21(3):494-506

The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials

Randomized controlled trials checklist (archive), systematic reviews  , checklist for systematic reviews.

Aromataris E, Fernandez R, Godfrey C, Holly C, Kahlil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an Umbrella review approach. Int J Evid Based Healthc. 2015;13(3):132-40.

Chapter 10: Umbrella Reviews

Textual Evidence: Expert Opinion  

Checklist for textual evidence: expert opinion.

McArthur A, Klugarova J, Yan H, Florescu S. Chapter 4: Systematic reviews of text and opinion. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020

Chapter 4: Systematic reviews of text and opinion

Textual Evidence: Narrative  

Checklist for textual evidence: narrative, textual evidence: policy  , checklist for textual evidence: policy.

The Federal Register

The daily journal of the united states government, request access.

Due to aggressive automated scraping of FederalRegister.gov and eCFR.gov, programmatic access to these sites is limited to access to our extensive developer APIs.

If you are human user receiving this message, we can add your IP address to a set of IPs that can access FederalRegister.gov & eCFR.gov; complete the CAPTCHA (bot test) below and click "Request Access". This process will be necessary for each IP address you wish to access the site from, requests are valid for approximately one quarter (three months) after which the process may need to be repeated.

An official website of the United States government.

If you want to request a wider IP range, first request access for your current IP, and then use the "Site Feedback" button found in the lower left-hand side to make the request.