Qualitative data and quantitative data pdf

The completed instruments of data collection, viz. A vast mass of data. They cannot straightaway provide answers to research questions. They, like raw materials, need processing. Data processing involves classification and summarisal1on of data in order to make them amenable to analysis. Processing of data requires advance planning at the stage of planning the research design.

This advance planning may covey such aspects as categorization of variables and preparation of dummy tables. This type of preplanning ensures better identification of data needs and their adequate coverage in the tools for collection of data.

Data processing consists of a number of closely related operations, viz. The term data processing often used more specifically in the context of a research to refer to the class of qualitative and quantitative research application.

If we are actually doing a research study, we would by now have reached a stage where we have either extracted or collected the required information. The next step is what does with this information.

It has some aim and objectives. And taken some respondent who are living in a patriarchal based society of Bangladesh. Explanation and Example of Data: In my research I explained the opinions of differs level people about patriarchal based society of Bangladesh as well as the present situation of women decision making power in gender biased family and social structure pattern.

Editing is a process of checking to detect and or correct errors and omissions. Editing is done at two stages: first at the fieldwork stage and second at office. Field editing During the stress of interviewing the interviewer cannot always record responses completely and legibly. Therefore after each interview is over, he should review the schedule to complete abbreviated responses, rewrite illegible responses and correct omissions. Editing consists of scrutinizing the completed research instruments to identify and minimize, as far as possible, errors, incompleteness, misclassification and gaps in the information obtained from the respondents.

Irrespective of the method of data collection, the information collected is called raw data. In the case of questionnaire, similar problems can crop up. There are several ways of minimizing such problems. Of cause, we must be careful about making such inferences or we may introduce new errors into the data. Again we must be extremely careful. Examine answers to all questions or variable at a time 2.

Coding analysis: Coding means assigning numerals or other symbols to the categories or responses. For each question a coding scheme is designed on the basis of the con med categories. The coding schemes with their assigned symbols together with specific coding instructions may be assembled in a book. If the data are to be transferred to machine punch cards, the codebook will also identify the column in which it is entered.

The method of coding is largely dictated by two considerations: 1. The way a variable has been measures measurement scale in our research instrument e.

The way we want to communicate the findings about a variable to your readers. For coding, the first level of distinction is whether a set of data is qualitative or quantitative in nature. Steps in coding quantitative data: i. Developing a code book ii. Pretesting code book iii. Coding the data iv. Verifying the data Developing a code book: A codebook describes and documents the questions asked or items collected in a survey.

Codebooks and study documentation will provide you with crucial details to help you decide whether or not a particular data collection will be useful in your research. The codebook will describe the subject of the survey or data collection, the sample and how it was constructed, And how the data were coded, entered, and processed. The questionnaire or survey instrument will be included along with a description or layout of how the data file is organized.

Others are not electronic and must be used in a library or archive, or, depending on copyright, photocopied if you want your own for personal use. A code book provides a set of rules for assigning numerical values to answers obtained from respondents. Kindly tick the option. Your marital status: i Single ii Married iii Divorced 2. Level of achievement i. SSC ii. HSC iii. Masters v. PhD 2. Please tick i. Yes ii. Please give your answer i. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process.

Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.

When combined in a mixed-method synthesis, quantitative and qualitative evidence can potentially contribute to understanding how complex interventions work and for whom, and how the complex health systems into which they are implemented respond and adapt.

The different purposes and designs for combining quantitative and qualitative evidence in a mixed-method synthesis for a guideline process are described.

Questions relevant to gaining an understanding of the complexity of complex interventions and the wider health systems within which they are implemented that can be addressed by mixed-method syntheses are presented. The practical methodological guidance in this paper is intended to help guideline producers and review authors commission and conduct mixed-method syntheses where appropriate.

If more mixed-method syntheses are conducted, guideline developers will have greater opportunities to access this evidence to inform decision-making. Recognition has grown that while quantitative methods remain vital, they are usually insufficient to address complex health systems related research questions. Introducing change into a complex health system gives rise to emergent reactions, which cannot be fully predicted in advance.

Emergent reactions can often only be understood through combining quantitative methods with a more flexible qualitative lens. Publication of guidance on designing complex intervention process evaluations and other works advocating mixed-methods approaches to intervention research have stimulated better quality evidence for synthesis. Mixed-method research and review definitions are outlined in box 1. A mixed-method synthesis can integrate quantitative, qualitative and mixed-method evidence or data from primary studies.

Thomas and Harden further define three ways in which reviews are mixed. The types of synthesis method used eg, statistical meta-analysis and qualitative synthesis. Qualitative methods of data collection may include, for example, interviews, focus groups, observations and analysis of documents.

For example, qualitative data from case studies, grey literature reports and open-ended questions from surveys. This paper is one of a series that aims to explore the implications of complexity for systematic reviews and guideline development, commissioned by WHO.

This paper is concerned with the methodological implications of including quantitative and qualitative evidence in mixed-method systematic reviews and guideline development for complex interventions. The guidance was developed through a process of bringing together experts in the field, literature searching and consensus building with end users guideline developers, clinicians and reviewers.

We clarify the different purposes, review designs, questions and synthesis methods that may be applicable to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of WHO guidelines that incorporated quantitative and qualitative evidence are used to illustrate possible uses of mixed-method reviews and mechanisms of integration table 1 , online supplementary files 1—3.

Opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process are presented. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence in guidelines of complex interventions that take a complexity perspective and health systems focus.

Designs and methods and their use or applicability in guidelines and systematic reviews taking a complexity perspective.

The first paper in this series 17 outlines aspects of complexity associated with complex interventions and health systems that can potentially be explored by different types of evidence, including synthesis of quantitative and qualitative evidence. Petticrew et al 17 distinguish between a complex interventions perspective and a complex systems perspective.

Aspects of complexity associated with implementation of complex interventions in health systems that could potentially be addressed with a synthesis of quantitative and qualitative evidence are summarised in table 2. Another paper in the series outlines criteria used in a new evidence to decision framework for making decisions about complex interventions implemented in complex systems, against which the need for quantitative and qualitative evidence can be mapped. Mixed-method syntheses of quantitative and qualitative evidence can also help with understanding of whether there has been theory failure and or implementation failure.

The Cochrane Qualitative and Implementation Methods Group provide additional guidance on exploring implementation and theory failure that can be adapted to address aspects of complexity of complex interventions when implemented in health systems.

Health-system complexity-related questions that a synthesis of quantitative and qualitative evidence could address derived from Petticrew et al For a research question about Implementation: How and why does the implementation of this intervention vary across contexts?

For an effectiveness review: Do the effects of the intervention appear to be context dependent? It may not be apparent which aspects of complexity or which elements of the complex intervention or health system can be explored in a guideline process, or whether combining qualitative and quantitative evidence in a mixed-method synthesis will be useful, until the available evidence is scoped and mapped.

Following a scoping exercise and definition of key questions, the next step in the guideline development process is to identify existing or commission new systematic reviews to locate and summarise the best available evidence in relation to each question. Further understanding of health system complexity was facilitated through the conduct of additional country-level case studies that contributed to an overall understanding of what worked and what happened when lay health worker interventions were implemented.

See table 1 online supplementary file 2. There are a few existing examples, which we draw on in this paper, but integrating quantitative and qualitative evidence in a mixed-method synthesis is relatively uncommon in a guideline process. Box 2 includes a set of key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in mixed-methods design might ask.

Subsequent sections provide more information and signposting to further reading to help address these key questions. HOW: How easy is it to disaggregate quantitative and qualitative data from mixed-method studies? How will quantitative and qualitative evidence be integrated? Through a:. WHICH: Which mixed-method designs, methodologies and methods best fit into a guideline process to inform recommendations?

Petticrew et al 17 define the different aspects of complexity and examples of complexity-related questions that can potentially be explored in guidelines and systematic reviews taking a complexity perspective.

Relevant aspects of complexity outlined by Petticrew e t al 17 are summarised in table 2 below, together with the corresponding questions that could be addressed in a synthesis combining qualitative and quantitative evidence. Importantly, the aspects of complexity and their associated concepts of interest have however yet to be translated fully in primary health research or systematic reviews. There are few known examples where selected complexity concepts have been used to analyse or reanalyse a primary intervention study.

Most notable is Chandler e t al 26 who specifically set out to identify and translate a set of relevant complexity theory concepts for application in health systems research. Chandler then reanalysed a trial process evaluation using selected complexity theory concepts to better understand the complex causal pathway in the health system that explains some aspects of complexity in table 2. The criteria reflect WHO norms and values and take account of a complexity perspective.

The framework can be used by guideline development groups as a menu to decide which criteria to prioritise, and which study types and synthesis methods can be used to collect evidence for each criterion. Many of the criteria and their related questions can be addressed using a synthesis of quantitative and qualitative evidence: the balance of benefits and harms, human rights and sociocultural acceptability, health equity, societal implications and feasibility see table 3.

Similar aspects in the DECIDE framework 15 could also be addressed using synthesis of qualitative and quantitative evidence. Integrate evidence to decision framework criteria, example questions and types of studies to potentially address these questions derived from Rehfeuss et al Quantitative evidence can quantify the difference in effect, but does not answer the question of how.

For anchor and compass questions, additional application of a theory eg, complexity theory can help focus evidence synthesis and presentation to explore and explain complexity issues. If a complexity perspective, and certain criteria within evidence to decision frameworks, is deemed relevant and desirable by guideline developers, it is only possible to pursue a complexity perspective if the evidence is available. Careful scoping using knowledge maps or scoping reviews will help inform development of questions that are answerable with available evidence.

This should not mean that the original questions developed for which no evidence was found when scoping the literature were not important. An important function of creating a knowledge map is also to identify gaps to inform a future research agenda. The shift towards integration of qualitative and quantitative evidence in primary research has, in recent years, begun to be mirrored within research synthesis.

Advocating the integration of quantitative and qualitative evidence assumes a complementarity between research methodologies, and a need for both types of evidence to inform policy and practice.

Below, we briefly outline the current designs for integrating qualitative and quantitative evidence within a mixed-method review or synthesis. One of the early approaches to integrating qualitative and quantitative evidence detailed by Sandelowski et al 32 advocated three basic review designs: segregated, integrated and contingent designs, which have been further developed by Heyvaert et al 33 box 3. Conventional separate distinction between quantitative and qualitative approaches based on the assumption they are different entities and should be treated separately; can be distinguished from each other; their findings warrant separate analyses and syntheses.

Ultimately, the separate synthesis results can themselves be synthesised. The methodological differences between qualitative and quantitative studies are minimised as both are viewed as producing findings that can be readily synthesised into one another because they address the same research purposed and questions.

Transformation involves either turning qualitative data into quantitative quantitising or quantitative findings are turned into qualitative qualitising to facilitate their integration. Takes a cyclical approach to synthesis, with the findings from one synthesis informing the focus of the next synthesis, until all the research objectives have been addressed.

Studies are not necessarily grouped and categorised as qualitative or quantitative. A recent review of more than systematic reviews 34 combining quantitative and qualitative evidence identified two main synthesis designs—convergent and sequential.

In a convergent design, qualitative and quantitative evidence is collated and analysed in a parallel or complementary manner, whereas in a sequential synthesis, the collation and analysis of quantitative and qualitative evidence takes place in a sequence with one synthesis informing the other box 4.

Qualitative and quantitative research is collected and analysed at the same time in a parallel or complementary manner. Integration can occur at three points:. All included studies are analysed using the same methods and results presented together. As only one synthesis method is used, data transformation occurs qualitised or quantised.

Usually addressed one review question. Qualitative and quantitative data are analysed and presented separately but integrated using a further synthesis method; eg, narratively, tables, matrices or reanalysing evidence.

The results of both syntheses are combined in a third synthesis. Usually addresses an overall review question with subquestions.

Qualitative and quantitative data are analysed and presented separately with integration occurring in the interpretation of results in the discussion section. Usually addresses two or more complimentary review questions. A two-phase approach, data collection and analysis of one type of evidence eg, qualitative , occurs after and is informed by the collection and analysis of the other type eg, quantitative.

Usually addresses an overall question with subquestions with both syntheses complementing each other. The three case studies table 1 , online supplementary files 1—3 illustrate the diverse combination of review designs and synthesis methods that were considered the most appropriate for specific guidelines. In this section, we draw on examples where specific review designs and methods have been or can be used to explore selected aspects of complexity in guidelines or systematic reviews.

We also identify other review methods that could potentially be used to explore aspects of complexity.

Of particular note, we could not find any specific examples of systematic methods to synthesise highly diverse research designs as advocated by Petticrew e t al 17 and summarised in tables 2 and 3. For example, we could not find examples of methods to synthesise qualitative studies, case studies, quantitative longitudinal data, possibly historical data, effectiveness studies providing evidence of differential effects across different contexts, and system modelling studies eg, agent-based modelling to explore system adaptivity.

There are different ways that quantitative and qualitative evidence can be integrated into a review and then into a guideline development process. In practice, some methods enable integration of different types of evidence in a single synthesis, while in other methods, the single systematic review may include a series of stand-alone reviews or syntheses that are then combined in a cross-study synthesis. Table 1 provides an overview of the characteristics of different review designs and methods and guidance on their applicability for a guideline process.

Designs and methods that have already been used in WHO guideline development are described in part A of the table. Part B outlines a design and method that can be used in a guideline process, and part C covers those that have the potential to integrate quantitative, qualitative and mixed-method evidence in a single review design such as meta-narrative reviews and Bayesian syntheses , but their application in a guideline context has yet to be demonstrated.

Depending on the review design see boxes 3 and 4 , integration can potentially take place at a review team and design level, and more commonly at several key points of the review or guideline process. The following sections outline potential points of integration and associated practical considerations when integrating quantitative and qualitative evidence in guideline development. In a guideline process, it is common for syntheses of quantitative and qualitative evidence to be done separately by different teams and then to integrate the evidence.

A practical consideration relates to the organisation, composition and expertise of the review teams and ways of working. If the quantitative and qualitative reviews are being conducted separately and then brought together by the same team members, who are equally comfortable operating within both paradigms, then a consistent approach across both paradigms becomes possible.

If, however, a team is being split between the quantitative and qualitative reviews, then the strengths of specialisation can be harnessed, for example, in quality assessment or synthesis. Optimally, at least one, if not more, of the team members should be involved in both quantitative and qualitative reviews to offer the possibility of making connexions throughout the review and not simply at re-agreed junctures.

Clearly specified key question s , derived from a scoping or consultation exercise, will make it clear if quantitative and qualitative evidence is required in a guideline development process and which aspects will be addressed by which types of evidence. For the remaining stages of the process, as documented below, a review team faces challenges as to whether to handle each type of evidence separately, regardless of whether sequentially or in parallel, with a view to joining the two products on completion or to attempt integration throughout the review process.

In each case, the underlying choice is of efficiencies and potential comparability vs sensitivity to the underlying paradigm. Once key questions are clearly defined, the guideline development group typically needs to consider whether to conduct a single sensitive search to address all potential subtopics lumping or whether to conduct specific searches for each subtopic splitting. These two considerations often mean a trade-off between a single search process involving very large numbers of records or a more protracted search process retrieving smaller numbers of records.

Both approaches have advantages and choice may depend on the respective availability of resources for searching and sifting. Closely related to decisions around searching are considerations relating to screening and selecting studies for inclusion in a systematic review.

The risk of missing relevant reports might be minimised by whole team screening for empirical reports in the first instance and then coding them for a specific quantitative, qualitative or mixed-methods report at a subsequent stage. Within a guideline process, review teams may be more limited in their choice of instruments to assess methodological limitations of primary studies as there are mandatory requirements to use the Cochrane risk of bias tool 37 to feed into Grading of Recommendations Assessment, Development and Evaluation GRADE 38 or to select from a small pool of qualitative appraisal instruments in order to apply GRADE; Confidence in the Evidence from Reviews of Qualitative Research GRADE-CERQual 39 to assess the overall certainty or confidence in findings.

The Cochrane Qualitative and Implementation Methods Group has recently issued guidance on the selection of appraisal instruments and core assessment criteria. Alternatively, a more paradigm-sensitive approach would involve selecting the best instrument for each respective review while deferring challenges from later heterogeneity of reporting.

The way in which data and evidence are extracted from primary research studies for review will be influenced by the type of integrated synthesis being undertaken and the review purpose. Initially, decisions need to be made regarding the nature and type of data and evidence that are to be extracted from the included studies.

Method-specific reporting guidelines 43 44 provide a good template as to what quantitative and qualitative data it is potentially possible to extract from different types of method-specific study reports, although in practice reporting quality varies.

Online supplementary file 5 provides a hypothetical example of the different types of studies from which quantitative and qualitative evidence could potentially be extracted for synthesis.

For those reviews where the quantitative and qualitative findings of studies are synthesised separately and integrated at the point of findings eg, segregated or contingent approaches or sequential synthesis design , separate data extraction approaches will likely be used.

Where integration occurs during the process of the review eg, integrated approach or convergent synthesis design , an integrated approach to data extraction may be considered, depending on the purpose of the review. This may involve the use of a data extraction framework, the choice of which needs to be congruent with the approach to synthesis chosen for the review. The Cochrane Qualitative and Implementation Methods Group provide further guidance on extraction of qualitative data, including use of software.

Relatively few synthesis methods start off being integrated from the beginning, and these methods have generally been subject to less testing and evaluation particularly in a guideline context see table 1. A review design that started off being integrated from the beginning may be suitable for some guideline contexts such as in case study 3—risk communication in humanitarian disasters—where there was little evidence of effect , but in general if there are sufficient trials then a separate systematic review and meta-analysis will be required for a guideline.

Other papers in this series offer guidance on methods for synthesising quantitative 46 and qualitative evidence 14 in reviews that take a complexity perspective. Further guidance on integrating quantitative and qualitative evidence in a systematic review is provided by the Cochrane Qualitative and Implementation Methods Group.

It is highly likely unless there are well-designed process evaluations that the primary studies may not themselves seek to address the complexity-related questions required for a guideline process. In which case, review authors will need to configure the available evidence and transform the evidence through the synthesis process to produce explanations, propositions and hypotheses ie, findings that were not obvious at primary study level.

It is important that guideline commissioners, developers and review authors are aware that specific methods are intended to produce a type of finding with a specific purpose such as developing new theory in the case of meta-ethnography.

Descriptive findings —qualitative evidence-driven translated descriptive themes that do not move beyond the primary studies. Explanatory findings —may either be at a descriptive or theoretical level. At the descriptive level, qualitative evidence is used to explain phenomena observed in quantitative results, such as why implementation failed in specific circumstances.

At the theoretical level, the transformed and interpreted findings that go beyond the primary studies can be used to explain the descriptive findings. The latter description is generally the accepted definition in the wider qualitative community.

Hypothetical or theoretical finding —qualitative evidence-driven transformed themes or lines of argument that go beyond the primary studies. Although similar, Thomas and Harden 56 make a distinction in the purposes between two types of theoretical findings: analytical themes and the product of meta-ethnographies, third-order interpretations.

Analytical themes are a product of interrogating descriptive themes by placing the synthesis within an external theoretical framework such as the review question and subquestions and are considered more appropriate when a specific review question is being addressed eg, in a guideline or to inform policy. Third - order interpretations come from translating studies into one another while preserving the original context and are more appropriate when a body of literature is being explored in and of itself with broader or emergent review questions.

A critical element of guideline development is the formulation of recommendations by the Guideline Development Group, and EtD frameworks help to facilitate this process. It is commonly the EtD framework that enables the findings of the separate quantitative and qualitative reviews to be brought together in a guideline process.

Specific challenges when populating the DECIDE evidence to decision framework 15 were noted in case study 3 risk communication in humanitarian disasters as there was an absence of intervention effect data and the interventions to communicate public health risks were context specific and varied. A d ifferent type of EtD framework needs to be developed for reviews that do not include sufficient evidence of intervention effect. Mixed-method review and synthesis methods are generally the least developed of all systematic review methods.

It is acknowledged that methods for combining quantitative and qualitative evidence are generally poorly articulated. With the exception of case study 3 risk communication , the quantitative and qualitative reviews for these specific guidelines have been conducted separately, and the findings subsequently brought together in an EtD framework to inform recommendations.

Other mixed-method review designs have potential to contribute to understanding of complex interventions and to explore aspects of wider health systems complexity but have not been sufficiently developed and tested for this specific purpose, or used in a guideline process section C of table 1. Some methods such as meta-narrative reviews also explore different questions to those usually asked in a guideline process. Methods for processing eg, quality appraisal and synthesising the highly diverse evidence suggested in tables 2 and 3 that are required to explore specific aspects of health systems complexity such as system adaptivity and to populate some sections of the INTEGRATE EtD framework remain underdeveloped or in need of development.

In addition to the required methodological development mentioned above, there is no GRADE approach 38 for assessing confidence in findings developed from combined quantitative and qualitative evidence. Another paper in this series outlines how to deal with complexity and grading different types of quantitative evidence, 51 and the GRADE CERQual approach for qualitative findings is described elsewhere, 39 but both these approaches are applied to method-specific and not mixed-method findings.

An unofficial adaptation of GRADE was used in the risk communication guideline that reported mixed-method findings. Nor is there a reporting guideline for mixed-method reviews, 47 and for now reports will need to conform to the relevant reporting requirements of the respective method-specific guideline. When conducting quantitative and qualitative reviews that will subsequently be combined, there are specific considerations for managing and integrating the different types of evidence throughout the review process.

We have summarised different options for combining qualitative and quantitative evidence in mixed-method syntheses that guideline developers and systematic reviewers can choose from, as well as outlining the opportunities to integrate evidence at different stages of the review and guideline development process.

Review commissioners, authors and guideline developers generally have less experience of combining qualitative and evidence in mixed-methods reviews. In particular, there is a relatively small group of reviewers who are skilled at undertaking fully integrated mixed-method reviews. Commissioning additional qualitative and mixed-method reviews creates an additional cost.

Large complex mixed-method reviews generally take more time to complete. Careful consideration needs to be given as to which guidelines would benefit most from additional qualitative and mixed-method syntheses. More training is required to develop capacity and there is a need to develop processes for preparing the guideline panel to consider and use mixed-method evidence in their decision-making. This paper has presented how qualitative and quantitative evidence, combined in mixed-method reviews, can help understand aspects of complex interventions and the systems within which they are implemented.

There are further opportunities to use these methods, and to further develop the methods, to look more widely at additional aspects of complexity. There is a range of review designs and synthesis methods to choose from depending on the question being asked or the questions that may emerge during the conduct of the synthesis. Additional methods need to be developed or existing methods further adapted in order to synthesise the full range of diverse evidence that is desirable to explore the complexity-related questions when complex interventions are implemented into health systems.

We encourage review commissioners and authors, and guideline developers to consider using mixed-methods reviews and synthesis in guidelines and to report on their usefulness in the guideline development process.

Handling editor: Soumyadeep Bhaumik. All authors contributed to paper development and writing and agreed the final manuscript. Helen Smith was series Editor. We thank all those who provided feedback on various iterations. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions or policies of WHO. Competing interests: No financial interests declared.

Patient consent: Not required. Provenance and peer review: Not commissioned; externally peer reviewed. Data sharing statement: No additional data are available. National Center for Biotechnology Information , U. BMJ Glob Health. Published online Jan Author information Article notes Copyright and License information Disclaimer.

Corresponding author. Correspondence to Professor Jane Noyes; ku. Licensee BMJ. In any reproduction of this article there should not be any suggestion that WHO or this article endorse any specific organization or products.

The use of the WHO logo is not permitted. This notice should be preserved along with the article's original URL. This article has been cited by other articles in PMC. Associated Data Supplementary Materials Supplementary data. Supplementary data. Abstract Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems.

Keywords: health systems, systematic review, qualitative study, randomised control trial. Summary box. Introduction Recognition has grown that while quantitative methods remain vital, they are usually insufficient to address complex health systems related research questions. Box 1 Defining mixed-method research and reviews. Table 1 Designs and methods and their use or applicability in guidelines and systematic reviews taking a complexity perspective.

Case study examples and references Complexity-related questions of interest in the guideline Types of synthesis used in the guideline Mixed-method review design and integration mechanisms Observations, concerns and considerations A. Mixed-method review designs used in WHO guideline development Antenatal Care ANC guidelines online supplementary file 1 14 15 27 45 48 49 What do women in high-income, medium-income and low-income countries want and expect from antenatal care ANC , based on their own accounts of their beliefs, views, expectations and experiences of pregnancy?

The hypothetical nature of meta-ethnography findings may be challenging for guideline panel members to process without additional training See Flemming et al 14 for considerations when selecting meta-ethnography What are the evidence-based practices during ANC that improved outcomes and lead to positive pregnancy experience and how should these practices be delivered?

Quantitative review of trials Factors that influence the uptake of routine antenatal services by pregnant women Views and experiences of maternity care providers Qualitative synthesis Framework synthesis Meta-ethnography Task shifting guidelines online supplementary file 2 15 24 27 45 54 What are the effects of lay health worker interventions in primary and community healthcare on maternal and child health and the management of infectious diseases?

Quantitative review of trials Combination of a segregated design and sequential synthesis Several published quantitative reviews were used eg, Cochrane review of lay health worker interventions Additional new qualitative evidence syntheses were commissioned segregated Integration: quantitative and qualitative review findings on lay health workers were brought together in several DECIDE frameworks.

Tools included adapted SURE Framework and post hoc logic model An innovative approach to guideline development The post hoc logic model was developed after the guideline was completed What factors affect the implementation of lay health worker programmes for maternal and child health?

Qualitative evidence synthesis Framework synthesis Risk communication guideline online supplementary file 3 15 27 45 55 Quantitative review of quantitative evidence descriptive Qualitative using framework synthesis Results based convergent synthesis A knowledge map of studies was produced to identify the method, topic and geographical spread of evidence. Reviews first organised and synthesised evidence by method-specific streams and reported method-specific findings.

Then similar findings across method-specific streams were grouped and further developed using all the relevant evidence Integration: where possible, quantitative and qualitative evidence for the same intervention and question was mapped against core DECIDE domains.

Tools included framework using public health emergency model and disaster phases Very few trials were identified.

Quantitative and qualitative evidence was used to construct a high level view of what appeared to work and what happened when similar broad groups of interventions or strategies were implemented in different contexts Example of a fully integrated mixed-method synthesis.

Methodology is complex. Most subsequent examples have only partially operationalised the methodology An intervention effect review will still be required to feed into the guideline process Factors affecting childhood immunisation 59 What factors explain complexity and causal pathways? Complex methodology. Undergoing development and testing for a health context. Realist synthesis NB.

Other theory-informed synthesis methods follow similar processes Can be accommodated within various review designs Development of a theory from the literature, analysis of quantitative and qualitative evidence against the theory leads to development of context, mechanism and outcome chains that explain how outcomes come about Integration: programme theory and assembling mixed-method evidence to create Context, Mechanism and Outcome CMO configurations May be useful where there are few trials.

The hypothetical nature of findings may be challenging for guideline panel members to process without additional training. Critical interpretive synthesis Can be accommodated within various review designs Aims to generate theory from large and diverse body of literature Segregated sequential design Integration: integrative grid There are few examples and the methodology is complex.

The end product would need to be designed to feed into an evidence to decision framework and an intervention effect review will still be required Food sovereignty, food security and health equity 63 64 Examples have examined health system complexity To understand the state of knowledge on relationships between health equity—ie, health inequalities that are socially produced—and food systems, where the concepts of 'food security' and 'food sovereignty' are prominent Focused on eight pathways to health in equity through the food system: 1 Multi-Scalar Environmental, Social Context; 2 Occupational Exposures; 3 Environmental Change; 4 Traditional Livelihoods, Cultural Continuity; 5 Intake of Contaminants; 6 Nutrition; 7 Social Determinants of Health; 8 Political, Economic and Regulatory context Meta-narrative Aim is to review research on diffusion of innovation to inform healthcare policy Which research or epistemic traditions have considered this broad topic area?

The originators are calling for meta-narrative reviews to be used in a guideline process. Potential to provide a contextual overview within which to interpret other types of reviews in a guideline process. Open in a separate window. Supplementary data bmjghsupp Taking a complexity perspective The first paper in this series 17 outlines aspects of complexity associated with complex interventions and health systems that can potentially be explored by different types of evidence, including synthesis of quantitative and qualitative evidence.

How can it be described? What are the main influences on the health problem? How are they created and maintained? How do these influences interconnect? Where might one intervene in the system? Quantitative: previous systematic reviews of the causes of the problem ; epidemiological studies eg, cohort studies examining risk factors of obesity ; network analysis studies showing the nature of social and other systems Qualitative data: theoretical papers; policy documents Interactions of interventions with context and adaptation For a research question about Implementation: How and why does the implementation of this intervention vary across contexts?

Qualitative: 1 eg, qualitative studies; case studies Quantitative: 2 trials or other effectiveness studies from different contexts; multicentre trials, with stratified reporting of findings; other quantitative studies that provide evidence of moderating effects of context System adaptivity how does the system change? How does the system change when the intervention is introduced? Which aspects of the system are affected? Does this potentiate or dampen its effects? Quantitative: longitudinal data; possibly historical data; effectiveness studies providing evidence of differential effects across different contexts; system modelling eg, agent-based modelling Qualitative: qualitative studies; case studies Emergent properties What are the effects anticipated and unanticipated which follow from this system change?