Claire Glenton
Systematic reviews of qualitative studies
C o c h r a n e N o r w a y / C o c h r a n e E P O C
N o r w e g i a n I n s t i t u t e o f P u b l i c H e a l t h
Have you….
Carried out primary qualitative research?
Carried out a systematic review of qualitative research?
Used results from qualitative research in a decision
making process?
What is the aim of qualitative research?
To understand people’s underlying reasons, opinions, motivations
To describe the social world
To explain the social world by developing
hypotheses, theories or models
Common methods
Focus groups
Individual interviews
Document analysis
(Participant) observations
What is a systematic review of qualitative research (qualitative evidence synthesis)?
Like primary qualitative research, qualitative evidence syntheses aim to:
understand people’s underlying reasons, opinions, motivations
describe the social world
explain the social world by developing hypotheses,
theories or models
Qualitative evidence syntheses increasingly common…
1995 2000 2005 2010 2015
1995: 10 publications
2000: 38 publications
2005: 245 publications
2010: 985 publications
2015: 3250 publications
Research about sickness and health – where does qualitative evidence fit in?
How many people have this health
condition?
(prevalence)
Why do some people get this condition while
others do not?
(etiology) How can we decide
if someone has this condition?
(diagnostics)
What happens to people who have
this condition?
(prognosis) How do people
experience this condition?
(attitudes and experiences)
What can we do to treat or prevent this
problem?
(effect of
interventions)
Healthcare guidelines –
where does qualitative evidence fit in?
Is the treatment effective and does it have side-effects?
How much will the treatment cost?
Will this treatment be acceptable to people?
Will this treatment be feasible to implement?
Using qualitative evidence in decision making Example: WHO Guidelines
• Antenatal care guidelines
• 39 recommendations
• Example: Group antenatal care versus individual care
• Systematic reviews commissioned:
• Effect: Review of trials
• Acceptability/Feasibility:
Review of qualitative research
What are the benefits and harms of the intervention?
Outcomes Individual ANC Group ANC Certainty of the evidence
(GRADE)
Comments
Preterm birth
105 per 1000 79 per 1000
(60 to 105) Moderate
Group ANC may reduce preterm birth. However, the CI includes no difference
Low
birthweight
89 per 1000 82 per 1000
(60 to 109) Moderate
Group ANC probably has little or no effect on birth weight
Perinatal mortality
21 per 1000 14 per 1000
(7 to 27) Low
Group ANC may have little or no effect on perinatal mortality Women’s
satisfaction Moderate
Group ANC probably leads to higher satisfaction
Spontaneous vaginal birth
606 per 1000 582 per 1000
(485 to 697) High
Group ANC does not have an important effect on spontaneous vaginal birth
Catling et al, 2015
Is the intervention acceptable?
Evidence from high-income settings:
Most women enjoy the group format – use it to build socially supportive relationships
Most women appreciate the
additional time, but some women don't attend because of it
Some women have reservations about the lack of privacy during the group sessions, particularly during physical examinations
Providers find group sessions to be enjoyable and a more efficient use of their time
No evidence from low or middle-income settings. Indirect evidence suggests that in rural areas of some LMICs where traditional beliefs restrict pregnancy exposure, the group approach may be inappropriate (moderate confidence)
Downe et al, 2015
Is the intervention feasible to implement?
Providers view the facilitative
component of group antenatal care as a skill that requires additional
training and provider commitment
Some providers also feel that clinics need to be better equipped to deliver group sessions, i.e. clinics need to have large enough rooms with
adequate seating
Downe et al, 2015
What did the WHO recommend?
We suggest considering the option only in specific circumstances
Group antenatal care should be offered as an alternative to
standard (individual) antenatal care for pregnant women
depending on a woman’s preferences and provided that the
infrastructure and resources for delivery of group care are
available
Implementation considerations
The following should be considered when implementing group antenatal care:
Group antenatal care may take longer than individual antenatal care, and this may pose practical problems for some women in terms of work and childcare.
Women should be offered a variety of time slots for group sessions (morning, afternoon, evening) and should consider making individual care available as well (especially for women with complications in pregnancy)
Women’s need for privacy should be considered. A private space should be made available for physical examinations, and opportunities should be given for private conversations
Healthcare providers and their supervisors need to receive appropriate initial and refresher/booster training in group facilitation and communication
Pre-service training institutions and professional bodies should also be informed and involved so that training curricula and supervision guidelines are updated
Healthcare providers need to be have appropriate facilities to deal with group sessions, including access to large, well-ventilated rooms, or sheltered spaces and adequate seating
…..etc….
How do qualitative evidence syntheses differ from reviews of effectiveness?
Systematic search for all relevant qualitative studies
Data extraction and quality assessment of included studies
Synthesis of the
results of these
studies
Exercises
Carrying out a qualitative evidence synthesis
Stage 1: Formulating the question
Stage 2: Searching for the evidence
Stage 3: Critically appraising the evidence
Stage 4: Synthesising the evidence
Stage 5: Assessing confidence in the findings
New WHO Guidelines
Digital strategies for
reproductive, maternal,
newborn, child and adolescent
health
New WHO Guidelines
Recommendations about the use of:
Telemedicine
Digital tools for provider training
Digital decision support tools
Digital registration and tracking of client health records
Digital tracking of commodities
Digital birth or death notification
Etc
New WHO Guidelines
How effective are these
interventions for people’s health status and use of health services?
What resources will they require?
How acceptable are they to clients and health workers?
How feasible are they to implement at scale?
Reviews commissioned to answer
these questions
Exercise
Develop a systematic review of qualitative research
(“qualitative evidence synthesis”) to explore health
worker acceptability of digital health interventions
Stage 1: Formulating the question
SPICE:
Setting (Where? in what context?)
Population or Perspective (For whom?)
Intervention (What?)
Comparison (What else?)
Evaluation (How well? What result?)
SPIDER (Sample, Phenomenon of Interest, Design,
Evaluation, Research type).
Setting:
Population:
Intervention:
Comparison:
Evaluation:
Global
Health workers (in RMNCH?) Digital health interventions
(Usual practice) Acceptability
Stage 1: Formulating the question
Review already underway
How did the authors formulate the question?
Setting: Global
Population: Health workers in primary care Intervention: mHealth technologies
Comparison: not mentioned
Evaluation: perceptions and experiences
Stage 2: Searching for the evidence
Decisions need to be made about:
Databases
Methods filters
“Sibling” versus “non-sibling” studies
Language
Date of publication
Where did the authors search?
Stage 2: Searching for the evidence - Sampling
Effect reviews: Important to identify and include all studies
Qualitative research: Too much data can threaten the quality of the analysis
Sampling – map all relevant studies and then select -
still not clear which approach is best – but sampling
framework must address the review objective
How did the authors sample?
Stage 3: Critically appraising the evidence
No consensus (e.g. RoB tool for trials)
CAMELOT project
Critical Appraisal Skills Programme (CASP) tool
Minimum quality requirement: Must have
qualitative data collection AND qualitative data
analysis
Stage 3: Critically appraising the evidence
Stage 4: Synthesising the evidence
– many different names and approaches
Different approaches transform the data to different extent. They are more or less….
configurative aggregative
descriptive explanatory
Approaches that are primarily aggregative*…..
Here, review authors “add up”
the data from primary studies to answer a review question
(Thanks to Karin Hannes, Leuven University, for slide concept)
*Gough D; Thomas J; Oliver S (2012) Clarifying differences between review designs and methods. Systematic Reviews. 1(28)
and approaches that are primarily configurative*
Here, review authors arrange the data from primary studies to answer the review question
(Thanks to Karin Hannes, Leuven University, for slide concept)
*Gough D; Thomas J; Oliver S (2012) Clarifying differences between review designs and methods. Systematic Reviews. 1(28)
Degree of transformation
Different degrees of transformation
Varying degrees of transformation: Example
How do children in
institutions experience staffing arrangements?
What are their attitudes to long and short shifts?
Munthe-Kaas, HM, Hammerstrøm, KT, Kurtze, N, Nordlund, KR. Effekt av og erfaringer med kontinuitetsfremmende tiltak i barnevernsinstitusjoner. Rapport fra Kunnskapssenteret nr. 4 – 2013.
Varying degrees of transformation: Example
Aggregative / Descriptive:
Most children preferred staff to have week-long shifts because they liked the stability and
structure and the opportunity to form attachment. Children in one study preferred short shifts, but these children had poor
relationships with their caregivers. In one study the experiences of the children were unclear.
Configurative / Explanatory:
In situations where children have good relations with their caregivers, they prefer longer shifts because these provide stability and structure and opportunities to form
attachment.
Different approaches are also more or less….
deductive inductive
Approaches that are primarily deductive…
Review authors already have a theory or framework that they use to organise their data
(testing this theory in the
process)
…and approaches that are primarily inductive
Review authors build a
theory or framework using
the data they have
Stage 4: Synthesising the evidence
The review authors planned to identify an existing theoretical framework; extract and cod the data according to this framework;
and revise the framework where necessary.
(However, they ended up using an
inductive approach – a “thematic
synthesis”, instead…)
Exercise
You have been provided with data from different studies that the review authors have extracted. The authors have decided that this data belongs in the same category
Read through the data
How would you synthesise this data? Try and create
a finding
Data extracts
Chang 2011 ‘‘The phones have greatly helped us because when they give drugs [ART]…the patient kind of gets mad, funny dreams and starts developing nightmares. I have even found a patient tied up by ropes in the room. They called me at my home in the night at 2:00 am, they told me that the patient is tied….The next day when it came to about 9:00 am when the health workers have reached the office I made a phone call. I told them that the patient at this
number has run mad. The medical staff…called me and told me to first withdraw the patient from that type of drug and get him on Nevirapine.’’ (PHW, Focus Group).
Hampshire
2016 Finally, being permanently reachable by phone can carry significant time and emotional burdens. Almost all interviewees had become accustomed to receiving patients’ calls out of working hours, sometimes late at night:
Sometimes calls from patients can be a bother. When you close from work very tired and want to rest a little, patients will be calling. Sometimes they are asking for information that you have already given but they might have forgotten. [. . .] Sometimes too they call even as late as midnight when you are tired and want to sleep. (Ghana, male clinic-based nurse, urban) Sometimes, clients call at odd hours. Last year a patient called me at 5 am. I was deeply asleep when she called and said she was having menstrual pains and did not know what to do. (Ghana, female clinic-based nurse, urban) Only one interviewee—a Malawian HSA—had stopped giving out his personal number to community members so freely, ‘to avoid them bothering me with trivial issues or issues that are not related to health’. However, his attempts were thwarted because other people continued to pass on his number.
Huq 2014 “When a mother called at late night it disturbed me. But if I switched off the mobile at night, I feel uncomfortable. A mother would fail to inform me if she was in a state of complication at night and she would suffer. So I always put my mobile in a switched on mode”
Jennings
2013 CHWs also indicated that clients often called or sent SMS texts to them in order to reach a nurse for help. “As for me, I call and also send SMS. Some people send ‘please call me’ [SMS texts] because they have something to say but do not have the airtime to call.”
