You are viewing the site in preview mode

Skip to content

Advertisement

  • Research article
  • Open Access
  • Open Peer Review

Evaluation of a novel vital sign device to reduce maternal mortality and morbidity in low-resource settings: a mixed method feasibility study for the CRADLE-3 trial

  • 1Email authorView ORCID ID profile,
  • 1,
  • 1,
  • 1,
  • 2,
  • 4,
  • 3,
  • 3,
  • 5,
  • 3,
  • 1,
  • 1,
  • 1 and
BMC Pregnancy and Childbirth201818:115

https://doi.org/10.1186/s12884-018-1737-x

  • Received: 4 January 2018
  • Accepted: 9 April 2018
  • Published:
Open Peer Review reports

Abstract

Background

The CRADLE-3 trial is a stepped-wedge randomised controlled trial aiming to reduce maternal mortality and morbidity by implementing a novel vital sign device (CRADLE Vital Sign Alert) and training package into routine maternity care in 10 low-income sites. The MRC Guidance on complex interventions proposes that interventions and implementation strategies be shaped by early phase piloting and development work. We present the findings of a three-month mixed-methodology feasibility study for this trial, describe how this was informed by the MRC guidance and the study design was refined.

Methods

The fidelity, dose, feasibility and acceptability of implementation and training materials were assessed in three representative non-trial sites (Zimbabwe, Ethiopia, India) using multiple-choice questionnaires, evaluation of clinical management (action log), healthcare provider (HCP) semi-structured interviews and focus groups 4–10 weeks after implementation. Simultaneously, the 10 sites included in the main trial (eight countries) collected primary outcome data to inform the power calculation and randomisation allocation and assess the feasibility of data collection.

Results

The package was implemented with high fidelity (85% of HCP trained, n = 204). The questionnaires indicated a good understanding of device use with 75% of participants scoring > 75% (n = 97; 90% of those distributed). Action logs were inconsistently completed but indicated that the majority of HCP responded appropriately to abnormal results. From 18 HCP interviews and two focus groups it was widely reported that the intervention improved capacity to make clinical decisions, escalate care and make appropriate referrals. Nine of the ten main trial sites achieved ethical approval for pilot data collection. Intensive care was an inconsistent marker of morbidity and stroke an infrequent outcome and therefore they were removed from the main trial composite outcome. Tools and methods of data collection were optimized and event rates used to inform randomisation.

Conclusions

This feasibility study demonstrates that the components of the intervention were acceptable, methods of implementing were successful and the main trial design would be feasible. Qualitative work identified key moderators that informed the main trial process evaluation. Changes to the training package, implementation strategy, study design and processes were identified to refine the implementation in the main trial.

Trial registration

ISRCTN41244132; Registered 24/11/2015.

Keywords

  • Complex intervention
  • Maternal mortality
  • Low resource
  • Feasibility
  • Pilot
  • Implementation

Background

Maternal mortality remains a challenge in the post-Millennium Development Goal (MDG) era, especially in low resource settings where in 2013, 800 women died every day from complications of pregnancy and childbirth [1]. The most common causes of maternal mortality worldwide are hypertensive disorders, obstetric haemorrhage, sepsis and abortion complications. There are simple interventions that are readily available to treat these conditions. However, in low-resource settings, delays in presenting to care, reaching care and receiving this care all contribute to high maternal mortality. Measurement of vital signs is the first step in recognising women at risk of deterioration and therefore in initiating life-saving treatments [2]. Despite this, access to accurate equipment to measure vital signs and adequate healthcare provider (HCP) training on escalation pathways are frequently lacking in low and middle income countries (LMIC) [3].

The CRADLE-3 trial aims to determine whether implementation of a novel vital sign alert (VSA) device into routine maternity care at both community and facility levels, will reduce a composite outcome of maternal mortality and morbidity in LMIC. This is a stepped wedge randomised controlled trial (RCT) based in seven low-income and one middle-income country over 20 months. The CRADLE VSA is a semi automated device that measures blood pressure, pulse and calculates the mothers risk of shock. It has been extensively validated for accuracy [4, 5] and usability [6] and the need defined in this context through field work and stakeholder engagement.

The trial development was informed by the Medical Research Council (MRC) guidance for complex interventions [7]. Although the stages of development and evaluation can take many different forms, it is recommended that key uncertainties in the design are systematically studied in a development phase. Procedures should be tested for their acceptability and the likely rates of recruitment estimated to inform sample size calculations. The guidance also states that a mixture of qualitative and quantitative methods is likely to be needed in order to understand barriers to participation and estimated responses. Guidance on how best to undertake this was provided by Moore et al. in 2015 [8].

Whilst the MRC guidance is heavily cited there are few published papers that consider its practical application during this stage especially taking into context the challenges of working in multiple LMIC. Prior to the CRADLE 3 trial start a mixed-methodology feasibility study was undertaken to finalise the intervention and implementation processes which were guided by the Expert Recommendations for Implementing Change (ERIC) project [9]. Potential causal mechanisms and contextual factors that may influence the success of the trial were also identified. A logic model was created to describe these components and to identify the key research questions that inform the process evaluation of the main trial [10]. Through presenting results of this feasibility study, we aim to provide a worked example of the application of the MRC guidance in finalising the subsequent trial protocol and process evaluation.

Method

The study took place over 3 months from November 2015 to January 2016 with a further 3 months to analyse and adapt the intervention and protocol prior to the trial start in April 2016. It consisted of three key objectives:
  1. 1.

    Exploration of the acceptability and feasibility of the CRADLE programme components and the development of its implementation strategies by a mixed-method evaluation in three non-trial sites representative of the 10 main trial clusters.

     
  2. 2.

    Collection of primary outcome data in the 10 main trial clusters in order to evaluate the methods of data collection, and assess factors related to the randomisation programme (number of deliveries per month) and the sample size calculation.

     
  3. 3.

    Utilise results to optimize final CRADLE 3 protocol including training materials and implementation strategy for main trial.

     

Setting

Implementation was undertaken in three areas representative of the main trial clusters. These were a convenience sample of facilities meeting the study’s inclusion criteria but geographically distant to avoid contamination of the main trial. All facilities approached agreed to participate. These were sites based around Ramdurg in Karnataka, India, Bishoftu in Ethiopia, and Masvingo in Zimbabwe. In accordance with the 10 main trial clusters (list in Additional file 1) these included one or more secondary or tertiary facilities that provided comprehensive emergency obstetric care and the surrounding primary care facilities that referred to these higher facilities and were urban or semi-urban.

Participants

The CRADLE VSA was incorporated into routine maternity care. All HCPs working in these services were eligible for training and all women identified as pregnant or within the 6 weeks post-partum period, presenting for antenatal, intrapartum or postpartum care within the three areas were eligible for inclusion.

Intervention

The intervention, described in accordance with the TiDieR guidance [11], included two key components. The Microlife CRADLE VSA is a novel device that accurately measures blood pressure and pulse [4, 5, 12] and calculates the pregnant mother’s risk of hypovolaemic or septic shock [13, 14]. It has been specifically developed to meet the World Health Organisation’s criteria for use in a low resource setting. A traffic light Early Warning System display alerts users to abnormalities in the vital signs results. The lights are triggered by standard thresholds of hypertension as well as by shock index (heart rate divided by systolic blood pressure) [13], as shown in Table 1. The CRADLE VSA was incorporated into routine maternity care.
Table 1

Thresholds that trigger the CRADLE VSA Early Warning System

Hypertension Thresholds

Light and Arrow Results

Category

Blood Pressure (mmHg)

RED LIGHT & UP ARROW

Severe hypertension

≥ 160 and / or ≥ 110

YELLOW LIGHT & UP ARROW

Hypertension

≥ 140 & ≤ 159 and / or ≥ 90 & ≤ 109

GREEN LIGHT

Normal

<  140 and < 90

Shock Index (SI) Thresholds

Light and Arrow Results

Category

Shock Index (HR / sBP)

RED LIGHT & DOWN ARROW

Severe shock

≥ 1.7

YELLOW LIGHT & DOWN ARROW

Shock

≥ 0.9 and < 1.7

GREEN LIGHT

Normal

<  0.9

Primary, secondary and tertiary facilities were allocated devices according to their delivery rate, staffing numbers and number of beds per ward. Pre-existing BP devices were removed from clinical areas, unless existing equipment has functionality designed for that area e.g. repeated automated measures in a high dependency area. This was supported with a CRADLE training package consisting of short animated film, interactive sessions, booklet and posters. There were two sets of training materials available, one for facility HCP and one for community HCP with very limited resources or no formal training.

The CRADLE package content covered:
  • How to use the CRADLE VSA.

  • Maintenance of the CRADLE VSA.

  • Basic overview of clinical assessment and management of pre-eclampsia/eclampsia and shock in relation to the traffic light alerts.

The local implementation team and research team delivered one-off interactive group training sessions lasting 2–4 h to local stakeholders and representative HCP from each of the clinical areas in the cluster. This included a presentation on the background of the CRADLE VSA and the importance of measuring vital signs in pregnancy. This was followed by a demonstration of the features and use of the CRADLE VSA and small group practice using the CRADLE VSA. Training finished with interactive clinical scenarios exploring the use of the CRADLE VSA with available guidance and resources.

Attendees were given training materials and CRADLE VSA devices to disseminate to their clinical areas. The implementation team attended each clinical area over the subsequent days (depending on cluster size) to support dissemination and hold interactive sessions.

The core components of the intervention (provision of the CRADLE VSA devices, animated films, posters and content of the training presentation) were standardized across all sites. The way the core components were delivered was adapted to meet the needs of the site. This intervention can be described as complex [7] because it comprises multiple interacting components that require considerable shift in the behavior of recipients. It is also being implemented at multiple levels of low-resource health services and will affect multiple outcome measures [7]. In the main trial, the intervention will be compared to routine maternity care with clinical assessment and management according to local guidelines. Vital signs measurement is normally undertaken with a variety of blood pressure devices where these are available, which are rarely validated in pregnancy.

Study design

For this feasibility study, the 10 clusters involved in the main trial collected the primary outcome as defined at that time: a composite of maternal mortality or major morbidity (one of maternal death, Intensive Care Unit admission (or predefined equivalent), eclampsia, stroke, or hysterectomy, with no double counting). Data were collected from existing healthcare facility registers, maternity records, maternal mortality reports and healthcare providers at the discretion of the local research team. Anonymised data were recorded onto a paper form and transferred by the research team onto a purpose-built online database (MedSciNet).

The components of the mixed methodology assessment of the acceptability and feasibility of the CRADLE programme components, undertaken in the three non-trial sites are shown in Table 2.
Table 2

Outcomes, method of measurement and time of measurement in the CRADLE-3 feasibility study

Outcome

Method of Measurement

Time of measurement

Feasibility

Fidelity

Duration of training

At training

Dose

Proportion of staff trained

At training

Number of facilities included

At training

Adaptations to fit context

Observation of training

At training

Understanding of training materials

Questionnaires (n = 30 each site)

Post training

Action Log of Clinical Practice (n = 30 each site)

For 1 month post training

Acceptability

Stakeholder Interviews (n = 5 each site)

3 months post implementation

Stakeholder Focus Group (n = 1 each site)

3 months post implementation

In order to explore potential mediators of action a purposive sample of HCP were requested to complete action logs of clinical practice to evaluate their referral practice. In these action logs, HCP were requested to record any “yellow” or “red” Early Warning System with the action they took, for example ‘administered anti-hypertensives and referred to hospital today’. This was compared to baseline information of available resources to interpret how HCP had understood and interpreted the training. Questionnaires assessing knowledge of when and how to use the CRADLE VSA and how to interpret the results were filled after training by a random sample of participants.

We undertook semi-structured interviews and focus groups (guides available on request) 4–10 weeks after implementation. This time was selected to allow sufficient experience of the VSA in clinical practice whilst allowing time to make improvements to the intervention within the funding timeline. The aim of these was to further knowledge on the acceptability of the device features gained from a previous observational study of the CRADLE VSA (CRADLE 2) [15] by exploring the feasibility of implementing the CRADLE package and how incorporating it into routine care might change behaviour. Participants were selected through purposive sampling to ensure representation of different HCP cadres in hospital and community care. Participants were approached face-to-face and gave written informed consent to participate in the qualitative study. The interviews were recorded and transcribed verbatim for further analysis and field notes were recorded. Snowball sampling was utilised until data saturation was achieved in each site. Two data coders that were independent to the interviewers undertook analysis using QSR NVivo 11 software (QRS, Vic, Australia). We used the framework method with a coding framework that draws upon the study objectives, logic model and interview guide [16, 17]. New concepts initiated by participants that could not be categorized within the initial scheme were coded using an inductive approach and added to the framework. The COnsolidated criteria for REporting Qualitative research (COREQ) Checklist was used to report the methodology and findings [18].

Results

Development of trial tools

We developed the training materials and data collection tools based on the experiences of the previous CRADLE projects [4, 5, 1214]. It was necessary to create a training package that could be disseminated quickly in multiple contexts and easily understood by every cadre of HCP with different resources available. Therefore, we selected animated film as the main component so that voice overs could be translated at minimal additional cost and it could be disseminated widely onto smart phones by Bluetooth. Feedback from end-users, health service managers and implementation experts was incorporated.

We created a logic model to depict the components of the intervention, how they may interact to produce change and what the anticipated outcome of that change may be (Fig. 1) [10, 19]. This was developed based on the evidence described in the introduction with input of the research team and key stakeholders. These included representative healthcare professionals and service managers from our low-income trial sites and implementation experts. The moderating factors and anticipated changes to practice in the logic model were utilized to inform the process evaluation outcome measures.
Fig. 1
Fig. 1

Logic model for the CRADLE-3 intervention

Feasibility of implementation

The intervention was implemented in three regional hospitals, one from each site, together with 61 associated primary healthcare facilities. The proportion of staff trained and the duration of training is shown in Table 3. In India and Ethiopia, we translated the training materials and in India the graphics of the poster and booklets were altered to be more culturally appropriate. In all countries the guidance for management of pre-eclampsia, haemorrhage and shock, which was based on WHO guidelines [20, 21], was adapted to incorporate locally available resources and infrastructure.
Table 3

Fidelity and dose of implementation

Study Site

Number of primary health care facilities involved

Proportion of HCP trained

Duration of training

Devices Distributed

Masvingo, Zimbabwe

21

90% (n = 92)

8 days

62

Bishoftu, Ethiopia

3

85% (n = 37)

3 days

29

Ramadurg, India

37

95% (n = 75)

2 days

53

In total, 108 questionnaires were distributed and 97 fully completed by HCP and analysed, 16% by HCP working in primary level facilities and 84% in secondary level facilities. This represents 48% of those trained. Of these, five were doctors, 78 were nurses and midwives and 15 were allied HCP. The majority of respondents had more than 5 years of service (1–5 years n = 28, 29%; 6–10 years n = 35, 36%; > 10 years n = 34, 35%). Results indicated a good understanding use of the VSA and interpretation of results with 73 participants (75%) scoring more than 75% correct answers. HCP working in secondary care scored higher in questions on how to act in response to abnormal vital signs than HCP working in primary care (74% (n = 60) scoring over 75% correct compared to 56% (n = 9) of clinic participants.

Action logs were distributed to 90 HCP and completed by 68 (76%). The level of detail completed was diverse with many participants failing to complete details on medication given or the time frame taken for action. This was reported to be due to the additional burden of work required to complete these logs and this is explored further in the discussion. Analysis of these records was therefore challenging and results should be interpreted with caution. It was possible to determine that in total, 62 red lights were recorded, indicating severely abnormal vital signs. The majority (n = 50, 81%) had optimal care appropriate for the available resources. Of the 12 that did not, five were referred to hospital but within a longer timeframe and seven were reported to look well so no further action was taken.

The action logs demonstrated varying practice in response to a yellow light with a downward pointing arrow. This result indicates a relative low blood pressure compared to the heart rate. In Ramadurg, India, the majority of these patients were referred to higher level care, compared to fewer in Bishoftu, Ethiopia. This variance highlighted ambiguity in the training materials. This was explored widely with stakeholders and changes made to the training materials accordingly. A yellow down light may be caused by multiple clinical causes of varying importance (such as maternal anaemia which can be common in pregnancy) and therefore greater emphasis was made on assessment of the patient’s clinical condition in the final training materials.

Acceptability of the intervention and implementation strategies

A total of 18 interviews and two focus group were undertaken across the three sites in January and February 2016. Participants were approached face to face and provided written informed consent. The participants had limited prior relationship with the researchers from training and outreach support visits. Individual interviews were undertaken with nine nurses, five midwives, one maternity manager, two health assistants and one doctor. The median age of participants was 25–34 years (n = 12) (18–24, n = 1; 34–44, n = 3; 45–54, n-2), the most common number of years of service was 1–5 (n = 8) (6–10 years, n = 5; 11–15 years, n = 2; > 15, n = 3). Findings are presented under themes below. Interviews and focus groups were undertaken in private, quiet locations near the participant’s place of work with no non-participant observers to ensure confidentiality and allow for an open discussion. Sessions lasted between 15 and 75 min and were undertaken, translated and transcribed by experienced local research coordinators (clinical background) following training from the trial coordinator and senior social scientist (JS).

Use of CRADLE VSA in clinical decision making

It was widely reported that the CRADLE VSA package improved capacity to make decisions about which women require treatment and referral. This was true for HCP in the community who initiated referral as well as those working in facilities who initiated treatment or escalated care to senior staff. The suggested reasons for this were varied. HCP who were previously using auscultatory BP devices reported lack of confidence in accurately auscultating the Korotkoff sounds and therefore reported regularly rounding the BP to the nearest ten or delaying taking action. This was alleviated by the digital display of systolic and diastolic blood pressure and pulse on the CRADLE VSA.

“There, if we get high BP, we used to think and get the BP checked by others, as we had less confidence in that machine. We used to ask others saying that it has shown high BP in our machine and we want to check by you also. Here with this device (CRADLE VSA) we can take decision immediately.” Staff Nurse, Hospital, India

HCP also reported that the Early Warning System alerted them to abnormalities in the results which may otherwise not be recognised. This was reported to be mainly because in pressured environments with fatigued staff abnormal results may be missed whereas a red light is an instantly recognisable alarm. This, in combination with the training materials was felt to provide valuable guidance in the action required to manage abnormalities. The majority of HCP reported that this has improved the time taken to recognise pregnancy complications and initiate treatment or referral. These views are illustrated in the quote below:

“Yes, more interventions are done and done in a more prompt manner because of the indicators which one can easily refer to and take action promptly. Interventions are done early or immediately because of the indicators that alert the clinicians. They are forced to act quickly even if they refer, but they will have taken some action” Staff Nurse Masvingo, Zimbabwe

Whilst the majority of HCP felt that the results are accurate and the lights beneficial, there were two HCP who reported that they did not always act on the lights because they were confused. They felt that the yellow light was reported too frequently when patients appeared well. The quote below demonstrates this:

“But what we are observing is that for majority of women it is showing yellow with down arrow. But the patient is stable, normal, she is not sick and she is well. In such situations we feel “why it so? The woman is normal, but still why it is showing yellow down”?” Staff nurse, PHC, India.

Other HCP acknowledged that this was a common result but demonstrated understanding of potential causes and how to manage this result. This finding triangulates the results of the Action Log and is explored further in the discussion.

Aid of the CRADLE VSA in escalating care and referral

Nursing HCP reported that when they were previously using auscultatory devices they reported doubt or being challenged when referring cases to medical staff. The improved confidence in the vital signs measurement and the understanding conveyed by the training package was beneficial in escalating care to senior staff. This was also true in convincing women and their families of the need for further monitoring or referral.

“It is helpful in providing immediate care to the patient. As everything is given there (booklets and poster) like the patient has to be referred within 4 hours or within 1 week...etc., we call the attenders and convince them, to refer the patient at the earliest.” Staff Nurse, Gokak, India

The majority of HCP reported an increase in the number of vital sign abnormalities detected; however, the reported impact on the rate of referrals varied between countries. All participants from India reported an increase in the number of referrals. The number of hypertensive patients referred was increased as mild hypertension was considered more seriously. The number of patients with low blood pressure detected and referred was also noted to increase. In Zimbabwe and Ethiopia responses were mixed with some reporting a reduction in referrals due to improved understanding of vital signs and confidence in initiating treatment peripherally. A quote to illustrate this is:

“because of the indicators the referrals have decreased, now we know when to refer and we now don’t refer all patients for the sake of referring.” Nurse, Clinic, Zimbabwe

Use of results to finalise implementation strategies

The MRC Guidance and supporting documents highlight that the pilot data should be used to shape the intervention and implementation strategies. Following the completion of this feasibility study, the experiences of implementation and input from a stakeholder meeting across all sites in February 2016 were used to agree a number of changes prior to the main trial. The choice of implementation strategy was guided by the Expert Recommendations for Implementing Change project [9] with the aim of improving understanding of the training materials and overcoming identified barriers to use. Table 4 presents the key issues experienced through the feasibility study alongside quotes to support this and the implementation strategies or changes we have selected.
Table 4

Barriers to measuring the vital signs and using the VSA and actions made prior to the main trial

Issue from Feasibility study

Supporting quotes from Interviews and Focus Group

Changes required after feasibility study

Understanding of yellow light with down arrow

“When the pregnant woman comes to us she is well and it shows yellow down (laughs); in that case how to interpret?” Staff Nurse, India

- Training materials updated to explain the reason for each colour light.

- Guidance updated to place greater emphasis on how to assess the woman to decide if further action or referral is required.

Problems with charging the VSA

“… some workers say that the battery life is good but I have to charge it for daily. If I have to go for BP check-up today I have to charge it first.” Staff Nurse, India

- Interactive training session developed which incorporates guidance on charging and accountability for charging.

- Including explanation that more than 100 readings can be taken even when the battery low sign is showing and overcharging will damage the battery.

Provision of Charger

“the package should include the charger rather than USB cable only so that we can charge it easily” Staff Nurse, Zimbabwe

Chargers provided in addition to the cable that comes as standard in the package.

High staff turnover / incapacity to train all staff at once

“Initially, we faced difficulty. There were nurses using it during the night shift and they were not trained.” Staff Nurse, Bishoftu

Designated CRADLE Champions in each facility identified to provide ongoing local training and support for CRADLE VSA

Unsupportive seniors

“Recently during an ANC clinic he inquired which device we are using; we said “we are using the CRADLE device given by the research unit” then he said “oh… it shows yellow down to all pregnant women, so better do not use it”.” SN, India

- Designated CRADLE Champions identified to provide ongoing local training and support for CRADLE VSA

- Engage Local Opinion leaders prior to implementation

Need for equipment

“…no BP machines at all, the one we had was no longer working.” Staff Nurse, Zimbabwe

Ensure adequate supply of VSA available

A key implementation strategy that we selected as a result of the feasibility findings was the identification and training of CRADLE champions. These are clinical staff and local leaders who receive more in-depth training about device and are equipped with the tools to support the use of the VSA in their area. Whilst we wanted the trial to remain pragmatic and replicable, this strategy was chosen to support ongoing local training and build local capacity for maintenance. The importance of the support of the implementation team on the effectiveness of the intervention was also noted during the study and added to the logic model as a core component of the intervention. In order to maintain the pragmatic trial design, it was decided not to quantify this support but for the research team to observe it during the routine monthly site monitoring and be aware of its potential impact.

Main trial primary outcomes

As the main trial has a stepped wedge design with an intervention incorporated into routine care, cluster level consent instead of individual was appropriate. This was agreed as preferable by all sites during the development work. All ten trial sites gained ethical and local approval for participation in the main trial. Nine of them achieved this before or during the feasibility phase and were therefore able to commence data collection, with six sites completing a full 3 months of data collection. During this formative phase a total of 844 outcome events (any one of eclampsia, maternal death, hysterectomy, ICU admission or stroke) occurred in 681 women from nine of the ten sites. On average, the number of women experiencing events each month varied from five in Gokak, India to 63 in Zomba and South-Eastern Malawi (median 19.6, standard deviation 19.6). The most common outcome was eclampsia (n = 386, 46%) followed by ICU admission (n = 271, 32%), hysterectomy and maternal death (n = 96, 11%; n = 91, 11% respectively) and stroke (n = 2, 0.2%). The tools for data collection were found to be appropriate. Methods of data collection were discussed and optimised based on the existing resources available in each site. Outcomes were triangulated across multiple sources (including referral registers, ward registers, patient records, local mortality and morbidity records, active case finding) to ensure data completeness and all outcomes checked to avoid double counting.

These outcomes had been intended as unequivocal markers of severe maternal mortality and morbidity. However, during the process of data collection it became apparent that ICU admission was dependent on availability of services which varied greatly between clusters and therefore this was not a reliable proxy marker of maternal morbidity. The number of strokes reported across all sites was just two, both from one site. It was therefore prospectively decided that for the main trial the composite primary outcome would include only eclampsia, maternal death and emergency hysterectomy. Given that the size of clusters varies greatly, it was agreed that the number of deliveries would be counted as a surrogate denominator so that results for the main trial could be presented as an event rate per 10,000 deliveries. These prospective amendments were added to the trial registry (ISRCTN41244132).

Randomisation and power calculation

An estimated event rate from the data was used to inform the randomisation sequence and power calculation of the main trial. With so few centres to be randomised there was a risk of unbalance – for example that by chance the centres with the highest event rates might be allocated to the intervention at the end of the study, giving a false impression of a low event rate in centres which received the intervention early, and hence a biased estimate of the treatment effect. Although the planned analysis would correct for this, to further minimize the problem, a restricted form of randomisation was used, so that the rank correlation between the initial event rate, and the order of randomisation was zero. For logistic reasons, the two centres outside Africa: Cap Hatien (Haiti) and Gokak (India) were treated as one centre for randomisation (but not for analysis).

First the overall rate of the primary outcome was determined in the nine clusters and each cluster given a corresponding rank r between 1 and 9. Then a provisional list l of the order of intervention was determined for each cluster using computer-generated random numbers based on an arbitrary random number seed. Spearman’s rank correlation rho between r and l was determined. If the rank correlation was not equal to 0, the list was discarded, and the process repeated until an acceptable order was determined. The first acceptable randomly-generated list was retained. This resulted in a near-zero Pearson’s product-moment correlation of − 0.019 between the event rate and the order of randomisation.

Based on these data, our sample size estimation was carried out by the trial statistician, using Stata version 13.1 and the methods of Hemming and Girling [22]. For the purpose of the power calculation, an assumption that there are at least 4000 deliveries per cluster per month was made (or 8000 per cluster period of 2 months) and at least nine clusters, each observed for 20 months (ten time periods of 2 months each). Our feasibility data indicated a baseline event rate of 1% and we have anticipated a 25% reduction in this to 0.75% post intervention. We would require a total of 2450 outcome events with a coefficient of variation of 0.4 and an Intra Cluster Correlation of 0.0085, to have power of 95%.

Discussion

This paper aims to describe the mixed methodology feasibility study exploring the acceptability and feasibility of introducing the CRADLE package into routine maternity care in LMIC along with the feasibility of the main trial data collection processes. The key findings are that the intervention can be delivered with high fidelity and dose and incorporated into routine maternity care successfully in different contexts without major adaptation. In addition, we have confirmed that the methods of collecting the main outcome data are feasible and can be maintained for consistency and this data has successfully been used to improve the study design, randomisation and power calculation.

The MRC framework stresses the importance of quality development work to avoid problems with acceptability, compliance, recruitment and retention. Yet methodological research suggests that pilot studies are often poorly performed and few are published [23]. Most published examples of complex interventions demonstrate valuable learning from pilots; for example, the exploratory RCT of an intervention to reduce alcohol related harm demonstrated that whilst the trial was methodologically feasible, poor enthusiasm resulted in low fidelity thus the conclusion that the intervention would need to be enforced in future work [24]. However, published description of its use in the field of maternity and in LMIC is scarce.

One key learning from our feasibility study was the practicalities of a pragmatic process evaluation in a low-resource setting. The MRC supporting guidance provides explanation for each of the key dimensions of implementation that could be measured and guides researchers to select the most important questions to investigate. Whilst this is a strength of the framework as it ensures comprehensive evaluation of implementation, it provides little guidance on how to select suitable outcome measures for each dimension. This is especially challenging when working in a resource-poor environment with additional burden of high workload and heterogeneous routine data collection. In this feasibility study, measuring multiple components compared to baseline was not possible within the short time frame. The proportion of people trained centrally and time taken for training were selected as simple surrogates of fidelity and dose. However, in the qualitative work the majority of HCP reported the VSA was easy to use and it was possible to commence use just on reading the training materials so the extent that these measures may impact the effectiveness of the program is unknown. In the main trial the number of core components delivered in training and how they were adapted to context were added to the measure of fidelity.

The MRC Guidance advises using the logic model to identify causal assumptions and guide selection of research questions. In the CRADLE Logic model the assumed changes to practice include improving access to equipment therefore in the trial we will capture the proportion of HCP that have access to a working BP machine before and after implementation. A further assumed change is improving vital sign measurement and management of complications. In this feasibility study, the action log of referral practice was poorly received and completed due to the extra burden of work it demanded. For the main trial, we chose to explore change in practice during the qualitative work rather than during structured observation due to the large variety of normal practice in different countries. The qualitative work also highlighted that number of women referred to higher level care was a potential key moderator. Therefore, after stakeholder discussion across our sites we agreed to measure the proportion of women presenting for maternity care that had their blood pressure measured and the proportion that were referred to higher level care. Due to the intensity of this data collection it was agreed this would be measured for a period of 4 weeks before implementation and 4 weeks, 3 months after the device was introduced to allow time for familiarization of the VSA in routine maternity care. We found the process of identifying potential factors that may impact on the success of the trial and tailoring outcome measures accordingly was achievable for a team of predominantly newcomers to the field of implementation. It was also beneficial in assuring that assumptions are shared between stakeholders.

Strengths and limitations of the study

We describe the feasibility testing of the CRADLE intervention prior to a fully powered trial. This was guided by the MRC guidance and was undertaken over a 6-month period; results are therefore applicable to others working in restricted funding periods. Feasibility and pilot research is often skipped or inadequate to fully inform the main trial and rarely published to inform others researching similar areas. In a situation where a complete development phase is not possible due to the stepped wedge design we demonstrated that our event rate in the main trial sites is sufficient to have adequate power. We undertook formal qualitative analysis to explore the acceptability of the program components and feasibility of implementation. We did not collect cost-effectiveness data in the feasibility study but will collect this data for an economic evaluation in the full trial. We did not assess knowledge of vital signs measurement prior to training and therefore results should be interpreted with caution.

Conclusions

This feasibility study demonstrates that the components of the intervention were acceptable and the methods of implementing successful. Qualitative work identified key moderators that have informed the main trial process evaluation. Changes to the training package, implementation strategy, study design and processes were identified to optimize the main trial implementation. Carrying out these changes in practice, (for example to the data collection forms, database and graphics of the training materials were challenging) within the six-month allocated time frame of this feasibility study. If larger changes had been required, for example to the components of the intervention which would require re-testing, this would not have been possible. Funders recommending that a development phase is incorporated into a standard 3-year funding period might therefore take into consideration the time taken to undertake the work, analyse results and incorporate changes into the study to ensuring meaningful improvements can be made.

Concurrently assessing the feasibility of the trial outcomes in the 10 main trial sites and the implementation processes in geographically distant sites was necessary to avoid contamination of the trial area. Whilst undertaking this in multiple sites required additional work, it allowed for greater understanding of how the intervention could be adapted in each context and how the main trial will run. The MRC Guidance and supporting documents provided a valuable tool to guide the overall design of this study and the development of the process evaluation for the main trial. This study presents a real world worked example that has utilised this guidance to refine the intervention, main trial outcomes and process measures.

Abbreviations

COREQ: 

COnsolidated criteria for reporting qualitative research

ERIC: 

Expert recommendations for implementing change

HCP: 

Health care provider

LMIC: 

Low and middle income countrie

MDG: 

Millennium development goal

MRC: 

Medical research council

RCT: 

Randomised controlled trial

SI: 

Shock index

VSA: 

Vital sign alert

WHO: 

World Health Organisation

Declarations

Acknowledgements

John Hopkins Public Health and implementation department, JHPIEGO, contributed $10,000 towards the development of the training videos and provided academic input and review during their development. We would like to acknowledge all authors of the CRADLE Trial Collaborative Group for their contribution to this work: Kate E Duhig, King’s College London, UK; Natasha L Hezelgrave, King’s College London, UK; Umesh Charantimath, KLE University’s J N Medical College, India; Chandrappa C Karadiguddi, KLE University’s J N Medical College, India;Sphoorthi S Mastiholi, KLE University’s J N Medical College, India; Geetanjali M Mungarwadi, KLE University’s J N Medical College, India; Feiruz Surur, St Paul’s Millennium Medical School,Ethiopia; Lomi Yadeta, Maternity Worldwide, Ethiopia; Yonas Guchale, Maternity Worldwide, Ethiopia; Violet Mambo, University of Zimbabwe, Zimbabwe; Sebastian Chinkoyo, Ndola Teaching Hospital, Zambia; Thokozile Musonda, Ndola Teaching Hospital, Zambia; Christine Jere, Ndola Teaching Hospital, Zambia; Bellington Vwalika, University of Zambia, Zambia; Mercy Kopeka, University Teaching Hospital, Zambia; Martina Chima, University Teaching Hospital, Zambia; Josephine Miti, University Teaching Hospital, Zambia; Rebecca Best, Welbodi Partnership, Sierra Leone; Matthew Clarke, Welbodi Partnership, Sierra Leone; Jesse Kamara, Welbodi Partnership, Sierra Leone; Jeneba Conteh, Princess Christian Maternity Hospital, Sierra Leone; Patricia Sandi, Princess Christian Maternity Hospital, Sierra Leone; Margaret Sesay, Princess Christian Maternity Hospital, Sierra Leone; Fatmata Momodou, Princess Christian Maternity Hospital, Sierra Leone; Julius Wandabwa, Busitema University, Uganda; James Ditai, Sanyu Africa Research Institute, Uganda; Nathan Mackayi Odeke, Sanyu Africa Research Institute, Uganda; Annettee Nakimuli, Makerere University College of Health Sciences, Uganda; Josaphat Byamugisha, Makerere University College of Health Sciences, Uganda; Dorothy Namakula, Mulago National Referral Hospital, Uganda; Noela Kalyowa, Mulago National Referral Hospital, Uganda; Doreen Birungi, Mulago National Referral Hospital, Uganda; Emily Nakirijja, Mulago National Referral Hospital, Uganda; Carwyn Hill, Hope Health Action, Haiti; Grace Greene, Hope Health Action, Haiti; Adeline Vixama, Hope Health Action, Haiti; Paul Toussaint, Hospital Convention Baptiste d’Haiti, Haiti; Grace Makonyola, Maternity Worldwide, Malawi; Doreen Bukani, Maternity Worldwide, Malawi; Monice Kachinjika, Maternity Worldwide, Malawi; Jane Makwakwa, Maternity Worldwide, Malawi.

Jane Sandall and Paul Seed were supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South London at King’s College Hospital NHS Foundation Trust. Lucy Chappell is supported by the National Institute for Health Research Professorship, (RP-2014-05-019). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Funding

This study is funded by Medical Research Council (UK), Department for International Development (UK) and Department of Biotechnology (India) through the Department for International Development (UK) Global Research Programme: Addressing the health needs of women and children in the most disadvantaged populations globally. Neither funding body had any role in the design of the study or collection, analysis, and interpretation of data or in writing the manuscripts.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

AS is primary investigator for the CRADLE-3 trial and has academic oversight in all actions. SG and FG are co-PIs for India and low –resource settings respectively. JS is senior social scientist who has supported NV and EL to design and undertake the process evaluation and qualitative components of this study. PS as trial statistician has undertaken all statistical work. AB, TM, BM and UC have overseen data collection in their sites. NV as trial coordinator has overseen all data collection, created training package and data collection tools with assistance from EL. LC and HN are co-investigators supporting trial management. All authors have contributed to the writing of this paper.

Ethics approval and consent to participate

Ethical approval has been granted by the Biomedical Sciences, Dentistry, Medicine and Natural & Mathematical Sciences Research Ethics Subcommittee at King’s College London (LRS-14/15–1484). Local ethical approval from each cluster has been granted by:
  • Ethiopian Public Health Institute, Ethiopia: EPHI6.4/185

  • K.L.E Society’s Jawaharlal Nehru Medical College, Belgaum, India: MDC/IECHSR/2015–16/A-59

  • Medical Research Council of Zimbabwe; Zimbabwe: MRCZ/A/1999

Institutional level consent was sought prior to implementation of the CRADLE intervention. Informed written consent was sought from HCP participating in interviews, focus group discussions and the action logs of clinical practice.

Competing interests

The CRADLE VSA has been developed by Microlife with academic input from King’s College London and funding from Bill and Melinda Gates. The device is provided at manufacturing cost for this trial and is available to purchase at low cost to all LMIC. King’s College London receive no funds from the sale of devices to LMIC.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Women and Children’s Health, School of Life Course Sciences, Faculty of Life Sciences and Medicine, King’s College London, London, SE1 7EH, UK
(2)
Maternity Worldwide Community Base, 113 Queens Road, Brighton, BN1 3XG, UK
(3)
Women’s and Children’s Health Research Unit, KLE Academy of Higher Education and Research Jawaharlal Nehru Medical College, Belgaum, Karnataka, 590010, India
(4)
Morgenster Mission Hospital, Masvingo, Zimbabwe
(5)
Department of Obstetrics and Gynaecology, College of Health Sciences, University of Zimbabwe, Harare, Zimbabwe

References

  1. World Health Organisation. Trends in maternal mortality: 1990 to 2013. Estimates by WHO, UNICEF, UNFPA, the World Bank and the United Nations population division. 2015 http://apps.who.int/iris/bitstream/10665/194254/1/9789241565141_eng.pdf?ua=1. [Accessed 20th November 2017].
  2. World Health Organisation. The world health report 2005: make every mother and child count, vol. 2005. Geneva: World Health Organization.Google Scholar
  3. Baker EC, van der Westhuizen A, Duhig KE, Shennan A. Can a low-resource-use blood pressure device impact on global maternal mortality? Hypertension in pregnancy : official journal of the International Society for the Study of Hypertension in Pregnancy. 2011;30(3):359–63.View ArticleGoogle Scholar
  4. Nathan HL, de Greeff A, Hezelgrave NL, Chappell LC, Shennan AH. An accurate semiautomated oscillometric blood pressure device for use in pregnancy (including pre-eclampsia) in a low-income and middle-income country population: the microlife 3AS1-2. Blood Press Monit. 2015;20(1):52–5.View ArticlePubMedGoogle Scholar
  5. Nathan HL, de Greeff A, Hezelgrave NL, Chappell LC, Shennan AH. Accuracy validation of the microlife 3AS1-2 blood pressure device in a pregnant population with low blood pressure. Blood Press Monit. 2015;20(5):299–302.View ArticlePubMedGoogle Scholar
  6. Nathan HL, Boene H, Munguambe K, Sevene E, Akeju D, Adetoro OO, et al. The CRADLE vital signs alert: qualitative evaluation of a novel device designed for use in pregnancy by healthcare workers in low-resource settings. Reprod Health. 2018;15(1):5.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10(1):21.View ArticlePubMedPubMed CentralGoogle Scholar
  10. W.K. Kellogg Foundation. Logic Model Development Guide: Using logic models to bring together planning, evaluation and action. 2004 https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide. [Accessed 20th November 2017].
  11. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.View ArticlePubMedGoogle Scholar
  12. de Greeff A, Nathan H, Stafford N, Liu B, Shennan AH. Development of an accurate oscillometric blood pressure device for low resource settings. England: Blood Press Monit. 13; 2008. p. 342–8.Google Scholar
  13. Nathan HL, El Ayadi A, Hezelgrave NL, Seed P, Butrick E, Miller S, et al. Shock index: an effective predictor of outcome in postpartum haemorrhage? BJOG. 2015;122(2):268–75.View ArticlePubMedGoogle Scholar
  14. El Ayadi AM, Nathan HL, Seed PT, Butrick EA, Hezelgrave NL, Shennan AH, et al. Vital sign prediction of adverse maternal outcomes in women with hypovolemic shock: the role of shock index. PLoS One. 2016;11(2):e0148729.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Nathan HL, Boene H, Munguambe K, Sevene E, Akeju D, Adetoro OO, Charanthimath U, Bellad MB, de Greeff A, Anthony J, Hall DR, Steyn W, Vidler M, von Dadelszen P, Chappell LC, Sandall J, Shennan AH, and the CLIP Working Group. The CRADLE vital signs alert: qualitative evaluation of a novel device designed for use in pregnancy by healthcare workers in low-resource settings. BMC Reproductive Health. 2018;15(1):5.Google Scholar
  16. Ritchie JLJ. Qualitative research practice: a guide for social science students and researchers. London: Sage; 2003.Google Scholar
  17. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.View ArticlePubMedGoogle Scholar
  19. Kirby D. BDI logic models: a useful tool for designing, strengthening and evaluating programs to reduce adolescent sexual risk taking, pregnancy, HIV and other STDs. 2004 http://recapp.etr.org/recapp/documents/BDILOGICMODEL20030924.pdf. [Accessed 14th June 2017].
  20. World Health Organisation. WHO recommendations for prevention and treatment of pre-eclampsia and eclampsia. 2011 http://apps.who.int/iris/bitstream/10665/44703/1/9789241548335_eng.pdf. [Accessed 3rd April 2017].
  21. World Health Organisation. WHO recommendations for the prevention and treatment of postpartum haemorrhage. 2012 http://apps.who.int/iris/bitstream/10665/75411/1/9789241548502_eng.pdf. [Accessed 30th October 2017].
  22. Hemming K, Girling A. A menu-driven facility for power and detectable-difference calculations in stepped-wedge cluster-randomized trials. Stata J. 2014;14(2):363–80.Google Scholar
  23. Eldridge SM, Ashby D, Feder GS, Rudnicka AR, Ukoumunne OC. Lessons for cluster randomized trials in the twenty-first century: a systematic review of trials in primary care. Clinical Trials. 2004;1(1):80–90.View ArticlePubMedGoogle Scholar
  24. Moore SC, Murphy S, Moore SN, Brennan I, Byrne E, Shepherd J, et al. An exploratory randomised controlled trial of a premises-level intervention to reduce alcohol-related harm including violence in the United Kingdom. BMC Public Health. 2012;12:412.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement