Studies were reviewed to identify sources of potential bias including selection bias, detection bias, performance bias, attrition bias, and reporting bias.
Selection bias refers to the systematic differences between baseline characteristics of the different treatment groups. It was determined by examining sequence generation, allocation concealment, and baseline imbalances.
Sequence generation
All studies were reviewed to determine how patients were allocated to interventions.
Ideally all studies would have some predetermined method to randomly assign patients to the study groups. However, several studies in this review did not have the opportunity to randomly assign patients to a specific study group (treatment). By design, they reported data before-and-after the implementation of a new service level in the area of study, in order to assess the effectiveness of the new service.
Other studies reported a change in the treatment regime which took place on a limited basis within their area of coverage. These studies had the possibility to assign subjects randomly into different treatment groups. However, it was noted in these studies that the possibility to include the subject into the predetermined randomized care group was not always possible if the allotted care provider group was already dispatched to another incident, or were too far away to respond in a timely manner. Furthermore these studies did not report how frequently they were able to follow their randomization schedules. Overall, it was decided that studies which had this limitation could still be included in the final review, as
long as they noted this as a limitation in the study, and discussed the steps they took to limit this source of bias as much as possible.
Allocation concealment
When the studies did have a method of random assignment defined, it was necessary to assess the adherence level to the allocation plan.
Randomization schedules were predetermined in studies where different levels of EMS service were concurrently available. It was not always possible to adhere to the randomization schedule for ethical and logistical reasons. When the allocated service was too far away to respond in a timely manner, or was already engaged with a patient, the allocation table was not followed. Several studies mentioned this phenomenon as a limitation, and sited the ethical obligation to treat patients in a timely manner. However, none of the studies reported the level of adherence to the randomization schedule. It was determined that this type of selection bias was unavoidable, and that adhering to the schedule in this manner would not introduce a significant bias.
Baseline imbalance
Studies involving OHCA potentially use many different aspects to define the baselines of their study groups, since many studies investigate factors which may influence the primary outcome of this review (survival to hospital discharge). Articles also included data regarding the time to EMS arrival, time until CPR is started, time spent on the scene before attempting to transport the patient to a hospital, type of cardiac rhythm, presence of a shockable rhythm. Descriptive statistics such as age, gender, bystander CPR started, witnessed arrest, and type of cardiac rhythm were used to compare for baseline differences between different study groups.
For the purpose of this review, assessment of baseline similarity was sufficient when age and gender were evaluated. No significant differences were observed in any of the included studies.
Detection bias refers to systematic differences between groups in how outcomes are determined. For this type of primary endpoint it would appear that there is very little possibility for any discrepancy, since the patient either died or was discharged from the hospital alive. However, there are differences in the underlying criteria for when EMS
care is or is not provided, which in turn can influence the outcomes. Areas which may differ from study to study include the period of time a patient with a witnessed arrest remains untreated before EMS assistance arrives, and the initial cardiac rhythm upon arrival of EMS.
These cut-off times are determined by the practice of the local area or state. There is not always worldwide and timely agreement about what consists the best medical practice in all situations. For the purpose of this review, studies which clearly defined these underlying criteria were rated as “low” risk for bias, while studies which did not clearly or completely define these were marked as “uncertain” risk for bias.
Performance bias refers to the measures used to blind the study participants and care providers from knowing which treatment a patient received, and whether or not the desired blinding methods were effective. It is not possible to blind the patients from which kind of care they received in this instance. Neither is it possible to blind the care providers on what level of care they will provide. They provide care based solely on the need of the patient.
The only area where some level of blinding could have taken place is in the dispatch of services area. Here some studies did have randomization tables defining which type of EMS service should be dispatched. However, it was not always possible to follow these schedules when the allocated service was not in range, or was already engaged with another patient. It was felt that there is a low risk of this type of bias. Studies which did not try to address or discuss this issue were listed as “uncertain” level of risk.
Attrition bias refers to the completeness of data, regarding the outcomes. In this review there is only one endpoint, which is survival to hospital discharge. It is important to know what exclusions were made in each study, and the reasons for exclusion.
Assessment of incomplete outcome data.
The completeness of the data being reported was assessed for every included study. All included articles described the overall number of included/excluded patients, and described the reasons for exclusions. Sighted reasons for excluding patients from studies included;
patients with traumatic injuries or who clearly had non-cardiac etiology, EMS did not attempt resuscitation due to illness or presence of Do Not Attempt Resuscitation (DNAR) order, no bystander resuscitation had been attempted for 15 minutes or longer before EMS
arrival, pediatric patients. One study also excluded pregnant women, and cardiac arrests that were witnessed by emergency medical personnel (Mitchell 2000). This study doesn’t state how many pregnant women were excluded. Even lacking this description, it was considered to have a “low risk” for bias, since it is highly unlikely that there were a large number of pregnant women excluded, and their exclusion would have been for reasons other than to influence the true outcome.
Reporting bias refers to the possibility of selecting only specific data to report, instead of reporting all of the data which was listed in the study design or protocol.
Assessment of Selective outcome data.
Only studies which listed the primary outcome as one of the study objectives, and also included said data were included into the review. It was checked in each article that the complete data set regarding the primary outcome was described, and that it was described in the manner stated in the objectives. In one study there was a subset of patients who were treated by more than one type of EMS service, and they were later excluded from the study results. In an attempt to limit any bias from this, the number of patients in this subcategory, and the rationale for receiving combined treatments were stated in the article.