Introduction

Sedation, analgesia and delirium management are key elements of ICU care influencing patients’ outcome. Numerous studies have proved influence of sedation management, patients’ pain levels or incidence of delirium on patients’ outcome [1–5]. These are associated with prolonged mechanical ventilation, extended length of ICU stay (LOS), higher morbidity and mortality and increased costs [1–4].

Scoring systems to monitor the use of sedatives and analgesics and to screen for delirium in the ICU are recommended in current national and international guidelines [5]. Martin et al. [6] reported that only 8 % of the German ICUs used sedation scores prior to the publication of the German S2e Guidelines which recommend monitoring of sedation, pain and delirium using validated scoring systems every 8 h.

For our implementation protocol we integrated five principle educational techniques identified by Landry and Sibbald [7]:

  • academic detailing,

  • audit and feedback,

  • reminder systems,

  • local opinion leaders and

  • printed material.

The aim of this study is to compare the effectiveness of two training strategies (a modified, extended approach compared to our regular, standard training approach) on the implementation rate of scoring instruments in an ICU.

Additionally the influence on patients’ outcome was analyzed.

Methods

Study design and population

This study was approved by the local ethics committee (Approval No. EA1/083/06) and the need for written informed consent was waived.

The intervention and the score-data documentation were performed prospectively. The patient data management system (PDMS, COPRA®, version 5, Sasbachwalden, Germany) was used in all three ICUs to collect the adherence data. Study data were extracted manually by paired research assistance/study nurses blinded to the specific aim of the study. Integrity of the extracted data was assured by the documentation and instant control method (one research assistant actually documenting the data while the other supervised the process for correctness). Overall plausibility analyses were performed by the study investigators at regular intervals.

We compared two training strategies (see Fig. 1):

Fig. 1
figure 1

Algorithm standard and extended training

Full size image
  1. 1.

    the modified, extended training (ICU 1 and 2)

  2. 2.

    standard training (ICU 3)

The frequency of scoring prior to and after training on all ICUs was recorded [for sedation monitoring = frequency of Richmond agitation sedation score (RASS); for pain documentation = the frequency of numeric rating scale (NRS) or behaviour pain scale (BPS), and for delirium monitoring = the frequency of delirium detection score (DDS) scoring]. Subsequently at the end of post-training-data collection period and prior to the follow-up-data collection period, the staff of ICU 3 underwent in addition the same modified extended training that the staff on ICU 1 and 2 received. In a 1 year follow-up, the frequency of scoring was re-analyzed on all three ICUs (Fig. 2).

Fig. 2
figure 2

Study flow chart; screening, pre-training, post-training and follow-up. ICU intensive care unit, LOS length of stay

Full size image

In the pre-training period from October until November 2005, baseline data were collected from all patients included in the study, including acute physiology and chronic health evaluation (APACHE) II scores, sequential organ failure assessment (SOFA) and therapeutic intervention scoring system (TISS). Basic patient characteristics such as age, gender, admission diagnoses, pre-existing diseases as well as data regarding duration of mechanical ventilation, ICU length of stay (LOS), hospital length of stay and mortality were recorded.

The training started in December 2005 for both groups. Data for the post-training phase were collected from April 1st to May 31st 2006.

Subsequently the staff of ICU 3 (having only received standard training) underwent an additional training (see Fig. 2) according to our extended, modified protocol.

In July 2007 patient data were collected for the follow-up period (see Fig. 2).

Scores and protocol

Prior to this study, the Ramsay sedation scale (RSS) was used to monitor sedation and the NRS was used to document pain.

In our training, the Richmond agitation sedation scale (RASS) was introduced to monitor sedation [8]. The 11-point self-rating NRS continued to be used to monitor pain in non-intubated patients. For sedated patients our protocol introduced the Behavioral pain scale (BPS) a validated 3-item score [9, 10]. For delirium monitoring in patients with a RASS ≥ −2 we used the DDS [11]. All scores were to be documented in the PDMS by the nurses for every patient once every shift at least every 8 h.

We used the frequency of RASS for sedation monitoring, the frequency of NRS or BPS for pain documentation and the frequency of DDS for delirium respectively for analysis.

Staff member training

Experienced nurses and physicians experts for sedation, pain and delirium monitoring conducted the training sessions, for nurses and physicians.

Standard training

The standard program for staff member training consisted of four parts (Fig. 1):

  • Lectures provided information about the scores. The content of the lectures were generally designed not to exceed 45 min of presentation time. All three topics were integrated into one presentation and divided into theoretical and practical aspects. The emphasis was placed on delirium theory and measurement.

  • A movie for each test demonstrated the detailed conduct of the respective scores.

  • Each trainee received handouts including relevant literature.

  • A one to one instruction at the patients’ bedside was performed after the lecture.

During a 1 week piloting phase all staff members tested the patients on their own. During this period staff members were able to contact the monitoring experts regarding any issues with the scores. Each trained staff member had to evaluate five patients that were also tested by a monitoring expert. Different results in assessment between the trained staff members and the expert were analyzed.

Modified extended training

For the modified extended training, a local support team was established. The support team consisted of two nurses and one physician who were versed in monitoring sedation, pain and delirium in the intensive care setting and were validated by the monitoring experts. The team was available and provided support to the staff during normal working hours helping to resolve immediate problems and collect problems needing further attention and detailing.

The modified extended training consisted of three consecutive repetitive training cycles. The second and third lectures in the extended training group elaborated more on the theoretical background of sedation, analgesia and delirium, in addition to practical aspects of monitoring, and was intended to provide in-depth theoretical knowledge of these three topics. The second and third one-on-one instruction at patients’ bedsides again consisted of the evaluation of five patients and following discussion which served additionally as a quality control measure that all staff members continued to apply scoring criteria in a similar fashion. Upcoming problems and open questions were discussed in meetings between the local support team and the monitoring experts.

New members to the staff were trained individually according to the same protocol as the regular staff.

Statistical analysis

Results are expressed as means and standard deviations (SD) in cases of continuous variables, medians and interquartile ranges (shown as the interval of the 25–75 % quartiles) for categorical variables, and absolute and relative frequencies for dichotomous variables. The Kolmogorov–Smirnov test was used and histograms and normal-quartile plots were examined to verify the normality of distribution of the given observations. Because of deviations from normal distribution we exclusively applied nonparametric statistics and tested differences between the three documentation periods for each ICU using the non-parametric Kruskal–Wallis or Wilcoxon–Mann–Whitney tests. Frequencies were tested using the Chi-square-test in contingency tables. In cases of either small samples or of large but unbalanced sample sizes, tests were carried out in their exact versions. Multiple logistic regression analyses were carried out with mortality as response and various influencing factors of clinical interest such as age, gender, SOFA, TISS, pre-existing relevant organ dysfunctions, multiple trauma, carcinoma, and monitoring sedation, pain and delirium. Because of the strong association between simplified acute physiology score (SAPS) II and APACHE II, a new common feature, “APACHE-SAPS”, was created using factor analysis. Logistic regressions were accomplished in two steps: First, separate univariate logistic regressions for every influencing characteristic, and, second, selection of those variables with significant impact to mortality and their inclusion into multiple logistic regressions. Odds ratios (OR) with 95 % confidence intervals (CI) were determined in the logistic regression. A two-tailed p value <0.05 was considered statistically significant. All tests were conducted in the context of exploratory data analysis and no adjustments for multiple testing were made. All numerical calculations were carried out with SPSS® for Windows®, version 15 (SPSS Inc., Chicago, IL, USA) and SAS® (Release 9.1, 3, SAS Institute Inc., Cary, NC, USA).

Results

During our study period, in total 1,063 patients were admitted into the three ICUs. Only patients older than 18 years and spending at least 3 days in the ICU were included. Four hundred forty-three patients were excluded due to ICU length of stay (LOS) <3 days. One patient’s LOS in the ICU overlapped the pre- and post-training periods and was excluded as well. Two hundred forty-one consecutive patients were included in the pre-training period, and 228 consecutive patients in the post-training period. The follow-up period comprised 150 consecutive patients. Over all periods, data from a total of 619 patients were analyzed (Fig. 2). There were no significant differences concerning age and gender between the three periods for each ICU. Except for the follow-up data collection period in ICU 2, the severity of illness (APACHE II, SOFA) was without significant difference (Table 1).

Table 1 Basic patient characteristics and outcome in the 3 ICUs
Full size table

In ICU 1 and 2 (modified, extended training group), all scores were documented significantly more often in the post-training period (16 weeks after the training began) compared to the pre-training period (p < 0.001 for all scores). Likewise, the scoring frequency in the follow-up period (20 months after the training began) was significantly higher than in the pre-training period (Table 2). However, in the follow-up period the frequency of sedation scoring on ICU 2 decreased when compared to the post-training period (15 % p < 0.01, Table 2), whereas the frequency of pain scoring (Table 2) remained stable. The rate of delirium scoring (Table 2) remained the same in ICU 1 and increased in ICU 2.

Table 2 Frequency of monitoring per patient and day in pre-, post- and follow-up-period
Full size table

In ICU 3 (having received standard training), there was no significant change in the frequency of scoring when comparing the pre- and post-training periods (16 weeks after the training).

In the follow-up period on ICU 3 (12 months after modified extended training), all scores were documented significantly more often when compared to the post-training period (p < 0.001).

Monitoring and outcome

On ICU 1 in multivariate analysis the frequency of delirium scoring was associated with reduced mortality (odds ratio (DDS) = 0.451 [95 % CI: 0.220–0.924], p = 0.030; odds ratio (APACHE-SAPS) = 5.283 [95 % CI: 2.483–11.244], p < 0.001).

On ICU 2, the measurement of NRS/BPS were associated with reduced mortality (odds ratio (pain) = 0.365 [95 % CI: 0.147–0.866], p = 0.022; odds ratio (TISS) = 1.137 [95 % CI: 1.016–1.279], p = 0.026).

On ICU 3, we found no correlation of the frequency of scoring with mortality.

Discussion

The most important result is that the modified extended training program leads to higher intermediate and long-term implementation rates. Implementation rates were significantly increased and—even more importantly—sustained by implementing a new training algorithm based on the five educational techniques to change provider behavior identified by Landry and Sibbald [7]: Academic detailing, local opinion leaders, audit and feedback, reminder systems and printed material. Effective training strategies and protocols need to be multifaceted, repetitive and adapted to the local setting [12].

Similar results also with high implementation rates were achieved with graded, staged educational interventions occurring at regular time points in a study by Pun and colleagues [13].

Because of the work conditions on ICUs, with shift work, reaching all staff members can only be achieved by repeating training sessions, with successful attendance of each team member. Monitoring performance and maintaining feedback channels are keys to delivering change [14]. The support team was almost constantly available (at least two out of three shifts) to answer questions.

Another interesting result was that the low implementation rates after standard training on ICU 3 also increased significantly after receiving training according to the modified, extended training algorithm. The implementation rate on ICU 3 increased to a level equal to ICU 1 and 2 and may be interpreted as a proof of concept. The implementation rates for sedation and pain monitoring was higher in ICUs 1 and 3 compared to ICU 2, reflecting the higher rate of ventilator time and severity of illness.

Pain and delirium monitoring remained high even 12 months after starting the modified implementation algorithm. However, the frequency of sedation scoring on ICU 2 decreased by 15 % after 12 months. Nevertheless, it remained on a notably higher level than in the pre-training period. Staff debriefing indicated that the relevance for recording levels of sedation in non-sedated patients was not fully understood and supported. We adjusted our current training protocols in order to stress the value of the RASS evaluation and documentation in detecting non-pharmacological sedative states and agitated states, both of which may be indicators of subsyndromal delirium or delirium [15]. Feedback was provided in regard to the implementation rates as well as the influence on patients’ outcome as soon as the data was available.

Another important finding was that we confirmed the relevance of implemented protocols for sedation, analgesia and delirium on outcome. Increased monitoring rates were independently associated with improved outcome. Delirium screening was an independent predictor for decrease in mortality on ICU 1 while increased pain monitoring was independently predictive for reduced mortality on ICU 2. Improvement in delirium and pain management without advising on any specific pharmacotherapy was already shown to be associated with a better outcome [16–19]. Inadequate management is associated with prolonged mechanical ventilation, extended length of ICU stay (LOS), higher morbidity and mortality, and increased costs [2–4, 20, 21].

Benefits of accurate and regular patient assessment of sedative and analgesic needs include decreased ventilation time and nosocomial infection rates [22], a decrease of duration and severity of delirium [23, 24], a shorter hospital stay and a lower mortality [25]. Additionally, drug expenditure on sedative agents is significantly decreased [16].

Limitations of the study and possible source for bias were the inhomogeneous patient group sizes: 241 patients in the pre-training, 228 in the post-training, and 150 patients in the follow-up period. In addition, there were only two defined observation periods after training at 4 months and 12 months. Also, no further evaluation of invested time and potential costs benefit for patients’ outcome were performed. Additionally, changes in case-mix and/or nursing staff might have confounded the results, especially care provider characteristics such as attitude and motivation, as well as other factors such as the perception of two small time-points of the performing screener we re not the aim of this investigation. As patients with an ICU LOS <3 days were excluded, this possibly limits the generalizability and does not address a large proportion of patients who do indeed have delirium and require sedation during short ICU stays. Pain, sedation, and delirium monitoring should, of course, be performed for all ICU patients, not just those who are suspected to require a longer ICU stay. Additionally, due to the single-center design of the study results need to be confirmed in a multicenter study.

Conclusions

Intensified team training effectively facilitates the adoption of new evidence based protocols by ICU physicians and nurses. Changes are possible and durable if a modified, extended training strategy is followed. Although a complex implementation process will possibly procure higher short-term costs, it does result in better patient outcome.