Are observation selection methods important when comparing early warning score performance?
Authors: Jarvis, S.W., Kovacs, C., Briggs, J., Meredith, P., Schmidt, P.E., Featherstone, P.I., Prytherch, D.R. and Smith, G.B.
Journal: Resuscitation
Volume: 90
Pages: 1-6
eISSN: 1873-1570
ISSN: 0300-9572
DOI: 10.1016/j.resuscitation.2015.01.033
Abstract:Introduction: Sicker patients generally have more vital sign assessments, particularly immediately before an adverse outcome, and especially if the vital sign monitoring schedule is driven by an early warning score (EWS) value. This lack of independence could influence the measured discriminatory performance of an EWS. Methods: We used a population of 1564,143 consecutive vital signs observation sets collected as a routine part of patients' care. We compared 35 published EWSs for their discrimination of the risk of death within 24. h of an observation set using (1) all observations in our dataset, (2) one observation per patient care episode, chosen at random and (3) one observation per patient care episode, chosen as the closest to a randomly selected point in time in each episode. We compared the area under the ROC curve (AUROC) as a measure of discrimination for each of the 35 EWSs under each observation selection method and looked for changes in their rank order. Results: There were no significant changes in rank order of the EWSs based on AUROC between the different observation selection methods, except for one EWS that included age among its components. Whichever method of observation selection was used, the National Early Warning Score (NEWS) showed the highest discrimination of risk of death within 24. h. AUROCs were higher when only one observation set was used per episode of care (significantly higher for many EWSs, including NEWS). Conclusions: Vital sign measurements can be treated as if they are independent - multiple observations can be used from each episode of care - when comparing the performance and ranking of EWSs, provided no EWS includes age.
Source: Scopus
Are observation selection methods important when comparing early warning score performance?
Authors: Jarvis, S.W., Kovacs, C., Briggs, J., Meredith, P., Schmidt, P.E., Featherstone, P.I., Prytherch, D.R. and Smith, G.B.
Journal: Resuscitation
Volume: 90
Pages: 1-6
eISSN: 1873-1570
DOI: 10.1016/j.resuscitation.2015.01.033
Abstract:INTRODUCTION: Sicker patients generally have more vital sign assessments, particularly immediately before an adverse outcome, and especially if the vital sign monitoring schedule is driven by an early warning score (EWS) value. This lack of independence could influence the measured discriminatory performance of an EWS. METHODS: We used a population of 1564,143 consecutive vital signs observation sets collected as a routine part of patients' care. We compared 35 published EWSs for their discrimination of the risk of death within 24h of an observation set using (1) all observations in our dataset, (2) one observation per patient care episode, chosen at random and (3) one observation per patient care episode, chosen as the closest to a randomly selected point in time in each episode. We compared the area under the ROC curve (AUROC) as a measure of discrimination for each of the 35 EWSs under each observation selection method and looked for changes in their rank order. RESULTS: There were no significant changes in rank order of the EWSs based on AUROC between the different observation selection methods, except for one EWS that included age among its components. Whichever method of observation selection was used, the National Early Warning Score (NEWS) showed the highest discrimination of risk of death within 24h. AUROCs were higher when only one observation set was used per episode of care (significantly higher for many EWSs, including NEWS). CONCLUSIONS: Vital sign measurements can be treated as if they are independent - multiple observations can be used from each episode of care--when comparing the performance and ranking of EWSs, provided no EWS includes age.
Source: PubMed
Are observation selection methods important when comparing early warning score performance?
Authors: Jarvis, S.W., Kovacs, C., Briggs, J., Meredith, P., Schmidt, P.E., Featherstone, P.I., Prytherch, D.R. and Smith, G.B.
Journal: RESUSCITATION
Volume: 90
Pages: 1-6
eISSN: 1873-1570
ISSN: 0300-9572
DOI: 10.1016/j.resuscitation.2015.01.033
Source: Web of Science (Lite)
Are observation selection methods important when comparing early warning score performance?
Authors: Jarvis, S.W., Kovacs, C., Briggs, J., Meredith, P., Schmidt, P.E., Featherstone, P.I., Prytherch, D.R. and Smith, G.B.
Journal: Resuscitation
Volume: 90
Pages: 1-6
eISSN: 1873-1570
ISSN: 0300-9572
DOI: 10.1016/j.resuscitation.2015.01.033
Abstract:Introduction
Sicker patients generally have more vital sign assessments, particularly immediately before an adverse outcome, and especially if the vital sign monitoring schedule is driven by an early warning score (EWS) value. This lack of independence could influence the measured discriminatory performance of an EWS.Methods
We used a population of 1564,143 consecutive vital signs observation sets collected as a routine part of patients' care. We compared 35 published EWSs for their discrimination of the risk of death within 24h of an observation set using (1) all observations in our dataset, (2) one observation per patient care episode, chosen at random and (3) one observation per patient care episode, chosen as the closest to a randomly selected point in time in each episode. We compared the area under the ROC curve (AUROC) as a measure of discrimination for each of the 35 EWSs under each observation selection method and looked for changes in their rank order.Results
There were no significant changes in rank order of the EWSs based on AUROC between the different observation selection methods, except for one EWS that included age among its components. Whichever method of observation selection was used, the National Early Warning Score (NEWS) showed the highest discrimination of risk of death within 24h. AUROCs were higher when only one observation set was used per episode of care (significantly higher for many EWSs, including NEWS).Conclusions
Vital sign measurements can be treated as if they are independent - multiple observations can be used from each episode of care--when comparing the performance and ranking of EWSs, provided no EWS includes age.Source: Europe PubMed Central