Purpose: To evaluate the functionality of drug safety alerting in hospital computerized physician order entry (CPOE) systems by a newly developed comprehensive test. Methods: Comparative evaluation of drug safety alerting quality in 6 different CPOEs used in Dutch hospitals, by means of 29 test items for sensitivity and 19 for specificity in offices of CPOE system vendors. Sensitivity and specificity were calculated for the complete test, and for the categories "within-order checks", "patient-specific checks", and "checks related to laboratory data and new patient conditions". Qualitative interviews with 16 hospital pharmacists evaluating missing functionality and corresponding pharmacy checks. Results: Sensitivity ranged from 0.38 to 0.79 and specificity from 0.11 to 0.84. The systems achieved the same ranking for sensitivity as for specificity. Within-order checks and patient-specific checks were present in all systems; alert generation or suppression due to laboratory data and new patient conditions was largely absent. Hospital pharmacists unanimously rated checks on contra-indications (absent in 2 CPOEs) and dose regimens less than once a day (absent in 4 CPOEs) as important. Pharmacists' opinions were more divergent for other test items. A variety of pharmacy checks were used, and clinical rules developed, to address missing functionality. Conclusions: Our test revealed widely varying functionality and appeared to be highly discriminative. Basic clinical decision support was partly absent in two CPOEs. Hospital pharmacists did not rate all test items as important and tried to accommodate the lacking functionality by performing additional checks and developing clinical rules.

Additional Metadata
Keywords Artificial intelligence, Clinical decision support, Clinical decision support systems, Comparative evaluations, Computer assisted, Computerized physician order entry systems, Decision making, Decision support systems, Decision theory, Drug safety, Drug therapy, Drug therapy computer-assisted, Dutch hospitals, Functionality tests, Hospitals, Medical order entry systems, Order entry system, Patient condition, Qualitative interviews, Quality control, Safety management, Sensitivity and specificity, Testing, article, computer assisted drug therapy, computerized provider order entry, drug contraindication, drug interaction, drug overdose, drug safety, hospital information system, medication error, pharmacist, pharmacist attitude, pharmacy, priority journal, sensitivity and specificity
Persistent URL dx.doi.org/10.1016/j.ijmedinf.2010.01.005, hdl.handle.net/1765/19839
Citation
van der Sijs, I.H, Bouamar, R, van Gelder, T, Aarts, J.E.C.M, Berg, M, & Vulto, A.G. (2010). Functionality test for drug safety alerting in computerized physician order entry systems. International Journal of Medical Informatics, 79(4), 243–251. doi:10.1016/j.ijmedinf.2010.01.005