Informatics and the American College of Surgeons National Surgical Quality Improvement Program: automated processes could replace manual record review

J Am Coll Surg. 2009 Jan;208(1):37-41. doi: 10.1016/j.jamcollsurg.2008.08.030. Epub 2008 Oct 31.

Abstract

Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) provides reliable, risk-adjusted outcomes data using standardized definitions and end points. Collection of the data is time consuming, and the surgical clinical nurse reviewers (SCNRs) can sample only a subset of all surgical cases. We sought to test the feasibility of using an informatics tool to automatically identify postoperative complications stored as free-text documents in our electronic medical record.

Study design: We used a locally developed electronic medical record search engine (EMERSE) to build sets of terminology that could accurately identify postoperative complications of both myocardial infarction (MI) and pulmonary embolism (PE) as defined by the ACS-NSQIP. All complications had been previously identified by our SCNRs and these were considered the gold standard. We used 5,894 cases from 2001 to 2004 from our institution's ACS-NSQIP dataset for building the terminology and 4,898 cases from 2005 to 2006 for validation. False-positive cases were then further reviewed manually.

Results: We achieved sensitivities of 100.0% and 92.8% for identifying postoperative myocardial infarction and pulmonary embolism, respectively, with somewhat lower specificities of 93.0% and 95.9%, respectively. These results compared favorably with results from the SCNRs, especially because our manual review uncovered cases previously missed.

Conclusions: Informatics has the potential to improve the efficiency and accuracy of chart abstraction by SCNRs for the ACS-NSQIP. Using such tools may eventually allow all cases at an institution to be reviewed rather than a small subset.

MeSH terms

  • Data Collection / standards
  • Electronic Data Processing*
  • Feasibility Studies
  • Forms and Records Control
  • General Surgery / standards*
  • Humans
  • Medical Audit
  • Medical Informatics*
  • Medical Records Systems, Computerized
  • Medical Records*
  • Myocardial Infarction / epidemiology
  • Outcome Assessment, Health Care / statistics & numerical data
  • Postoperative Complications / epidemiology*
  • Pulmonary Embolism / epidemiology
  • Quality Assurance, Health Care
  • Quality of Health Care*
  • United States