Study Sounds Death Knell for TRISS

LAKE BUENA VISTA, FLA. – The Trauma-Related Injury Severity Score is outdated and should be replaced by a survival prediction model derived from more contemporary mortality rates, a study showed.

The score, known as TRISS, is the workhorse of outcome prediction in trauma. It is integral to the federal Performance Improvement and Patient Safety plan for trauma, is used for benchmarking hospital performance and comparing interhospital performance, and has been incorporated into most commercial trauma registries, including the National Trauma Registry System and the National Trauma Data Bank (NTDB).

Unfortunately, it is based on data collected in the Major Trauma Outcome Study (MTOS) during the 1980s, and its coefficients of survival prediction were last updated in 1995, said lead author Dr. Frederick Rogers, medical director of the Lancaster (Pa.) General Health Trauma Center.

"Few of us have really complained about TRISS up until this point because TRISS is what we consider an easy grader, making our results much better than they really are by comparing ourselves to trauma care in the last century," he said at the annual meeting of the Eastern Association for the Surgery of Trauma (EAST). "We really must be honest with ourselves.

"We must retire TRISS and create a mortality prediction model for the next century that reflects contemporary results."

Invited discussant Dr. Avery Nathens, director of the American College of Surgeons’ Trauma Quality Improvement Program and trauma director at St. Michael’s Hospital in Toronto, said, "I feel like we just heard the obituary for TRISS."

He said the study confirms many of the concerns regarding TRISS and pointed out that no fewer than 93 publications over the last 5 years have used TRISS to compare their outcomes.

"Dr. Rogers’ work could not have come too soon," he remarked.

To get an overall snapshot of mortality, the first part of the study compared outcomes from the 1982-1987 MTOS database for 80,544 patients from 139 U.S. and Canadian hospitals with those from 1.9 million patients from 900 U.S. trauma centers in the 2002-2006 NTDB. Patients were then stratified by mortality and age into 5-year age groups, up to 85-89 years.

In all age groups examined, there was a significant decrease in mortality in the NTDB data set, compared with the MTOS data set (P less than .001), he said. Overall mortality was more than double in the MTOS at 9.0% vs. just 4.4% in the NTDB.

To determine how well TRISS predicts survival over time, the researchers then compared observed to expected (O/E) mortality ratios using TRISS longitudinally and data on 451,868 patients in the Pennsylvania Trauma Outcome Study (PTOS) from 1990 to 2010.

PTOS entry criteria are ICU admission, a hospital length of stay greater than 48 hours, transfers in, and transfers out. The data are drawn from 31 accredited trauma centers and are maintained with strict internal and external auditing, Dr. Rogers said.

The O/E mortality ratios declined significantly over time for all patients in the PTOS database compared with TRISS. This was also true for the 403,935 patients with blunt trauma injuries (P less than .001 for both), indicative of improved outcomes over what would be predicted.

There was a more gradual decline in the O/E mortality ratio among the 47,933 patients with penetrating trauma injuries (P = .73), suggesting that mortality has not improved as much in this population, he said.

If TRISS were accurate, it would have an O/E ratio equal to 1, but the scoring system fell short of this in the PTOS analysis by about 20%, Dr. Rogers said.

"TRISS is drifting out of calibration," he said.

The obvious reason is that trauma care has significantly improved over the last 30 years, particularly in blunt trauma management, and, as expected, O/E mortality ratios have dropped.

In addition, TRISS is based in part on the Injury Severity Score (ISS) and Revised Trauma Score, and as such inherits their limitations. Dr. Rogers pointed out that the ISS isn’t monotonic with respect to death and isn’t linear in the logit of death. Also, the Glasgow Coma Scale can’t be computed for many patients at the highest risk of dying because of endotracheal tubes and chemical paralytic agents. Attempts to fill this gap have been made by data imputation, which he described as a form of mathematical trickery.

"In short, we believe continued attempts to resuscitate TRISS as an outcome predictor are untenable," he said. "With the incredible talent in EAST and other trauma organizations, we believe we can do better."

Dr. Rogers called for a prediction model with a fundamentally sound statistical underpinning that would minimize missing variables such as Glasgow Coma Scale scores, maintain accuracy over time, and contain a clear mechanism for periodic updating.

During the discussion of the study, Dr. Rogers said they compared mortality across ISS categories in the NTDB and PCOS databases and that their comparison confirmed the PTOS finding that mortality has declined over time, after controlling for both ISS and age, in blunt trauma and less convincingly for penetrating trauma.

He suggested that outcomes may have improved in blunt trauma patients because CT scanning has allowed more accurate probing of injuries and that improved field care may be allowing more penetrating trauma patients to survive long enough to get into the data set, but not necessarily out of the hospital.

Finally, TRISS founder and former EAST president Dr. Howard Champion rose from the audience to congratulate the authors on the study and for pointing out the "obvious frailties of TRISS." He suggested the issue revolves around three problematic elements: taxonomy coding, the model, and the coefficients attached to the model.

He pointed out that the United States uses the ICD-9 code, while the rest of the world uses the ICD-10 code and the ICD-11 code is on the horizon. "In other words, we are about two generations behind."

He went on to say that the Abbreviated Injury Score, which is integral to the ISS, was developed in the 1960s and 1970s for engineers and had 78 codes. That number has mushroomed to more than 2,000, and up to 80% of those codes are not used. "There are huge inter-reliability problems with it, suggesting that the Abbreviated Injury Score needs another careful look, as well as the models and the coefficients."

He closed by saying, "There’s a lot to be done. It is not a simple problem; it’s a very challenging problem."

Dr. Rogers said progress will be made by identifying better, more robust predictors for use in trauma outcome models.

Dr. Rogers, his coauthors, and Dr. Nathens reported no conflicts of interest.

LAKE BUENA VISTA, FLA. – The Trauma-Related Injury Severity Score is outdated and should be replaced by a survival prediction model derived from more contemporary mortality rates, a study showed.

The score, known as TRISS, is the workhorse of outcome prediction in trauma. It is integral to the federal Performance Improvement and Patient Safety plan for trauma, is used for benchmarking hospital performance and comparing interhospital performance, and has been incorporated into most commercial trauma registries, including the National Trauma Registry System and the National Trauma Data Bank (NTDB).

Unfortunately, it is based on data collected in the Major Trauma Outcome Study (MTOS) during the 1980s, and its coefficients of survival prediction were last updated in 1995, said lead author Dr. Frederick Rogers, medical director of the Lancaster (Pa.) General Health Trauma Center.

"Few of us have really complained about TRISS up until this point because TRISS is what we consider an easy grader, making our results much better than they really are by comparing ourselves to trauma care in the last century," he said at the annual meeting of the Eastern Association for the Surgery of Trauma (EAST). "We really must be honest with ourselves.

"We must retire TRISS and create a mortality prediction model for the next century that reflects contemporary results."

Invited discussant Dr. Avery Nathens, director of the American College of Surgeons’ Trauma Quality Improvement Program and trauma director at St. Michael’s Hospital in Toronto, said, "I feel like we just heard the obituary for TRISS."

He said the study confirms many of the concerns regarding TRISS and pointed out that no fewer than 93 publications over the last 5 years have used TRISS to compare their outcomes.

"Dr. Rogers’ work could not have come too soon," he remarked.

To get an overall snapshot of mortality, the first part of the study compared outcomes from the 1982-1987 MTOS database for 80,544 patients from 139 U.S. and Canadian hospitals with those from 1.9 million patients from 900 U.S. trauma centers in the 2002-2006 NTDB. Patients were then stratified by mortality and age into 5-year age groups, up to 85-89 years.

In all age groups examined, there was a significant decrease in mortality in the NTDB data set, compared with the MTOS data set (P less than .001), he said. Overall mortality was more than double in the MTOS at 9.0% vs. just 4.4% in the NTDB.

To determine how well TRISS predicts survival over time, the researchers then compared observed to expected (O/E) mortality ratios using TRISS longitudinally and data on 451,868 patients in the Pennsylvania Trauma Outcome Study (PTOS) from 1990 to 2010.

PTOS entry criteria are ICU admission, a hospital length of stay greater than 48 hours, transfers in, and transfers out. The data are drawn from 31 accredited trauma centers and are maintained with strict internal and external auditing, Dr. Rogers said.

The O/E mortality ratios declined significantly over time for all patients in the PTOS database compared with TRISS. This was also true for the 403,935 patients with blunt trauma injuries (P less than .001 for both), indicative of improved outcomes over what would be predicted.

There was a more gradual decline in the O/E mortality ratio among the 47,933 patients with penetrating trauma injuries (P = .73), suggesting that mortality has not improved as much in this population, he said.

If TRISS were accurate, it would have an O/E ratio equal to 1, but the scoring system fell short of this in the PTOS analysis by about 20%, Dr. Rogers said.

"TRISS is drifting out of calibration," he said.

The obvious reason is that trauma care has significantly improved over the last 30 years, particularly in blunt trauma management, and, as expected, O/E mortality ratios have dropped.

In addition, TRISS is based in part on the Injury Severity Score (ISS) and Revised Trauma Score, and as such inherits their limitations. Dr. Rogers pointed out that the ISS isn’t monotonic with respect to death and isn’t linear in the logit of death. Also, the Glasgow Coma Scale can’t be computed for many patients at the highest risk of dying because of endotracheal tubes and chemical paralytic agents. Attempts to fill this gap have been made by data imputation, which he described as a form of mathematical trickery.

"In short, we believe continued attempts to resuscitate TRISS as an outcome predictor are untenable," he said. "With the incredible talent in EAST and other trauma organizations, we believe we can do better."

Dr. Rogers called for a prediction model with a fundamentally sound statistical underpinning that would minimize missing variables such as Glasgow Coma Scale scores, maintain accuracy over time, and contain a clear mechanism for periodic updating.

During the discussion of the study, Dr. Rogers said they compared mortality across ISS categories in the NTDB and PCOS databases and that their comparison confirmed the PTOS finding that mortality has declined over time, after controlling for both ISS and age, in blunt trauma and less convincingly for penetrating trauma.

He suggested that outcomes may have improved in blunt trauma patients because CT scanning has allowed more accurate probing of injuries and that improved field care may be allowing more penetrating trauma patients to survive long enough to get into the data set, but not necessarily out of the hospital.

Finally, TRISS founder and former EAST president Dr. Howard Champion rose from the audience to congratulate the authors on the study and for pointing out the "obvious frailties of TRISS." He suggested the issue revolves around three problematic elements: taxonomy coding, the model, and the coefficients attached to the model.

He pointed out that the United States uses the ICD-9 code, while the rest of the world uses the ICD-10 code and the ICD-11 code is on the horizon. "In other words, we are about two generations behind."

He went on to say that the Abbreviated Injury Score, which is integral to the ISS, was developed in the 1960s and 1970s for engineers and had 78 codes. That number has mushroomed to more than 2,000, and up to 80% of those codes are not used. "There are huge inter-reliability problems with it, suggesting that the Abbreviated Injury Score needs another careful look, as well as the models and the coefficients."

He closed by saying, "There’s a lot to be done. It is not a simple problem; it’s a very challenging problem."

Dr. Rogers said progress will be made by identifying better, more robust predictors for use in trauma outcome models.

Dr. Rogers, his coauthors, and Dr. Nathens reported no conflicts of interest.

LAKE BUENA VISTA, FLA. – The Trauma-Related Injury Severity Score is outdated and should be replaced by a survival prediction model derived from more contemporary mortality rates, a study showed.

The score, known as TRISS, is the workhorse of outcome prediction in trauma. It is integral to the federal Performance Improvement and Patient Safety plan for trauma, is used for benchmarking hospital performance and comparing interhospital performance, and has been incorporated into most commercial trauma registries, including the National Trauma Registry System and the National Trauma Data Bank (NTDB).

Unfortunately, it is based on data collected in the Major Trauma Outcome Study (MTOS) during the 1980s, and its coefficients of survival prediction were last updated in 1995, said lead author Dr. Frederick Rogers, medical director of the Lancaster (Pa.) General Health Trauma Center.

"Few of us have really complained about TRISS up until this point because TRISS is what we consider an easy grader, making our results much better than they really are by comparing ourselves to trauma care in the last century," he said at the annual meeting of the Eastern Association for the Surgery of Trauma (EAST). "We really must be honest with ourselves.

"We must retire TRISS and create a mortality prediction model for the next century that reflects contemporary results."

Invited discussant Dr. Avery Nathens, director of the American College of Surgeons’ Trauma Quality Improvement Program and trauma director at St. Michael’s Hospital in Toronto, said, "I feel like we just heard the obituary for TRISS."

He said the study confirms many of the concerns regarding TRISS and pointed out that no fewer than 93 publications over the last 5 years have used TRISS to compare their outcomes.

"Dr. Rogers’ work could not have come too soon," he remarked.

To get an overall snapshot of mortality, the first part of the study compared outcomes from the 1982-1987 MTOS database for 80,544 patients from 139 U.S. and Canadian hospitals with those from 1.9 million patients from 900 U.S. trauma centers in the 2002-2006 NTDB. Patients were then stratified by mortality and age into 5-year age groups, up to 85-89 years.

In all age groups examined, there was a significant decrease in mortality in the NTDB data set, compared with the MTOS data set (P less than .001), he said. Overall mortality was more than double in the MTOS at 9.0% vs. just 4.4% in the NTDB.

To determine how well TRISS predicts survival over time, the researchers then compared observed to expected (O/E) mortality ratios using TRISS longitudinally and data on 451,868 patients in the Pennsylvania Trauma Outcome Study (PTOS) from 1990 to 2010.

PTOS entry criteria are ICU admission, a hospital length of stay greater than 48 hours, transfers in, and transfers out. The data are drawn from 31 accredited trauma centers and are maintained with strict internal and external auditing, Dr. Rogers said.

The O/E mortality ratios declined significantly over time for all patients in the PTOS database compared with TRISS. This was also true for the 403,935 patients with blunt trauma injuries (P less than .001 for both), indicative of improved outcomes over what would be predicted.

There was a more gradual decline in the O/E mortality ratio among the 47,933 patients with penetrating trauma injuries (P = .73), suggesting that mortality has not improved as much in this population, he said.

If TRISS were accurate, it would have an O/E ratio equal to 1, but the scoring system fell short of this in the PTOS analysis by about 20%, Dr. Rogers said.

"TRISS is drifting out of calibration," he said.

The obvious reason is that trauma care has significantly improved over the last 30 years, particularly in blunt trauma management, and, as expected, O/E mortality ratios have dropped.

In addition, TRISS is based in part on the Injury Severity Score (ISS) and Revised Trauma Score, and as such inherits their limitations. Dr. Rogers pointed out that the ISS isn’t monotonic with respect to death and isn’t linear in the logit of death. Also, the Glasgow Coma Scale can’t be computed for many patients at the highest risk of dying because of endotracheal tubes and chemical paralytic agents. Attempts to fill this gap have been made by data imputation, which he described as a form of mathematical trickery.

"In short, we believe continued attempts to resuscitate TRISS as an outcome predictor are untenable," he said. "With the incredible talent in EAST and other trauma organizations, we believe we can do better."

Dr. Rogers called for a prediction model with a fundamentally sound statistical underpinning that would minimize missing variables such as Glasgow Coma Scale scores, maintain accuracy over time, and contain a clear mechanism for periodic updating.

During the discussion of the study, Dr. Rogers said they compared mortality across ISS categories in the NTDB and PCOS databases and that their comparison confirmed the PTOS finding that mortality has declined over time, after controlling for both ISS and age, in blunt trauma and less convincingly for penetrating trauma.

He suggested that outcomes may have improved in blunt trauma patients because CT scanning has allowed more accurate probing of injuries and that improved field care may be allowing more penetrating trauma patients to survive long enough to get into the data set, but not necessarily out of the hospital.

Finally, TRISS founder and former EAST president Dr. Howard Champion rose from the audience to congratulate the authors on the study and for pointing out the "obvious frailties of TRISS." He suggested the issue revolves around three problematic elements: taxonomy coding, the model, and the coefficients attached to the model.

He pointed out that the United States uses the ICD-9 code, while the rest of the world uses the ICD-10 code and the ICD-11 code is on the horizon. "In other words, we are about two generations behind."

He went on to say that the Abbreviated Injury Score, which is integral to the ISS, was developed in the 1960s and 1970s for engineers and had 78 codes. That number has mushroomed to more than 2,000, and up to 80% of those codes are not used. "There are huge inter-reliability problems with it, suggesting that the Abbreviated Injury Score needs another careful look, as well as the models and the coefficients."

He closed by saying, "There’s a lot to be done. It is not a simple problem; it’s a very challenging problem."

Dr. Rogers said progress will be made by identifying better, more robust predictors for use in trauma outcome models.

Dr. Rogers, his coauthors, and Dr. Nathens reported no conflicts of interest.