User login
SAN FRANCISCO - Hospital procedure volume, which is commonly used as a proxy measure for hospital quality, is not significantly associated with in-hospital mortality for four common surgical procedures, based on a rigorous statistical analysis of data from the Nationwide Inpatient Sample.
Furthermore, "no identifiable threshold values exist for hospital procedure volume at which mortality risk significantly increased. Mortality risk was primarily attributable to patient-level risk factors," said Dr. Damian J. LaPar of the University of Virginia, Charlottesville.
He and his colleagues examined the relative strength of association between hospital volume and mortality vs. other modeled variables by comparing model covariate likelihood ratios for four high-risk procedures: pancreatic resection, abdominal aortic aneurysm (AAA) repair, esophageal resection, and coronary artery bypass graft (CABG).
Using data from the Nationwide Inpatient Sample in 2008, they obtained weighted discharge records for 261,142 patients: 19,194 patients who had pancreatic resection, 15,266 who had AAA repair, 4,764 who had esophageal resection, and 222,122 who had CABG. The primary outcome of interest was the estimated risk-adjusted effect of hospital procedure volume on mortality (in-hospital death). Comorbid disease was assessed based on Agency for Healthcare Research and Quality (AHRQ) comorbidity categories.
"In all four models, hospital volume was associated with the lowest statistical strength of association with mortality," compared with all other factors, Dr. LaPar said at the annual meeting of the American Surgical Association. Alternatively, other operation and patient-related risk factors -- including elective vs. nonelective status, age, sex, hypertension, weight loss, heart failure, chronic obstructive pulmonary disease, liver disease, and renal failure -- had higher strengths of association with mortality.
Dr. LaPar noted that procedure volume is an attractive metric for regulatory bodies to use as a predictor of outcomes; it is easy to measure and intuitive in nature. In addition, higher-volume hospitals are more likely to have established system-based processes and the infrastructure in place to improve patient outcomes.
The Leapfrog Group and the AHRQ both have adopted procedure volume as a quality indicator for the four high-risk surgical procedures. Arbitrarily defined volume thresholds have been adopted as a metric of quality for these procedures. However, many previous statistical methods that are used to define these thresholds have drawn criticism in the recent surgical literature. In many former series, volume is represented as arbitrarily defined categories, rather than as a continuous variable. Furthermore, there has been little attempt to rigorously assess and compare statistical model performance; to assess the relative strength of the association of procedure volume with other outcome predictors; and to utilize hierarchical, multilevel, statistical modeling techniques for complex, multicenter patient samples.
Dr. LaPar and his colleagues used hierarchical general linear modeling and created separate models for each procedure, which were adjusted for patient and operative factors as potential confounders. Patient factors included age, sex, and comorbid disease. Operative factors included procedure volume and elective/nonelective status. All model covariates were selected a priori.
The researchers used hospital volume as a continuous variable with restricted cubic spline regression, which uses all data points to estimate the shape of the association between hospital volume and mortality, and is considered to be the best way to visually identify threshold values. They also assessed the relative strength of association between hospital volume and mortality, compared with other factors (likelihood ratio). Model performance was assessed by looking at discrimination, calibration, and predictive capacity.
AAA repair was associated with the greatest in-hospital death. Patients having AAA repair had the greatest burden of comorbid disease, including peripheral vascular disease, chronic obstructive pulmonary disease, and renal failure.
Patient age was 60 years or greater. Most procedures were elective.
Dr. LaPar noted that the study did not investigate the impact of surgeon volume, nor did it adjust for surgical risk factors such as tumor type/stage, pulmonary function, performance status, surgical technique, preoperative medications, and neoadjuvant therapy. They were also unable to assess the effects of hospital volume on long-term survival, resource utilization, and readmission.
The findings have several implications. Previous reports using conventional modeling techniques may have overestimated the significance of hospital volume as a predictor of mortality. "However, these data do not intend to declare that hospital volume is irrelevant, but rather that hospital procedure volume may be a surrogate for other unidentified institutional factors that influence quality," said Dr. LaPar. "Most importantly, these data do not support the current policy of using hospital procedure volume as a proxy measure for quality."
Invited discussant Dr. Edward Livingston praised the group's rigorous statistical analysis of the association between hospital procedure volume and quality of care (mortality). He noted that earlier papers showed a statistical association between procedure volume and mortality. "Where the volume outcome research efforts took a left turn is that, instead of trying to understand what it was about volume that's associated with outcomes, there have been 2 decades of papers published looking at and reconfirming a statistical association between procedure volume and outcomes. Procedure volume itself does not translate into better outcomes. It is the things associated with procedure volume, such as surgeon experience, better functioning [operating room teams, and the like]. We really haven't looked into those causative factors."
If the causative factors could be identified, "then we could take the experience of high-volume centers and translate that to everybody else, so everybody could have good outcomes," he said.
According to Dr. Livingston, of the University of Texas, Dallas, previous studies relied on statistical modeling of the mortality relationship. "Those models are only as good as the model can represent the data," he said, and very few have been rigorously assessed to see how well they describe the phenomenon that they're trying to describe. This paper "should serve as the template for what everyone should do when they're performing volume outcome studies or any kind of regression analysis," he added.
The authors reported that they have no financial disclosures.
Procedural volume and its relationship to outcome has enormous implications, not just for patients undergoing surgical treatments, but in planning future surgical services. This study is important in respect of the large data set analysed and the finding that volume was only weakly associated with mortality compared to other factors such as patient co-morbidity. This raises questions about the current trend across the developed world to move towards high volume surgical units.
As impressive as the study initially appears there are some weaknesses that need to be considered. We of course only know about patients who were operated on and know nothing about patients declined surgery. There are studies that indicate patients are more likely to be declined appropriate surgery in units with less experience and lower volumes. Also of course many low volume units will refer high risk or complex cases to larger more experienced centres. It may be expected that some of this effect would be picked up in the Agency for Health Care Research and Quality comorbidities category (AHRQ) but the most usual reason for transfer to a regional unit is because of surgical complexity rather than patient co-morbidity.
This study also does not allow for any clinical governance measures that may be in force. Most if not all units with outcomes worse than average will take measures to address the problem ranging from ceasing undertaking the procedure to better case selection or improved team structure. At the time this study was conducted it is inconceivable to assume that no such measures were in action. It is incorrect then to conclude that this is an unselected study of the effects of volume. In the United Kingdom following concerns raised about outcomes from aortic surgery prospectively collected national data showed a very strong association with volume and outcome. The highest volume units had mortality 2.5 times lower than the lowest volume units (Outcomes after elective repair of infra-renal abdominal aortic aneurysm, March 2012; www.vascularsociety.org).
The authors conclude that volume is a poor indicator of quality of health care. This may be correct, but it is hard to suggest that based on this study as the authors did not measure factors such as length of hospital stay, re-admission rates, costs or patient experience of their care. In a review of cardiovascular services in London, not only did unit volume for AAA repair influence mortality it had a strong effect on efficiency. While patient outcome must be the main factor in planning service configurations, other issues such as cost and manpower need also to be considered.
Finally there are some practical issues. Small volume units can get good results. A unit that does 5 aortic aneurysm repairs in a year with no mortality will look very good. However, if they have a single mortality they will become one of the worst performing units overnight. Performance of units can only be assessed on large numbers. If a small volume unit is underperforming it may take a long time to identify that. There cannot be a surgical team in the world that is not aware that the more they work together to undertake complex surgical interventions the better they function across the board. So while the results of the study need to be considered carefully they do fly in the face of what most surgeons know in their hearts.
Dr. C. P. Shearman is Professor of Vascular Surgery, University of Southampton, United Kingdom, and an associate medical editor of Vascular Specialist.
Procedural volume and its relationship to outcome has enormous implications, not just for patients undergoing surgical treatments, but in planning future surgical services. This study is important in respect of the large data set analysed and the finding that volume was only weakly associated with mortality compared to other factors such as patient co-morbidity. This raises questions about the current trend across the developed world to move towards high volume surgical units.
As impressive as the study initially appears there are some weaknesses that need to be considered. We of course only know about patients who were operated on and know nothing about patients declined surgery. There are studies that indicate patients are more likely to be declined appropriate surgery in units with less experience and lower volumes. Also of course many low volume units will refer high risk or complex cases to larger more experienced centres. It may be expected that some of this effect would be picked up in the Agency for Health Care Research and Quality comorbidities category (AHRQ) but the most usual reason for transfer to a regional unit is because of surgical complexity rather than patient co-morbidity.
This study also does not allow for any clinical governance measures that may be in force. Most if not all units with outcomes worse than average will take measures to address the problem ranging from ceasing undertaking the procedure to better case selection or improved team structure. At the time this study was conducted it is inconceivable to assume that no such measures were in action. It is incorrect then to conclude that this is an unselected study of the effects of volume. In the United Kingdom following concerns raised about outcomes from aortic surgery prospectively collected national data showed a very strong association with volume and outcome. The highest volume units had mortality 2.5 times lower than the lowest volume units (Outcomes after elective repair of infra-renal abdominal aortic aneurysm, March 2012; www.vascularsociety.org).
The authors conclude that volume is a poor indicator of quality of health care. This may be correct, but it is hard to suggest that based on this study as the authors did not measure factors such as length of hospital stay, re-admission rates, costs or patient experience of their care. In a review of cardiovascular services in London, not only did unit volume for AAA repair influence mortality it had a strong effect on efficiency. While patient outcome must be the main factor in planning service configurations, other issues such as cost and manpower need also to be considered.
Finally there are some practical issues. Small volume units can get good results. A unit that does 5 aortic aneurysm repairs in a year with no mortality will look very good. However, if they have a single mortality they will become one of the worst performing units overnight. Performance of units can only be assessed on large numbers. If a small volume unit is underperforming it may take a long time to identify that. There cannot be a surgical team in the world that is not aware that the more they work together to undertake complex surgical interventions the better they function across the board. So while the results of the study need to be considered carefully they do fly in the face of what most surgeons know in their hearts.
Dr. C. P. Shearman is Professor of Vascular Surgery, University of Southampton, United Kingdom, and an associate medical editor of Vascular Specialist.
Procedural volume and its relationship to outcome has enormous implications, not just for patients undergoing surgical treatments, but in planning future surgical services. This study is important in respect of the large data set analysed and the finding that volume was only weakly associated with mortality compared to other factors such as patient co-morbidity. This raises questions about the current trend across the developed world to move towards high volume surgical units.
As impressive as the study initially appears there are some weaknesses that need to be considered. We of course only know about patients who were operated on and know nothing about patients declined surgery. There are studies that indicate patients are more likely to be declined appropriate surgery in units with less experience and lower volumes. Also of course many low volume units will refer high risk or complex cases to larger more experienced centres. It may be expected that some of this effect would be picked up in the Agency for Health Care Research and Quality comorbidities category (AHRQ) but the most usual reason for transfer to a regional unit is because of surgical complexity rather than patient co-morbidity.
This study also does not allow for any clinical governance measures that may be in force. Most if not all units with outcomes worse than average will take measures to address the problem ranging from ceasing undertaking the procedure to better case selection or improved team structure. At the time this study was conducted it is inconceivable to assume that no such measures were in action. It is incorrect then to conclude that this is an unselected study of the effects of volume. In the United Kingdom following concerns raised about outcomes from aortic surgery prospectively collected national data showed a very strong association with volume and outcome. The highest volume units had mortality 2.5 times lower than the lowest volume units (Outcomes after elective repair of infra-renal abdominal aortic aneurysm, March 2012; www.vascularsociety.org).
The authors conclude that volume is a poor indicator of quality of health care. This may be correct, but it is hard to suggest that based on this study as the authors did not measure factors such as length of hospital stay, re-admission rates, costs or patient experience of their care. In a review of cardiovascular services in London, not only did unit volume for AAA repair influence mortality it had a strong effect on efficiency. While patient outcome must be the main factor in planning service configurations, other issues such as cost and manpower need also to be considered.
Finally there are some practical issues. Small volume units can get good results. A unit that does 5 aortic aneurysm repairs in a year with no mortality will look very good. However, if they have a single mortality they will become one of the worst performing units overnight. Performance of units can only be assessed on large numbers. If a small volume unit is underperforming it may take a long time to identify that. There cannot be a surgical team in the world that is not aware that the more they work together to undertake complex surgical interventions the better they function across the board. So while the results of the study need to be considered carefully they do fly in the face of what most surgeons know in their hearts.
Dr. C. P. Shearman is Professor of Vascular Surgery, University of Southampton, United Kingdom, and an associate medical editor of Vascular Specialist.
SAN FRANCISCO - Hospital procedure volume, which is commonly used as a proxy measure for hospital quality, is not significantly associated with in-hospital mortality for four common surgical procedures, based on a rigorous statistical analysis of data from the Nationwide Inpatient Sample.
Furthermore, "no identifiable threshold values exist for hospital procedure volume at which mortality risk significantly increased. Mortality risk was primarily attributable to patient-level risk factors," said Dr. Damian J. LaPar of the University of Virginia, Charlottesville.
He and his colleagues examined the relative strength of association between hospital volume and mortality vs. other modeled variables by comparing model covariate likelihood ratios for four high-risk procedures: pancreatic resection, abdominal aortic aneurysm (AAA) repair, esophageal resection, and coronary artery bypass graft (CABG).
Using data from the Nationwide Inpatient Sample in 2008, they obtained weighted discharge records for 261,142 patients: 19,194 patients who had pancreatic resection, 15,266 who had AAA repair, 4,764 who had esophageal resection, and 222,122 who had CABG. The primary outcome of interest was the estimated risk-adjusted effect of hospital procedure volume on mortality (in-hospital death). Comorbid disease was assessed based on Agency for Healthcare Research and Quality (AHRQ) comorbidity categories.
"In all four models, hospital volume was associated with the lowest statistical strength of association with mortality," compared with all other factors, Dr. LaPar said at the annual meeting of the American Surgical Association. Alternatively, other operation and patient-related risk factors -- including elective vs. nonelective status, age, sex, hypertension, weight loss, heart failure, chronic obstructive pulmonary disease, liver disease, and renal failure -- had higher strengths of association with mortality.
Dr. LaPar noted that procedure volume is an attractive metric for regulatory bodies to use as a predictor of outcomes; it is easy to measure and intuitive in nature. In addition, higher-volume hospitals are more likely to have established system-based processes and the infrastructure in place to improve patient outcomes.
The Leapfrog Group and the AHRQ both have adopted procedure volume as a quality indicator for the four high-risk surgical procedures. Arbitrarily defined volume thresholds have been adopted as a metric of quality for these procedures. However, many previous statistical methods that are used to define these thresholds have drawn criticism in the recent surgical literature. In many former series, volume is represented as arbitrarily defined categories, rather than as a continuous variable. Furthermore, there has been little attempt to rigorously assess and compare statistical model performance; to assess the relative strength of the association of procedure volume with other outcome predictors; and to utilize hierarchical, multilevel, statistical modeling techniques for complex, multicenter patient samples.
Dr. LaPar and his colleagues used hierarchical general linear modeling and created separate models for each procedure, which were adjusted for patient and operative factors as potential confounders. Patient factors included age, sex, and comorbid disease. Operative factors included procedure volume and elective/nonelective status. All model covariates were selected a priori.
The researchers used hospital volume as a continuous variable with restricted cubic spline regression, which uses all data points to estimate the shape of the association between hospital volume and mortality, and is considered to be the best way to visually identify threshold values. They also assessed the relative strength of association between hospital volume and mortality, compared with other factors (likelihood ratio). Model performance was assessed by looking at discrimination, calibration, and predictive capacity.
AAA repair was associated with the greatest in-hospital death. Patients having AAA repair had the greatest burden of comorbid disease, including peripheral vascular disease, chronic obstructive pulmonary disease, and renal failure.
Patient age was 60 years or greater. Most procedures were elective.
Dr. LaPar noted that the study did not investigate the impact of surgeon volume, nor did it adjust for surgical risk factors such as tumor type/stage, pulmonary function, performance status, surgical technique, preoperative medications, and neoadjuvant therapy. They were also unable to assess the effects of hospital volume on long-term survival, resource utilization, and readmission.
The findings have several implications. Previous reports using conventional modeling techniques may have overestimated the significance of hospital volume as a predictor of mortality. "However, these data do not intend to declare that hospital volume is irrelevant, but rather that hospital procedure volume may be a surrogate for other unidentified institutional factors that influence quality," said Dr. LaPar. "Most importantly, these data do not support the current policy of using hospital procedure volume as a proxy measure for quality."
Invited discussant Dr. Edward Livingston praised the group's rigorous statistical analysis of the association between hospital procedure volume and quality of care (mortality). He noted that earlier papers showed a statistical association between procedure volume and mortality. "Where the volume outcome research efforts took a left turn is that, instead of trying to understand what it was about volume that's associated with outcomes, there have been 2 decades of papers published looking at and reconfirming a statistical association between procedure volume and outcomes. Procedure volume itself does not translate into better outcomes. It is the things associated with procedure volume, such as surgeon experience, better functioning [operating room teams, and the like]. We really haven't looked into those causative factors."
If the causative factors could be identified, "then we could take the experience of high-volume centers and translate that to everybody else, so everybody could have good outcomes," he said.
According to Dr. Livingston, of the University of Texas, Dallas, previous studies relied on statistical modeling of the mortality relationship. "Those models are only as good as the model can represent the data," he said, and very few have been rigorously assessed to see how well they describe the phenomenon that they're trying to describe. This paper "should serve as the template for what everyone should do when they're performing volume outcome studies or any kind of regression analysis," he added.
The authors reported that they have no financial disclosures.
SAN FRANCISCO - Hospital procedure volume, which is commonly used as a proxy measure for hospital quality, is not significantly associated with in-hospital mortality for four common surgical procedures, based on a rigorous statistical analysis of data from the Nationwide Inpatient Sample.
Furthermore, "no identifiable threshold values exist for hospital procedure volume at which mortality risk significantly increased. Mortality risk was primarily attributable to patient-level risk factors," said Dr. Damian J. LaPar of the University of Virginia, Charlottesville.
He and his colleagues examined the relative strength of association between hospital volume and mortality vs. other modeled variables by comparing model covariate likelihood ratios for four high-risk procedures: pancreatic resection, abdominal aortic aneurysm (AAA) repair, esophageal resection, and coronary artery bypass graft (CABG).
Using data from the Nationwide Inpatient Sample in 2008, they obtained weighted discharge records for 261,142 patients: 19,194 patients who had pancreatic resection, 15,266 who had AAA repair, 4,764 who had esophageal resection, and 222,122 who had CABG. The primary outcome of interest was the estimated risk-adjusted effect of hospital procedure volume on mortality (in-hospital death). Comorbid disease was assessed based on Agency for Healthcare Research and Quality (AHRQ) comorbidity categories.
"In all four models, hospital volume was associated with the lowest statistical strength of association with mortality," compared with all other factors, Dr. LaPar said at the annual meeting of the American Surgical Association. Alternatively, other operation and patient-related risk factors -- including elective vs. nonelective status, age, sex, hypertension, weight loss, heart failure, chronic obstructive pulmonary disease, liver disease, and renal failure -- had higher strengths of association with mortality.
Dr. LaPar noted that procedure volume is an attractive metric for regulatory bodies to use as a predictor of outcomes; it is easy to measure and intuitive in nature. In addition, higher-volume hospitals are more likely to have established system-based processes and the infrastructure in place to improve patient outcomes.
The Leapfrog Group and the AHRQ both have adopted procedure volume as a quality indicator for the four high-risk surgical procedures. Arbitrarily defined volume thresholds have been adopted as a metric of quality for these procedures. However, many previous statistical methods that are used to define these thresholds have drawn criticism in the recent surgical literature. In many former series, volume is represented as arbitrarily defined categories, rather than as a continuous variable. Furthermore, there has been little attempt to rigorously assess and compare statistical model performance; to assess the relative strength of the association of procedure volume with other outcome predictors; and to utilize hierarchical, multilevel, statistical modeling techniques for complex, multicenter patient samples.
Dr. LaPar and his colleagues used hierarchical general linear modeling and created separate models for each procedure, which were adjusted for patient and operative factors as potential confounders. Patient factors included age, sex, and comorbid disease. Operative factors included procedure volume and elective/nonelective status. All model covariates were selected a priori.
The researchers used hospital volume as a continuous variable with restricted cubic spline regression, which uses all data points to estimate the shape of the association between hospital volume and mortality, and is considered to be the best way to visually identify threshold values. They also assessed the relative strength of association between hospital volume and mortality, compared with other factors (likelihood ratio). Model performance was assessed by looking at discrimination, calibration, and predictive capacity.
AAA repair was associated with the greatest in-hospital death. Patients having AAA repair had the greatest burden of comorbid disease, including peripheral vascular disease, chronic obstructive pulmonary disease, and renal failure.
Patient age was 60 years or greater. Most procedures were elective.
Dr. LaPar noted that the study did not investigate the impact of surgeon volume, nor did it adjust for surgical risk factors such as tumor type/stage, pulmonary function, performance status, surgical technique, preoperative medications, and neoadjuvant therapy. They were also unable to assess the effects of hospital volume on long-term survival, resource utilization, and readmission.
The findings have several implications. Previous reports using conventional modeling techniques may have overestimated the significance of hospital volume as a predictor of mortality. "However, these data do not intend to declare that hospital volume is irrelevant, but rather that hospital procedure volume may be a surrogate for other unidentified institutional factors that influence quality," said Dr. LaPar. "Most importantly, these data do not support the current policy of using hospital procedure volume as a proxy measure for quality."
Invited discussant Dr. Edward Livingston praised the group's rigorous statistical analysis of the association between hospital procedure volume and quality of care (mortality). He noted that earlier papers showed a statistical association between procedure volume and mortality. "Where the volume outcome research efforts took a left turn is that, instead of trying to understand what it was about volume that's associated with outcomes, there have been 2 decades of papers published looking at and reconfirming a statistical association between procedure volume and outcomes. Procedure volume itself does not translate into better outcomes. It is the things associated with procedure volume, such as surgeon experience, better functioning [operating room teams, and the like]. We really haven't looked into those causative factors."
If the causative factors could be identified, "then we could take the experience of high-volume centers and translate that to everybody else, so everybody could have good outcomes," he said.
According to Dr. Livingston, of the University of Texas, Dallas, previous studies relied on statistical modeling of the mortality relationship. "Those models are only as good as the model can represent the data," he said, and very few have been rigorously assessed to see how well they describe the phenomenon that they're trying to describe. This paper "should serve as the template for what everyone should do when they're performing volume outcome studies or any kind of regression analysis," he added.
The authors reported that they have no financial disclosures.
Major Finding: Hospital procedure volume is not significantly associated with in-hospital mortality for four common surgical procedures.
Data Source: The researchers used data from the NIS in 2008. They obtained weighted-discharge records for 19,194 patients who had pancreatic resection, 15,266 who had AAA repair, 4,764 who had esophageal resection, and 222,122 who had CABG. The primary outcome of interest was the estimated risk-adjusted effect of hospital procedure volume on mortality (in-hospital death).
Disclosures: The authors reported that they have no financial disclosures.
