Tucker BS

Anterior cruciate ligament (ACL) injuries affect >175,000 people each year,¹ with >100,000 Americans undergoing ACL reconstruction annually.² Due to the high impact this injury has on the general population, and especially on athletes, it is important to determine the factors that influence a patient’s selection of a particular graft type. With increasing access to information and other outside influences, surgeons should attempt to provide as much objective information as possible in order to allow patients to make appropriate informed decisions regarding their graft choice for ACL surgery.

While autografts are used in >60% of primary ACL reconstructions, allografts are used in >80% of revision procedures.³ Both autografts and allografts offer advantages and disadvantages, and the advantages of each may depend on patient age, activity level, and occupation.⁴ For example, graft rerupture rates have been shown to be higher in patients with ACL allografts⁴, while kneeling pain has been shown to be worse in patients with bone-patellar tendon-bone (BPTB) autografts compared to hamstring autografts⁵ as well as BPTB allografts.⁴

Patient satisfaction rates are high for ACL autografts and allografts. Boonriong and Kietsiriroje⁶ have shown visual analog scale (VAS) patient satisfaction score averages to be 88 out of 100 for BPTB autografts and 93 out of 100 for hamstring tendon autografts. Fox and colleagues⁷ showed that 87% of patients were completely or mostly satisfied following revision ACL reconstruction with patellar tendon allograft. Cohen and colleagues⁸ evaluated 240 patients undergoing primary ACL reconstruction; 63.3% underwent ACL reconstruction with an allograft and 35.4% with an autograft. Of all patients enrolled in the study, 93% were satisfied with their graft choice, with 12.7% of patients opting to choose another graft if in the same situation again. Of those patients, 63.3% would have switched from an autograft to allograft. Although these numbers represent high patient satisfaction following a variety of ACL graft types, it is important to continue to identify graft selection factors in order to maximize patient outcomes.

The purposes of this prospective study were to assess patients’ knowledge of their graft type used for ACL reconstruction, to determine the most influential factors involved in graft selection, and to determine the level of satisfaction with the graft of choice at a minimum of 1-year follow-up. Based on a previous retrospective study,8 we hypothesized that physician recommendation would be the most influential factor in ACL graft selection. We also hypothesized that patients receiving an autograft would be more accurate in stating their graft harvest location compared to allograft patients.

Materials and Methods

We prospectively enrolled 304 patients who underwent primary ACL reconstruction from January 2008 to September 2013. Surgery was performed by 9 different surgeons within the same practice. All patients undergoing primary ACL reconstruction were eligible for the study.

All surgeons explained to each patient the pros and cons of each graft choice based upon peer-reviewed literature. Each patient was allowed to choose autograft or allograft, although most of the surgeons strongly encourage patients under age 25 years to choose autograft. One of the surgeons specifically encourages a patellar tendon autograft in patients under age 30 to 35 years, except for those patients with a narrow patellar tendon on magnetic resonance imaging, in which case he recommends a hamstring autograft. Another surgeon also specifically encourages patellar tendon autograft in patients under 35 years, except in skeletally immature patients, for whom he encourages hamstring autograft. However, none of the surgeons prohibited patients from choosing autograft or allograft, regardless of age.

The Institutional Review Board at our institution provided approval for this study. At the first postoperative follow-up appointment, each patient completed a questionnaire asking to select from a list the type (“your own” or “a cadaver”) and harvest site of the graft that was used for the surgery. Patients were also asked how they decided upon that graft type by ranking a list of 4 factors from 1 to 4. These included (1) physician recommendation, (2) family/friend’s recommendation, (3) coach’s recommendation, and (4) the media. Patients had the option of ranking more than one factor as most important in their decision. In addition, patients were asked to list any other factors that influenced their decision regarding graft type.

At a minimum of 1 year following surgery, patients completed the same questionnaire described above. In addition, patients were asked if they were satisfied with their graft and whether they would choose the same graft type if undergoing ACL reconstruction again. Patients who would have chosen a different graft were asked which graft they would have chosen and why. Any patient who experienced graft rupture prior to follow-up was included in the analysis.

Statistical Analysis

Chi square tests were used to compare dichotomous outcomes. A type I error of less than 5% (P < .05) was considered statistically significant.

Results

At least 1 year following ACL reconstruction, 213 of 304 patients (70%) successfully completed the same questionnaire as they did at their first postoperative follow-up appointment. The mean age of these patients at the time of surgery was 31.9 ± 11.0 years (range, 13.9 to 58.0 years). The mean follow-up time was 1.4 ± 0.4 years (range, 1.0 to 2.6 years), and 59% of these patients were male.

Autografts were used for 139 patients (139/304, 46%), allografts for 156 patients (156/304, 51%), and hybrid grafts for 9 patients (9/304, 3%). Overall, 77% of patients were accurate in stating the type of graft used for their ACL reconstruction, including 88% of autograft patients, 71% of allograft patients, and 11% of hybrid graft patients (Table 1). Patients who underwent reconstruction with an autograft were significantly more accurate in stating their graft type compared to patients with an allograft (P < .001). Graft type by surgeon is shown in Table 2. A statistically significant difference was found in the proportion of patients choosing autograft versus allograft based on surgeon (P < .0001).

When asked which type of graft was used for their surgery, 12 of 304 patients (4%) did not know their graft type or harvest location. Twenty-nine patients stated that their graft was an allograft but did not know the harvest location. Five patients stated that their graft was an autograft but did not know the harvest location. The 34 patients who classified their choice of graft but did not know the harvest site (11%) stated their surgeon never told them where their graft was from or they did not remember. A complete list of graft type responses is shown in Table 3.

Of the 29 patients who stated that their graft was an allograft but did not know the harvest location, 19 (66%) had a tibialis anterior allograft, 7 (24%) had a BPTB allograft, 2 (7%) had an Achilles tendon allograft, and 1 (3%) had a tibialis anterior autograft.

Physician recommendation was the most important decision-making factor listed for 82% of patients at their first postoperative appointment (Table 4). In addition to the 4 factors listed on our survey, patients were allowed to write in other factors involved in their decision. The most popular answers included recovery time, personal research on graft types, and prior personal experience with ACL reconstruction on the contralateral knee.

At the time of 1-year follow-up, 205 of 213 patients (96%) said they were satisfied with their graft choice (Table 5). All 4 unsatisfied autograft patients received a hamstring autograft, 3 of which were performed by the same surgeon. No significant difference was found in satisfaction rates between patients with autograft vs allograft (P = .87). There was a higher satisfaction rate among patients with a BPTB autograft compared to those with a hamstring autograft (P = .043). Of the unsatisfied patients, 3 patients stated that their graft had failed in the time prior to follow-up and 2 patients stated that they were having donor site pain following surgery with hamstring autograft and would consider an allograft if the reconstruction were repeated (Table 6). Two patients stated that they were unsatisfied with their graft but would need to do more research before deciding on a different graft type.

As shown in Tables 5 and 6, there is a discrepancy between the number of patients who were unsatisfied with their graft and the number of patients who stated that they would switch to a different graft type if they were to have ACL reconstruction again. A number of patients stated that they were satisfied with their graft, yet they would switch to a different graft. The main reasons for this related to issues from a hamstring autograft harvest site. One patient noted that although she was satisfied with her graft, she would switch after doing further research.

Discussion

Determining the decision-making factors for patients choosing between graft types for ACL reconstruction is important to ensure that patients can make a decision based on objective information. Several previous studies have evaluated patient selection of ACL grafts.^8-10 All 3 of these studies showed that surgeon recommendation is the primary factor in a patient’s decision. Similar to previous studies, we also found that physician recommendation is the most influential factor involved in this decision.

At an average follow-up of 41 months, Cohen and colleagues⁸ found that 1.3% of patients did not know whether they received an autograft or allograft for their ACL reconstruction. Furthermore, 50.7% of patients stating they received an allograft in Cohen’s study⁸ were unsure of the harvest location. In our study, 4% of patients at their first postoperative visit did not know whether they had received an autograft or allograft and 10% of patients stating they received an allograft selected an unknown harvest site. In contrast, only 2% of autograft patients in our study were unsure of the harvest location at their first postoperative appointment. It is likely that, over time, patients with an allograft forget the harvest location, whereas autograft patients are more likely to remember the location of harvest. This is especially true in patients with anterior knee pain or hamstring pain following ACL reconstruction with a BPTB or hamstring tendon autograft, respectively.

In terms of patients’ knowledge of their graft type, we found an overall accuracy of 77%, with 88% of autograft patients, 71% of allograft patients, and 11% of hybrid graft patients remembering their graft type and harvest location. Although we do not believe it to be critical for patients to remember these details, we do believe that patients who do not know their graft type likely relied on the recommendation of their physician.

We found a significant difference in the proportion of patients choosing autograft vs allograft based on surgeon, despite these surgeons citing available data in the literature to each patient and ultimately allowing each patient to make his or her own decision. This is partly due to the low sample size of most of the surgeons involved. However, the main reason for this distortion is likely that different surgeons may highlight different aspects of the literature to “spin” patients towards one graft or another in certain cases.

Currently, there remains a lack of clarity in the literature on appropriate ACL graft choices for patients. With constant new findings being published on different aspects of various grafts, it is important for surgeons to remain up to date with the literature. Nevertheless, we believe that certain biases are inevitable among surgeons due to unique training experiences as well as experience with their own patients.

Cohen and colleagues8 found that only 7% of patients reported that their own personal research influenced their decision, and only 6.4% of patients reported the media as their primary decision-making factor. Cheung and colleagues⁹ conducted a retrospective study and found that more than half of patients did significant personal research prior to making a decision regarding their graft type. Most of this research was done using medical websites and literature. Koh and colleagues¹⁰ noted that >80% of patients consulted the internet for graft information before making a decision. Koh’s study¹⁰ was performed in Korea and therefore the high prevalence of internet use may be culturally-related.

Overall, quality of information for patients undergoing ACL reconstruction is mixed across the internet, with only 22.5% of top websites being affiliated with an academic institution and 35.5% of websites authored by private physicians or physician groups.¹¹ Although a majority of internet websites offer discussion into the condition and surgical procedure of ACL reconstruction, less than half of these websites share the equally important information on the eligibility for surgery and concomitant complications following surgery.¹¹In our study, only 39 patients (13%) listed the media as either the first (13, 4%) or second (26, 9%) most important factor in their graft decision. Clearly there is some discrepancy between studies regarding the influence of personal research and media. There are a few potential reasons for this. First, we did not explicitly ask patients if their own personal research had any influence on their graft decision. Rather, we asked patients to rank their decision-making factors, and few patients ranked the media as their first or second greatest influence. Second, the word “media” was used in our questionnaire rather than “online research” or “internet.” It may seem somewhat vague to patients what the word “media” really means in terms of their own research, whereas listing “online research” or “internet” as selection options may have influenced patient responses.

In our study, we asked patients for any additional factors that influenced their graft choice. Thirteen patients (4%) noted that “personal research” through internet, orthopaedic literature, and the media influenced their graft decision. This corroborates the idea that “media” may have seemed vague to some patients. Of these patients, 9 chose an autograft and 4 chose an allograft. The relative ease in accessing information regarding graft choice in ACL reconstruction should be noted. Numerous websites offer advice, graft options, and commentary from group practices and orthopaedic surgeons. Whether or not these sources provide reasonable support for one graft vs another graft remains to be answered. The physician should be responsible for providing the patient with this collected objective information.

In our study, 205 patients (96%) were satisfied with their graft choice at the time of follow-up, with 15 patients (7%) stating that they would have chosen a different graft type if they could redo the operation. Cheung and colleagues⁹ found a satisfaction rate of 87.4% at an average follow-up time of 19 months, with 4.6% stating they would have chosen a different graft type. Many factors can contribute to patient satisfaction after ACL reconstruction. Looking at patient variables such as age, demographics, occupation, activity level, surgical technique including tunnel placement and fixation, postoperative rehabilitation, and graft type may influence the success of the patient after ACL reconstruction.

The strengths of this study include the patient population size with 1-year follow-up as well as the prospective study design. In comparison to a previous retrospective study in 2009 by Cohen and colleagues⁸with a sample size of 240 patients, our study collected 213 patients with 70% follow-up at minimum 1 year. Collecting data prospectively ensures accurate representation of the factors influencing each patient’s graft selection, while follow-up data was useful for patient satisfaction.

The limitations of this study include the percentage of patients lost from follow-up as well as any bias generated from the organization of the questionnaire. Unfortunately, with a younger, transient population of patients undergoing ACL reconstruction in a major metropolitan area, a percentage of patients are lost to follow-up. Many attempts were made to locate these patients. Another potential limitation was the order of decision factors listed on the questionnaire. These factors were not ordered randomly on each survey, but were listed in the following order: (1) physician recommendation (2) family/friend’s recommendation (3) coach’s recommendation and (4) the media. This may have influenced patient responses. The organization of these factors in the questionnaire started with physician recommendation, which may have influenced the patient’s initial thought process of which factor had the greatest influence in their graft decision. In addition, for the surveys completed at least 1 year following surgery, some patients were contacted via e-mail and others via telephone. Thus, some patients may have changed their answers if they were able to see the questions rather than hearing the questions. We believe this is particularly true of the question regarding graft harvest site.

Our study indicates that the majority of patients undergoing ACL reconstruction are primarily influenced by the physician’s recommendation.

Anterior cruciate ligament (ACL) injuries affect >175,000 people each year,¹ with >100,000 Americans undergoing ACL reconstruction annually.² Due to the high impact this injury has on the general population, and especially on athletes, it is important to determine the factors that influence a patient’s selection of a particular graft type. With increasing access to information and other outside influences, surgeons should attempt to provide as much objective information as possible in order to allow patients to make appropriate informed decisions regarding their graft choice for ACL surgery.

While autografts are used in >60% of primary ACL reconstructions, allografts are used in >80% of revision procedures.³ Both autografts and allografts offer advantages and disadvantages, and the advantages of each may depend on patient age, activity level, and occupation.⁴ For example, graft rerupture rates have been shown to be higher in patients with ACL allografts⁴, while kneeling pain has been shown to be worse in patients with bone-patellar tendon-bone (BPTB) autografts compared to hamstring autografts⁵ as well as BPTB allografts.⁴

Patient satisfaction rates are high for ACL autografts and allografts. Boonriong and Kietsiriroje⁶ have shown visual analog scale (VAS) patient satisfaction score averages to be 88 out of 100 for BPTB autografts and 93 out of 100 for hamstring tendon autografts. Fox and colleagues⁷ showed that 87% of patients were completely or mostly satisfied following revision ACL reconstruction with patellar tendon allograft. Cohen and colleagues⁸ evaluated 240 patients undergoing primary ACL reconstruction; 63.3% underwent ACL reconstruction with an allograft and 35.4% with an autograft. Of all patients enrolled in the study, 93% were satisfied with their graft choice, with 12.7% of patients opting to choose another graft if in the same situation again. Of those patients, 63.3% would have switched from an autograft to allograft. Although these numbers represent high patient satisfaction following a variety of ACL graft types, it is important to continue to identify graft selection factors in order to maximize patient outcomes.

The purposes of this prospective study were to assess patients’ knowledge of their graft type used for ACL reconstruction, to determine the most influential factors involved in graft selection, and to determine the level of satisfaction with the graft of choice at a minimum of 1-year follow-up. Based on a previous retrospective study,8 we hypothesized that physician recommendation would be the most influential factor in ACL graft selection. We also hypothesized that patients receiving an autograft would be more accurate in stating their graft harvest location compared to allograft patients.

Materials and Methods

We prospectively enrolled 304 patients who underwent primary ACL reconstruction from January 2008 to September 2013. Surgery was performed by 9 different surgeons within the same practice. All patients undergoing primary ACL reconstruction were eligible for the study.

All surgeons explained to each patient the pros and cons of each graft choice based upon peer-reviewed literature. Each patient was allowed to choose autograft or allograft, although most of the surgeons strongly encourage patients under age 25 years to choose autograft. One of the surgeons specifically encourages a patellar tendon autograft in patients under age 30 to 35 years, except for those patients with a narrow patellar tendon on magnetic resonance imaging, in which case he recommends a hamstring autograft. Another surgeon also specifically encourages patellar tendon autograft in patients under 35 years, except in skeletally immature patients, for whom he encourages hamstring autograft. However, none of the surgeons prohibited patients from choosing autograft or allograft, regardless of age.

The Institutional Review Board at our institution provided approval for this study. At the first postoperative follow-up appointment, each patient completed a questionnaire asking to select from a list the type (“your own” or “a cadaver”) and harvest site of the graft that was used for the surgery. Patients were also asked how they decided upon that graft type by ranking a list of 4 factors from 1 to 4. These included (1) physician recommendation, (2) family/friend’s recommendation, (3) coach’s recommendation, and (4) the media. Patients had the option of ranking more than one factor as most important in their decision. In addition, patients were asked to list any other factors that influenced their decision regarding graft type.

At a minimum of 1 year following surgery, patients completed the same questionnaire described above. In addition, patients were asked if they were satisfied with their graft and whether they would choose the same graft type if undergoing ACL reconstruction again. Patients who would have chosen a different graft were asked which graft they would have chosen and why. Any patient who experienced graft rupture prior to follow-up was included in the analysis.

Statistical Analysis

Chi square tests were used to compare dichotomous outcomes. A type I error of less than 5% (P < .05) was considered statistically significant.

Results

At least 1 year following ACL reconstruction, 213 of 304 patients (70%) successfully completed the same questionnaire as they did at their first postoperative follow-up appointment. The mean age of these patients at the time of surgery was 31.9 ± 11.0 years (range, 13.9 to 58.0 years). The mean follow-up time was 1.4 ± 0.4 years (range, 1.0 to 2.6 years), and 59% of these patients were male.

Autografts were used for 139 patients (139/304, 46%), allografts for 156 patients (156/304, 51%), and hybrid grafts for 9 patients (9/304, 3%). Overall, 77% of patients were accurate in stating the type of graft used for their ACL reconstruction, including 88% of autograft patients, 71% of allograft patients, and 11% of hybrid graft patients (Table 1). Patients who underwent reconstruction with an autograft were significantly more accurate in stating their graft type compared to patients with an allograft (P < .001). Graft type by surgeon is shown in Table 2. A statistically significant difference was found in the proportion of patients choosing autograft versus allograft based on surgeon (P < .0001).

When asked which type of graft was used for their surgery, 12 of 304 patients (4%) did not know their graft type or harvest location. Twenty-nine patients stated that their graft was an allograft but did not know the harvest location. Five patients stated that their graft was an autograft but did not know the harvest location. The 34 patients who classified their choice of graft but did not know the harvest site (11%) stated their surgeon never told them where their graft was from or they did not remember. A complete list of graft type responses is shown in Table 3.

Of the 29 patients who stated that their graft was an allograft but did not know the harvest location, 19 (66%) had a tibialis anterior allograft, 7 (24%) had a BPTB allograft, 2 (7%) had an Achilles tendon allograft, and 1 (3%) had a tibialis anterior autograft.

Physician recommendation was the most important decision-making factor listed for 82% of patients at their first postoperative appointment (Table 4). In addition to the 4 factors listed on our survey, patients were allowed to write in other factors involved in their decision. The most popular answers included recovery time, personal research on graft types, and prior personal experience with ACL reconstruction on the contralateral knee.

At the time of 1-year follow-up, 205 of 213 patients (96%) said they were satisfied with their graft choice (Table 5). All 4 unsatisfied autograft patients received a hamstring autograft, 3 of which were performed by the same surgeon. No significant difference was found in satisfaction rates between patients with autograft vs allograft (P = .87). There was a higher satisfaction rate among patients with a BPTB autograft compared to those with a hamstring autograft (P = .043). Of the unsatisfied patients, 3 patients stated that their graft had failed in the time prior to follow-up and 2 patients stated that they were having donor site pain following surgery with hamstring autograft and would consider an allograft if the reconstruction were repeated (Table 6). Two patients stated that they were unsatisfied with their graft but would need to do more research before deciding on a different graft type.

As shown in Tables 5 and 6, there is a discrepancy between the number of patients who were unsatisfied with their graft and the number of patients who stated that they would switch to a different graft type if they were to have ACL reconstruction again. A number of patients stated that they were satisfied with their graft, yet they would switch to a different graft. The main reasons for this related to issues from a hamstring autograft harvest site. One patient noted that although she was satisfied with her graft, she would switch after doing further research.

Discussion

Determining the decision-making factors for patients choosing between graft types for ACL reconstruction is important to ensure that patients can make a decision based on objective information. Several previous studies have evaluated patient selection of ACL grafts.^8-10 All 3 of these studies showed that surgeon recommendation is the primary factor in a patient’s decision. Similar to previous studies, we also found that physician recommendation is the most influential factor involved in this decision.

At an average follow-up of 41 months, Cohen and colleagues⁸ found that 1.3% of patients did not know whether they received an autograft or allograft for their ACL reconstruction. Furthermore, 50.7% of patients stating they received an allograft in Cohen’s study⁸ were unsure of the harvest location. In our study, 4% of patients at their first postoperative visit did not know whether they had received an autograft or allograft and 10% of patients stating they received an allograft selected an unknown harvest site. In contrast, only 2% of autograft patients in our study were unsure of the harvest location at their first postoperative appointment. It is likely that, over time, patients with an allograft forget the harvest location, whereas autograft patients are more likely to remember the location of harvest. This is especially true in patients with anterior knee pain or hamstring pain following ACL reconstruction with a BPTB or hamstring tendon autograft, respectively.

In terms of patients’ knowledge of their graft type, we found an overall accuracy of 77%, with 88% of autograft patients, 71% of allograft patients, and 11% of hybrid graft patients remembering their graft type and harvest location. Although we do not believe it to be critical for patients to remember these details, we do believe that patients who do not know their graft type likely relied on the recommendation of their physician.

We found a significant difference in the proportion of patients choosing autograft vs allograft based on surgeon, despite these surgeons citing available data in the literature to each patient and ultimately allowing each patient to make his or her own decision. This is partly due to the low sample size of most of the surgeons involved. However, the main reason for this distortion is likely that different surgeons may highlight different aspects of the literature to “spin” patients towards one graft or another in certain cases.

Currently, there remains a lack of clarity in the literature on appropriate ACL graft choices for patients. With constant new findings being published on different aspects of various grafts, it is important for surgeons to remain up to date with the literature. Nevertheless, we believe that certain biases are inevitable among surgeons due to unique training experiences as well as experience with their own patients.

Cohen and colleagues8 found that only 7% of patients reported that their own personal research influenced their decision, and only 6.4% of patients reported the media as their primary decision-making factor. Cheung and colleagues⁹ conducted a retrospective study and found that more than half of patients did significant personal research prior to making a decision regarding their graft type. Most of this research was done using medical websites and literature. Koh and colleagues¹⁰ noted that >80% of patients consulted the internet for graft information before making a decision. Koh’s study¹⁰ was performed in Korea and therefore the high prevalence of internet use may be culturally-related.

Overall, quality of information for patients undergoing ACL reconstruction is mixed across the internet, with only 22.5% of top websites being affiliated with an academic institution and 35.5% of websites authored by private physicians or physician groups.¹¹ Although a majority of internet websites offer discussion into the condition and surgical procedure of ACL reconstruction, less than half of these websites share the equally important information on the eligibility for surgery and concomitant complications following surgery.¹¹In our study, only 39 patients (13%) listed the media as either the first (13, 4%) or second (26, 9%) most important factor in their graft decision. Clearly there is some discrepancy between studies regarding the influence of personal research and media. There are a few potential reasons for this. First, we did not explicitly ask patients if their own personal research had any influence on their graft decision. Rather, we asked patients to rank their decision-making factors, and few patients ranked the media as their first or second greatest influence. Second, the word “media” was used in our questionnaire rather than “online research” or “internet.” It may seem somewhat vague to patients what the word “media” really means in terms of their own research, whereas listing “online research” or “internet” as selection options may have influenced patient responses.

In our study, we asked patients for any additional factors that influenced their graft choice. Thirteen patients (4%) noted that “personal research” through internet, orthopaedic literature, and the media influenced their graft decision. This corroborates the idea that “media” may have seemed vague to some patients. Of these patients, 9 chose an autograft and 4 chose an allograft. The relative ease in accessing information regarding graft choice in ACL reconstruction should be noted. Numerous websites offer advice, graft options, and commentary from group practices and orthopaedic surgeons. Whether or not these sources provide reasonable support for one graft vs another graft remains to be answered. The physician should be responsible for providing the patient with this collected objective information.

In our study, 205 patients (96%) were satisfied with their graft choice at the time of follow-up, with 15 patients (7%) stating that they would have chosen a different graft type if they could redo the operation. Cheung and colleagues⁹ found a satisfaction rate of 87.4% at an average follow-up time of 19 months, with 4.6% stating they would have chosen a different graft type. Many factors can contribute to patient satisfaction after ACL reconstruction. Looking at patient variables such as age, demographics, occupation, activity level, surgical technique including tunnel placement and fixation, postoperative rehabilitation, and graft type may influence the success of the patient after ACL reconstruction.

The strengths of this study include the patient population size with 1-year follow-up as well as the prospective study design. In comparison to a previous retrospective study in 2009 by Cohen and colleagues⁸with a sample size of 240 patients, our study collected 213 patients with 70% follow-up at minimum 1 year. Collecting data prospectively ensures accurate representation of the factors influencing each patient’s graft selection, while follow-up data was useful for patient satisfaction.

The limitations of this study include the percentage of patients lost from follow-up as well as any bias generated from the organization of the questionnaire. Unfortunately, with a younger, transient population of patients undergoing ACL reconstruction in a major metropolitan area, a percentage of patients are lost to follow-up. Many attempts were made to locate these patients. Another potential limitation was the order of decision factors listed on the questionnaire. These factors were not ordered randomly on each survey, but were listed in the following order: (1) physician recommendation (2) family/friend’s recommendation (3) coach’s recommendation and (4) the media. This may have influenced patient responses. The organization of these factors in the questionnaire started with physician recommendation, which may have influenced the patient’s initial thought process of which factor had the greatest influence in their graft decision. In addition, for the surveys completed at least 1 year following surgery, some patients were contacted via e-mail and others via telephone. Thus, some patients may have changed their answers if they were able to see the questions rather than hearing the questions. We believe this is particularly true of the question regarding graft harvest site.

Our study indicates that the majority of patients undergoing ACL reconstruction are primarily influenced by the physician’s recommendation.

Anterior cruciate ligament (ACL) injuries affect >175,000 people each year,¹ with >100,000 Americans undergoing ACL reconstruction annually.² Due to the high impact this injury has on the general population, and especially on athletes, it is important to determine the factors that influence a patient’s selection of a particular graft type. With increasing access to information and other outside influences, surgeons should attempt to provide as much objective information as possible in order to allow patients to make appropriate informed decisions regarding their graft choice for ACL surgery.

While autografts are used in >60% of primary ACL reconstructions, allografts are used in >80% of revision procedures.³ Both autografts and allografts offer advantages and disadvantages, and the advantages of each may depend on patient age, activity level, and occupation.⁴ For example, graft rerupture rates have been shown to be higher in patients with ACL allografts⁴, while kneeling pain has been shown to be worse in patients with bone-patellar tendon-bone (BPTB) autografts compared to hamstring autografts⁵ as well as BPTB allografts.⁴

Patient satisfaction rates are high for ACL autografts and allografts. Boonriong and Kietsiriroje⁶ have shown visual analog scale (VAS) patient satisfaction score averages to be 88 out of 100 for BPTB autografts and 93 out of 100 for hamstring tendon autografts. Fox and colleagues⁷ showed that 87% of patients were completely or mostly satisfied following revision ACL reconstruction with patellar tendon allograft. Cohen and colleagues⁸ evaluated 240 patients undergoing primary ACL reconstruction; 63.3% underwent ACL reconstruction with an allograft and 35.4% with an autograft. Of all patients enrolled in the study, 93% were satisfied with their graft choice, with 12.7% of patients opting to choose another graft if in the same situation again. Of those patients, 63.3% would have switched from an autograft to allograft. Although these numbers represent high patient satisfaction following a variety of ACL graft types, it is important to continue to identify graft selection factors in order to maximize patient outcomes.

The purposes of this prospective study were to assess patients’ knowledge of their graft type used for ACL reconstruction, to determine the most influential factors involved in graft selection, and to determine the level of satisfaction with the graft of choice at a minimum of 1-year follow-up. Based on a previous retrospective study,8 we hypothesized that physician recommendation would be the most influential factor in ACL graft selection. We also hypothesized that patients receiving an autograft would be more accurate in stating their graft harvest location compared to allograft patients.

Materials and Methods

We prospectively enrolled 304 patients who underwent primary ACL reconstruction from January 2008 to September 2013. Surgery was performed by 9 different surgeons within the same practice. All patients undergoing primary ACL reconstruction were eligible for the study.

All surgeons explained to each patient the pros and cons of each graft choice based upon peer-reviewed literature. Each patient was allowed to choose autograft or allograft, although most of the surgeons strongly encourage patients under age 25 years to choose autograft. One of the surgeons specifically encourages a patellar tendon autograft in patients under age 30 to 35 years, except for those patients with a narrow patellar tendon on magnetic resonance imaging, in which case he recommends a hamstring autograft. Another surgeon also specifically encourages patellar tendon autograft in patients under 35 years, except in skeletally immature patients, for whom he encourages hamstring autograft. However, none of the surgeons prohibited patients from choosing autograft or allograft, regardless of age.

The Institutional Review Board at our institution provided approval for this study. At the first postoperative follow-up appointment, each patient completed a questionnaire asking to select from a list the type (“your own” or “a cadaver”) and harvest site of the graft that was used for the surgery. Patients were also asked how they decided upon that graft type by ranking a list of 4 factors from 1 to 4. These included (1) physician recommendation, (2) family/friend’s recommendation, (3) coach’s recommendation, and (4) the media. Patients had the option of ranking more than one factor as most important in their decision. In addition, patients were asked to list any other factors that influenced their decision regarding graft type.

At a minimum of 1 year following surgery, patients completed the same questionnaire described above. In addition, patients were asked if they were satisfied with their graft and whether they would choose the same graft type if undergoing ACL reconstruction again. Patients who would have chosen a different graft were asked which graft they would have chosen and why. Any patient who experienced graft rupture prior to follow-up was included in the analysis.

Statistical Analysis

Chi square tests were used to compare dichotomous outcomes. A type I error of less than 5% (P < .05) was considered statistically significant.

Results

At least 1 year following ACL reconstruction, 213 of 304 patients (70%) successfully completed the same questionnaire as they did at their first postoperative follow-up appointment. The mean age of these patients at the time of surgery was 31.9 ± 11.0 years (range, 13.9 to 58.0 years). The mean follow-up time was 1.4 ± 0.4 years (range, 1.0 to 2.6 years), and 59% of these patients were male.

Autografts were used for 139 patients (139/304, 46%), allografts for 156 patients (156/304, 51%), and hybrid grafts for 9 patients (9/304, 3%). Overall, 77% of patients were accurate in stating the type of graft used for their ACL reconstruction, including 88% of autograft patients, 71% of allograft patients, and 11% of hybrid graft patients (Table 1). Patients who underwent reconstruction with an autograft were significantly more accurate in stating their graft type compared to patients with an allograft (P < .001). Graft type by surgeon is shown in Table 2. A statistically significant difference was found in the proportion of patients choosing autograft versus allograft based on surgeon (P < .0001).

When asked which type of graft was used for their surgery, 12 of 304 patients (4%) did not know their graft type or harvest location. Twenty-nine patients stated that their graft was an allograft but did not know the harvest location. Five patients stated that their graft was an autograft but did not know the harvest location. The 34 patients who classified their choice of graft but did not know the harvest site (11%) stated their surgeon never told them where their graft was from or they did not remember. A complete list of graft type responses is shown in Table 3.

Of the 29 patients who stated that their graft was an allograft but did not know the harvest location, 19 (66%) had a tibialis anterior allograft, 7 (24%) had a BPTB allograft, 2 (7%) had an Achilles tendon allograft, and 1 (3%) had a tibialis anterior autograft.

Physician recommendation was the most important decision-making factor listed for 82% of patients at their first postoperative appointment (Table 4). In addition to the 4 factors listed on our survey, patients were allowed to write in other factors involved in their decision. The most popular answers included recovery time, personal research on graft types, and prior personal experience with ACL reconstruction on the contralateral knee.

At the time of 1-year follow-up, 205 of 213 patients (96%) said they were satisfied with their graft choice (Table 5). All 4 unsatisfied autograft patients received a hamstring autograft, 3 of which were performed by the same surgeon. No significant difference was found in satisfaction rates between patients with autograft vs allograft (P = .87). There was a higher satisfaction rate among patients with a BPTB autograft compared to those with a hamstring autograft (P = .043). Of the unsatisfied patients, 3 patients stated that their graft had failed in the time prior to follow-up and 2 patients stated that they were having donor site pain following surgery with hamstring autograft and would consider an allograft if the reconstruction were repeated (Table 6). Two patients stated that they were unsatisfied with their graft but would need to do more research before deciding on a different graft type.

As shown in Tables 5 and 6, there is a discrepancy between the number of patients who were unsatisfied with their graft and the number of patients who stated that they would switch to a different graft type if they were to have ACL reconstruction again. A number of patients stated that they were satisfied with their graft, yet they would switch to a different graft. The main reasons for this related to issues from a hamstring autograft harvest site. One patient noted that although she was satisfied with her graft, she would switch after doing further research.

Discussion

Determining the decision-making factors for patients choosing between graft types for ACL reconstruction is important to ensure that patients can make a decision based on objective information. Several previous studies have evaluated patient selection of ACL grafts.^8-10 All 3 of these studies showed that surgeon recommendation is the primary factor in a patient’s decision. Similar to previous studies, we also found that physician recommendation is the most influential factor involved in this decision.

At an average follow-up of 41 months, Cohen and colleagues⁸ found that 1.3% of patients did not know whether they received an autograft or allograft for their ACL reconstruction. Furthermore, 50.7% of patients stating they received an allograft in Cohen’s study⁸ were unsure of the harvest location. In our study, 4% of patients at their first postoperative visit did not know whether they had received an autograft or allograft and 10% of patients stating they received an allograft selected an unknown harvest site. In contrast, only 2% of autograft patients in our study were unsure of the harvest location at their first postoperative appointment. It is likely that, over time, patients with an allograft forget the harvest location, whereas autograft patients are more likely to remember the location of harvest. This is especially true in patients with anterior knee pain or hamstring pain following ACL reconstruction with a BPTB or hamstring tendon autograft, respectively.

In terms of patients’ knowledge of their graft type, we found an overall accuracy of 77%, with 88% of autograft patients, 71% of allograft patients, and 11% of hybrid graft patients remembering their graft type and harvest location. Although we do not believe it to be critical for patients to remember these details, we do believe that patients who do not know their graft type likely relied on the recommendation of their physician.

We found a significant difference in the proportion of patients choosing autograft vs allograft based on surgeon, despite these surgeons citing available data in the literature to each patient and ultimately allowing each patient to make his or her own decision. This is partly due to the low sample size of most of the surgeons involved. However, the main reason for this distortion is likely that different surgeons may highlight different aspects of the literature to “spin” patients towards one graft or another in certain cases.

Currently, there remains a lack of clarity in the literature on appropriate ACL graft choices for patients. With constant new findings being published on different aspects of various grafts, it is important for surgeons to remain up to date with the literature. Nevertheless, we believe that certain biases are inevitable among surgeons due to unique training experiences as well as experience with their own patients.

Cohen and colleagues8 found that only 7% of patients reported that their own personal research influenced their decision, and only 6.4% of patients reported the media as their primary decision-making factor. Cheung and colleagues⁹ conducted a retrospective study and found that more than half of patients did significant personal research prior to making a decision regarding their graft type. Most of this research was done using medical websites and literature. Koh and colleagues¹⁰ noted that >80% of patients consulted the internet for graft information before making a decision. Koh’s study¹⁰ was performed in Korea and therefore the high prevalence of internet use may be culturally-related.

Overall, quality of information for patients undergoing ACL reconstruction is mixed across the internet, with only 22.5% of top websites being affiliated with an academic institution and 35.5% of websites authored by private physicians or physician groups.¹¹ Although a majority of internet websites offer discussion into the condition and surgical procedure of ACL reconstruction, less than half of these websites share the equally important information on the eligibility for surgery and concomitant complications following surgery.¹¹In our study, only 39 patients (13%) listed the media as either the first (13, 4%) or second (26, 9%) most important factor in their graft decision. Clearly there is some discrepancy between studies regarding the influence of personal research and media. There are a few potential reasons for this. First, we did not explicitly ask patients if their own personal research had any influence on their graft decision. Rather, we asked patients to rank their decision-making factors, and few patients ranked the media as their first or second greatest influence. Second, the word “media” was used in our questionnaire rather than “online research” or “internet.” It may seem somewhat vague to patients what the word “media” really means in terms of their own research, whereas listing “online research” or “internet” as selection options may have influenced patient responses.

In our study, we asked patients for any additional factors that influenced their graft choice. Thirteen patients (4%) noted that “personal research” through internet, orthopaedic literature, and the media influenced their graft decision. This corroborates the idea that “media” may have seemed vague to some patients. Of these patients, 9 chose an autograft and 4 chose an allograft. The relative ease in accessing information regarding graft choice in ACL reconstruction should be noted. Numerous websites offer advice, graft options, and commentary from group practices and orthopaedic surgeons. Whether or not these sources provide reasonable support for one graft vs another graft remains to be answered. The physician should be responsible for providing the patient with this collected objective information.

In our study, 205 patients (96%) were satisfied with their graft choice at the time of follow-up, with 15 patients (7%) stating that they would have chosen a different graft type if they could redo the operation. Cheung and colleagues⁹ found a satisfaction rate of 87.4% at an average follow-up time of 19 months, with 4.6% stating they would have chosen a different graft type. Many factors can contribute to patient satisfaction after ACL reconstruction. Looking at patient variables such as age, demographics, occupation, activity level, surgical technique including tunnel placement and fixation, postoperative rehabilitation, and graft type may influence the success of the patient after ACL reconstruction.

The strengths of this study include the patient population size with 1-year follow-up as well as the prospective study design. In comparison to a previous retrospective study in 2009 by Cohen and colleagues⁸with a sample size of 240 patients, our study collected 213 patients with 70% follow-up at minimum 1 year. Collecting data prospectively ensures accurate representation of the factors influencing each patient’s graft selection, while follow-up data was useful for patient satisfaction.

The limitations of this study include the percentage of patients lost from follow-up as well as any bias generated from the organization of the questionnaire. Unfortunately, with a younger, transient population of patients undergoing ACL reconstruction in a major metropolitan area, a percentage of patients are lost to follow-up. Many attempts were made to locate these patients. Another potential limitation was the order of decision factors listed on the questionnaire. These factors were not ordered randomly on each survey, but were listed in the following order: (1) physician recommendation (2) family/friend’s recommendation (3) coach’s recommendation and (4) the media. This may have influenced patient responses. The organization of these factors in the questionnaire started with physician recommendation, which may have influenced the patient’s initial thought process of which factor had the greatest influence in their graft decision. In addition, for the surveys completed at least 1 year following surgery, some patients were contacted via e-mail and others via telephone. Thus, some patients may have changed their answers if they were able to see the questions rather than hearing the questions. We believe this is particularly true of the question regarding graft harvest site.

Our study indicates that the majority of patients undergoing ACL reconstruction are primarily influenced by the physician’s recommendation.

As defined by the American Academy of Orthopaedic Surgeons (AAOS) in 1996, conflict of interest (COI) is the “circumstance that exists when, because of personal financial gain, an individual has the potential to be less than objective when called on to reach a judgment or interpret a result.”¹ In medical research, COIs often occur in relationships between physician-researchers and pharmaceutical, medical device, and biotechnology companies. These relationships usually take the form of research grants but can also arise when the researcher has a financial interest in the product being tested or in the company that manufactures the product.

 Although constructive collaboration between academic medicine and industry has worked to improve health care and ultimately benefit patients, potential drawbacks of such relationships include sequestration and suppression of results that may be disadvantageous to the industry sponsor,² increased likelihood of reporting positive results (pro-industry),^3-7 and biased study designs.⁸ The nature of such relationships may threaten the integrity of scientific studies, the objectivity of medical education, the quality of patient care, and the public’s trust in medicine.⁹

Financial relationships and affiliations are increasing as we seek to answer a growing number of clinical questions—with funding often being a limiting factor. At national scientific meetings, the number of presentations reporting COIs reflects this trend. Paper and poster presentations accepted for annual meetings of the Orthopaedic Trauma Association (OTA) and reporting a COI increased from 7.6% in 1993 to 12.6% in 2002 (P = .0129).²

Medical subspecialties outside of orthopedics are experiencing similar trends. Most notable is the American Psychiatric Association (APA). After the APA published a mandatory financial COI disclosure policy in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), the percentage of task force members reporting industry relationships increased by 12%.¹⁰ Analysis of the DSM-5 panels demonstrated that the panels with the largest percentage of reported COIs are those for which pharmacological treatment is the first-line intervention, including the panels for mood disorders (67%), psychotic disorders (83%) and sleep/wake disorders (100%).¹⁰ Moreover, the industry ties reported are to the pharmaceutical companies that manufacture the medications used to treat these disorders or to companies that service the pharmaceutical industry.¹⁰

The degree to which financial COIs affect the interpretation of the orthopedic literature has never been quantified. Although it is clear that COIs can confound the results and reporting of data, how the medical community uses disclosures when interpreting the literature and when formulating opinions that may or may not affect their practice patterns is largely unknown.

We conducted a study to evaluate how a hypothetical financial COI disclosure would influence the interpretation of data by orthopedic clinicians. We also wanted to determine the reliability of the data as perceived in association with different study designs, levels of evidence, research institutional settings, and reporting of positive or negative results.

Methods

We asked members of the Arthroscopy Association of North America (AANA) and the American Orthopaedic Society for Sports Medicine (AOSSM) to complete a multiple-choice situational questionnaire (Table). The questionnaire assesses the degree to which respondents use COI disclosures when interpreting the literature. It further explores the perceived clinical value of a study with a given reported COI, assuming variations in study design, research institutional setting, and significance of results. The fictional research team disclosed the project was funded by a pharmaceutical company and all team members received consulting compensation. The survey and study were reviewed and approved by our institutional review board. The survey consisted of 14 multiple-choice questions that allowed for only 1 answer selection per person and allowed survey takers to skip questions they did not wish to answer. The survey questions and associated response options appear in edited form in the Table. A link to the questionnaire (https://www.surveymonkey.com/s/MPCCLCX) was sent with a message explaining the study. The responses to the questionnaire constituted the data.

Results

We sent a request to participate in the survey to 750 physicians and received 522 responses (overall response rate, 70%). The response rate for each question equaled or exceeded 98%.

The majority of respondents (95.6%) were male. Ninety-nine percent of respondents were orthopedic surgeons. The Northeast (US) was the most common geographical practice location of respondents (32%), followed by the Midwest (19.1%) and the Southeast (16.6%). Most respondents (40%) had been in practice for more than 20 years; 67% had been in practice a minimum of 10 years. The majority (68.8%) were employed by private practice groups, either single specialty (57.8%) or multispecialty (11%).

Eighty percent of respondents strongly agreed that COI disclosure is important when interpreting study results, 62% reported always reading the disclosure slide during academy or other meeting presentations, and 41% reported always using this information when deciding how to interpret scientific data.

Seventy-five percent of respondents thought the study—an academic-center case series with significant results in favor of the pharmaceutical company funding the study—was biased (42% indicated biased with merit, 33% biased without merit). Twenty-three percent thought the study was possibly biased, but likely trustworthy given the academic institutional affiliation. When the study setting was changed to community hospital, 95% thought the study was biased (51% biased with merit, 44% biased without merit). With the same study performed at an academic center, and no statistically significant results (not in favor of the pharmaceutical company funding the study), 88% thought the study had merit (46% biased with merit, 42% unbiased with merit).

When the study design was changed to a randomized controlled trial (level I evidence) conducted at an academic center with negative results, an overwhelming 95% of respondents thought the study had merit (33% biased with merit, 62% unbiased with merit). Given the same study design at an academic center, with positive results, 78% still thought the study had merit (39% biased with merit, 39% unbiased with merit). An additional 18% thought the study was biased, but still likely trustworthy given the academic institutional affiliation. Finally, given a randomized controlled trial and positive results, but with the research setting a small community practice, 90% thought the study had merit (51% biased with merit, 39% unbiased with merit). The percentage of respondents who found the study biased and likely without merit increased from 3.7% to 9.5% when the institutional affiliation changed from academic to community.

Discussion

As governmental funding sources become increasingly limited, the role of industry sponsorship of orthopedic research has grown. Potential drawbacks and biases of such research support have been well described—most notably, increased positive result reporting, suppression of results that may be disadvantageous to the industry sponsor, and biased study designs.^2-8 However, the extent to which financial COIs affect the orthopedic medical community’s interpretation of the literature has never been quantified. To our knowledge, the present study is the first to quantify the impact of reported COI on study interpretation.

Our goal was to examine how reported financial COIs influence the interpretation of the literature by the orthopedic medical community. Moreover, we wanted to determine the perceived reliability of the data when variables (study design, institutional affiliation, positive vs negative results) were changed. The results of our survey indicate that, when a financial COI is reported, study reliability is perceived as highest when negative results were found.

Our survey noted a discrepancy between the documented importance of the hypothetical research team’s COI disclosure and the use of such disclosures when interpreting study results. Eighty percent of respondents agreed that COI disclosure is important when interpreting study results, but only 62% reported always reading disclosures, and even fewer (41%) reported always using the information when interpreting results. It is unclear exactly why this trend exists, as one would expect the percentages to be more similar. These particular survey questions were formed around using COI disclosures when interpreting study results during academic presentations at national meetings and not during the review of published literature. It is possible that positioning the COI disclosure at the beginning of a presentation has an effect, but only 3.7% of respondents indicated they seldom remembered the disclosure by the end of the presentation. The results of our survey may have varied if the questions had targeted reading and interpreting the literature.

Interestingly, the results of these survey questions tended to be more consistent with rates of reported financial COI by presenters at national orthopedic meetings. A study published in the New England Journal of Medicine found that less than 80% of orthopedic surgeons reported their disclosures at a large annual meeting (AAOS), even when the disclosure involved payments pertinent to the research they were presenting.⁵ When the payments were indirectly related to the research, the percentage of surgeons reporting disclosures was 50%, almost the same as the disclosure rate for unrelated payments.⁵

When the study was changed to a level I randomized controlled trial, more survey respondents found it to be less biased and have more merit. Although it would seem intuitive for a study with a higher level of evidence to carry more clinical value during interpretation, this may not hold true in the setting of industry-sponsored clinical trials. Several studies have documented a significant association between the reporting of positive results and industry-sponsored randomized clinical trials. In 2008, Khan and colleagues³ examined 100 orthopedic randomized clinical trials reported in 5 major orthopedic subspecialty journals over a 2-year period. The association between industry funding and favorable outcome in all original randomized clinical trials was strong and significant (P < .001). This is not surprising, given the amount of time and money required for a well-designed clinical study. Commercial products with preclinical promise are pushed to testing in a clinical trial, whereas resources would not be wasted on products lacking preclinical merit.

The most important variable affecting interpretation of study merit by survey respondents was the reporting of negative results. As more researchers are developing COIs, many studies are discovering a relationship between COIs and outcomes of research studies. Reviewing the adult total joint literature, Ezzet⁸ found an industry funding rate of 50%. Positive results were reported in 93% of cases in commercially funded studies versus 37% of cases in independently funded studies. Furthermore, no negative results were reported by investigators who were receiving royalties from the respective companies.

Studies across the medical literature have also found this association between industry sponsorship and reporting of positive results. One such study, reported by Valachis and colleagues⁷ in the Journal of Clinical Oncology, examined more than 80 economic analyses of targeted oncologic therapies and found the studies funded by pharmaceutical companies were more likely to report favorable qualitative cost estimates. In addition, when studies with a COI disclosure were examined, those reporting any financial relationship with a manufacturer (eg, author affiliation, funding) were more likely than those without such a relationship to report favorable results.

Our study had several limitations. First, as most of the survey respondents were orthopedic surgeons, extrapolating their data to the medical community at large may not be appropriate, as each specialty may view industry affiliations differently. In addition, respondents were asked to base their interpretations of a study on conclusions we predetermined—no direct visualization of the data set or statistical testing methods. It is possible that these responses may have been different had the respondents had the opportunity to further evaluate the study in question. In a recent study, Altwairgi and colleagues¹¹ found that 10% of randomized clinical trials involving lung cancer treatment were reported with different conclusions in their full manuscripts relative to their abstracts. We think our survey design perhaps best mimics an annual meeting environment in which participants have very limited ability to interpret studies and may rely more heavily on the factors we investigated—study design, significance of findings, and setting, all similar to information presented in an abstract—when making informed decisions. Although our response rate was only 70%, this is comparable to or better than the rates in similar survey studies that used email-based questionnaires.^12,13

Another limitation was that our survey may have forced respondents into answers they did not entirely agree with, given the limited options of the multiple-choice response format and the specific wording of the questions. Our conclusions may have been more dramatic when we were evaluating whether the study was deemed meritorious or not. However, there is no adopted standard for evaluating the extent of bias perceived by a clinician. We thought it was important to include answer options indicating a study had merit despite obvious bias in design and execution. That a study had merit can mean different things. It may change clinical practice, may require further study and reproducibility, or may not be significant enough to matter. Asking follow-up questions to evaluate this perception among the respondents could have provided validity to the term merit. Further studies in this field are needed to determine how studies are interpreted and translated into clinical practice by various clinicians.

Conclusion

Although the present study is not a quantitative analysis of the determination of bias in the orthopedic community, it is the first to evaluate orthopedic surgeons’ perceptions on the basis of key fundamentals of orthopedic research relative to COI. It is clear from our study results that introducing levels of evidence to the orthopedic milieu has had a significant impact both on the quality of research and on the foundational use of deductive reasoning when interpreting the literature. Reporting negative outcomes is perhaps the most important factor in eliminating the perception of bias among orthopedic surgeons. To what extent a perceived COI plays into medical decision-making and the ultimate treatment of patients is still relatively unknown.

As defined by the American Academy of Orthopaedic Surgeons (AAOS) in 1996, conflict of interest (COI) is the “circumstance that exists when, because of personal financial gain, an individual has the potential to be less than objective when called on to reach a judgment or interpret a result.”¹ In medical research, COIs often occur in relationships between physician-researchers and pharmaceutical, medical device, and biotechnology companies. These relationships usually take the form of research grants but can also arise when the researcher has a financial interest in the product being tested or in the company that manufactures the product.

 Although constructive collaboration between academic medicine and industry has worked to improve health care and ultimately benefit patients, potential drawbacks of such relationships include sequestration and suppression of results that may be disadvantageous to the industry sponsor,² increased likelihood of reporting positive results (pro-industry),^3-7 and biased study designs.⁸ The nature of such relationships may threaten the integrity of scientific studies, the objectivity of medical education, the quality of patient care, and the public’s trust in medicine.⁹

Financial relationships and affiliations are increasing as we seek to answer a growing number of clinical questions—with funding often being a limiting factor. At national scientific meetings, the number of presentations reporting COIs reflects this trend. Paper and poster presentations accepted for annual meetings of the Orthopaedic Trauma Association (OTA) and reporting a COI increased from 7.6% in 1993 to 12.6% in 2002 (P = .0129).²

Medical subspecialties outside of orthopedics are experiencing similar trends. Most notable is the American Psychiatric Association (APA). After the APA published a mandatory financial COI disclosure policy in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), the percentage of task force members reporting industry relationships increased by 12%.¹⁰ Analysis of the DSM-5 panels demonstrated that the panels with the largest percentage of reported COIs are those for which pharmacological treatment is the first-line intervention, including the panels for mood disorders (67%), psychotic disorders (83%) and sleep/wake disorders (100%).¹⁰ Moreover, the industry ties reported are to the pharmaceutical companies that manufacture the medications used to treat these disorders or to companies that service the pharmaceutical industry.¹⁰

The degree to which financial COIs affect the interpretation of the orthopedic literature has never been quantified. Although it is clear that COIs can confound the results and reporting of data, how the medical community uses disclosures when interpreting the literature and when formulating opinions that may or may not affect their practice patterns is largely unknown.

We conducted a study to evaluate how a hypothetical financial COI disclosure would influence the interpretation of data by orthopedic clinicians. We also wanted to determine the reliability of the data as perceived in association with different study designs, levels of evidence, research institutional settings, and reporting of positive or negative results.

Methods

We asked members of the Arthroscopy Association of North America (AANA) and the American Orthopaedic Society for Sports Medicine (AOSSM) to complete a multiple-choice situational questionnaire (Table). The questionnaire assesses the degree to which respondents use COI disclosures when interpreting the literature. It further explores the perceived clinical value of a study with a given reported COI, assuming variations in study design, research institutional setting, and significance of results. The fictional research team disclosed the project was funded by a pharmaceutical company and all team members received consulting compensation. The survey and study were reviewed and approved by our institutional review board. The survey consisted of 14 multiple-choice questions that allowed for only 1 answer selection per person and allowed survey takers to skip questions they did not wish to answer. The survey questions and associated response options appear in edited form in the Table. A link to the questionnaire (https://www.surveymonkey.com/s/MPCCLCX) was sent with a message explaining the study. The responses to the questionnaire constituted the data.

Results

We sent a request to participate in the survey to 750 physicians and received 522 responses (overall response rate, 70%). The response rate for each question equaled or exceeded 98%.

The majority of respondents (95.6%) were male. Ninety-nine percent of respondents were orthopedic surgeons. The Northeast (US) was the most common geographical practice location of respondents (32%), followed by the Midwest (19.1%) and the Southeast (16.6%). Most respondents (40%) had been in practice for more than 20 years; 67% had been in practice a minimum of 10 years. The majority (68.8%) were employed by private practice groups, either single specialty (57.8%) or multispecialty (11%).

Eighty percent of respondents strongly agreed that COI disclosure is important when interpreting study results, 62% reported always reading the disclosure slide during academy or other meeting presentations, and 41% reported always using this information when deciding how to interpret scientific data.

Seventy-five percent of respondents thought the study—an academic-center case series with significant results in favor of the pharmaceutical company funding the study—was biased (42% indicated biased with merit, 33% biased without merit). Twenty-three percent thought the study was possibly biased, but likely trustworthy given the academic institutional affiliation. When the study setting was changed to community hospital, 95% thought the study was biased (51% biased with merit, 44% biased without merit). With the same study performed at an academic center, and no statistically significant results (not in favor of the pharmaceutical company funding the study), 88% thought the study had merit (46% biased with merit, 42% unbiased with merit).

When the study design was changed to a randomized controlled trial (level I evidence) conducted at an academic center with negative results, an overwhelming 95% of respondents thought the study had merit (33% biased with merit, 62% unbiased with merit). Given the same study design at an academic center, with positive results, 78% still thought the study had merit (39% biased with merit, 39% unbiased with merit). An additional 18% thought the study was biased, but still likely trustworthy given the academic institutional affiliation. Finally, given a randomized controlled trial and positive results, but with the research setting a small community practice, 90% thought the study had merit (51% biased with merit, 39% unbiased with merit). The percentage of respondents who found the study biased and likely without merit increased from 3.7% to 9.5% when the institutional affiliation changed from academic to community.

Discussion

As governmental funding sources become increasingly limited, the role of industry sponsorship of orthopedic research has grown. Potential drawbacks and biases of such research support have been well described—most notably, increased positive result reporting, suppression of results that may be disadvantageous to the industry sponsor, and biased study designs.^2-8 However, the extent to which financial COIs affect the orthopedic medical community’s interpretation of the literature has never been quantified. To our knowledge, the present study is the first to quantify the impact of reported COI on study interpretation.

Our goal was to examine how reported financial COIs influence the interpretation of the literature by the orthopedic medical community. Moreover, we wanted to determine the perceived reliability of the data when variables (study design, institutional affiliation, positive vs negative results) were changed. The results of our survey indicate that, when a financial COI is reported, study reliability is perceived as highest when negative results were found.

Our survey noted a discrepancy between the documented importance of the hypothetical research team’s COI disclosure and the use of such disclosures when interpreting study results. Eighty percent of respondents agreed that COI disclosure is important when interpreting study results, but only 62% reported always reading disclosures, and even fewer (41%) reported always using the information when interpreting results. It is unclear exactly why this trend exists, as one would expect the percentages to be more similar. These particular survey questions were formed around using COI disclosures when interpreting study results during academic presentations at national meetings and not during the review of published literature. It is possible that positioning the COI disclosure at the beginning of a presentation has an effect, but only 3.7% of respondents indicated they seldom remembered the disclosure by the end of the presentation. The results of our survey may have varied if the questions had targeted reading and interpreting the literature.

Interestingly, the results of these survey questions tended to be more consistent with rates of reported financial COI by presenters at national orthopedic meetings. A study published in the New England Journal of Medicine found that less than 80% of orthopedic surgeons reported their disclosures at a large annual meeting (AAOS), even when the disclosure involved payments pertinent to the research they were presenting.⁵ When the payments were indirectly related to the research, the percentage of surgeons reporting disclosures was 50%, almost the same as the disclosure rate for unrelated payments.⁵

When the study was changed to a level I randomized controlled trial, more survey respondents found it to be less biased and have more merit. Although it would seem intuitive for a study with a higher level of evidence to carry more clinical value during interpretation, this may not hold true in the setting of industry-sponsored clinical trials. Several studies have documented a significant association between the reporting of positive results and industry-sponsored randomized clinical trials. In 2008, Khan and colleagues³ examined 100 orthopedic randomized clinical trials reported in 5 major orthopedic subspecialty journals over a 2-year period. The association between industry funding and favorable outcome in all original randomized clinical trials was strong and significant (P < .001). This is not surprising, given the amount of time and money required for a well-designed clinical study. Commercial products with preclinical promise are pushed to testing in a clinical trial, whereas resources would not be wasted on products lacking preclinical merit.

The most important variable affecting interpretation of study merit by survey respondents was the reporting of negative results. As more researchers are developing COIs, many studies are discovering a relationship between COIs and outcomes of research studies. Reviewing the adult total joint literature, Ezzet⁸ found an industry funding rate of 50%. Positive results were reported in 93% of cases in commercially funded studies versus 37% of cases in independently funded studies. Furthermore, no negative results were reported by investigators who were receiving royalties from the respective companies.

Studies across the medical literature have also found this association between industry sponsorship and reporting of positive results. One such study, reported by Valachis and colleagues⁷ in the Journal of Clinical Oncology, examined more than 80 economic analyses of targeted oncologic therapies and found the studies funded by pharmaceutical companies were more likely to report favorable qualitative cost estimates. In addition, when studies with a COI disclosure were examined, those reporting any financial relationship with a manufacturer (eg, author affiliation, funding) were more likely than those without such a relationship to report favorable results.

Our study had several limitations. First, as most of the survey respondents were orthopedic surgeons, extrapolating their data to the medical community at large may not be appropriate, as each specialty may view industry affiliations differently. In addition, respondents were asked to base their interpretations of a study on conclusions we predetermined—no direct visualization of the data set or statistical testing methods. It is possible that these responses may have been different had the respondents had the opportunity to further evaluate the study in question. In a recent study, Altwairgi and colleagues¹¹ found that 10% of randomized clinical trials involving lung cancer treatment were reported with different conclusions in their full manuscripts relative to their abstracts. We think our survey design perhaps best mimics an annual meeting environment in which participants have very limited ability to interpret studies and may rely more heavily on the factors we investigated—study design, significance of findings, and setting, all similar to information presented in an abstract—when making informed decisions. Although our response rate was only 70%, this is comparable to or better than the rates in similar survey studies that used email-based questionnaires.^12,13

Another limitation was that our survey may have forced respondents into answers they did not entirely agree with, given the limited options of the multiple-choice response format and the specific wording of the questions. Our conclusions may have been more dramatic when we were evaluating whether the study was deemed meritorious or not. However, there is no adopted standard for evaluating the extent of bias perceived by a clinician. We thought it was important to include answer options indicating a study had merit despite obvious bias in design and execution. That a study had merit can mean different things. It may change clinical practice, may require further study and reproducibility, or may not be significant enough to matter. Asking follow-up questions to evaluate this perception among the respondents could have provided validity to the term merit. Further studies in this field are needed to determine how studies are interpreted and translated into clinical practice by various clinicians.

Conclusion

Although the present study is not a quantitative analysis of the determination of bias in the orthopedic community, it is the first to evaluate orthopedic surgeons’ perceptions on the basis of key fundamentals of orthopedic research relative to COI. It is clear from our study results that introducing levels of evidence to the orthopedic milieu has had a significant impact both on the quality of research and on the foundational use of deductive reasoning when interpreting the literature. Reporting negative outcomes is perhaps the most important factor in eliminating the perception of bias among orthopedic surgeons. To what extent a perceived COI plays into medical decision-making and the ultimate treatment of patients is still relatively unknown.

As defined by the American Academy of Orthopaedic Surgeons (AAOS) in 1996, conflict of interest (COI) is the “circumstance that exists when, because of personal financial gain, an individual has the potential to be less than objective when called on to reach a judgment or interpret a result.”¹ In medical research, COIs often occur in relationships between physician-researchers and pharmaceutical, medical device, and biotechnology companies. These relationships usually take the form of research grants but can also arise when the researcher has a financial interest in the product being tested or in the company that manufactures the product.

 Although constructive collaboration between academic medicine and industry has worked to improve health care and ultimately benefit patients, potential drawbacks of such relationships include sequestration and suppression of results that may be disadvantageous to the industry sponsor,² increased likelihood of reporting positive results (pro-industry),^3-7 and biased study designs.⁸ The nature of such relationships may threaten the integrity of scientific studies, the objectivity of medical education, the quality of patient care, and the public’s trust in medicine.⁹

Financial relationships and affiliations are increasing as we seek to answer a growing number of clinical questions—with funding often being a limiting factor. At national scientific meetings, the number of presentations reporting COIs reflects this trend. Paper and poster presentations accepted for annual meetings of the Orthopaedic Trauma Association (OTA) and reporting a COI increased from 7.6% in 1993 to 12.6% in 2002 (P = .0129).²

Medical subspecialties outside of orthopedics are experiencing similar trends. Most notable is the American Psychiatric Association (APA). After the APA published a mandatory financial COI disclosure policy in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), the percentage of task force members reporting industry relationships increased by 12%.¹⁰ Analysis of the DSM-5 panels demonstrated that the panels with the largest percentage of reported COIs are those for which pharmacological treatment is the first-line intervention, including the panels for mood disorders (67%), psychotic disorders (83%) and sleep/wake disorders (100%).¹⁰ Moreover, the industry ties reported are to the pharmaceutical companies that manufacture the medications used to treat these disorders or to companies that service the pharmaceutical industry.¹⁰

The degree to which financial COIs affect the interpretation of the orthopedic literature has never been quantified. Although it is clear that COIs can confound the results and reporting of data, how the medical community uses disclosures when interpreting the literature and when formulating opinions that may or may not affect their practice patterns is largely unknown.

We conducted a study to evaluate how a hypothetical financial COI disclosure would influence the interpretation of data by orthopedic clinicians. We also wanted to determine the reliability of the data as perceived in association with different study designs, levels of evidence, research institutional settings, and reporting of positive or negative results.

Methods

We asked members of the Arthroscopy Association of North America (AANA) and the American Orthopaedic Society for Sports Medicine (AOSSM) to complete a multiple-choice situational questionnaire (Table). The questionnaire assesses the degree to which respondents use COI disclosures when interpreting the literature. It further explores the perceived clinical value of a study with a given reported COI, assuming variations in study design, research institutional setting, and significance of results. The fictional research team disclosed the project was funded by a pharmaceutical company and all team members received consulting compensation. The survey and study were reviewed and approved by our institutional review board. The survey consisted of 14 multiple-choice questions that allowed for only 1 answer selection per person and allowed survey takers to skip questions they did not wish to answer. The survey questions and associated response options appear in edited form in the Table. A link to the questionnaire (https://www.surveymonkey.com/s/MPCCLCX) was sent with a message explaining the study. The responses to the questionnaire constituted the data.

Results

We sent a request to participate in the survey to 750 physicians and received 522 responses (overall response rate, 70%). The response rate for each question equaled or exceeded 98%.

The majority of respondents (95.6%) were male. Ninety-nine percent of respondents were orthopedic surgeons. The Northeast (US) was the most common geographical practice location of respondents (32%), followed by the Midwest (19.1%) and the Southeast (16.6%). Most respondents (40%) had been in practice for more than 20 years; 67% had been in practice a minimum of 10 years. The majority (68.8%) were employed by private practice groups, either single specialty (57.8%) or multispecialty (11%).

Eighty percent of respondents strongly agreed that COI disclosure is important when interpreting study results, 62% reported always reading the disclosure slide during academy or other meeting presentations, and 41% reported always using this information when deciding how to interpret scientific data.

Seventy-five percent of respondents thought the study—an academic-center case series with significant results in favor of the pharmaceutical company funding the study—was biased (42% indicated biased with merit, 33% biased without merit). Twenty-three percent thought the study was possibly biased, but likely trustworthy given the academic institutional affiliation. When the study setting was changed to community hospital, 95% thought the study was biased (51% biased with merit, 44% biased without merit). With the same study performed at an academic center, and no statistically significant results (not in favor of the pharmaceutical company funding the study), 88% thought the study had merit (46% biased with merit, 42% unbiased with merit).

When the study design was changed to a randomized controlled trial (level I evidence) conducted at an academic center with negative results, an overwhelming 95% of respondents thought the study had merit (33% biased with merit, 62% unbiased with merit). Given the same study design at an academic center, with positive results, 78% still thought the study had merit (39% biased with merit, 39% unbiased with merit). An additional 18% thought the study was biased, but still likely trustworthy given the academic institutional affiliation. Finally, given a randomized controlled trial and positive results, but with the research setting a small community practice, 90% thought the study had merit (51% biased with merit, 39% unbiased with merit). The percentage of respondents who found the study biased and likely without merit increased from 3.7% to 9.5% when the institutional affiliation changed from academic to community.

Discussion

As governmental funding sources become increasingly limited, the role of industry sponsorship of orthopedic research has grown. Potential drawbacks and biases of such research support have been well described—most notably, increased positive result reporting, suppression of results that may be disadvantageous to the industry sponsor, and biased study designs.^2-8 However, the extent to which financial COIs affect the orthopedic medical community’s interpretation of the literature has never been quantified. To our knowledge, the present study is the first to quantify the impact of reported COI on study interpretation.

Our goal was to examine how reported financial COIs influence the interpretation of the literature by the orthopedic medical community. Moreover, we wanted to determine the perceived reliability of the data when variables (study design, institutional affiliation, positive vs negative results) were changed. The results of our survey indicate that, when a financial COI is reported, study reliability is perceived as highest when negative results were found.

Our survey noted a discrepancy between the documented importance of the hypothetical research team’s COI disclosure and the use of such disclosures when interpreting study results. Eighty percent of respondents agreed that COI disclosure is important when interpreting study results, but only 62% reported always reading disclosures, and even fewer (41%) reported always using the information when interpreting results. It is unclear exactly why this trend exists, as one would expect the percentages to be more similar. These particular survey questions were formed around using COI disclosures when interpreting study results during academic presentations at national meetings and not during the review of published literature. It is possible that positioning the COI disclosure at the beginning of a presentation has an effect, but only 3.7% of respondents indicated they seldom remembered the disclosure by the end of the presentation. The results of our survey may have varied if the questions had targeted reading and interpreting the literature.

Interestingly, the results of these survey questions tended to be more consistent with rates of reported financial COI by presenters at national orthopedic meetings. A study published in the New England Journal of Medicine found that less than 80% of orthopedic surgeons reported their disclosures at a large annual meeting (AAOS), even when the disclosure involved payments pertinent to the research they were presenting.⁵ When the payments were indirectly related to the research, the percentage of surgeons reporting disclosures was 50%, almost the same as the disclosure rate for unrelated payments.⁵

When the study was changed to a level I randomized controlled trial, more survey respondents found it to be less biased and have more merit. Although it would seem intuitive for a study with a higher level of evidence to carry more clinical value during interpretation, this may not hold true in the setting of industry-sponsored clinical trials. Several studies have documented a significant association between the reporting of positive results and industry-sponsored randomized clinical trials. In 2008, Khan and colleagues³ examined 100 orthopedic randomized clinical trials reported in 5 major orthopedic subspecialty journals over a 2-year period. The association between industry funding and favorable outcome in all original randomized clinical trials was strong and significant (P < .001). This is not surprising, given the amount of time and money required for a well-designed clinical study. Commercial products with preclinical promise are pushed to testing in a clinical trial, whereas resources would not be wasted on products lacking preclinical merit.

The most important variable affecting interpretation of study merit by survey respondents was the reporting of negative results. As more researchers are developing COIs, many studies are discovering a relationship between COIs and outcomes of research studies. Reviewing the adult total joint literature, Ezzet⁸ found an industry funding rate of 50%. Positive results were reported in 93% of cases in commercially funded studies versus 37% of cases in independently funded studies. Furthermore, no negative results were reported by investigators who were receiving royalties from the respective companies.

Studies across the medical literature have also found this association between industry sponsorship and reporting of positive results. One such study, reported by Valachis and colleagues⁷ in the Journal of Clinical Oncology, examined more than 80 economic analyses of targeted oncologic therapies and found the studies funded by pharmaceutical companies were more likely to report favorable qualitative cost estimates. In addition, when studies with a COI disclosure were examined, those reporting any financial relationship with a manufacturer (eg, author affiliation, funding) were more likely than those without such a relationship to report favorable results.

Our study had several limitations. First, as most of the survey respondents were orthopedic surgeons, extrapolating their data to the medical community at large may not be appropriate, as each specialty may view industry affiliations differently. In addition, respondents were asked to base their interpretations of a study on conclusions we predetermined—no direct visualization of the data set or statistical testing methods. It is possible that these responses may have been different had the respondents had the opportunity to further evaluate the study in question. In a recent study, Altwairgi and colleagues¹¹ found that 10% of randomized clinical trials involving lung cancer treatment were reported with different conclusions in their full manuscripts relative to their abstracts. We think our survey design perhaps best mimics an annual meeting environment in which participants have very limited ability to interpret studies and may rely more heavily on the factors we investigated—study design, significance of findings, and setting, all similar to information presented in an abstract—when making informed decisions. Although our response rate was only 70%, this is comparable to or better than the rates in similar survey studies that used email-based questionnaires.^12,13

Another limitation was that our survey may have forced respondents into answers they did not entirely agree with, given the limited options of the multiple-choice response format and the specific wording of the questions. Our conclusions may have been more dramatic when we were evaluating whether the study was deemed meritorious or not. However, there is no adopted standard for evaluating the extent of bias perceived by a clinician. We thought it was important to include answer options indicating a study had merit despite obvious bias in design and execution. That a study had merit can mean different things. It may change clinical practice, may require further study and reproducibility, or may not be significant enough to matter. Asking follow-up questions to evaluate this perception among the respondents could have provided validity to the term merit. Further studies in this field are needed to determine how studies are interpreted and translated into clinical practice by various clinicians.

Conclusion

Although the present study is not a quantitative analysis of the determination of bias in the orthopedic community, it is the first to evaluate orthopedic surgeons’ perceptions on the basis of key fundamentals of orthopedic research relative to COI. It is clear from our study results that introducing levels of evidence to the orthopedic milieu has had a significant impact both on the quality of research and on the foundational use of deductive reasoning when interpreting the literature. Reporting negative outcomes is perhaps the most important factor in eliminating the perception of bias among orthopedic surgeons. To what extent a perceived COI plays into medical decision-making and the ultimate treatment of patients is still relatively unknown.