Stephen A. Wilson, MD

Survival curves illustrate prognosis. The percentage of patients reaching an endpoint (eg, death, recurrence of disease, or cure) is plotted on the y(vertical) axis against time on the x(horizontal) axis.

Plotting a survival curve

Two common plotting methods are used. With the actuarial method, the x axis is divided into regular intervals (eg, by month) and percent survival is calculated for each interval. With the Kaplan-Meier method, percent survival is recalculated each time a patient dies (or reaches a different endpoint). Consider the example here (Figure).¹

Time zerois when each patient entered the trial. Survival is the percentage of patients still alive thereafter. Median survivalis found by extending a horizontal line from the 50% survival point until it intersects the curve (24 months in this case).

FIGURE
Sample survival curve

Survival curves have limitations. Consider a study that enrolls patients between 1996 and 2002 and ends in 2005. All that is known about a patient enrolled in 2002 who survived until 2005 is that he or she survived 3 years. Some patients also drop out of the study early or are lost to follow-up. Some patients die from causes other than the one under study.

Censoringis the process of excluding data from survival curves when information about survival is unknown. For a patient who drops out early, for example, only data obtained when the patient was followed would be included. The result is a more accurate picture of survival for the patients under study.

Correspondence
Stephen A. Wilson, MD, UPMC Family Praxctice Residency, 200 Lothrop St., Pittsburgh, PA 15213-2582. E-mail: wilsons2@upmc.edu.

Survival curves illustrate prognosis. The percentage of patients reaching an endpoint (eg, death, recurrence of disease, or cure) is plotted on the y(vertical) axis against time on the x(horizontal) axis.

Plotting a survival curve

Two common plotting methods are used. With the actuarial method, the x axis is divided into regular intervals (eg, by month) and percent survival is calculated for each interval. With the Kaplan-Meier method, percent survival is recalculated each time a patient dies (or reaches a different endpoint). Consider the example here (Figure).¹

Time zerois when each patient entered the trial. Survival is the percentage of patients still alive thereafter. Median survivalis found by extending a horizontal line from the 50% survival point until it intersects the curve (24 months in this case).

FIGURE
Sample survival curve

Survival curves have limitations. Consider a study that enrolls patients between 1996 and 2002 and ends in 2005. All that is known about a patient enrolled in 2002 who survived until 2005 is that he or she survived 3 years. Some patients also drop out of the study early or are lost to follow-up. Some patients die from causes other than the one under study.

Censoringis the process of excluding data from survival curves when information about survival is unknown. For a patient who drops out early, for example, only data obtained when the patient was followed would be included. The result is a more accurate picture of survival for the patients under study.

Correspondence
Stephen A. Wilson, MD, UPMC Family Praxctice Residency, 200 Lothrop St., Pittsburgh, PA 15213-2582. E-mail: wilsons2@upmc.edu.

Survival curves illustrate prognosis. The percentage of patients reaching an endpoint (eg, death, recurrence of disease, or cure) is plotted on the y(vertical) axis against time on the x(horizontal) axis.

Plotting a survival curve

Two common plotting methods are used. With the actuarial method, the x axis is divided into regular intervals (eg, by month) and percent survival is calculated for each interval. With the Kaplan-Meier method, percent survival is recalculated each time a patient dies (or reaches a different endpoint). Consider the example here (Figure).¹

Time zerois when each patient entered the trial. Survival is the percentage of patients still alive thereafter. Median survivalis found by extending a horizontal line from the 50% survival point until it intersects the curve (24 months in this case).

FIGURE
Sample survival curve

Survival curves have limitations. Consider a study that enrolls patients between 1996 and 2002 and ends in 2005. All that is known about a patient enrolled in 2002 who survived until 2005 is that he or she survived 3 years. Some patients also drop out of the study early or are lost to follow-up. Some patients die from causes other than the one under study.

Censoringis the process of excluding data from survival curves when information about survival is unknown. For a patient who drops out early, for example, only data obtained when the patient was followed would be included. The result is a more accurate picture of survival for the patients under study.

Correspondence
Stephen A. Wilson, MD, UPMC Family Praxctice Residency, 200 Lothrop St., Pittsburgh, PA 15213-2582. E-mail: wilsons2@upmc.edu.

Evidence summary

Several randomized controlled trials and a meta-analysis demonstrated that the children most likely to benefit from tympanostomy tubes are those more than 6 months old with middle-ear effusion who have had 3 or more episodes of acute otitis media in 6 months, or 4 or more episodes in 12 months.^1-4 Data are inadequate to determine the lowest rate of recurrence that would suggest a benefit from tube placement.

A meta-analysis of 5 randomized trials comparing no surgery with placement of tubes for recurrent acute otitis media with or without middle-ear effusion showed that the placement of tubes resulted in a mean absolute decrease in acute otitis media incidence of 1.0 per year (95% confidence interval [CI], 0.4–1.6), and a decrease in the prevalence of middle-ear effusion by 115 days per year (95% CI, 11–220).⁴ The benefit of tubes for recurrent acute otitis media was demonstrated only in studies in which middle-ear effusion was present:^2,3 one found 3.01 (95% CI, 2.18–3.84) fewer acute episodes per year;^1,4 the other found 2.27 (95% CI, 1.03–3.51) fewer.^2,4

One randomized controlled trial of 264 children, aged 7 to 35 months, with a history of recurrent acute otitis media but free of middle-ear effusion, compared tubes with medical therapy and found no difference in recurrence over 2 years.³ The medical therapy arm received prophylaxis with either amoxicillin or placebo. The amoxicillin arm had 0.6 fewer episodes of acute otitis media per year compared with the other 2, a statistically significant 40% decrease (relative risk reduction=0.4).³

The average time with otitis media of any type (acute otitis media, otitis media with effusion, or ottorhea) also decreased—15.0% in the placebo group, 10.0% in the amoxicillin group, and 6.6% in the tympanostomy tube group (amoxicillin vs. placebo, P=.03; tubes vs. placebo, P<.001).³ Higher dropout rates occurred in the amoxicillin and medical treatment groups.³

In prospective studies of patients receiving tubes for recurrent acute otitis media and otitis media with effusion, measures of quality of life—physical suffering, emotional distress, activity limitation, hearing loss, speech development, caregiver concern/worry, parental post-tube satisfaction,^4,5,6 and an ear symptom score⁶ —improved after tube placement. Within several weeks of tube placement, 79% of children had improved quality of life, 17% had trivial change, and 4% were worse.⁴

A meta-analysis reporting sequelae of tympanostomy tubes found an absolute complication rate of 26% for transient otorrhea and 4% for chronic otorrhea.⁴

Compared with nonsurgical treatment, complication rates for tube placement were reported in 0.7% of surgically treated ears.⁷ Complications included:

• tympanosclerosis (relative risk [RR]=3.5 [95% CI, 2.6–4.9])
• focal atrophy (RR=1.7 [95% CI, 1.1–2.7])
• perforation (RR=3.5 [95% CI, 1.5–7.1])
  • 2% with short-term tubes
  • 16% with long-term tubes
• cholesteatoma (RR=2.6 [95% CI, 1.5–4.4]).

Recommendations from others

The Institute for Clinical Systems Improvement 2001 guidelines for recurrent acute otitis media treatment in children recommends initial antibiotic prophylaxis with amoxicillin (20 mg/kg/day) for 2 to 6 months (based on randomized controlled trial data). If there are 2 recurrences of acute otitis media during that time, then referral to an otorhinolaryngologist for possible tympanostomy tube placement is recommended.⁸

Evidence summary

Several randomized controlled trials and a meta-analysis demonstrated that the children most likely to benefit from tympanostomy tubes are those more than 6 months old with middle-ear effusion who have had 3 or more episodes of acute otitis media in 6 months, or 4 or more episodes in 12 months.^1-4 Data are inadequate to determine the lowest rate of recurrence that would suggest a benefit from tube placement.

A meta-analysis of 5 randomized trials comparing no surgery with placement of tubes for recurrent acute otitis media with or without middle-ear effusion showed that the placement of tubes resulted in a mean absolute decrease in acute otitis media incidence of 1.0 per year (95% confidence interval [CI], 0.4–1.6), and a decrease in the prevalence of middle-ear effusion by 115 days per year (95% CI, 11–220).⁴ The benefit of tubes for recurrent acute otitis media was demonstrated only in studies in which middle-ear effusion was present:^2,3 one found 3.01 (95% CI, 2.18–3.84) fewer acute episodes per year;^1,4 the other found 2.27 (95% CI, 1.03–3.51) fewer.^2,4

One randomized controlled trial of 264 children, aged 7 to 35 months, with a history of recurrent acute otitis media but free of middle-ear effusion, compared tubes with medical therapy and found no difference in recurrence over 2 years.³ The medical therapy arm received prophylaxis with either amoxicillin or placebo. The amoxicillin arm had 0.6 fewer episodes of acute otitis media per year compared with the other 2, a statistically significant 40% decrease (relative risk reduction=0.4).³

The average time with otitis media of any type (acute otitis media, otitis media with effusion, or ottorhea) also decreased—15.0% in the placebo group, 10.0% in the amoxicillin group, and 6.6% in the tympanostomy tube group (amoxicillin vs. placebo, P=.03; tubes vs. placebo, P<.001).³ Higher dropout rates occurred in the amoxicillin and medical treatment groups.³

In prospective studies of patients receiving tubes for recurrent acute otitis media and otitis media with effusion, measures of quality of life—physical suffering, emotional distress, activity limitation, hearing loss, speech development, caregiver concern/worry, parental post-tube satisfaction,^4,5,6 and an ear symptom score⁶ —improved after tube placement. Within several weeks of tube placement, 79% of children had improved quality of life, 17% had trivial change, and 4% were worse.⁴

A meta-analysis reporting sequelae of tympanostomy tubes found an absolute complication rate of 26% for transient otorrhea and 4% for chronic otorrhea.⁴

Compared with nonsurgical treatment, complication rates for tube placement were reported in 0.7% of surgically treated ears.⁷ Complications included:

• tympanosclerosis (relative risk [RR]=3.5 [95% CI, 2.6–4.9])
• focal atrophy (RR=1.7 [95% CI, 1.1–2.7])
• perforation (RR=3.5 [95% CI, 1.5–7.1])
  • 2% with short-term tubes
  • 16% with long-term tubes
• cholesteatoma (RR=2.6 [95% CI, 1.5–4.4]).

Recommendations from others

The Institute for Clinical Systems Improvement 2001 guidelines for recurrent acute otitis media treatment in children recommends initial antibiotic prophylaxis with amoxicillin (20 mg/kg/day) for 2 to 6 months (based on randomized controlled trial data). If there are 2 recurrences of acute otitis media during that time, then referral to an otorhinolaryngologist for possible tympanostomy tube placement is recommended.⁸

Evidence summary

Several randomized controlled trials and a meta-analysis demonstrated that the children most likely to benefit from tympanostomy tubes are those more than 6 months old with middle-ear effusion who have had 3 or more episodes of acute otitis media in 6 months, or 4 or more episodes in 12 months.^1-4 Data are inadequate to determine the lowest rate of recurrence that would suggest a benefit from tube placement.

A meta-analysis of 5 randomized trials comparing no surgery with placement of tubes for recurrent acute otitis media with or without middle-ear effusion showed that the placement of tubes resulted in a mean absolute decrease in acute otitis media incidence of 1.0 per year (95% confidence interval [CI], 0.4–1.6), and a decrease in the prevalence of middle-ear effusion by 115 days per year (95% CI, 11–220).⁴ The benefit of tubes for recurrent acute otitis media was demonstrated only in studies in which middle-ear effusion was present:^2,3 one found 3.01 (95% CI, 2.18–3.84) fewer acute episodes per year;^1,4 the other found 2.27 (95% CI, 1.03–3.51) fewer.^2,4

One randomized controlled trial of 264 children, aged 7 to 35 months, with a history of recurrent acute otitis media but free of middle-ear effusion, compared tubes with medical therapy and found no difference in recurrence over 2 years.³ The medical therapy arm received prophylaxis with either amoxicillin or placebo. The amoxicillin arm had 0.6 fewer episodes of acute otitis media per year compared with the other 2, a statistically significant 40% decrease (relative risk reduction=0.4).³

The average time with otitis media of any type (acute otitis media, otitis media with effusion, or ottorhea) also decreased—15.0% in the placebo group, 10.0% in the amoxicillin group, and 6.6% in the tympanostomy tube group (amoxicillin vs. placebo, P=.03; tubes vs. placebo, P<.001).³ Higher dropout rates occurred in the amoxicillin and medical treatment groups.³

In prospective studies of patients receiving tubes for recurrent acute otitis media and otitis media with effusion, measures of quality of life—physical suffering, emotional distress, activity limitation, hearing loss, speech development, caregiver concern/worry, parental post-tube satisfaction,^4,5,6 and an ear symptom score⁶ —improved after tube placement. Within several weeks of tube placement, 79% of children had improved quality of life, 17% had trivial change, and 4% were worse.⁴

A meta-analysis reporting sequelae of tympanostomy tubes found an absolute complication rate of 26% for transient otorrhea and 4% for chronic otorrhea.⁴

Compared with nonsurgical treatment, complication rates for tube placement were reported in 0.7% of surgically treated ears.⁷ Complications included:

• tympanosclerosis (relative risk [RR]=3.5 [95% CI, 2.6–4.9])
• focal atrophy (RR=1.7 [95% CI, 1.1–2.7])
• perforation (RR=3.5 [95% CI, 1.5–7.1])
  • 2% with short-term tubes
  • 16% with long-term tubes
• cholesteatoma (RR=2.6 [95% CI, 1.5–4.4]).

Recommendations from others

The Institute for Clinical Systems Improvement 2001 guidelines for recurrent acute otitis media treatment in children recommends initial antibiotic prophylaxis with amoxicillin (20 mg/kg/day) for 2 to 6 months (based on randomized controlled trial data). If there are 2 recurrences of acute otitis media during that time, then referral to an otorhinolaryngologist for possible tympanostomy tube placement is recommended.⁸

Evidence summary

In studies that evaluated onetime softening in the office to ease or eliminate the need for irrigation, a presoak with docusate sodium (Colace) was most effective, although its effects were not compared with those of water.¹ Both triethanolamine (Cerumenex) and olive oil were the next most effective treatments.² Carbamide peroxide (Debrox, Murine Ear) was least effective (see Table and Table W1*).³ In 1 small, carefully done study of ear candles, more candle wax was added than earwax was removed in the 8 ears tested.⁴

In studies that evaluated 3 to 14 days of home ceruminolysis to obviate or ease irritation, 5% urea hydrogen peroxide in glycerol was most effective.⁵ Sterile water, sodium bicarbonate in glycerol, 2% acetic acid (VoSoL, Domeboro), ethylene oxide polyoxypropylene (Addax), and acpd (arachis oil, chlorobutanol, p-dichlorobenzene [Cerumenol]) were all of equal efficacy.^6-8 All were more effective than no treatment. Notably, 5% of cases resolved completely and 26% became moderately clear after 5 days of no treatment (Table W2*).⁶

No direct comparisons exist of same-day in-office softening followed by irrigation or disimpaction against home softening followed by irrigation and manual disimpaction. Until more placebo-controlled data are generated, recommendations should be based on relative safety and on the direct comparison trials within each strategy. Complications of irrigation include otitis externa, perforation, canal trauma, pain, cough, tinnitus, vertigo, otitis media, treatment failure, and time consumption.⁹ Harm done by wax softeners is minimal.

TABLE
EVIDENCE SUMMARY FOR IN-OFFICE CERUMEN REMOVAL

Recommendations from others

The 5-Minute Clinical Consult 2001 recommends Cerumenex followed by irrigation in office. Clinical Evidence 2001 reports that clinically accepted standards are ear syringing and manual disimpaction, although no randomized clinical trials addressing benefit or harm have been conducted. No specific recommendation made because of inconsistent, unclear study design or undefined terms (eg, impaction).

Evidence summary

In studies that evaluated onetime softening in the office to ease or eliminate the need for irrigation, a presoak with docusate sodium (Colace) was most effective, although its effects were not compared with those of water.¹ Both triethanolamine (Cerumenex) and olive oil were the next most effective treatments.² Carbamide peroxide (Debrox, Murine Ear) was least effective (see Table and Table W1*).³ In 1 small, carefully done study of ear candles, more candle wax was added than earwax was removed in the 8 ears tested.⁴

In studies that evaluated 3 to 14 days of home ceruminolysis to obviate or ease irritation, 5% urea hydrogen peroxide in glycerol was most effective.⁵ Sterile water, sodium bicarbonate in glycerol, 2% acetic acid (VoSoL, Domeboro), ethylene oxide polyoxypropylene (Addax), and acpd (arachis oil, chlorobutanol, p-dichlorobenzene [Cerumenol]) were all of equal efficacy.^6-8 All were more effective than no treatment. Notably, 5% of cases resolved completely and 26% became moderately clear after 5 days of no treatment (Table W2*).⁶

No direct comparisons exist of same-day in-office softening followed by irrigation or disimpaction against home softening followed by irrigation and manual disimpaction. Until more placebo-controlled data are generated, recommendations should be based on relative safety and on the direct comparison trials within each strategy. Complications of irrigation include otitis externa, perforation, canal trauma, pain, cough, tinnitus, vertigo, otitis media, treatment failure, and time consumption.⁹ Harm done by wax softeners is minimal.

TABLE
EVIDENCE SUMMARY FOR IN-OFFICE CERUMEN REMOVAL

Recommendations from others

The 5-Minute Clinical Consult 2001 recommends Cerumenex followed by irrigation in office. Clinical Evidence 2001 reports that clinically accepted standards are ear syringing and manual disimpaction, although no randomized clinical trials addressing benefit or harm have been conducted. No specific recommendation made because of inconsistent, unclear study design or undefined terms (eg, impaction).

Evidence summary

In studies that evaluated onetime softening in the office to ease or eliminate the need for irrigation, a presoak with docusate sodium (Colace) was most effective, although its effects were not compared with those of water.¹ Both triethanolamine (Cerumenex) and olive oil were the next most effective treatments.² Carbamide peroxide (Debrox, Murine Ear) was least effective (see Table and Table W1*).³ In 1 small, carefully done study of ear candles, more candle wax was added than earwax was removed in the 8 ears tested.⁴

In studies that evaluated 3 to 14 days of home ceruminolysis to obviate or ease irritation, 5% urea hydrogen peroxide in glycerol was most effective.⁵ Sterile water, sodium bicarbonate in glycerol, 2% acetic acid (VoSoL, Domeboro), ethylene oxide polyoxypropylene (Addax), and acpd (arachis oil, chlorobutanol, p-dichlorobenzene [Cerumenol]) were all of equal efficacy.^6-8 All were more effective than no treatment. Notably, 5% of cases resolved completely and 26% became moderately clear after 5 days of no treatment (Table W2*).⁶

No direct comparisons exist of same-day in-office softening followed by irrigation or disimpaction against home softening followed by irrigation and manual disimpaction. Until more placebo-controlled data are generated, recommendations should be based on relative safety and on the direct comparison trials within each strategy. Complications of irrigation include otitis externa, perforation, canal trauma, pain, cough, tinnitus, vertigo, otitis media, treatment failure, and time consumption.⁹ Harm done by wax softeners is minimal.

TABLE
EVIDENCE SUMMARY FOR IN-OFFICE CERUMEN REMOVAL

Recommendations from others

The 5-Minute Clinical Consult 2001 recommends Cerumenex followed by irrigation in office. Clinical Evidence 2001 reports that clinically accepted standards are ear syringing and manual disimpaction, although no randomized clinical trials addressing benefit or harm have been conducted. No specific recommendation made because of inconsistent, unclear study design or undefined terms (eg, impaction).

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.