User login
“How come you retired?” I asked.
A few years my junior, Marty had taught in public school forever. “It was the MCAS,” he said. That’s the Massachusetts Comprehensive Assessment System, a standardized test meant to gauge student performance and teacher competence.
“They demanded that my students test at a fifth-grade level,” Marty said. “But they were all at a second-grade level.
“Plus, I had been teaching for thirty years, and some kid right out of college was telling me how to do my job. So I left.”
Of course, this tale will sound familiar to physicians. Pay for performance. Bean counters calling the shots. Dismissal of clinical experience as useless and self-serving.
A recent book lays it all out: Jerry Z. Muller’s The Tyranny of Metrics. This book is punchy, witty, and succinct – you can read it in a day. A historian of economics and culture, Muller shows the extent of what I had guessed at from chats with people in different fields. In all of these, if you don’t count it, it doesn’t count.
Metrics, it is assumed, are “hard” and “objective.” They must “replace judgment based on experience with standardized measurement.” Their promise is transparency, efficiency, accountability.
Muller began to study this when he became chair of his academic department. He thought his job was to nurture scholars and help students learn, only to find much of his time taken up with feeding often worthless data to remote administrators. He traces the metrical impulse, at root, to lack of trust. It’s not only doctors whom society doesn’t trust, but experts of all kinds.
Principal agents ... “employed in institutions are not to be trusted … their activity must be monitored and measured ... those measures need to be transparent to those without firsthand knowledge of the institutions ... and ... pecuniary rewards and punishments are the best way to motivate ‘agents.’ ”
What this analysis ignores, argues Muller, is that professionals respond not just to “extrinsic motivation[s]” (money) but to intrinsic ones: commitment to profession and clients, doing the right thing, making people happier and better, being recognized and honored by peers, doing interesting and stimulating work. When society denigrates and dismisses those considerations, professionals become demoralized. They leave, or they learn to game the system.
Muller gives many examples. Punish hospitals for readmissions within 30 days of discharge? Fine – readmit patients under “observation status” and call them outpatients. Dock hospitals for deaths within 30 days of leaving? Keep the respirator on for an extra day, and let the patient die on day 31. Risky case? Don’t operate. “Juking the stats” – arresting many small-fry drug pushers instead of focusing on the kingpins. Does U.S. News and World Report rank a college higher for classes with under 20 students? Schedule seminars with a maximum of 19. (My example, not Muller’s.) Teach to the MCAS (unless, like Marty, you decide that’s hopeless and just quit). Buff the numbers.
You know the drill. And if you need to learn it to succeed – or not be judged a failure – you’ll learn it.
Studies show that “pay for performance” often doesn’t work. Metric advocates ignore these and call for more studies. In Muller’s words, “Metric fixation, which aspires to imitate science, too often resembles faith.”
Muller argues with balance and nuance. He affirms that objective measurement has helped sweep away old dogmas no one had ever tested and culled markedly substandard teachers. But he shows that over the past 30 years just counting what you know how to count, counting things that cannot be counted, and privileging belief over disconfirming evidence has conferred on metrics “elements of a cult,” whose baleful effects doctors and others toiling in their professional vineyards know too well.
Faith in metrics will wane and its cult will pass away, though likely well after we have done so ourselves. At some point, so-called situated knowledge – what people who actually do something know – will again be valued.
In the meantime, please rate this column highly (give it a 6 on a scale of 1-5), and confirm that there are no barriers to your implementing its wisdom, which comes unsullied by any financial conflicts of interest.
And check out Muller’s book. You have nothing to lose but your chains.
Measurement without meaning is tyranny!
Dr. Rockoff practices dermatology in Brookline, Mass., and is a longtime contributor to Dermatology News. He serves on the clinical faculty at Tufts University, Boston, and has taught senior medical students and other trainees for 30 years. His second book, “Act Like a Doctor, Think Like a Patient,” is available at amazon.com and barnesandnoble.com. Write to him at dermnews@mdedge.com.
“How come you retired?” I asked.
A few years my junior, Marty had taught in public school forever. “It was the MCAS,” he said. That’s the Massachusetts Comprehensive Assessment System, a standardized test meant to gauge student performance and teacher competence.
“They demanded that my students test at a fifth-grade level,” Marty said. “But they were all at a second-grade level.
“Plus, I had been teaching for thirty years, and some kid right out of college was telling me how to do my job. So I left.”
Of course, this tale will sound familiar to physicians. Pay for performance. Bean counters calling the shots. Dismissal of clinical experience as useless and self-serving.
A recent book lays it all out: Jerry Z. Muller’s The Tyranny of Metrics. This book is punchy, witty, and succinct – you can read it in a day. A historian of economics and culture, Muller shows the extent of what I had guessed at from chats with people in different fields. In all of these, if you don’t count it, it doesn’t count.
Metrics, it is assumed, are “hard” and “objective.” They must “replace judgment based on experience with standardized measurement.” Their promise is transparency, efficiency, accountability.
Muller began to study this when he became chair of his academic department. He thought his job was to nurture scholars and help students learn, only to find much of his time taken up with feeding often worthless data to remote administrators. He traces the metrical impulse, at root, to lack of trust. It’s not only doctors whom society doesn’t trust, but experts of all kinds.
Principal agents ... “employed in institutions are not to be trusted … their activity must be monitored and measured ... those measures need to be transparent to those without firsthand knowledge of the institutions ... and ... pecuniary rewards and punishments are the best way to motivate ‘agents.’ ”
What this analysis ignores, argues Muller, is that professionals respond not just to “extrinsic motivation[s]” (money) but to intrinsic ones: commitment to profession and clients, doing the right thing, making people happier and better, being recognized and honored by peers, doing interesting and stimulating work. When society denigrates and dismisses those considerations, professionals become demoralized. They leave, or they learn to game the system.
Muller gives many examples. Punish hospitals for readmissions within 30 days of discharge? Fine – readmit patients under “observation status” and call them outpatients. Dock hospitals for deaths within 30 days of leaving? Keep the respirator on for an extra day, and let the patient die on day 31. Risky case? Don’t operate. “Juking the stats” – arresting many small-fry drug pushers instead of focusing on the kingpins. Does U.S. News and World Report rank a college higher for classes with under 20 students? Schedule seminars with a maximum of 19. (My example, not Muller’s.) Teach to the MCAS (unless, like Marty, you decide that’s hopeless and just quit). Buff the numbers.
You know the drill. And if you need to learn it to succeed – or not be judged a failure – you’ll learn it.
Studies show that “pay for performance” often doesn’t work. Metric advocates ignore these and call for more studies. In Muller’s words, “Metric fixation, which aspires to imitate science, too often resembles faith.”
Muller argues with balance and nuance. He affirms that objective measurement has helped sweep away old dogmas no one had ever tested and culled markedly substandard teachers. But he shows that over the past 30 years just counting what you know how to count, counting things that cannot be counted, and privileging belief over disconfirming evidence has conferred on metrics “elements of a cult,” whose baleful effects doctors and others toiling in their professional vineyards know too well.
Faith in metrics will wane and its cult will pass away, though likely well after we have done so ourselves. At some point, so-called situated knowledge – what people who actually do something know – will again be valued.
In the meantime, please rate this column highly (give it a 6 on a scale of 1-5), and confirm that there are no barriers to your implementing its wisdom, which comes unsullied by any financial conflicts of interest.
And check out Muller’s book. You have nothing to lose but your chains.
Measurement without meaning is tyranny!
Dr. Rockoff practices dermatology in Brookline, Mass., and is a longtime contributor to Dermatology News. He serves on the clinical faculty at Tufts University, Boston, and has taught senior medical students and other trainees for 30 years. His second book, “Act Like a Doctor, Think Like a Patient,” is available at amazon.com and barnesandnoble.com. Write to him at dermnews@mdedge.com.
“How come you retired?” I asked.
A few years my junior, Marty had taught in public school forever. “It was the MCAS,” he said. That’s the Massachusetts Comprehensive Assessment System, a standardized test meant to gauge student performance and teacher competence.
“They demanded that my students test at a fifth-grade level,” Marty said. “But they were all at a second-grade level.
“Plus, I had been teaching for thirty years, and some kid right out of college was telling me how to do my job. So I left.”
Of course, this tale will sound familiar to physicians. Pay for performance. Bean counters calling the shots. Dismissal of clinical experience as useless and self-serving.
A recent book lays it all out: Jerry Z. Muller’s The Tyranny of Metrics. This book is punchy, witty, and succinct – you can read it in a day. A historian of economics and culture, Muller shows the extent of what I had guessed at from chats with people in different fields. In all of these, if you don’t count it, it doesn’t count.
Metrics, it is assumed, are “hard” and “objective.” They must “replace judgment based on experience with standardized measurement.” Their promise is transparency, efficiency, accountability.
Muller began to study this when he became chair of his academic department. He thought his job was to nurture scholars and help students learn, only to find much of his time taken up with feeding often worthless data to remote administrators. He traces the metrical impulse, at root, to lack of trust. It’s not only doctors whom society doesn’t trust, but experts of all kinds.
Principal agents ... “employed in institutions are not to be trusted … their activity must be monitored and measured ... those measures need to be transparent to those without firsthand knowledge of the institutions ... and ... pecuniary rewards and punishments are the best way to motivate ‘agents.’ ”
What this analysis ignores, argues Muller, is that professionals respond not just to “extrinsic motivation[s]” (money) but to intrinsic ones: commitment to profession and clients, doing the right thing, making people happier and better, being recognized and honored by peers, doing interesting and stimulating work. When society denigrates and dismisses those considerations, professionals become demoralized. They leave, or they learn to game the system.
Muller gives many examples. Punish hospitals for readmissions within 30 days of discharge? Fine – readmit patients under “observation status” and call them outpatients. Dock hospitals for deaths within 30 days of leaving? Keep the respirator on for an extra day, and let the patient die on day 31. Risky case? Don’t operate. “Juking the stats” – arresting many small-fry drug pushers instead of focusing on the kingpins. Does U.S. News and World Report rank a college higher for classes with under 20 students? Schedule seminars with a maximum of 19. (My example, not Muller’s.) Teach to the MCAS (unless, like Marty, you decide that’s hopeless and just quit). Buff the numbers.
You know the drill. And if you need to learn it to succeed – or not be judged a failure – you’ll learn it.
Studies show that “pay for performance” often doesn’t work. Metric advocates ignore these and call for more studies. In Muller’s words, “Metric fixation, which aspires to imitate science, too often resembles faith.”
Muller argues with balance and nuance. He affirms that objective measurement has helped sweep away old dogmas no one had ever tested and culled markedly substandard teachers. But he shows that over the past 30 years just counting what you know how to count, counting things that cannot be counted, and privileging belief over disconfirming evidence has conferred on metrics “elements of a cult,” whose baleful effects doctors and others toiling in their professional vineyards know too well.
Faith in metrics will wane and its cult will pass away, though likely well after we have done so ourselves. At some point, so-called situated knowledge – what people who actually do something know – will again be valued.
In the meantime, please rate this column highly (give it a 6 on a scale of 1-5), and confirm that there are no barriers to your implementing its wisdom, which comes unsullied by any financial conflicts of interest.
And check out Muller’s book. You have nothing to lose but your chains.
Measurement without meaning is tyranny!
Dr. Rockoff practices dermatology in Brookline, Mass., and is a longtime contributor to Dermatology News. He serves on the clinical faculty at Tufts University, Boston, and has taught senior medical students and other trainees for 30 years. His second book, “Act Like a Doctor, Think Like a Patient,” is available at amazon.com and barnesandnoble.com. Write to him at dermnews@mdedge.com.