User login
Until more is known about the potential for unblinding, clinicians need to keep in mind that patients and physicians could often guess accurately who was getting placebo or active drug, first author Cody Bruggemeyer, MD, a resident at the Medical College of Wisconsin, Milwaukee, said in an interview.
“It’s important that rheumatologists be aware of this potential issue and use their clinical reasoning and their ability to critically assess papers to evaluate the study design” of research on treatments, he said in an interview.
Dr. Bruggemeyer and coauthors at the Medical College of Wisconsin presented their assessment of the potential for unblinding in a Viewpoint article in The Lancet Rheumatology.
A sample of pivotal clinical trials
The authors selected a sample of pivotal studies of 14 commonly prescribed drugs for rheumatic conditions for which double-blind randomized controlled trials (RCTs) that compared the active ingredient with a placebo were available.
The 14 trials involved treatments classified as disease-modifying antirheumatic drugs (DMARDs), some of which were likely to produce side effects that placebos would not mimic, such as injection site and infusion reactions and difference in readings in lab reports, the authors wrote.
In their analysis, Dr. Bruggemeyer and colleagues evaluated discrepancies in the rates of adverse events reported between active drugs and placebos and classified the 14 studies as follows:
- High unblinding risk: Nine studies had a high estimated risk of unblinding, including trials of adalimumab with citrate (Humira), anakinra (Kineret), anifrolumab (Saphnelo), apremilast (Otezla), ixekizumab (Taltz), leflunomide (Arava), methotrexate, risankizumab (Skyrizi) and tofacitinib (Xeljanz).
- Moderate unblinding risk: Three studies had a moderate estimated risk of unblinding, including trials of azathioprine (Imuran), mycophenolate mofetil and tocilizumab (Actemra).
- Low unblinding risk: Two studies had a low estimated risk of unblinding. These involved tests of belimumab (Benlysta) and rituximab (Rituxan).
Many of the effectiveness measurements of treatments used in rheumatology depend on patients’ reports of relief of pain and other disease symptoms. For example, the widely used American College of Rheumatology 20% response for rheumatoid arthritis includes components that rely on patient and physician assessment of disease activity.
Unblinding risk to clinical trial validity
CTs are the highest level of evidence to establish efficacy, because the study design aims to mask whether the experimental treatment is a drug or placebo. In cases where patients and physicians are more likely to correctly detect use of an active drug, there can be biases that skew results toward reports of symptom improvement. Other patients’ views of their treatment may be distorted by accurate guesses that they have been given placebo, Dr. Bruggemeyer and coauthors wrote.
“The degree of these effects cannot be predicted, but they tend to erroneously inflate the perceived benefit of novel interventions,” they wrote.
The consequences of this unblinding may be minimal in cases where there’s a clear difference between the placebo and active drug, they said. As an example, they cited trials of interleukin-23 inhibitors for psoriasis, where skin clearance as measured by the Psoriatic Area and Severity Index 75 differed by more than 50% in absolute terms between the treatment and placebo groups.
But in other cases, there needs to be more attention paid to the potential role of unblinding, they wrote.
“Studies where effect sizes were small, contradictory, or dependent on subgroup analyses might be especially problematic, but commentary rarely reflects this issue or acknowledges the potential influence of unblinding,” they wrote.
In the paper, they call for more analysis of previous trials to look for unreported assessments of unblinding, while also asking that researchers consider surveying participants in future trials to evaluate the degree to which unblinding occurs.
“Advocacy from professional societies and the U.S. Food and Drug Administration itself might be necessary, but in the interim, rheumatologists should assume unblinding has occurred to some degree in most trials,” they wrote.
Unblinding measure needs validation
In an interview, Roy M. Fleischmann, MD, co–medical director of the Metroplex Clinical Research Center in Dallas, raised some objections to the paper. The paper addresses an interesting question about unblinding, but there should have been more work done, such as finding “a measure that is validated that can say whether you’ve been unblinded or not.”
He added that he was surprised the paper on unblinding in rheumatology trials was published in its current form.
“I would have sent it for a major rewrite” if asked to review this paper before publication, said Dr. Fleischmann, who as a reviewer for Lancet Rheumatology. “I would have said: ‘Okay, 90% of this paper is okay, but your gist is not correct.’ It should be: ‘Is this a problem?’ ”
Dr. Fleischmann said he would have recommended a different perspective to the paper. “That is, this could occur. Should we be looking at this, and how would we look at this?”
In the paper, the authors acknowledge their approach has not been validated, “but it highlights the potential effect of idiosyncratic adverse events,” they wrote.
There’s less funding in general for meta-research than for studies involving treatments, so researchers look for approaches that can be handled without requiring significant funding, and much of the research on the quality of research is conducted like this analysis of rheumatology trials, Michael Putman, MD, the corresponding author and is a rheumatologist and an assistant professor at the Medical College of Wisconsin, said in an interview.
“You’re mostly doing on a shoestring budget with yourself and trainees,” he said. Dr. Putman is an associate editor at the journal Rheumatology and also involved in meta-research, or efforts to understand how studies and trials answer questions about how medical treatments work.
In an Aug. 16 tweet, Dr. Putman said this issue of unintentional unblinding with rheumatology trials was something he’d “been ruminating about for awhile; took two all star trainees to push it over the top!”
One of the barriers to funding of meta-research is a tendency for major funding for medical studies to be focused on specific diseases or targets. With meta-research, it may be more difficult to explain how a specific project will advance efforts to treat or prevent a certain disease, Dr. Putman said.
“It’s a little more esoteric and maybe not quite as clear how these projects will move things forward,” Dr. Putman said.
In addition, the nature of meta-research is to question and often be critical of work that’s already been published, adding another hurdle in attempts to secure funding, he said.
Dr. Putman is supported by a Rheumatology Research Foundation Scientist Development Grant, receives research funding related to clinical trials by AbbVie and AstraZeneca, and consulting fees from Novartis. The other authors declared no competing interests.
Until more is known about the potential for unblinding, clinicians need to keep in mind that patients and physicians could often guess accurately who was getting placebo or active drug, first author Cody Bruggemeyer, MD, a resident at the Medical College of Wisconsin, Milwaukee, said in an interview.
“It’s important that rheumatologists be aware of this potential issue and use their clinical reasoning and their ability to critically assess papers to evaluate the study design” of research on treatments, he said in an interview.
Dr. Bruggemeyer and coauthors at the Medical College of Wisconsin presented their assessment of the potential for unblinding in a Viewpoint article in The Lancet Rheumatology.
A sample of pivotal clinical trials
The authors selected a sample of pivotal studies of 14 commonly prescribed drugs for rheumatic conditions for which double-blind randomized controlled trials (RCTs) that compared the active ingredient with a placebo were available.
The 14 trials involved treatments classified as disease-modifying antirheumatic drugs (DMARDs), some of which were likely to produce side effects that placebos would not mimic, such as injection site and infusion reactions and difference in readings in lab reports, the authors wrote.
In their analysis, Dr. Bruggemeyer and colleagues evaluated discrepancies in the rates of adverse events reported between active drugs and placebos and classified the 14 studies as follows:
- High unblinding risk: Nine studies had a high estimated risk of unblinding, including trials of adalimumab with citrate (Humira), anakinra (Kineret), anifrolumab (Saphnelo), apremilast (Otezla), ixekizumab (Taltz), leflunomide (Arava), methotrexate, risankizumab (Skyrizi) and tofacitinib (Xeljanz).
- Moderate unblinding risk: Three studies had a moderate estimated risk of unblinding, including trials of azathioprine (Imuran), mycophenolate mofetil and tocilizumab (Actemra).
- Low unblinding risk: Two studies had a low estimated risk of unblinding. These involved tests of belimumab (Benlysta) and rituximab (Rituxan).
Many of the effectiveness measurements of treatments used in rheumatology depend on patients’ reports of relief of pain and other disease symptoms. For example, the widely used American College of Rheumatology 20% response for rheumatoid arthritis includes components that rely on patient and physician assessment of disease activity.
Unblinding risk to clinical trial validity
CTs are the highest level of evidence to establish efficacy, because the study design aims to mask whether the experimental treatment is a drug or placebo. In cases where patients and physicians are more likely to correctly detect use of an active drug, there can be biases that skew results toward reports of symptom improvement. Other patients’ views of their treatment may be distorted by accurate guesses that they have been given placebo, Dr. Bruggemeyer and coauthors wrote.
“The degree of these effects cannot be predicted, but they tend to erroneously inflate the perceived benefit of novel interventions,” they wrote.
The consequences of this unblinding may be minimal in cases where there’s a clear difference between the placebo and active drug, they said. As an example, they cited trials of interleukin-23 inhibitors for psoriasis, where skin clearance as measured by the Psoriatic Area and Severity Index 75 differed by more than 50% in absolute terms between the treatment and placebo groups.
But in other cases, there needs to be more attention paid to the potential role of unblinding, they wrote.
“Studies where effect sizes were small, contradictory, or dependent on subgroup analyses might be especially problematic, but commentary rarely reflects this issue or acknowledges the potential influence of unblinding,” they wrote.
In the paper, they call for more analysis of previous trials to look for unreported assessments of unblinding, while also asking that researchers consider surveying participants in future trials to evaluate the degree to which unblinding occurs.
“Advocacy from professional societies and the U.S. Food and Drug Administration itself might be necessary, but in the interim, rheumatologists should assume unblinding has occurred to some degree in most trials,” they wrote.
Unblinding measure needs validation
In an interview, Roy M. Fleischmann, MD, co–medical director of the Metroplex Clinical Research Center in Dallas, raised some objections to the paper. The paper addresses an interesting question about unblinding, but there should have been more work done, such as finding “a measure that is validated that can say whether you’ve been unblinded or not.”
He added that he was surprised the paper on unblinding in rheumatology trials was published in its current form.
“I would have sent it for a major rewrite” if asked to review this paper before publication, said Dr. Fleischmann, who as a reviewer for Lancet Rheumatology. “I would have said: ‘Okay, 90% of this paper is okay, but your gist is not correct.’ It should be: ‘Is this a problem?’ ”
Dr. Fleischmann said he would have recommended a different perspective to the paper. “That is, this could occur. Should we be looking at this, and how would we look at this?”
In the paper, the authors acknowledge their approach has not been validated, “but it highlights the potential effect of idiosyncratic adverse events,” they wrote.
There’s less funding in general for meta-research than for studies involving treatments, so researchers look for approaches that can be handled without requiring significant funding, and much of the research on the quality of research is conducted like this analysis of rheumatology trials, Michael Putman, MD, the corresponding author and is a rheumatologist and an assistant professor at the Medical College of Wisconsin, said in an interview.
“You’re mostly doing on a shoestring budget with yourself and trainees,” he said. Dr. Putman is an associate editor at the journal Rheumatology and also involved in meta-research, or efforts to understand how studies and trials answer questions about how medical treatments work.
In an Aug. 16 tweet, Dr. Putman said this issue of unintentional unblinding with rheumatology trials was something he’d “been ruminating about for awhile; took two all star trainees to push it over the top!”
One of the barriers to funding of meta-research is a tendency for major funding for medical studies to be focused on specific diseases or targets. With meta-research, it may be more difficult to explain how a specific project will advance efforts to treat or prevent a certain disease, Dr. Putman said.
“It’s a little more esoteric and maybe not quite as clear how these projects will move things forward,” Dr. Putman said.
In addition, the nature of meta-research is to question and often be critical of work that’s already been published, adding another hurdle in attempts to secure funding, he said.
Dr. Putman is supported by a Rheumatology Research Foundation Scientist Development Grant, receives research funding related to clinical trials by AbbVie and AstraZeneca, and consulting fees from Novartis. The other authors declared no competing interests.
Until more is known about the potential for unblinding, clinicians need to keep in mind that patients and physicians could often guess accurately who was getting placebo or active drug, first author Cody Bruggemeyer, MD, a resident at the Medical College of Wisconsin, Milwaukee, said in an interview.
“It’s important that rheumatologists be aware of this potential issue and use their clinical reasoning and their ability to critically assess papers to evaluate the study design” of research on treatments, he said in an interview.
Dr. Bruggemeyer and coauthors at the Medical College of Wisconsin presented their assessment of the potential for unblinding in a Viewpoint article in The Lancet Rheumatology.
A sample of pivotal clinical trials
The authors selected a sample of pivotal studies of 14 commonly prescribed drugs for rheumatic conditions for which double-blind randomized controlled trials (RCTs) that compared the active ingredient with a placebo were available.
The 14 trials involved treatments classified as disease-modifying antirheumatic drugs (DMARDs), some of which were likely to produce side effects that placebos would not mimic, such as injection site and infusion reactions and difference in readings in lab reports, the authors wrote.
In their analysis, Dr. Bruggemeyer and colleagues evaluated discrepancies in the rates of adverse events reported between active drugs and placebos and classified the 14 studies as follows:
- High unblinding risk: Nine studies had a high estimated risk of unblinding, including trials of adalimumab with citrate (Humira), anakinra (Kineret), anifrolumab (Saphnelo), apremilast (Otezla), ixekizumab (Taltz), leflunomide (Arava), methotrexate, risankizumab (Skyrizi) and tofacitinib (Xeljanz).
- Moderate unblinding risk: Three studies had a moderate estimated risk of unblinding, including trials of azathioprine (Imuran), mycophenolate mofetil and tocilizumab (Actemra).
- Low unblinding risk: Two studies had a low estimated risk of unblinding. These involved tests of belimumab (Benlysta) and rituximab (Rituxan).
Many of the effectiveness measurements of treatments used in rheumatology depend on patients’ reports of relief of pain and other disease symptoms. For example, the widely used American College of Rheumatology 20% response for rheumatoid arthritis includes components that rely on patient and physician assessment of disease activity.
Unblinding risk to clinical trial validity
CTs are the highest level of evidence to establish efficacy, because the study design aims to mask whether the experimental treatment is a drug or placebo. In cases where patients and physicians are more likely to correctly detect use of an active drug, there can be biases that skew results toward reports of symptom improvement. Other patients’ views of their treatment may be distorted by accurate guesses that they have been given placebo, Dr. Bruggemeyer and coauthors wrote.
“The degree of these effects cannot be predicted, but they tend to erroneously inflate the perceived benefit of novel interventions,” they wrote.
The consequences of this unblinding may be minimal in cases where there’s a clear difference between the placebo and active drug, they said. As an example, they cited trials of interleukin-23 inhibitors for psoriasis, where skin clearance as measured by the Psoriatic Area and Severity Index 75 differed by more than 50% in absolute terms between the treatment and placebo groups.
But in other cases, there needs to be more attention paid to the potential role of unblinding, they wrote.
“Studies where effect sizes were small, contradictory, or dependent on subgroup analyses might be especially problematic, but commentary rarely reflects this issue or acknowledges the potential influence of unblinding,” they wrote.
In the paper, they call for more analysis of previous trials to look for unreported assessments of unblinding, while also asking that researchers consider surveying participants in future trials to evaluate the degree to which unblinding occurs.
“Advocacy from professional societies and the U.S. Food and Drug Administration itself might be necessary, but in the interim, rheumatologists should assume unblinding has occurred to some degree in most trials,” they wrote.
Unblinding measure needs validation
In an interview, Roy M. Fleischmann, MD, co–medical director of the Metroplex Clinical Research Center in Dallas, raised some objections to the paper. The paper addresses an interesting question about unblinding, but there should have been more work done, such as finding “a measure that is validated that can say whether you’ve been unblinded or not.”
He added that he was surprised the paper on unblinding in rheumatology trials was published in its current form.
“I would have sent it for a major rewrite” if asked to review this paper before publication, said Dr. Fleischmann, who as a reviewer for Lancet Rheumatology. “I would have said: ‘Okay, 90% of this paper is okay, but your gist is not correct.’ It should be: ‘Is this a problem?’ ”
Dr. Fleischmann said he would have recommended a different perspective to the paper. “That is, this could occur. Should we be looking at this, and how would we look at this?”
In the paper, the authors acknowledge their approach has not been validated, “but it highlights the potential effect of idiosyncratic adverse events,” they wrote.
There’s less funding in general for meta-research than for studies involving treatments, so researchers look for approaches that can be handled without requiring significant funding, and much of the research on the quality of research is conducted like this analysis of rheumatology trials, Michael Putman, MD, the corresponding author and is a rheumatologist and an assistant professor at the Medical College of Wisconsin, said in an interview.
“You’re mostly doing on a shoestring budget with yourself and trainees,” he said. Dr. Putman is an associate editor at the journal Rheumatology and also involved in meta-research, or efforts to understand how studies and trials answer questions about how medical treatments work.
In an Aug. 16 tweet, Dr. Putman said this issue of unintentional unblinding with rheumatology trials was something he’d “been ruminating about for awhile; took two all star trainees to push it over the top!”
One of the barriers to funding of meta-research is a tendency for major funding for medical studies to be focused on specific diseases or targets. With meta-research, it may be more difficult to explain how a specific project will advance efforts to treat or prevent a certain disease, Dr. Putman said.
“It’s a little more esoteric and maybe not quite as clear how these projects will move things forward,” Dr. Putman said.
In addition, the nature of meta-research is to question and often be critical of work that’s already been published, adding another hurdle in attempts to secure funding, he said.
Dr. Putman is supported by a Rheumatology Research Foundation Scientist Development Grant, receives research funding related to clinical trials by AbbVie and AstraZeneca, and consulting fees from Novartis. The other authors declared no competing interests.
FROM THE LANCET RHEUMATOLOGY