What is the difference between a non inferiority trial and a superiority trial

An adequately powered superiority trial allows one to conclude that a new treatment is superior to placebo. Conclusions from non-inferiority trials, however, are based on assumptions that cannot be verified directly. The effect of the active control in relation to the placebo could be different from historical data. In case of the example, the PCI vs. This is frequently not the case as treatment effects can be heterogeneous due to differences in patient populations, outcome definitions, treatment allocation, or other study factors.

Conclusions from non-inferiority trials are highly sensitive to the method of analysis. The intention-to-treat analysis, typically preferred as the more robust analytical framework in a superiority trial, can be biased towards non-inferiority. In a superiority trial this strengthens the final effect of a difference, because the analysis makes the results of two arms more similar and thus harder to detect a significant difference.

Loss to follow-up will also increase the similarity between groups, because of the assumption that none of these patients met the primary endpoint. Other protocol deviations such as non-adherence to the assigned therapy can bias the results towards non-inferiority. However, the intention-to-treat analysis should be the primary analysis as it preserves the advantages of randomization, while the per-protocol analysis can be used as the supporting sensitivity analysis for non-inferiority assessment.

Patients who cross over or drop out need close examination. If a specific reason for a cross-over or drop-out is found in one treatment group, this shows that the two treatments are not similar by concept, thereby providing evidence of lack of non-inferiority.

It is, however, often misinterpreted as equivalence. Non-inferiority means that the new treatment is not significantly worse inferior than the active control, while equivalence means that the new treatment is not significantly worse inferior or better superior Figure 1. If non-inferior, the new treatment can be preferred because of an associated ancillary benefit in terms of invasiveness, cost, or convenience.

If the non-inferiority endpoint is not met, the interpretation becomes more difficult. Frequently one concludes that the new treatment is inferior to the active control. To conclude which is the case, it depends on the side of the CI being considered Figure 1. From a statistical point of view, a trial can show both non-inferiority and inferiority at the same time Figure 1.

Although rare, it is often the result of a poor trial design and should be avoided. From a clinical standpoint, a treatment can be inferior and non-inferior when non-inferiority is met but the margin might have been chosen too generously.

Excluding these transfusions, the rate of major adverse events in the MitraClip group was not significantly lower 5 vs. Thus, one can reasonably argue that MitraClip is less effective than surgery while not demonstrating a clinically relevant safety advantage. Sequential testing for superiority is only justified after non-inferiority has been successfully demonstrated. Although somewhat obvious, post hoc non-inferiority testing in a negative superiority trial is not appropriate, as the margins are not pre-specified and the trial not adequately powered for non-inferiority.

Table 1 provides an overview of recent non-inferiority trials. It demonstrates the differences in trial design, conduct, and analysis based on the expected event rate, power, sample size, non-inferiority margin, and preservation of the effect of standard therapy. The design and interpretation of non-inferiority trials is more complex than for superiority trials.

Therefore, many readers and investigators have difficulties understanding the full concept of these trials. When starting a non-inferiority trial, investigators need to make several assumptions and should be aware of not choosing inaccurate or unreasonably generous active control event rates or non-inferiority margins.

For readers, to objectively interpret non-inferiority trial results, one must be conscious of several pitfalls of the methodology. Assay sensitivity and trial inconsistency impede conclusions from non-inferiority trials.

Google Scholar. We thank Simon Day for his comment on our recent manuscript, and agree that a non-inferiority trial with low power and a small sample size does not easier results in making a claim of non-inferiority.

In contrary, a study with low power would produce large confidence intervals and therefore a greater likelihood of crossing the non-inferiority margin - delta. This could result in a type II error: a false-negative rejecting of non-inferiority [1].

In our discussion about the one-sided alpha of 2. We do want to emphasize that there may be misconduct in performing underpowered subgroup analyses in non-inferiority trial. The predefined 'rules' of analysis are more often ignored and the reporting of confidence intervals is often lacking, providing no basis for validity of the findings. For readers it is no longer possible to interpret whether the treatment is indeed non-inferior, irrespective of whether the confidence interval exceeds -delta.

In a subgroup analysis of left main patients only, it was concluded that PCI was non-inferior to CABG because there was no difference in major adverse cardiac or cerebrovascular events at 1 year Kaul S, Diamond GA. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med ; Percutaneous coronary intervention versus coronary-artery bypass grafting for severe coronary artery disease.

N Engl J Med ; Circulation ; Non-inferiority study design: lessons to be learned from cardiovascular trials. Eur Heart J ; Kappetein AP. Editorial comment: Is there enough evidence that proves clinical equipoise between stenting and coronary surgery for patients with left main coronary artery disease? Eur J Cardiothorac Surg ; Dr Stuart Head and colleagues present an excellent and readable summary of many difficult issues with non-inferiority trials.

However, whilst I agree with almost all that they say, I believe that the comments about low power in non-inferiority trials need clarifying. I am certainly an advocate of appropriately powered studies but the statement that "lower power biases the results towards non-inferiority" is not true. In a superiority study, it is certainly true that low power is likely to lead to failing to reject the null hypothesis and, hence, a conclusion of "no statistically significant difference".

But as I am sure Dr Head would agree, that is not sufficient to claim that there truly is no difference between two interventions the old adage: "absence of evidence is not evidence of absence".

In an under-powered superiority study, the confidence interval for the treatment effect will likely include the null i. In a non-inferiority study, the null hypothesis is that the new or test treatment is worse than the reference treatment, by an amount at least -?. As with an under-powered superiority study, an under-powered non-inferiority study risks failing to reject its null hypothesis and the confidence interval for the treatment effect is likely to include the null i.

Hence, we would not be sure if the test treatment is non- inferior to the comparator or not, and so non-inferiority could not be claimed. Under-powered studies should be discouraged but readers of non- inferiority studies should not be concerned that a small study, with low power, but that does demonstrate non-inferiority, may have erroneously done so because of the low power. There are many other risks that may lead to an erroneous conclusion of non-inferiority as the paper clearly identifies , but small size and low power is not one of them.

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide. Sign In or Create an Account. Sign In. Advanced Search. Search Menu. Article Navigation. Close mobile search navigation Article Navigation. Volume Non-inferiority studies are used to show that a minimum level of efficacy has been achieved. In comparison studies with a current therapy, non-inferiority is used to demonstrate that the new therapy provides at least the same benefit to the patient.

Superiority trials are always used when comparisons are made to placebo or vehicle treatments. In these studies, it is critical that the effect in the treatment group be clearly superior to any effects in the placebo groups. Failure to demonstrate superiority over vehicle suggests that the drug is not effective. Bioequivalence trials are used to show that a new treatment is identical within an acceptable range to a current treatment.

This is used in the registration and approval of generic drugs that are shown to be bioequivalent to their branded reference drugs. In the end, ask yourself which hypothesis am I trying to address, then use the appropriate study design. Best of luck! Making the right choices in drug development often means the difference between getting a new medication to patients and it ending up in the scrap heap of failed programs.

Read this white paper to learn how MBMA can be used to predict the likelihood of a new drug showing superiority or non-inferiority to the competition. Featured Product. Simcyp: 20 Years of Innovation. Certara Appoints James E. Contact Investors Careers Support. It is easy to understand the difference between superiority and equivalence trials, but the challenge is to understand the difference between equivalence and non-inferiority designs.

These are inherently very different from each other but figuring out the difference can feel like being stuck in a vortex! At least it was the case for me personally. When is it appropriate to choose a non-inferiority design instead of superiority? A typical RCT is usually a superiority trial but is not always labelled so and the aim is to show that a new intervention, treatment or drug is better than the active control or placebo.

CREDENCE showed that in patients with type 2 diabetes and kidney disease, the risk of kidney failure and cardiovascular events was lower in the canagliflozin group than in the placebo.

Non inferiority trials are usually designed to show that a particular intervention may be acceptably worse than the current standard of care treatment control but has ancillary benefits over the control such as lower costs, lesser side effects, easier administration etc.

In case of PIVOTAL trial the investigators hypothesized and proved that a proactive high dose IV iron regimen will be at least non-inferior to current practice of dosing iron reactively, with the added advantage of ability to decrease exposure to ESAs and reducing cardiovascular complications associated with ESAs.

Finally, non-inferiority trial design usually requires a smaller sample size as compared to a superiority trial and hence fewer resources. This plays a huge role in lowering the cost of a trial. Lower cost and sample size also makes it a more convenient endeavour! Assessing non-inferiority is actually more complex than assessing and proving superiority both in terms of trial design and analysis of results.

The NI margin is determined with both clinical and statistical considerations. An acceptable non-inferiority margin should be pre-specified otherwise it may lead to bias. From Schumi and Wittes. Trials Figure 1 beautifully elaborates the differences between the 3 designs. The positive and negative triangles represent the upper and lower bound pre-specified margins.

Sometimes the goal is to show that the two treatments are equivalent. If the CI of the effect size between the two treatments lies strictly between within the upper and lower margin, the two treatments are concluded to be equivalent. Equivalence trial may be used in showing a new antimicrobial which has less resistance is biologically the same as the generic antibiotic or in showing lot consistency in vaccine trials.

In a non-inferiority trial, the focus is on the lower bound margin, what happens at the upper end is not of primary concern in this type of trial design. One can also declare superiority in a non-inferiority trial if the lower limit of CI of the new treatment is above the non-inferiority margin and above zero. This is acceptable because it usually takes into account and controls for the Type 1 error and does not penalize for multiple testing.

Check out our post on multiple testing and Bonferonni correction here. But claiming non-inferiority after you have failed to prove superiority is usually not acceptable unless the trial is designed this way from the start and the non-inferiority margin has been set from the get go.

From Schumi and Wittes, Trials, Figure 2 elaborates all the possible outcomes of a non-inferiority trial.