Clinical relevance in medical studies: What does that actually mean?

In the world of medical research and clinical trials, we often come across the term “clinical relevance”. This is particularly required in trials of medical devices when reimbursement by health insurance companies ais sought and, where applicable, when price negotiations are to take place.

A central question is at what point a difference can be considered meaningful for the individual patient. In other words: what threshold must be reached to assume a response to an intervention, and to use the difference in response rates between two groups as an effect measure for benefit assessment?

This requirement is especially prominent for Digital Health Applications (DiGA). Therefore, the following will address this issue based on a statement by the independent Institute for Quality and Efficiency in Health Care (IQWiG), see sources below.

But what does this term “clinical relevance” actually mean, and why is it so important? Put simply, it concerns whether statistically significant results of a study actually represent a meaningful difference for patients. It is therefore not sufficient for a clinical trial to show that a treatment is statistically significantly superior to another or to no treatment; it must also have a measurable and tangible positive effect on health or quality of life.

Researchers and manufacturers should therefore consider clinical relevance to ensure that a potentially meaningful product truly achieves significance.

The problem with Clinical Relevance

The challenge lies in the fact that there is no universal formula for determining clinical relevance. What is relevant for one disease or one patient may be meaningless for another. For example, a slight improvement in a severe, life-threatening disease could make a significant difference, while the same improvement in a mild, self-limiting illness would hardly be noticeable.

When is an effect even considered clinically relevant?

The answer to this question depends on various factors:

• Severity of the disease: The more severe the disease, the more likely even a small effect will be considered relevant.
• Prevalence of the disease: For common diseases, even a minor effect can have a major impact on the population as a whole.
• Benefits and risks of the treatment: A treatment with significant side effects must offer greater benefit to be deemed clinically relevant.

How is the assessment of clinical relevance performed?

There are various approaches to evaluating the clinical relevance of study results. Some common methods include:

• Effect size: Here, the magnitude of a treatment effect is quantified using the value of Cohen’s d or the effect size correlation r. Cohen originally proposed a three-tier classification, with 0.2 interpreted as a small, up to 0.5 as a medium, and above 0.8 as a large effect. However, it is important to note that a large effect size does not automatically equate to clinical relevance.
• Significance test for relevant superiority: This method tests whether a new treatment is better than an existing one to an extent considered clinically relevant.
• Assessment based on the observed effect: The actual difference between treatment groups is considered and assessed for its practical significance.
• Responder analyses: This method examines the proportion of patients who respond to a treatment, i.e., who experience a certain improvement.
• Relative effect: This approach considers the likelihood that a patient will achieve a better outcome with a particular treatment compared to another.

This overview already shows that there is (too) much room for interpretation, which can be exploited depending on one’s own interests. The literature on this topic is extensive.

The Importance of Pre-Specification

An important point is that the method for assessing clinical relevance, the threshold for relevance, and the type of decision (whether the observed effect or the confidence interval is considered) should be defined before the study begins. This helps to avoid bias and ensures that results are interpreted objectively.

Further Approach by IQWiG

For a long time, the methodological discussion revolved around so-called Minimal Important Differences (MID), i.e., the smallest meaningful differences. The basic idea was to identify a threshold for each questionnaire that represents the smallest change meaningful to patients. However, methodological issues have recently emerged with this approach. The MID is not a fixed value but variable, depending, for example, on the type and severity of the disease, the direction of change being studied (improvement or deterioration), or the methodology used. Moreover, many studies determining MID no longer meet today’s methodological standards or do not adequately describe the methods applied.

Against this background, IQWiG developed another approach in 2020 for determining a threshold. The aim of this specification was to provide clarity for manufacturers and to make arbitrary responder analyses based on non-transparent responder definitions unattractive. IQWiG defined a value of 15% of the range of the respective scale as a plausible threshold for a still small but sufficiently perceptible change. In the General Methods 6.0, published in November 2020, the institute stipulated that in future, responder analyses for assessments will be based on a threshold of at least 15% of the scale range of the instrument used.

In practice, an improvement in treatment or diagnosis of 30% to 50% compared to the control group is also considered clinically relevant. However, this also means a large difference, which in reality has only been demonstrated for very few of today’s established treatment or diagnostic methods.

If you are interested in clinical trials for approval or reimbursement purposes in compliance with the requirements for clinical relevance, please feel free to arrange an initial non-binding consultation with us. We are here for you!

Source: https://www. iqwig.de/ events/11-11-25_ kieser_bewertung_klinischer_relevanz.pdf, https://www. iqwig.de /methoden /allgemeine-methoden-v6-1.pdf (please remove spaces for links)