Category: Evidence

Judging the research behind a health claim: study design, dose, funding and outcomes.

  • “Clinically proven” on a supplement box: a checklist for the study behind it

    “Clinically proven,” “clinically studied,” and “science-backed” appear on supplement packaging constantly. Sometimes there is a serious trial behind the phrase. Sometimes there is a study of twelve people, funded by the manufacturer, measuring something that is not the outcome you care about.

    The phrase itself will not tell you which. These six questions will.

    1. Was the study on this product, or on one ingredient?

    The most common gap. A trial establishing that a compound helps at 500 mg says nothing about a blend containing 50 mg of it alongside eleven other things. Ingredient research is legitimate evidence about the ingredient. It is not evidence about the formula.

    2. Was it randomized, placebo-controlled and blinded?

    Without a control group you cannot separate the supplement from the placebo effect, from regression to the mean, or from the fact that people who start taking a sleep supplement often also start going to bed earlier. Blinding — neither participant nor assessor knowing who got what — prevents expectation from leaking into the measurements.

    An uncontrolled study where participants report feeling better is not evidence that a product works. It is evidence that people who buy a sleep aid expect to sleep better.

    3. How many people, and for how long?

    Small trials produce unstable results in both directions. A twenty-person study finding a large effect is a reason to run a bigger study, not a reason to make a claim.

    Duration matters just as much and is quietly skipped in marketing. A four-week study cannot support a claim about long-term outcomes.

    4. Who paid for it?

    Industry funding does not make a study wrong, and it is worth saying plainly that much of the useful nutrition research in existence was industry-funded because nobody else would pay for it. But funding is disclosed in reputable journals for a reason, and it belongs in your weighting — particularly when the sponsor also chose which outcomes to report.

    5. Was the outcome the one that matters to you?

    This is where the most sophisticated sleight of hand happens. A study might report a statistically significant change in a surrogate marker — a blood value, a lab measurement — and the marketing translates that into a promise about how you will feel.

    A change in a biomarker is a legitimate scientific finding. It is not the same as a change in symptoms, and the distance between them is often where a product’s entire value proposition sits.

    6. Was it published and peer-reviewed, or is it “data on file”?

    “Data on file” means the company has a result it has not submitted to outside scrutiny. That is not automatically dishonest — but it means no independent reviewer has examined the methods, and you are being asked to trust the interested party’s summary of its own unexamined work.

    Statistical significance is not size

    One last trap that survives every other filter. A result can be statistically significant — unlikely to be chance — and still be far too small to notice. With a large enough sample, a trivial difference becomes significant.

    When a claim cites significance but never states the actual magnitude of the change, the magnitude is usually the reason.

    The short version

    Ask what was tested, on how many people, against what, for how long, paid by whom, measuring what. Six questions, most of which can be answered from a study abstract in under five minutes. If the manufacturer will not identify the study at all, that is itself the answer.

    This article may contain affiliate links. If you buy through one we may earn a commission at no extra cost to you. It does not affect what we write or which products we cover — see our affiliate disclosure.