Ben Goldacre is a physician, academic and science writer. As of 2014 he is a Wellcome research fellow in epidemiology at the London School of Hygiene and Tropical Medicine and a founder of the AllTrials campaign to require open science practices in clinical trials.
As an epidemiologist, Ben’s job is to use evidence and science to decide what is good for the body. An example of bad science is newspaper headlines, which categorise things that increase or decrease cancer risk – and sometimes contradict themselves by declaring coffee in both categories. The thrust of this talk discusses ways evidence can be manipulated through either ignorance or deception.
Bad science uses authorities – people or experts while ignoring their evidence. Good science should be carried by the weight of argument or evidence rather than who says it. Authority status can be easily contrived – TV doctors can create a pHd after their name or sign up online for advanced certificates of something or other.
Evidence can also be confused: a newspaper headline declared red wine reduces breast cancer risk. The study this was sourced from looked at a single chemical extracted from grape skins fighting some cancer cells in a petri dish – it has no relevance outside of this scenario. In truth the alcohol content of wine increases your cancer risk.
Another example was a study showing decreased skin wrinkles in people who eat olive oil and vegetables. The paper was correct that people who ate olive oil and vegetables had fewer wrinkles, but they also tended to be wealthier, better educated, do less manual labour, smoke less, drink less. These other factors had far more to do with fewer wrinkles.
The medical trial is one of the bases of epidemiology, and should be the basis for a doctor’s decisions, but so many people still get it wrong.
- A trial on fish oil tablets in school children didn’t use a control group – instead comparing their results against a projection of their results taken a year ago.
- The placebo is a place to abuse trials: the placebo is a powerful effect but new medicines should be testing themselves against the current best medicines rather than just a placebo. It is much more useful to see how a drug compares to the best competitors – since we would never prescribe a placebo.
- New drugs can compare themselves to a competitor that is not dosed correctly. For example they can prove themselves more effective by taking the alternative in too low a dose. They can also prove they have fewer side effects by taking the competitor’s drug in too high a dose.
For these reasons industry sponsored trials give a flattering result 4 times more often than independent trials. But this is true even when the industry’s trial is done correctly, because negative data can go missing. This can be analysed with statistics, with normal data giving a mix of false positives and false negatives and a few large trials with low error. If data has been hidden, the small false negatives will not be visible – the worst results will be the largest low error trials. Ben discusses one pill he has prescribed to patients, and discovered 75% of all the trials had never been released. Likewise Tamiflu has had billions invested in it because governments want to show it reduces serious complications associated with flu, but none of the evidence for this has been released.
Sunlight is the best disinfectant. All results should be published, but they are currently protected by a forcefield of tediousness.
Interesting to see how pharmaceuticals can abuse evidence by not publishing. I can understand commercial pressures not to publish negative results though, and not sure what the solution is. Should it be compulsory for companies to run trials through a central organisation and under specific rules?
Even more concerning is the ethics of treating people with excessive or insufficient doses of a drug. This is unforgivable – effectively putting someone’s health at risk to prove that an opponent’s drug is dangerous or ineffective. Directly hurting people as a marketing tactic doesn’t sit well with me at all.