The My Health

children to either watch or not watch TV violence to see if years of exposure to TV violence leads to violent behaviour in adulthood.

In general, RCTs are not practical for studying long-term effects because adequate experimental control cannot be maintained for an extended duration. They would also be prohibitively expensive.

So, while an RCT is ideal for determining if a course of vitamin D can remediate rickets, the methodology is impractical for answering the broader health question of “what is the optimal daily intake of vitamin D for long life expectancy?”. RCTs are a group design and are analysed with group statistics such as group means. So, while an RCT might show that an antidepressant drug reduced depression by 40 percent (as a group average) that doesn’t tell us, and can’t tell us, if some people within that group improved much more, only a little, not at all, or even got worse.

Exactly how much the drug affects individuals, and who it affects, requires extra clinical analysis and judgment not provided by the RCT. That RCT “group average” result might not apply to you.

Groups sizes must be large for RCT results to be reliable and trustworthy. But it’s hard to get people to volunteer for studies, and testing participants is both expensive and time consuming. For all these reasons most studies are run with a less than an ideal number of participants.

Researchers can try to compensate for this by running fewer (but larger) groups. For instance, in testing an antidepressant drug researchers might run a bare minimum of just two groups: a 40-milligram dose treatment group and a zeromilligram dose placebo group. This might show that the drug is effective at lowering depression, but a third middle (20 milligram) group would have provided useful extra information.

Perhaps 20 milligrams would produce a moderate decrease in depression (establishing a dose-response effect), or perhaps it would produce no effect at all (establishing a threshold effect, so don’t bother using low doses), or perhaps it would produce a full therapeutic effect (so don’t use 40 milligrams, which is wasteful and likely to have more side effects). More groups give more information, but more groups require more participants if tolerable group sizes are to be maintained. n Health scientist Dr Steve Humphries is a director at Hebe Botanicals in Ōtaki. He was previously a lecturer at Massey University and director of the Health Science Programme

The situation becomes even worse when we consider that the causes of ill health, and the cures, are normally multifactorial.

If we want to investigate the potential interaction of two antidepressant drugs (perhaps there is a synergistic effect where small doses of both drugs combined is more effective than a large dose of either drug alone?) then, if we have three dose levels for each drug (0, 20 and 40 milligrams), we will require nine groups to test all possible dose combinations. Add a third experimental variable for treating depression (for example, physical exercise with three levels: none, a little, and a lot) and now we need 27 groups to test all possible combinations! That’s a big ask.

Consequently, RCTs struggle to capture the multifactorial complexity of real life. Evaluating RCTs includes spotting the compromises researchers make in running suboptimal group sizes (making the results less trustworthy) and leaving out groups that would have provided useful information.

RCTs are a reductionist approach to health research, examining only a small number of factors in relative isolation. While this is fine for the pharmaceutical industry with its focused interest on whether a particular drug is effective for a particular health problem, RCTs are less effective for answering more complex health issues, such as what is an optimal diet for good health? To answer these questions, researchers must draw on a broad range of methodologies (RCTs, epidemiological surveys and case studies) and piece together an understanding.

In my next article I will look at how RCTs have contributed to a crisis in science.