August 11, 2017
Associate Professor of Biostatistics Ken Kleinman has received a four-year, $1.24 million grant from the NIH’s National Institute of General Medical Sciences to provide medical researchers with new software, a web application tool and other support designed to help them determine proper sample size, improve statistical power and improve other methods when using cluster-randomized trials in their studies.
The key feature of cluster-randomized trials is that subjects are randomized in groups, rather than as individuals, so all members of a community, hospital or medical practice receive the same treatment. For example, to investigate whether a new approach to hospital infection control is effective, one might randomize a whole hospital to implement the new approach and evaluate outcomes for all the patients in that hospital and compare the result to patients in another hospital that did not adopt the new approach.
The problem, Kleinman says, is that many researchers do not understand how to use this type of study correctly. “The way they were trained to think about trials is that you recruit two patients, and give one an intervention and not give it to the other, in as many multiples as you need for good statistical power. With a cluster-randomized trial, in general you will need more subjects, but many researchers don’t know how to determine how much larger the sample should be so the statistical analysis has enough power to answer the question.”
One solution too many researchers take, he says, is to recruit many more study participants than are needed, which is unethical. “It’s wrong to use resources that are not required. If you just pick a number like 10 times the usual sample size, it’s not defensible. You might find a statistically significant difference but a clinically meaningless difference. Similarly, if you do a study with too few people you have no chance of finding the right answer. You’ve wasted everyone’s time and your own.”
The biostatistician says, “It’s a privilege to have access to people and their health care information and treatment, and it’s not OK to waste their time or to make them sicker or expose them to any risk without a good reason. It’s a moral and ethical question that NIH takes very seriously and that researchers should also pay attention to.”
With his colleague in biostatistics Nicholas Reich and several postdoctoral researchers and graduate students, Kleinman plans to introduce a new tool this month, plus a web application, using the open access software known as R. They will test and enhance these products over the coming years. “We propose to generate a comprehensive free and open-source software suite to provide approximate, analytic and simulation-based power assessment. In addition, we will develop a web app for the code to allow users who have less computing knowledge to make use of the software. Finally, we will make use of the software to answer outstanding questions in the design of cluster-randomized trials.”
Kleinman says, “Right now it’s frankly disturbing to see what some studies propose for power and sample size calculations. Even scientists can be afraid of statistics, and there are too many people writing the statistics sections for medical studies who don’t know what they don’t know. When you read a lot of grant proposals as we do on an NIH study section, you see the need for improvement. I hope that someday the grant proposals I read on my study section will have better power and sample size, and they will include accurate and plausible assessments of how many subjects they require. That would be a great outcome of our project.”