As
a biomedical researcher, I consider the research I did during my Ph.D.
in India to be the most rigorous by far. It was the only project where
statistics were appropriately and correctly applied right from the first
step, the experiment design, continuing with blinding of the samples
through to data analysis.
Goal of my Ph.D. project was to figure out if prior exposure to environmental mycobacteria (NTM, Nontuberculous mycobacteria)
could explain why the largest TB vaccine trial had failed to protect
against adult pulmonary TB. Conducted from 1967 to 1980 on ~360000
people across 209 villages and 1 town in South India, prior exposure to
environmental mycobacteria emerged as a plausible reason. Only there was
no data on NTM in this environment, if yes, what species and where, in
the soil/water/dust. I was just one person. How could I cover such a
vast population over such a vast area? That's where statistics entered
the picture, exactly where it should, in the experimental design itself.
A professional statistician crunched the numbers to determine how many
villages I should cover, how many houses per village, which villages,
i.e., make sure I comprehensively sampled the entire trial area in as
unbiased a manner as possible. Starting with this design, he carefully
shepherded every step of my Ph.D. project and even blinded the samples I
brought back from the field, only decoding them after I'd generated all
the data. Since I don't have any other experience on basic research in
India, I don't know if my experience if generalizable so I'll leave it
at that.
Moving on from differences between
India and US, I'll highlight two dubious practices that are rampant in
basic biomedical research the world over, at least if we go by the
published literature. Overarching problem consists of two features
1. Statistics are misused, usually applied only at the back end to
analyze the data after it's been generated, instead of the optimal
approach which is to apply them from the beginning in the experiment
design itself.
2.Definition of scientific
misconduct is too narrow, completely ignoring the most prevalent
practice, which isn't outright fraud but rather data selection.
Compared
to basic research, rigorous statistical science applied to human
clinical trials is the norm. Only very slowly is this mindset permeating
into basic research to replace this ridiculous state of affairs. Last
year, we saw the publication of the first randomized clinical trial in
mice (1).
The US ORI (United States Office of Research Integrity) defines Scientific misconduct
as consisting of data fabrication, data falsification or plagiarism.
But far more than any of these, the most prevalent practice is something
that's not even on the radar, data selection, i.e., cherry-picking data.
Practice is rampant. Rarely do animal model studies show data combined
from different experiments. Take a look at any recent paper, even ones
published in Nature or Science. Invariably a figure legend would say
something along the lines of, 'Data from one representative experiment
out of 3, 4 or 5 different experiments is shown'. Why not show combined
data from all experiments performed? How could such a shoddy practice be
the norm? Simply means intra-group variation between experiments was
greater than inter-group variation within one single experiment. Either
experimenters are shoddy or techniques too unrefined. Either way, cannot
trust such data. And this is still the norm in basic biomedical
research.
Bibliography:
1.
Llovera, Gemma, et al. "Results of a preclinical randomized controlled
multicenter trial (pRCT): Anti-CD49d treatment for acute brain
ischemia." Science Translational Medicine 7.299 (2015):
299ra121-299ra121. http://stm.sciencemag.org /conten...
https://www.quora.com/What-is-Tirumalai-Kamalas-personal-experience-of-the-difference-in-the-way-basic-science-research-is-conducted-in-the-USA-and-India/answer/Tirumalai-Kamala
No comments:
Post a Comment