As
 a biomedical researcher, I consider the research I did during my Ph.D. 
in India to be the most rigorous by far. It was the only project where 
statistics were appropriately and correctly applied right from the first
 step, the experiment design, continuing with blinding of the samples 
through to data analysis.
Goal of my Ph.D. project was to figure out if prior exposure to environmental mycobacteria (NTM, Nontuberculous mycobacteria)
 could explain why the largest TB vaccine trial had failed to protect 
against adult pulmonary TB. Conducted from 1967 to 1980 on ~360000 
people across 209 villages and 1 town in South India, prior exposure to 
environmental mycobacteria emerged as a plausible reason. Only there was
 no data on NTM in this environment, if yes, what species and where, in 
the soil/water/dust. I was just one person. How could I cover such a 
vast population over such a vast area? That's where statistics entered 
the picture, exactly where it should, in the experimental design itself.
 A professional statistician crunched the numbers to determine how many 
villages I should cover, how many houses per village, which villages, 
i.e., make sure I comprehensively sampled the entire trial area in as 
unbiased a manner as possible. Starting with this design, he carefully 
shepherded every step of my Ph.D. project and even blinded the samples I
 brought back from the field, only decoding them after I'd generated all
 the data. Since I don't have any other experience on basic research in 
India, I don't know if my experience if generalizable so I'll leave it 
at that. 
Moving on from differences between 
India and US, I'll highlight two dubious practices that are rampant in 
basic biomedical research the world over, at least if we go by the 
published literature. Overarching problem consists of two features
 1. Statistics are misused, usually applied only at the back end to 
analyze the data after it's been generated, instead of the optimal 
approach which is to apply them from the beginning in the experiment 
design itself.
 2.Definition of scientific 
misconduct is too narrow, completely ignoring the most prevalent 
practice, which isn't outright fraud but rather data selection.
Compared
 to basic research, rigorous statistical science applied to human 
clinical trials is the norm. Only very slowly is this mindset permeating
 into basic research to replace this ridiculous state of affairs. Last 
year, we saw the publication of the first randomized clinical trial in 
mice (1). 
The US ORI (United States Office of Research Integrity) defines Scientific misconduct
 as consisting of data fabrication, data falsification or plagiarism. 
But far more than any of these, the most prevalent practice is something
 that's not even on the radar, data selection, i.e., cherry-picking data.
 Practice is rampant. Rarely do animal model studies show data combined 
from different experiments. Take a look at any recent paper, even ones 
published in Nature or Science. Invariably a figure legend would say 
something along the lines of, 'Data from one representative experiment 
out of 3, 4 or 5 different experiments is shown'. Why not show combined 
data from all experiments performed? How could such a shoddy practice be
 the norm? Simply means intra-group variation between experiments was 
greater than inter-group variation within one single experiment. Either 
experimenters are shoddy or techniques too unrefined. Either way, cannot
 trust such data. And this is still the norm in basic biomedical 
research.  
Bibliography:
1. 
 Llovera, Gemma, et al. "Results of a preclinical randomized controlled 
multicenter trial (pRCT): Anti-CD49d treatment for acute brain 
ischemia." Science Translational Medicine 7.299 (2015): 
299ra121-299ra121. http://stm.sciencemag.org /conten... 
https://www.quora.com/What-is-Tirumalai-Kamalas-personal-experience-of-the-difference-in-the-way-basic-science-research-is-conducted-in-the-USA-and-India/answer/Tirumalai-Kamala
No comments:
Post a Comment