Consideration of Sample Size in Neuroscience Studies
/ 4 min read
Abstract
Reproducibility of neuroscience studies is a primary goal of The Journal of Neuroscience. There are two main reasons for problems of reproducibility in the neuroscience literature. The first is the inflated false-positive rates that result in many studies falsely rejecting their null hypotheses.two main reasons for problems of reproducibility in the neuroscience literature [@NMPZXK65#Neuroscience-2020, p. 4076]
The first is the inflated false-positive rates that result in many studies falsely rejecting their null hypotheses. [@NMPZXK65#Neuroscience-2020, p. 4076]
Article aims to discuss low statistical power by discussing the issue of small sample sizes [@NMPZXK65#Neuroscience-2020, p. 4076]
focuses on a second reason for limited reproducibility in neuroscience studies: low statistical power, frequently caused by small sample sizes. [@NMPZXK65#Neuroscience-2020, p. 4076]
How can research determine the sample size needed to achieve a suitable effect size [@NMPZXK65#Neuroscience-2020, p. 4076]
provide suggestions on how to approach the determination of sample size in the context of the noisy and subtle effects often observed in neuroscience studies. [@NMPZXK65#Neuroscience-2020, p. 4076]
This is key theres clear differences in what an article wants to achieve! To find the presence of an effect or estimate what an effect is [@NMPZXK65#Neuroscience-2020, p. 4076]
determine the presence of an effect or to obtain accurate estimates of the effect. [@NMPZXK65#Neuroscience-2020, p. 4076]
Effects found in studies with low power are subject to the problem of low positive predictive value (Button et al., 2013). [@NMPZXK65#Neuroscience-2020, p. 4076]
The real effect sizes for most phenomena uncovered in exploratory studies are in fact smaller than reported, even without accounting for the procedural biases leading to inflated inferential statistics. [@NMPZXK65#Neuroscience-2020, p. 4076]
Unfortunately, this phenomenon is often missed given that many studies still fail to report effect size. [@NMPZXK65#Neuroscience-2020, p. 4076]
studies should accommodate and plan for two related experiments. [@NMPZXK65#Neuroscience-2020, p. 4076]
Studies should first provide evidence for the presence an effect exists first [@NMPZXK65#Neuroscience-2020, p. 4076]
First, an “exploratory” experiment provides provisional statistical evidence for the presence of an effect. [@NMPZXK65#Neuroscience-2020, p. 4076]
Second, an “estimation” experiment provides a more precise and accurate estimate of the real sizes of those effects. [@NMPZXK65#Neuroscience-2020, p. 4076]
We did actually do this in our 2nd paper. And this can be a response to our reviewer [@NMPZXK65#Neuroscience-2020, p. 4076]
The exploratory stage should also quantify the statistical power provided by the experimental design, either a priori or with post hoc simulations. [@NMPZXK65#Neuroscience-2020, p. 4076]
The sample size necessary to obtain an accurate estimate of an effect size is usually larger than the sample size necessary for adequate power to detect the presence of an effect (Maxwell et al., 2008). [@NMPZXK65#Neuroscience-2020, p. 4076]
Sample size of 100< is much too large in this case [@NMPZXK65#Neuroscience-2020, p. 4076]
The sample size necessary to obtain an accurate estimate of an effect size is usually larger than the sample size necessary for adequate power to detect the presence of an effect (Maxwell et al., 2008). [@NMPZXK65#Neuroscience-2020, p. 4076]
The suggested exploration-then-estimation procedure is functionally equivalent to practices already adopted by some subfields of neuroscience. For instance, in cognitive neuroscience, it is customary to separate the estimation phase of model fitting from the validation phase of the model parameters. [@NMPZXK65#Neuroscience-2020, p. 4076]
This is key for our revisions!! [@NMPZXK65#Neuroscience-2020, p. 4076]
While many of these practices typically rely on large sample sizes, some areas of neuroscience make statistical inferences on individual subjects, implementing a sort of exploration-then-estimation procedure across successive subjects (e.g., patients or nonhuman animal models in electrophysiology; machine-learning explorations of fMRI data; psychophysics and human brain lesion studies). These small4076 • The Journal of Neuroscience, May 20, 2020 • 40(21):4076 –407 [@NMPZXK65#Neuroscience-2020, p. 4076]
N approaches focus their statistical power on individual-level characterization of an effect; a finding is deemed present when all or a majority of a small pool of subjects show an effect, usually based on a large sample of trial-level observations (Smith and Little, 2018). [@NMPZXK65#Neuroscience-2020, p. 4077]