Notice: You have been identified as a bot, so no internal UID will be assigned to you. If you are a real person messing with your useragent, you should change it back to something normal.
> Functional MRI (fMRI) is 25 years old, yet surprisingly its most common statistical methods have not been validated using real data
> In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%
> a 15-year-old bug was found in 3dClustSim while testing the three software packages (the bug was fixed by the AFNI group as of May 2015, during preparation of this manuscript)
> It is not feasible to redo 40,000 fMRI studies, and lamentable archiving and data-sharing practices mean most could not be reanalyzed either
Imagine you did a scan, and the doc said: "We have to cut 20% of your brain out, shit is serious". And you relearn to walk and talk, and get back to a somewhat normal life after years and years of therapy. And it turns out, you were in the 70% of bogus results. Oh boy...
Anonymous B replied with this 10 years ago, 3 hours later, 5 hours after the original post[^][v]#698,683
Resting-state fMRI data from 499 healthy controls, downloaded from the 1,000 Functional Connectomes Project (4), were used for all analyses. Resting-state data should not contain systematic changes in brain activity, but our previous work (14) showed that the assumed activity paradigm can have a large impact on the degree of false positives. Several different activity paradigms were therefore used, two block based (B1 and B2) and two event related (E1 and E2); see Table 1 for details.
Fig. 1 presents the main findings of our study, summarized by a common analysis setting of a one-sample t test with 20 subjects and 6-mm smoothing [see SI Appendix, Figs. S1–S6 (20 subjects) and SI Appendix, Figs. S7–S12 (40 subjects) for the full results]. In broad summary, parametric software’s FWE rates for clusterwise inference far exceed their nominal 5% level, whereas parametric voxelwise inferences are valid but conservative, often falling below 5%. Permutation false positives are controlled at a nominal 5% for the two-sample t test, and close to nominal for the one-sample t test. The impact of smoothing and cluster-defining threshold (CDT) was appreciable for the parametric methods, with CDT P = 0.001 (SPM default) having much better FWE control than CDT P = 0.01 [FSL default; AFNI does not have a default setting, but P = 0.005 is most prevalent (21)]. Among the parametric software packages, FSL’s FLAME1 clusterwise inference stood out as having much lower FWE, often being valid (under 5%), but this comes at the expense of highly conservative voxelwise inference.
Yeah, if you do a bunch of voxel analyses willy-nilly the number of combinations and the statistics involved pretty much guarantee that you will find a bunch of random correlations. That's not a surprise. It also doesn't mean that if you go in with a specific hypothesis and are looking for a specific response that it is somehow just as accurate as looking for random noise.
I've used SPM on fMRI data that I have recorded. If you just go nuts with the statistics and look for random shit you will find it. That's not the same as trying to gauge a specific measure of blood flow in one area of the brain you know should be active.
DasSheeple (OP) replied with this 10 years ago, 1 hour later, 7 hours after the original post[^][v]#698,691
@previous (B) > If you just go nuts with the statistics and look for random shit you will find it.
That's the point. My example of cutting out 20% of your brain we're just for lulz. But all the sensationalistic studies about how "religious vs. atheist" brains work etc... it's literally what you said. You go nuts and ther's a 70% chance you'll find false positives.
> That's not the same as trying to gauge a specific measure of blood flow in one area of the brain you know should be active.
Yeah, my example in the @OP was perhaps bad. But still doesn't change what you said in the first sentence I quoted. What if you "know" it should be active, and it isn't, yet there's a 70% chance you'll see something anyway and vice versa...
I called some brain imaging buddies from Copengagen. They knew since late 2015 that this shit would hit the fan. Fun stuff...
Anonymous F joined in and replied with this 10 years ago, 2 hours later, 9 hours after the original post[^][v]#698,714
@previous (DasSheeple) > What if you "know" it should be active, and it isn't, yet there's a 70% chance you'll see something anyway and vice versa
No. Because you are comparing an active hypothesis with testing for random noise. If you ask a person a question and get a specific answer, then that is a result. If you ask a question and happen to get what sounds like an answer out of random crowd noise, then that's a false positive. The two are not the same, and you can't say that just because you get some measurable result from random crowd noise that asking questions of people is ineffective and prone to error. The two are not the same.
Anonymous G joined in and replied with this 10 years ago, 2 minutes later, 9 hours after the original post[^][v]#698,715
Why does everyone know/is a braom surgeon now? My life is an epoc fail
adam djensen joined in and replied with this 10 years ago, 8 minutes later, 9 hours after the original post[^][v]#698,716