S.Nieuwenhuis, B.U.Forstmann, and E-J Wagenmakers, Erroneous analyses of interactions in neuroscience: a problem of significance, Nature Neuroscience, vol. 14, no. 9, 2011
The paper mainly discusses the significant tests. However, I'd to say, when people apply machine learning techniques to neuroscience data (e.g. EEG, fMRI), erroneous analyses (even logically wrong) also exist. Sometimes the erroneous analyses are not explicitly, but more harmful.
One example is the application of ICA on the EEG/MEG/fMRI data. A key assumption of ICA is the independence or uncorrelation of "sources". This assumption is obviously violated in these neuroscience data. But some people seem to be too brave when using ICA to do analysis.
I am not saying using ICA to analyze neuroscience data is wrong. My point is: people should be more careful when using it:
(1) First, you should deeply understand ICA. You need to read enough classical papers, or even carefully read a book (e.g. A.Hyvarinen's book: independent component analysis).
I saw some people only read one or two papers and then jumped to the "ICA-analysis" job. Due to the availability of various ICA toolboxes for neuroscience, some people even didn't read any paper, and even could not correctly write the basic ICA model (really!).
It's very dangerous. This is because ICA is a complicated model and unfortunately, neuroscience is a more complicated field (probably the most complicated field in science). In the world there is nobody that have exact knowledge on the "sources" of EEG/MEG/fMRI data. As a result, people don't know whether the ICA separation is successful. This is different to other fields, where people can easily know whether their ICA is successful. For example, when people use ICA to separate speech signals, they can listen the separated signals to know whether the ICA separation is successful or not. But in neuroscience, you CAN NOT. We still lacks of much knowledge on these "sources" of EEG/MEG/fMRI data. This requires the analyzers to deeply understand the mathematical tools they are using: the sensitivity, the robustness, the all kinds of possibility of failure, etc.
It has been observed that ICA can split a signal emitted from an active brain area into two or more "independent sources". It has been observed that ICA only provides a temporal-averaged spatial distribution. It has also been observed that ICA fails when several brain activity are coupled. However, all these warnings are ignored by those brave people.
(2) Be careful when using two or more advanced machine learning analysis (e.g. ICA separation in a domain and then ICA separation in another domain, ICA followed by another exploring data analysis, etc). Due to the inconsistency of ICA models and neuroscience data, errors always exist. However, we don't have any knowledge on the errors from ICA. So, the errors from ICA is unpredictable, and such errors can also be unpredictably amplified when we use another advanced machine learning algorithm after ICA. The same goes to the use of other advanced algorithms successively.
In summary, ICA is a tiger, and to control it, the controller needs to be very skilled; otherwise, the controller will be seriously harmed by it.
---------------------------------------------------------------------------------------------
Nepenthes. x dyeriana.
This nepenthes was gaven by my friend, Bob, as a gift. It is a rare hybrid. Photo was taken by my friend Luo.
Nice post! I'm a begginer of ICA for multi-channel EEG analysis, so I'm really appreciated your thoughtful message.
ReplyDeleteCould u recommend any good article(s) that describe an ICA for neuroscience data in a good manner?(other than Hyvarinen's book)
Hi, ICALovers,
ReplyDeleteUnfortunately there are few ICA papers giving such warning and re-thinking on ICA's application to neuroscience data. What I've seen is the overwhelming cheer for the successful application of ICA on neuroscience (But there are also many excellent neuroscientists who refuse to use ICA on neuroscience data analysis or just use ICA to do artifact removal in neuroscience data).
My suggestion is:
First you can check the EEGLAB's website. There are many materials on how to use ICA to EEG/fMRI data. Also, you may want to register the EEGLAB maillist, since the EEGLAB's developers often answer people's questions, and there are many good discussions on how to use ICA to EEG.
Second, to carefully use ICA to EEG, you not only need to deeply understand ICA, but also need to deeply understand the neurophysics of EEG and the brain dynamics. Two books are must-to-read:
P.L.Nunez, R.Srinivasan, Electric Fields of The Brain: The Neurophysics of EEG, 2005
G.Buzsaki, Rhythms of the Brain, 2006
These two books have different emphasis. Nunez's book on EEG, while Buzsaki's book more on brain dynamics and more comprehensive.
Also, there are many excellent researchers on EEG dynamics, such as W.J. Freeman. I recommend you read some of his papers on brain dynamics.
After you get good knowledge on the physic models which ICA tries to capture (however, in my eyes, ICA does not capture such physic models well), and get good understanding on ICA's mathematical models, you will start to worry about the use of ICA on EEG for scientific research, and you may want to write your own papers to give such warnings and re-thinkings.
I am now co-working with some people in the Swartz Center for Computational Neuroscience to write a paper on the re-thinking of ICA's use to EEG. I think the paper will be finished soon in the coming months. Once it is been submitted, I will post it as a preprint in my homepage.
Btw: ICA has achieved a big success in BCI, which is an engineering field. And there is no things to worry about, since in this application we always know the "ground-truth" (e.g. the subject wants to move to the left or to the right).
Zhilin