More About the BSBL Codes and Demo Files for Telemonitoring of Physiological Signals

I got several emails asking some questions on the BSBL codes, especially the demo files for telemonitoring of fetal ECG in the package. I listed these questions and provided my answers:

1. Why provided the sensing matrix Phi in the software package?

     The Phi.mat in the software package stores a binary sparse matrix, which is used as the sensing matrix. But I have to emphasize that it is not an optimal sensing matrix. I just randomly generated it and then stored it and used it for all my experiments. This can help people exactly reproduce my results (When the dataset is given, different sensing matrices can result in slightly different performance).  As you know, designing an optimal sparse sensing matrix is an important direction in compressed sensing.

2. Can the demo files for telemonitoring of fetal ECG be directly used in a telemonitoring system?
     The demo files for telemonitoring of fetal ECG are just a mimicking of telemonitoring. In a real system, each channel recording should be processed by a BSBL-BO algorithm (parallel computation), and may need to use online ICA algorithms or at least on-line batch algorithms. The ICA algorithm used in the demo files is an off-line algorithm. You can use the (Extended-) Infomax algorithm, an online algorithm which can be accessed from the EEGLab.

3. Have you made  any real telemonitoring systems using the algorithm?

    The second author (Prof. Tzyy-Ping Jung) has led a team to build real-time telemonitoring systems for EEG, and some products have being sold in Taiwan.
    But these systems have not used compressed sensing technique. However, we are now considering to incorporate some BSBL algorithm. For your interests, please check the following links to see the videos on these telemonitoring systems for various applications:
      http://sccn.ucsd.edu/~jung/Site/Demos.html

      http://www.foxnews.com/scitech/2012/02/23/mind-reading-helmets-on-horizon-for-fighter-pilots/#ixzz1nbhinDD2

      http://health.universityofcalifornia.edu/2011/05/16/brain-monitoring-technology-makes-media-waves/

     

4. Can BSBL-BO work well when used in the in-direct recovery way (i.e. first recovering the representation coefficients of the signal in some transformed domain, and then recovering the original signal)?

    Yes, of course. In the DEMO_nonSparse.m I also give an example when BSBL-BO first recovers the DCT coefficients and then recovers the original fetal ECG. The performance is improved in this way. But such method does not work well for all physiological signals. In my experiments on some public EEG and EMG datasets, BSBL-BO using this method may have poorer performance than not using this method. 

5. Speed Improvement

     I am very glad that some people are interested in the BSBL-BO and want to try in their work. Please note the code is not optimized for speed. I chose a coding style for readability instead of for speed. There are some ways to improve the code:
     (1) Remove many "if-" in the code. The current code considers many simulation scenarios, such as block sparsity model with known partition (identical block size or random block size), with unknown partition, and recovery of non-sparse structured  signals, etc. According to your problem, you can remove those unnecessary parts.
     (2) There is a matrix inverse in each iteration. You can replace the current method by other fast matrix inverse method (check matrix computation textbooks), or consider the fast strategy used in Tipping and Faul (2003).
     (3) The updating of the intra-block correlation value in each iteration is not necessary. In some cases, you can figure out a good value according to a priori information. Also, the updating of noise variance in each iteration is not necessary.
     (4) Although I have said that BSBL-L1 is not suitable for telemonitoring of non-sparse physiological signals in the paper, later I found when replacing the group-Lasso type algorithms in the BSBL-L1 with some specific algorithms, we can get good performance with dramatically accelerated speed.
    Of course, as the author of the algorithm, I am responsible for making the code more efficient. When I have time, I will improve it along these ways. But in the near future I have no time to further improve it (Currently I am now deriving SBL algorithms to solve multi-modal/multi-dataset/longitudinal problems in neuroimaging). If you have improved it, or if you want to improve it, please let me know. I would like to share my ideas with you, and collaboration is welcome.

Codes of the BSBL Family and Demos for Telemonitoring of ECG have been updated

I have updated the codes of the BSBL family (including BSBL-EM, BSBL-BO, and EBSBL-BO). Their speed is twice of the old versions. Further, they are not sensitive to the scaling of the practical datasets, compared to the old versions.

The download link is here: http://dsp.ucsd.edu/~zhilin/BSBL_public.zip (Sometimes the link does not work. But you can download it from the bottom of the page: https://sites.google.com/site/researchbyzhang/bsbl)

Furthermore, I added a fold, which contains the demo files to perform the experiment in my post ("http://marchonscience.blogspot.com/2012/05/is-compressed-sensing-really-useful-for_5843.html", including using BSBL-BO to recover the raw recordings, and then using FastICA to do ICA decomposition. If you think there are other algorithms can do the work as BSBL-BO, you can use the demo files to try your favorite algorithms. And I will be very glad if you send your results to me.

I will release the code of BSBL-L1 soon.

Since tomorrow, I will modify the T-MSBL group. I am very sorry that the old version of T-MSBL is slow. There is due to some reasons. I have modified the code, and the speed is at least twice of the old one. In addition, I will release other variants of T-MSBL soon.

Is compressed sensing really useful for wireless telemonitoring of non-sparse physiological signals (Part 3)?

In the last post, the signal to recover is much easy, because the two fetal QRS complexes are relatively strong. Now I show you another example, in which the fetal QRS complexes are almost invisible.

Below are 8-channel abdominal recordings (download link can be found in [1]). The peaks you see are the maternal ECG's QRS complexes, which we are not interested in. What we are interested in are the QRS complexes of fetal ECG, which peak around 6, 217, 433, 643, 854, 1065, 1278, 1488, 1697, 1912, 2110, 2317 sampling points. However, you almost can't observe such peaks in each channel recording, since the fetal ECG is very weak and is buried in noise. So, any sparsifying processing can completely remove the fetal ECG QRS complexes.

The 8 recordings are compressed using the same binary sparse matrix A in my last post, and recovered by BSBL-BO using the same input parameters (using the direct recovery way). Results are shown below:

Apparently, the whole 8-channel recordings are recovered well. But note that the recovered recordings will be processed by independent component analysis (ICA) to extract a clean fetal ECG. Therefore, if there is any small distortion in the recovered recordings (the distortion may be not discernible but could destroy the mutual structure across the channel recordings), the ICA decomposition of the recovered recordings will be different from the ICA decomposition of the original recordings. To ensure the ICA decomposition is unchanged is crucial to practical clinical diagnosis.

Hence, we perform ICA decomposition on the recovered recordings using FastICA, and for comparison, we also perform the same ICA decomposition on the original recordings. The ICA decomposition of the original recordings is given below:

The ICA decomposition of the recovered recordings is given below:

Clearly, there is no significant difference between the two ICA decompositions. More importantly, the separated fetal ECG and maternal ECG from the recovered recordings are the same as those from the original recordings. For you convenience, I zoom in some of the separated components. Below are the separated maternal ECG (a)(b) and fetal ECG (c) from the original recordings:

And below are the separated maternal ECG (a)(b) and fetal ECG (c) from the recovered recordings:

Clearly, the fetal ECG separated from the recovered recordings is almost the same as the one separated from the original recordings.

Note that, in this task the BSBL-BO recovered the 8- recordings using the direct recovery method (without first recovering the representation coefficients and then reconstructing the original recordings) as stated in my last post. The results show the BSBL-BO is very powerful to recover non-sparse structured signals.

As I said early, the in-direct recovery method, i.e. first recovering the representation coefficients and then reconstructing the original recordings, is not effective in every situation, because for the applications considered here, small representation coefficients (in transformed domain) are very important and should be recovered. In [1] I have compared several well-known algorithms, which used the in-direct method. They all recovered the significant representation coefficients, and thus the maternal ECG could be obtained from the ICA decompostion. However, the ICA decompositions of their recovered recordings were still largely different from the genuine ICA decomposition, and no fetal ECG could be obtained. This is because the representation coefficients in the transformed domain are still not sparse enough, and these algorithms could not recover the majority of small representation coefficients. Interested readers can refer to Fig.13 in [1].

PS: Using the in-direct recovery method, BSBL-BO can get better performance.


We have other results when using BSBL-BO to recover EEG for brain-computer interface. We are now revising the paper. Once we have done, I will post the paper in my homepage.


[1] Zhilin Zhang, Tzyy-Ping Jung, Scott Makeig, Bhaskar D. Rao, Low Energy Wireless Body-Area Networks for Fetal ECG Telemonitoring via the Framework of Block Sparse Bayesian Learning, submitted to IEEE Trans. on Biomedical Engineering

Is compressed sensing really useful for wireless telemonitoring of non-sparse physiological signals (Part 2)?

In my last post, I have discussed the difficulties when using compressed sensing in wireless telemonitoring of non-sparse physiological signals. Essentially, these difficulties come from the requirements of practical use: the recovered signals  are generally further processed by other signal processing algorithms (e.g. independent component analysis) and/or pattern recognition algorithms (e.g. classification); some small characteristics in signals, although not salient, are very important for diagnosis (e.g. the QRS complexes of fetal ECG in raw abdominal recordings, the P-wave and T-wave in abnormal ECG recordings, the spike duration in EMG, and the fluctuation in some EEG signals). In the following, I will give you examples and show the results of the BSBL-BO algorithm.

The first example is the recovery of raw fetal ECG recordings (recorded from the mother's abdomen).

First, I use the fetal ECG recording in my last post to show the use of BSBL-BO. The recording contains one large peak (the QRS complex of the mother's ECG, which is not of interest) and two small peaks (the QRS complexes of the fetus' ECG, which is of interest), and large noise fluctuation. The recording can be viewed as a signal with block structure (the three peaks are blocks) but contaminated by strong noise (it's 'source noise', not the 'sensor noise' as in standard noisy compressed sensing model). Denote the recording by x. And it is compressed to y by: y = Ax, where A is a 125 x 250 binary sparse matrix (see [1] for details). .

Now I use BSBL-BO to recover x from y using A. Although BSBL-BO requires to know the block partition, we have pointed out that the block partition is in fact a regularization helping learn the covariance matrix of the signal in high dimensional space, and experiments also support that the user-defined block partition does not need to be consistent with the true block partition of the signal. Therefore, I randomly set the block partition like this: [1, 26, 51, 76, 101, ...], where each number indicates the starting location of each block.

If you are familiar with SBL algorithms, you must know SBL algorithms generally have a pruning-mechanism, namely they use a threshold to prune out some irrelevant coefficients. BSBL-BO also has such pruning-mechanism. But note that in this example, the recording x is a non-sparse signal; its every entry is non-zero. Therefore, I need to turn off the pruning-mechanism. This can be done by setting the input argument 'prune_gamma' to any non-positive constant (e.g. -1, 0).

Clearly, there is strong intra-block correlation in x. So we need to explore and exploit the intra-block correlation for better performance. This can be done by setting the input argument 'learntype' to 1 in the BSBL-BO code.

Then, running the algorithm, I get the results as shown below:

The top picture shows the original recording x, the second picture shows the result when exploiting the intra-block correlation, and the last one shows the result when ignoring the intra-block correlation. Clearly, when exploiting the intra-block correlation, the original signal can be recovered with high quality; the fetus' QRS complexes are recovered well.

I encourage you to repeat the above experiment using the demo code 'DEMO_nonSparse' in the package: dsp.ucsd.edu/~zhilin/BSBL_public.zip(Sometimes the link does not work. But you can download it from the bottom of the page: https://sites.google.com/site/researchbyzhang/bsbl)

I should emphasize: in the task BSBL-BO is directly recovering a non-sparse signal, namely, using y and A to directly recover the non-sparse x from the underdetermined inverse problem: y=Ax. I didn't see any other algorithms can successfully do this (In [1] I compared many representative algorithms).

Of course, we can assume that x is represented in other domain (i.e x = Bz), and first recover the representation coefficients z by solving the inverse problem y=(AB) z, and then recover the original signal x  by x = Bz. Using this method, BSBL-BO can get better result (using the same input parameters). And I believe, using this method, some existing algorithms may recover x as well. But note that, when other algorithms adopt this method to recover a non-sparse signal, they heavily rely on how sparse the representation coefficients z are. In fact, for more practical scenarios in telemonitoring, z is not sparse enough. In the next post, I will show you another example (should surprise you!).



[1] Zhilin Zhang, Tzyy-Ping Jung, Scott Makeig, Bhaskar D. Rao, Low Energy Wireless Body-Area Networks for Fetal ECG Telemonitoring via the Framework of Block Sparse Bayesian Learning, submitted to IEEE Trans. on Biomedical Engineering

Is compressed sensing really useful for wireless telemonitoring of non-sparse physiological signals (Part 1)?

Applying compressed sensing to wireless telemonitoring of physiological signals is not new. There have been a number of work published in recent years. However, the achievement is very limited.

This is because the following reasons:

1. Most published work were only successful on few types of physiological signals, such as adult ECG. This is because these signals are relatively sparse (or become sparse after some sparsifying processing). However, most physiological signals are non-sparse, such as fetal ECG, EEG, EMG, etc.

2. Current published work required that signals contain less noise and interference (otherwise the noisy signals are not sparse). However, the main goal of wireless telemonitoring is to allow people can walk freely at home or office. Thus artifacts/interference from muscle movement is unavoidable.

3. Most published work required some sparsifying pre-processing, namely setting the entries of signals with small values to zero. This sparsifying pre-processing requires some complicated techniques to set a threshold (used to set the entries to zero). But the sparsifying pre-processing has three crucial problems:
   (1) These techniques change the underlying structure in signals. For fetal ECG, which is very weak and even invisible in the recorded signals, this technique can completely remove the fetal ECG's QRS complexes.
   (2) These techniques need to occupy extra on-chip computation and thus consume extra energy, while energy consumption is a crucial problem in wireless telemonitoring.
   (3) The setting of thresholds may have problems in practical scenarios, such as when people are walking or in some abnormal situations (remember the target market of telemonitoring is for patients).

4. More importantly, practical wireless telemonitoring systems generally collect/send multi-channel signals. For example, in telemonitoring of EEG for BCI, the channel number generally ranges from 2 to 8; in telemonitoring of fetal ECG, the channel number generally ranges from 4 to 12. These multi-channel signals are sent to remote terminals and will be further processed by other advanced signal processing techniques, such as independent component analysis (ICA). These multi-channel signal processing techniques require that the underlying structure across multi-channel signals is intact. For example, ICA requires the underlying ICA mixing structure in collected multi-channel signals is intact. Therefore, a practical telemonitoring system requires that the raw physiological signals can be completely recovered even including the noise/interference contained in the recorded signals. In other words, this requires the non-sparse signals can be completely recovered, including its entries with small amplitudes. 

Clearly, we can see, if a compressed sensing algorithm can be widely used in wireless telemonitoring, it must has the ability to recover non-sparse physiological signals, which completely contradicts the sparsity assumption in all the compressed sensing algorithms.

One may ask, there is another way for compressed sensing of non-sparse signals: suppose a non-sparse signal can be represented in a transform domain (e.g. wavelet domains, DCT domains, etc), such that the representation coefficients are sparse, i.e.,
x = Bz,
where x is the signal, B is the basis of the transform domain, and z is the representation coefficients. Then, the signal x is compressed by,
y = Ax,
where y is the compressed signal, and A is the sensing matrix. In the remote terminal, the compressed data y and the matrix product AB are used to recover the sparse coefficients z  according to the relation: y = (AB) z. And then the original signal x is recovered by the relation x = Bz.

However, the success of this method relies on the sparsity of the representation coefficients z. Unfortunately, for most physiological signals, z is not sparse enough, and completely recovering z is still a challenge for most compressed sensing algorithms (especially the sensing matrix A is a binary sparse matrix, a requirement of telemonitoring with low energy consumption).

Here is an example. We transform a segment of fetal ECG signal into the DCT domain, and see the representation coefficients:

 

As we can see, the coefficients are still not sparse. There are many coefficients with small values. For any compressed sensing algorithms, it is not a problem to recover the big coefficients, but is a big problem to recover the small coefficients. Remember, recovering the small coefficients is very important to maintain the underlying structure of signals, especially for later multi-channel signal processing. For this example, recovering the small coefficients is important to recover the fetal ECG's QRS complexes (i.e. the peaks at 65 and 180 in figure(a)) and to maintain the underlying structure for later ICA decomposition (the goal is to use ICA to extract the weak fetal ECG, not just to recover the noisy recording).

In the next post, I will introduce my work:

Zhilin Zhang, Tzyy-Ping Jung, Scott Makeig, Bhaskar D. Rao, Low Energy Wireless Body-Area Networks for Fetal ECG Telemonitoring via the Framework of Block Sparse Bayesian Learning, submitted to IEEE Trans. on Biomedical Engineering

and I will show you how to use my BSBL-BO algorithm to break through the bottleneck in wireless telemonitoring using compressed sensing.

New Papers on Sparse Bayesian Learning (SBL)

I was super busy these days, and I am sorry that I have not updated my blog for a long time. Finally, I can find some time to write some things.


First, I've uploaded our CVPR 2012 paper on my website. I will upload the code tomorrow.

Jing Wan*, Zhilin Zhang*, Jingwen Yan, Taiyong Li, Bhaskar D. Rao, Shiaofen Fang, Sungeun Kim, Shannon Risacher, Andrew Saykin, Li Shen, Sparse Bayesian Multi-Task Learning for Predicting Cognitive Outcomes from Neuroimaging Measures in Alzheimer's Diseas, CVPR 2012 (*equal contribution)


I've uploaded my work on wireless telemonitoring of fetal ECG using compressed sensing. I planed to release the code once I have finally tested the code and optimized it. But since I listed the paper on my homepage, I received many emails asking for the code (Wow, it seems that many people are interested in telemonitoring/BCI, and are curious how compressed sensing is helpful to this field). So I am going to release the code of debug version tomorrow:

Zhilin Zhang, Tzyy-Ping Jung, Scott Makeig, Bhaskar D. Rao, Low Energy Wireless Body-Area Networks for Fetal ECG Telemonitoring via the Framework of Block Sparse Bayesian Learning, submitted to IEEE Trans. on Biomedical Engineering


We have an invited review paper on sparse Bayesian learning for structured signals. This paper reviewed our lab's work on SBL. We also hoped to include all the interesting SBL papers for structured signals. But since this is a short paper, we had to select limited papers.

Bhaskar D. Rao, Zhilin Zhang, Yuzhe Jin, Sparse Signal Recovery in the Presence of Intra-Vector and Inter-Vector Correlation, International Conference on Signal Processing and Communications (SPCOM 2012)


I (and my friend, Lei, together) was also invited to write a review paper on SBL for a Chinese journal. So, I posted my part in my homepage, which can be downloaded here: http://dsp.ucsd.edu/~zhilin/papers/SBL_introduction.pdf. I hope this short review can give Chinese people a bird-eye on my work on SBL.


Finally, I was selected to attend the Doctoral Consortium of CVPR 2012. I will have a  poster to present all my work on SBL. I am eager to see you and receive your comments.

Celebrating the 50th Anniversary of the Neurosciences Research Program: Neuroscience and Higher Brain Function: State of the Art

This Tuesday The Neurosciences Institute holds a workshop, titled "Neuroscience and Higher Brain Function: State of the Art". Here is the schedule: http://www.nsi.edu/nrp/index.html. See you tomorrow!

Celebrating the 50^th Anniversary of the Neurosciences Research Program

Neuroscience and Higher Brain Function: State of the Art
Tuesday, April 17, 2012
The Neurosciences Institute Auditorium
10640 John Jay Hopkins Drive, San Diego, CA 92121
8:30 am - 5:30 pm

Welcome and Introduction
8:30 am	Gerald Edelman
Cortex: A View from the Top Chairman: Gerald Edelman
8:40 am	Ranulfo Romo	Perceptual Decision Processes across Cortex
	Jon Kaas	Cortical Sensorimotor Networks
	Thomas Albright	On the Perception of Probable Things
Memory and Cognitive Control Chairman: Larry Squire
10:50 am	György Buzsaki	Neural Syntax
	John O'Keefe	Memory Systems of the Medial Temporal Lobes
	Mark D'Esposito	Modularity, Networks, and Cognitive Control
12:45 pm	Lunch
Value and Cognition Chairman: Einar Gall
1:45 pm	Michael Merzenich	"Cultural Neuroscience": How the World Changes Your Brain and Our Brains Change the World
	Joaquin Fuster	The Prefrontal Cortex: Executive, Controller, or Enabler of the Perception/Action Cycle?
	Yasushi Miyashita	Cognitive Memory and its Cellular/Network Machinery: How Global Brain-wide Networks Interact with Local Micro-circuits
Diagnosis and Repair Chairman: Gerald Edelman
3:55 pm	Eberhard Fetz	Bidirectional Interactions between the Brain and Implantable Computers
	Apostolos Georgopoulos	Prewhitening for Brain Discovery
Summary
5:20 pm	Gerald Edelman

Discovering oscillatory interaction networks with M/EEG: challenges and breakthroughs

Sometimes people asked me why I am interested in EEG/MEG source localization. In their eyes EEG/MEG source localization seems to be a direction with minor importance. However, when I researched in ICA with application to EEG/MEG data analysis before 2009, I realized the importance of EEG/MEG source localization, and this is why I started my research on sparse signal recovery in 2009.

I can spend more than 1000 words to detail the reasons. But the following paper well explained for me. Particularly, this nice paper implies the true value of EEG/MEG source localization study: the true value is not to pursuit higher spatial resolution, but serves as a crucial step for mining the brain connectivity. Here is the paper:

Satu Palva, J.Matias Palva, Discovering oscillatory interaction networks with M/EEG: challenges and breakthroughs, Trends in Cognitive Sciences, vol.16, no.4, 2012, pp.219-230

Abstract:

The systems-level neuronal mechanisms that coordinate temporally, anatomically and functionally distributed neuronal activity into coherent cognitive operations in the human brain have remained poorly understood. Synchronization of neuronal oscillations may regulate net- work communication and could thus serve as such a mechanism. Evidence for this hypothesis, however, was until recently sparse, as methodological challenges limit the investigation of interareal interactions with non- invasive magneto- and electroencephalography (M/EEG) recordings. Nevertheless, recent advances in M/EEG source reconstruction and clustering methods support complete phase-interaction mappings that are essential for uncovering the large-scale neuronal assemblies and their functional roles. These data show that synchroniza- tion is a robust and behaviorally significant phenomenon in task-relevant cortical networks and could hence bind distributed neuronal processing to coherent cognitive states.

Paper Club: Whatever Next? Predictive Brains, Situated agents and the future of cognitive science

Today 11 am at Ed's lab (3509 Mandler) we will discuss a very interesting and comprehensive paper:

Andy Clark (2012) "Whatever Next? Predictive Brains, Situated agents and the future of cognitive science", to appear in Behavioral and Brain Sciences

Here is the download link:  http://www.fil.lu.se/files/eventfile2811.pdf

Abstract: Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to adaptive success. The paper critically examines this ‘hierarchical prediction machine’ approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency.

My comment: Although the paper does not involve any computational tools or specific computational frameworks, it in fact points out the incapability of ICA and sparse coding in the computational vision field. Also, I further understand that why deep belief networks and other hierarchical Bayesian models are so promising in the computational vision field.

My New and Permanent Homepage

Considering I will graduate and will leave UCSD, I made another homepage: https://sites.google.com/site/researchbyzhang/,  and this should be my permanent homepage no matter where I am. Welcome to visit my new homepage.

Presentation: COMPRESSED SENSING AND SPARSE SIGNAL RECOVERY BY SPARSE BAYESIAN LEARNING: MODELS, ALGORITHMS, AND APPLICATIONS

In this Thursday's Research Expo 2012 (April 12), I will present my previous and on-going work on sparse signal recovery/compressed sensing using sparse Bayesian learning (SBL).

Here is the abstract:

Compressed sensing / sparse signal recovery is a hot field in signal processing. Numerous algorithms have been proposed and have shown promising successes in applications. Among these algorithms, sparse Bayesian learning (SBL) has outstanding performance. In this presentation I will summarize our lab's recent work on SBL. I will present four new models that data-adaptively learn and exploit signals' temporal, spatial, spatiotemporal, and dynamic information. The derived algorithms from these models have shown the best, or at least top-tier, performance among existing compressed sensing algorithms in both computer simulations and practical applications (e.g. telemonitoring, biomarker selection in gene expression, source localization, earthquake detection, neuroimaging). Particularly, some of them have:

(1) solved the challenge of non-sparse physiological signals (e.g. fetal ECG contaminated by strong noise, EEG, EMG, etc) telemonitoring via wireless body-area networks with ultra-low power consumption, which was not solved before;  (this application will show SBL's ability to recover non-sparse signals with little distortion, using simple sparse binary matrices as its sensing matrices)
(2) achieved higher EEG source localization accuracy than other famous algorithms in more complicated environments (this application will show SBL's excellent ability under strong noise ( 0dB), highly coherent sensing matrix, and frequently changing signals);
(3) broke the record of predicting cognitive outcomes from neuroimaging measures in Alzheimer's disease in 2011 (showing SBL's ability to  deal with highly coherent sensing matrix);
(4) Brain-Computer Interface (solving the speed problem of SBL)
(5) obtained the best accuracy in earthquake detection in some common datasets;

Some of these work have been published or submitted to: IEEE Trans. on Signal Processing, IEEE Journal of Selected Topics in Signal Processing, Proceedings of the IEEE, IEEE Trans. on Biomedical Engineering, NeuroImage, CVPR 2012, and ICASSP 2010, 2011, 2012. Also, a US patent is pending.

Welcome to hear my presentation on the recent progress on various spatiotemporal SBL models!

------------------------------------------------------------------------
Photo: N. talangensis (taken by April 7)

Literature-ICA/LVA: Recently published/accepted ICA papers

Here are other recently published/accepted ICA papers. Enjoying!

-------------------------------------------------------------------------------------------------------------------

Shashwath A. Meda, Balaji Narayanan, Jingyu Liu, Nora I. Perrone-Bizzozero, Michael C. Stevens, Vince D. Calhoun, David C. Glahn, Li Shen, Shannon L. Risacher, Andrew J. Saykin, Godfrey D. Pearlson, A large scale multivariate parallel ICA method reveals novel imaging-genetic relationships for Alzheimer's disease in the ADNI cohort, NeuroImage, vol.60, 2012

The underlying genetic etiology of late onset Alzheimer's disease (LOAD) remains largely unknown, likely due to its polygenic architecture and a lack of sophisticated analytic methods to evaluate complex genotype-phenotype models. The aim of the current study was to overcome these limitations in a bimultivariate fashion by linking intermediate magnetic resonance imaging (MRI) phenotypes with a genome-wide sample of common single nucleotide polymorphism (SNP) variants. We compared associations between 94 different brain regions of interest derived from structural MRI scans and 533,872 genome-wide SNPs using a novel multivariate statistical procedure, parallel-independent component analysis, in a large, national multi-center subject cohort. The study included 209 elderly healthy controls, 367 subjects with amnestic mild cognitive impairment and 181 with mild, early-stage LOAD, all of them Caucasian adults, from the Alzheimer's Disease Neuroimaging Initiative cohort. Imaging was performed on comparable 1.5 T scanners at over 50 sites in the USA/Canada. Four primary "genetic components" were associated significantly with a single structural network including all regions involved neuropathologically in LOAD. Pathway analysis suggested that each component included several genes already known to contribute to LOAD risk (e.g. APOE4) or involved in pathologic processes contributing to the disorder, including inflammation, diabetes, obesity and cardiovascular disease. In addition significant novel genes identified included ZNF673, VPS13, SLC9A7, ATP5G2 and SHROOM2. Unlike conventional analyses, this multivariate approach identified distinct groups of genes that are plausibly linked in physiologic pathways, perhaps epistatically. Further, the study exemplifies the value of this novel approach to explore large-scale data sets involving high-dimensional gene and endophenotype data.

[My comments: I really like this paper, not only the algorithm used but also the application. This application is closely related to the one in my CVPR paper. And, some of the authors here are also the authors of my CVPR paper.]



-------------------------------------------------------------------------------------------------------------------

Matthew Anderson, Tülay Adalı, Xi-Lin Li, Joint Blind Source Separation With Multivariate Gaussian Model: Algorithms and Performance Analysis, IEEE Trans. on Signal Processing, vol.60, no.4, 2012

Abstract: In this paper, we consider the joint blind source separation (JBSS) problem and introduce a number of algorithms to solve the JBSS problem using the independent vector analysis (IVA) framework. Source separation of multiple datasets simultaneously is possible when the sources within each and every dataset are independent of one another and each source is dependent on at most one source within each of the other datasets. In addition to source separation, the IVA framework solves an essential problem of JBSS, namely the identification of the dependent sources across the datasets. We propose to use the multivariate Gaussian source prior to achieve JBSS of sources that are linearly dependent across datasets. Analysis within the paper yields the local stability conditions, nonidentifiability conditions, and induced Cramér-Rao lower bound on the achievable interference to source ratio for IVA with multivariate Gaussian source priors. Additionally, by exploiting a novel nonorthogonal decoupling of the IVA cost function we introduce both Newton and quasi-Newton optimization algorithms for the general IVA framework.

[My comments: Joint analysis of multiple datasets is a very important and meaningful topic in biomedicine. The topic is a hot one not only in BSS but also in other machine learning subfields, even in sparse signal recovery/L1-penalized regression in high-dimensional space]

-------------------------------------------------------------------------------------------------------------------

Gautam V. Pendse, PMOG: The projected mixture of Gaussians model with application to blind source separation, Neural Networks, vol.28, 2012, pp.40-60

Abstract: We extend the mixtures of Gaussians (MOG) model to the projected mixture of Gaussians (PMOG) model. In the PMOG model, we assume that q dimensional input data points zi are projected by a q dimensional vector w into 1-D variables ui. The projected variables ui are assumed to follow a 1-D MOG model. In the PMOG model, we maximize the likelihood of observing ui to find both the model parameters for the 1-D MOG as well as the projection vectorw. First, we derive an EM algorithm for estimating the PMOG model. Next, we show how the PMOG model can be applied to the problem of blind source separation (BSS). In contrast to conventional BSS where an objective function based on an approximation to differential entropy is minimized, PMOG based BSS simply minimizes the differential entropy of projected sources by fitting a flexible MOG model in the projected 1-D space while simultaneously optimizing the projection vector w. The advantage of PMOG over conventional BSS algorithms is the more flexible fitting of non-Gaussian source densities without assuming near-Gaussianity (as in conventional BSS) and still retaining computational feasibility.

[My comments: MOG is a very useful probabilistic model for BSS algorithms. I am glad to read this paper]

 -------------------------------------------------------------------------------------------------------------------

Jen-Tzung Chien, Hsin-Lung Hsieh, Convex Divergence ICA for Blind Source Separation, IEEE Trans. on Audio, Speech, and Language Processing, vol.20, no.1, 2012

Abstract: Independent component analysis (ICA) is vital for unsupervised learning and blind source separation (BSS). The ICA unsupervised learning procedure attempts to demix the observation vectors and identify the salient features or mixture sources. This work presents a novel contrast function for evaluating the dependence among sources. A convex divergence measure is developed by applying the convex functions to the Jensen’s inequality. Adjustable with a convexity parameter, this inequality-based divergence measure has a wide range of the steepest descents to reach its minimum value. A convex divergence ICA (C-ICA) is constructed and a nonparametric C-ICA algorithm is derived with different convexity parameters where the non-Gaussianity of source signals is characterized by the Parzen window-based distribution. Experimental results indicate that the specialized C-ICA significantly reduces the number of learning epochs during estimation of the demixing matrix. The convergence speed is improved by using the scaled natural gradient algorithm. Experiments on the BSS of instantaneous, noisy and convolutive mixtures of speech and music signals further demonstrate the superiority of the proposed C-ICA to JADE, Fast-ICA, and the nonparametric ICA based on mutual information.

[My comments: A nice paper dealing with dependence among sources]

 -------------------------------------------------------------------------------------------------------------------

Martin Kleinsteuber, Hao Shen, Blind Source Separation With Compressively Sensed Linear Mixtures, IEEE Signal Processing Letters, vol.19, no.2, 2012

Abstract: This work studies the problem of simultaneously separating and reconstructing signals from compressively sensed linear mixtures. We assume that all source signals share a common sparse representation basis. The approach combines classical Compressive Sensing (CS) theory with a linear mixing model. It allows the mixtures to be sampled independently of each other. If samples are acquired in the time domain, this means that the sensors need not be synchronized. Since Blind Source Separation (BSS) from a linear mixture is only possible up to permutation and scaling, factoring out these ambiguities leads to a minimization problem on the so-called oblique manifold. We develop a geometric conjugate subgradient method that scales to large systems for solving the problem. Numerical results demonstrate the promising performance of the proposed algorithm compared to several state of the art methods.
[My comments: It's interesting to see the hybrid of compressed sensing and ICA]

 -------------------------------------------------------------------------------------------------------------------

Fasong Wang, Linrang Zhang, Rui Li, Harmonic retrieval by period blind source extraction method: Model and algorithm, accepted by Digital Signal Processing, 2012

Abstract: A frequently encountered problem in signal processing field is harmonic retrieval in additive colored Gaussian or non-Gaussian noise, especially when the frequency of the harmonic signals are closely spaced in frequency domain. The purpose of this paper is to develop novel harmonic retrieval algorithm based on blind source extraction(BSE) method from linear mixtures of harmonic signals using only one observed channel signal. First, we establish the blind source separation(BSS) based harmonic retrieval model in additive noise using the only one observed channel, at the same time the fundamental principle of BSE based harmonics retrieval algorithm is analyzed in detail. Then, based on the established harmonic BSS model, we propose a BSE approach to the harmonic retrieval using the concept of period BSE method, as a result, the harmonic retrieval algorithm using only one channel mixed signals is derived. Simulation results show that the proposed algorithm is able to separate the harmonic source signals and yield ideal performance.

[My comments: Glad to see another model for blind source extraction using only one channel signal. And more glad to see my previous work has been cited here. But I really hope to see what's the performance when used to extract FECG.]

Literature-ICA/LVA: Special Issue on Latent Variable Analysis and Signal Separation

Signal Processing has a special issue on Latent Variable Analysis and Signal Separation. (Volume 92, Issue 8, Pages 1765-1960 (August 2012)). Now the papers can be downloaded from the website: http://www.sciencedirect.com/science/journal/01651684/92/8



Below are the papers:

Consistency and asymptotic normality of FastICA and bootstrap FastICA
Nima Reyhani, Jarkko Ylipaavalniemi, Ricardo Vigário, Erkki Oja

Independent component analysis based on first-order statistics
V. Zarzoso, R. Martín-Clemente, S. Hornillo-Mellado

ICA-based and second-order separability of nonlinear models involving reference signals: General properties and application to quantum bits
Yannick Deville

ICA over finite fields—Separability and algorithms
Harold W. Gutch, Peter Gruber, Arie Yeredor, Fabian J. Theis

Stability of independent vector analysis
Takashi Itahashi, Kiyotoshi Matsuoka

Complex-valued independent vector analysis: Application to multivariate Gaussian model
Matthew Anderson, Xi-Lin Li, Tülay Adalı

Multiple-snapshots BSS with general covariance structures: A partial maximum likelihood approach involving weighted joint diagonalization
Arie Yeredor

Extraction of signals with higher order temporal structure using Correntropy
Eder Santana, Jose C. Principe, Ewaldo Santana, Allan Kardec Barros

Algorithms for probabilistic latent tensor factorization
Y. Kenan Yılmaz, A. Taylan Cemgil

Supervised input space scaling for non-negative matrix factorization
J. Driesen, H. Van hamme

ISI sparse channel estimation based on SL0 and its application in ML sequence-by-sequence equalization
Rad Niazadeh, Sina Hamidi Ghalehjegh, Massoud Babaie-Zadeh, Christian Jutten

A tractable framework for estimating and combining spectral source models for audio source separation
Simon Arberet, Alexey Ozerov, Frédéric Bimbot, Rémi Gribonval

Regulatory component analysis: A semi-blind extraction approach to infer gene regulatory networks with imperfect biological knowledge
Chen Wang, Jianhua Xuan, Ie-Ming Shih, Robert Clarke, Yue Wang

Use of bimodal coherence to resolve the permutation problem in convolutive BSS
Qingju Liu, Wenwu Wang, Philip Jackson

The signal separation evaluation campaign (2007–2010): Achievements and remaining challenges
Emmanuel Vincent, Shoko Araki, Fabian Theis, Guido Nolte, Pau Bofill, Hiroshi Sawada, Alexey Ozerov, Vikrham Gowreesunker, Dominik Lutter, Ngoc Q.K. Duong

Informed source separation through spectrogram coding and data embedding
Antoine Liutkus, Jonathan Pinel, Roland Badeau, Laurent Girin, Gaël Richard

Multi-source TDOA estimation in reverberant audio using angular spectra and clustering
Charles Blandin, Alexey Ozerov, Emmanuel Vincent

A blog post deeply explains the recent political event in China

http://blog.sina.com.cn/s/blog_4a405fd90100g4ce.html

Our three papers on BCI published in the Proceedings of the IEEE

In this May, the Proceedings of the IEEE will publish a special centennial celebration issue: Reviewing the Past, the Present, and the Future of Electrical Engineering Technology. In this issue our lab and our collaborative labs have three papers on brain-computer interface (BCI). They are:

Brent J. Lance, Scott E. Kerick, Anthony J. Ries, Kelvin S. Oie, and Kaleb McDowell, Brain-Computer Interface Technologies in the Coming Decades.

This paper focuses on using online brain–signal processing to enhance human–computer interactions; it highlights past and current BCI applications and proposes future technologies that will make significant expansion into education, entertainment, rehabilitation, and human–system performance domains.


Lun-De Liao, Chin-Teng Lin, Kaleb McDowell, Alma E. Wickenden, Klaus Gramann, Tzyy-Ping Jung, Li-Wei Ko, and Jyh-Yeong Chang, Biosensor Technologies for Augmented Brain-Computer Interfaces in the Next Decades.

This paper focuses on recent and projected advances of a wide range of sensor and acquisition neurotechnologies enabling online brain–signal processing in everyday, real-life environments, and highlights current and future approaches to address the challenges in this field. 


Scott Makeig, Christian Kothe, Tim Mullen, Nima Bigdely-Shamlo, Zhilin Zhang, Kenneth Kreutz-Delgado, Evolving Signal Processing for Brain-Computer Interface

This paper discuss the challenges associated with building robust and useful BCI models from accumulated biological knowledge and data, and the technical problems associated with incorporating multimodal physiological, behavioral, and contextual data that may become ubiquitous in the future.

The third paper has been introduced in my previous post. Now the final version can be downloaded from here.

The first two papers can be downloaded from the IEEE Xplore.

My comments: When talking about BCI, many people just think of such a scenario: a person, wearing a strange cap with electrodes, is sitting in front of a computer screen, watching the symbols appeared in the screen. But this is a traditional BCI. In fact, the BCI concept is largely widened now. Scott once gave a talk on one of the future directions, titled " Imaging Human Agency with Mobile Brain/Body Imaging (MoBI) ". The video is here http://sccn.ucsd.edu/eeglab/Online_EEGLAB_Workshop/EEGLAB12_MoBI.html

This direction is very important. In fact, I strongly feel such technique can open new worlds in several fields and encourage the hybrid of some traditional fields.