My blogs reporting quantitative financial analysis, artificial intelligence for stock investment & trading, and latest progress in signal processing and machine learning

## Thursday, February 24, 2011

### M-SBL Using Gaussian Scale Mixture Model ?

Recently, there are some interesting works that use the Gaussian scale mixture (GSM) model in the framework of sparse Bayesian learning (SBL) . But I have not seen any papers using this model to modify the original MSBL algorithm proposed by David, or to deal with the multiple measurement vector (MMV) model under the common sparsity assumption, the model that was proposed by Cotter & Rao (Although there are some papers considering MMV models, their models are different to here). It seems that the GSM model can model the spatial dependency in X in some degree. So, to satisfy my curiosity, I modified the David's MSBL using the GSM model.

The MMV model under the common sparsity assumption is given by:
Y=AX + V,
where Y is the available data matrix of size N x L, A is the known dictionary matrix, and X is the unknown solution matrix of size M x L. V is the noise matrix. Assume:
X_i = sqrt(z) U_i,    i=1,...,L,
where X_i is the i-th column of the solution matrix X, z is a positive scalar, U_i is a column vector with elements being independent. Each element of U_i satisfied a Gaussian distribution N(0,\gamma_i). Then, following the steps of the derivation of MSBL, we can easily obtain the GSM based MSBL algorithm.

The algorithm has the similar form to MSBL. Essentially, their only difference is that the $\lambda$ in MSBL becomes $\lambda / z$ in GSM-MSBL. So, it seems that GSM-MSBL may correct the learning of $\lambda$. It is known that SBL's learning rules for $\lambda$ are not robust in low SNR cases. So, I expect that GSM-MSBL can yield a better recovery performance than MSBL through the correction of the learning rule for $\lambda$.

However,  by observing the derived learning rule for z, i.e.

I found the learning rule is exactly equal to 1, i.e. z = 1! In this case, GSM-MSBL is the same to MSBL. Admittedly, the above learning rule is obtained from the data using the empirical Bayesian method. Maybe there exist other methods to estimate z. But currently I don't know.

If anybody has got a GSM based MSBL algorithm showing better performance, please let me know.