Zhou Wang, AlanC.Bovik, Mean Squared Error: Love it or Leave it? A New Look at Signal Fidelity Measures, IEEE Signal Processing Magazine, vol.26, no.1, 2009, pp.98-117
The authors list the implicit assumptions when using MSE, which I quote below:
1) Signal fidelity is independent of temporal or spatial relationships between the samples of the original signal. In other words, if the original and distorted signals are randomly re-ordered in the same way, then the MSE between them will be unchanged.
2) Signal fidelity is independent of any relationship between the original signal and the error signal. For a given error signal, the MSE remains unchanged, regardless of which original signal it is added to.
3) Signal fidelity is independent of the signs of the error signal samples.
4) All signal samples are equally important to signal fidelity.
Obviously, when we measure the recovery quality of structured signals, the above assumptions are violated.
The authors give a number of nice examples. Here is one of them:
(a) is the original image, and (b)-(d) are three images added noise. (b),(c),(d) almost have the same MSE, but clearly their recovery quality is different. MSE fails to show such difference. In contrast, the other two measurement indexes, SSIM and CW-SSIM, express well the difference.
SSIM, standing for Structural SIMilarity (SSIM) index, is proposed for structured signals, especially images. The basic form of SSIM (measuring small patches of an image) is:
There are many variants of the basic SSIM index. Interested people can read the paper and the references cited.
The codes of computing SSIM can be found here: https://ece.uwaterloo.ca/~z70wang/research/ssim/
Note that the SSIM index can be easily modified to measure 1-D structured signals.
Igor has two posts on SSIM in his Nuit Blanche:
And here is the most recent improvement on SSIM:
CALIBRATING MS-SSIM FOR COMPRESSION DISTORTIONS USING MLDS, by C. Charrier, K. Knoblauch, L. T. Maloney and A. C. Bovik, ICIP 2011.