Maximum Likelihood and Covariant Algorithms for Independent Component Analysis (abstract)

Maximum Likelihood and Covariant Algorithms for Independent Component Analysis

David J C MacKay

Bell and Sejnowski (1995) have derived a blind signal processing algorithm for a non-linear feedforward network from an information maximization viewpoint. This paper first shows that the same algorithm can be viewed as a maximum likelihood algorithm for the optimization of a linear generative model. Second, a covariant version of the algorithm is derived. This algorithm is simpler and somewhat more biologically plausible, involving no matrix inversions; and it converges in a smaller number of iterations. Third, this paper gives a partial proof of the `folk-theorem' that any mixture of sources with high-kurtosis histograms is separable by the classic ICA algorithm. Fourth, a collection of formulae are given that may be useful for the adaptation of the non-linearity in the ICA algorithm.

postscript (Cambridge UK).

postscript (Canada mirror).


David MacKay's: home page, publications. bibtex file.
Canadian mirrors: home page, publications. bibtex file.