Original title: $σ$-PCA: a unified neural model for linear and nonlinear principal component analysis
Authors: Fahdi Kanavati, Lucy Katsnith, Masayuki Tsuneki
In this article, three methods—linear PCA, nonlinear PCA, and linear ICA—use single-layer autoencoders to learn transformations from data. Linear PCA maximizes variance but faces issues when axes share the same variance, resulting in rotational indeterminacy. Nonlinear PCA and linear ICA solve this by maximizing statistical independence with unit variance, reducing the problem to permutations rather than rotations. Linear ICA’s transformation breaks down into rotations, scales, and rotations via singular value decomposition. Linear PCA learns the first rotation, nonlinear PCA the second. The challenge is that conventional nonlinear PCA can’t directly learn the crucial first rotation. Enter $\sigma$-PCA: a neural model unifying linear and nonlinear PCA as single-layer autoencoders. It tackles this by modeling not just rotation but also scale (variances), bridging the gap between linear and nonlinear PCA. Like linear PCA, $\sigma$-PCA reduces dimensionality and orders by variances without suffering from rotational indeterminacy.
Original article: https://arxiv.org/abs/2311.13580