Any bounded continuous PDF can be approximated with an arbitrary accuracy using a weighted sum of Gaussian densities, if the number of the Gaussians is large enough [Feller, 1966]. If no prior knowledge of the functional form of the PDF is assumed, the mixture density estimation can be reduced to weighted kernel density estimation with, for example, Gaussian kernel functions. The idea of the self-organizing reduced kernel density estimator (RKDE) [Holmström and Hämäläinen, 1993,Hämäläinen, 1995] is to reduce the number of kernels by using the centroids of the weighted Gaussian kernel functions trained by SOM. The covariances of the kernels in kernel density estimators are normally reduced to the so-called smoothing parameters that assume the feature vector components to be independent and already optimally scaled.