1. Independent Variable Group Analysis

We suggest an approach, independent variable group analysis (IVGA), where the dependences of variables within a group are modeled, whereas the dependences between the groups are neglected. This generates a pressure towards building groups whose variables are mutually dependent but are largely independent of the variables in other groups.

Usually such variable grouping is performed by a domain expert, prior to modeling with automatic, adaptive methods. However, we claim that it is worthwhile and feasible to try to obtain groupings automatically in order to create structured, efficient large-scale models automatically.

In IVGA, each separate group can be modeled using any method, as long as a cost function that measures the quality of the representation, including both compactness and faithfullness, is derived for the model family. In particular, such a cost function can be derived within the statistical framework to generative modeling. In this approach, the best model is the one that gives the highest probability to the observations.

Furthermore, we demonstrate the approach by describing and evaluating an algorithm, ${\rm IVGA_{VQ}}$, that can be used for computing IVGA when the model space used for modeling dependences between variables consists of vector quantizers (VQs). Variational EM-algorithm is used for adapting the VQs and for computing the cost function.