Overfitting

A trained PCA model can be used for reconstructing missing values:

$$\hat{x}_{ij} = \sum_{k=1}^c a_{ik} s_{kj} \, , \quad (i,j) \notin O \, .$$ (16)

Although PCA performs a linear transformation of data, overfitting is a serious problem for large-scale problems with lots of missing values. This happens when the value of the cost function $C$ in Eq. (10) is small for training data, but the quality of prediction (16) is poor for new data. For further details, see [12].