Next: Discussion Up: Simulations Previous: Learning artificial data

Learning natural data

Next the algorithm was tested with natural data. A sheet of hand-written text was digitised with 100 dpi resolution. The samples were obtained by taking small 16 by 16 pixels square areas corresponding to 4mm by 4mm area in the original text. In figure 4.11 (a) there are 100 samples from the image. In the sample vectors, background colour of the sheet was coded by zero and the text was coded with positive values. A 2-dimensional map was adapted using parameters in table 4.2. Parameters and were held constant throughout the learning. Figure 4.11 (b) shows the weight vectors of the resulting map. Each neuron seems to be sensitive to a dot of ink at a localised part of the sample. It is also evident that the finite sampling area has caused the distribution of the neurons to be far from uniform. The neurons which are sensitive to ink at the edge of the sample are much more localised than the neurons which are sensitive to ink at the centre of the sample. This happens probably because there are quite many samples where only a small part of a letter is visible at the edge of the sample.

Table 4.2: Parameters in the test runs with 100 neuron maps and natural data.

In order to decrease the effect of the finite sampling area, the samples were weighted with a Gaussian function, whose full width at half maximum was the width of the sample, i.e., the weights at the middle of the borders of a sample were 1/2. All parameters were the same as in the previous test. Figure 4.12 shows some of the samples and the resulting map. The centre of the sample received more neurons to represent it. Some neurons have elongated dots in their weight vectors. This means that these neurons are sensitive to location as well as orientation.

Next the effect of increasing the number of neurons was tested. The data was the same as in the previous test but the number of neurons was increased to 400. The parameters used in the training are shown in table 4.3. The resulting map is shown in figure 4.13. Due to the larger number of neurons there is more competition and the neurons become more specialised. The majority of the neurons are sensitive to both location and orientation.

Table 4.3: Parameters in the test run with 400 neuron map and natural data.

Figure 4.14 shows the linear reconstructions of the sample images from the outputs of the maps. The reconstructions have been computed using equation 3.8. The acuity of the reconstructions based on the map with 400 neurons is naturally better than with 100 neurons. It should be noted that equation 3.8 does not give the exact reconstruction of the inputs, but nevertheless it can be used to give an idea about the amount of information in the outputs of the network. Also, the only goal of the algorithm is not to find optimal representation with respect to reconstruction error, which could be minimised using dense coding. The outputs are also required to be sparse, which has several other advantages over dense coding.

Figure: 100 samples from hand-written text (a). The text has been digitised with 100 dpi resolution, and each sample has 16 by 16 pixels corresponding to 4mm by 4mm area. White colour corresponds to zero and dark colours to positive values. A 2-dimensional map with 100 neurons (b) has learned to represent the samples. Due to the fairly small area in the samples the distribution of the filters is far from uniform.

Figure: 100 samples from hand-written text (a). The samples are otherwise similar to those in figure 4.11 (a), but in order to decrease the effect of the finite sampling area they have been weighted with a Gaussian function, whose full width at half maximum has been the width of the sample, i.e., the weights at the middle of the borders of a sample have been 1/2. A 2-dimensional map with 100 neurons (b) has learned to represent the samples. The distribution of the filters is more uniform than in figure 4.11 (b).

Figure: A 2-dimensional map with 400 neurons has learned samples taken from hand-written text. The samples have been weighted with a Gaussian function (see figure 4.12 (a)).

Figure: 100 samples (a) have been linerly reconstructed from the outputs of a map with 100 neurons (b) (the map in figure 4.12) and 400 neurons (c) (the map in figure 4.13). The reconstructions give an idea about the amount of information in the outputs.

Next: Discussion Up: Simulations Previous: Learning artificial data

Harri Lappalainen
Thu May 9 14:06:29 DST 1996