Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position [25]

Original Abstract

A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by "learning without a teacher", and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname "neocognitron". After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consists of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of "S-cells', which show charac- teristics similar to simple cells or lower order hyper- complex cells, and the second layer consists of "C-cells" similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any "teacher" during the process of self- organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cells of the last layer, and conversely, this C-cell has become selectively responsive only to that stimulus pattern. That is, none of the C-cells of the last layer responds to more than one stimulus pattern. The response of the C-cells of the last layer is not affected by the pattern's position at all. Neither is it affected by a small change in shape nor in size of the stimulus pattern. 1.

Main points

• Reiteration of self-organized by ''learning without a teacher''
• Similar structure to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel.
• Network structure:
  • Input layer (photoreceptor array)
  • Cascade of modules each one with :
    • S-cells: in the first layer Simple cells or lower order hypercomplex cells
    • C-cells: in the second layer Complex cells or higher order hypercomplex cells
• Hubel and Wiesel : the neural network in the visual cortex has a hierarchy structure:
  • LGB (Lageral Geniculate Body)
  • Simple cells
  • Complex cells
  • Lower order hypercomplex cells
  • Higher order hypercomplex cells
• a cell in a higher stage generally has tendency to respond selectively to a more complicated feature of the stimulus pattern
• we extend the hierarchy model of Hubel and Wiesel, and hypothesize the existance of a similar hierarchy structure even in hte stages higher than hypercomplex cells.
• In the last module, the receptive field of each C-cell becomes so large as to cover the whole area of input layer , and each C-plane is so determined as to have only one C-cell
• The output of an S-cell in the -th S-plane in the l-th module is described below