Interpreting Imprecise Expressions:
Experiments with Kohonen's Self-Organizing Maps and Associative Memory
Timo Honkela and Ari M. Vepsäläinen
Proceedings of the International Conference on Artificial
Neural Networks, ICANN '91, Helsinki,
June 24th-28th, 1991, Vol 1, pp. 897-902.
Abstract
A vast majority of the present computer models of natural language
processing are based on symbol manipulation, and especially work
concerning semantics relies heavily on formal symbolic
logic. Resulting models consist of a set of entities and relations
connecting those entities. Words in natural languages, however, seldom
are entities with such precise meanings and, therefore, cannot be
accurately modelled with symbolic logic. The meaning of a word, say
'big', is not an entity with fixed boundaries precisely and constantly
separating what is big from everything that is not big. Much more
commonly, a meaning is fuzzy and changing, biased at any moment by the
particular context. As such, words and their meanings bear an
underlying similarity to self-organizing neural network computer
models. We argue that such connectionist models have substantial
advantages for dealing with knowledge acquisition and for handling
context-dependency, exceptions, and the relations between words and
meanings. In the present paper we present simulations in which
Kohonen's self-organizing feature maps and associative memory are used
to model the interpretation of some imprecise expressions. We also
show some results on how to model the subjectivity of meanings.
[...]
4. Conclusions
In the connectionist model, the computer acquired knowledge via
examples and the output neurons, through self-organization within the
network, became organized in a manner resembling the likely semantic
mapping of humans, with imprecise boundaries between concepts,
contextual dependency and individual differences. We argue that
connectionist techniques are an important method in the research
concerning the semantics of natural language. Possible application
areas are information retrieval (see e.g. [9]) and machine
translation.
References
[1] Baker, G.P. and Hacker, P.M.S.: Language, Sense & Nonsense - A Critical Investigation into Modern Theories of Language. Basil Blackwell, Oxford, England, 1984, 397 p.
[2] Hallett, G.L.: Language and Truth. New Haven, Yale University Press, 1988, 234 p.
[3] Thrane, T.: Symbolic Representation and Natural Language. Nordiske Datalingvistikdage 1987, Proceedings, Lambda, Institut för Datalingvistik, Handelshjskolen i Kbenhavn, 7, 1988, pp. 118-154.
[4] Ritter, H. and Kohonen, T. : Self-Organizing Semantic Maps. Biological Cybernetics, 61, 1989, pp. 241-254.
[5] McClelland, J.L. and Kawamoto, A.H.: Mechanisms of Sentence Processing: Assigning Roles to Constituents of Sentences. in: McClelland, J.L. and Rumelhart, D.E. (eds.): Parallel Distributed Processing; Explorations in the Microstructure of Cognition; Vol. 2: Psychological and Biological Models. MIT Press, Boston, 1986.
[6] Kohonen T.: Correlation Matrix Memories. IEEE Transaction on Computers, Vol. 21, 1972, pp. 353-359.
[7] Kohonen T.: Self-Organization and Associative Memory. 2nd Ed., New York, Springer, 1988.
[8] Press, W.H., Flannery, B.P., Teukolsky, S.A. and Vetterling, W.T.: Numerical Recipes. Cambridge University Press, Cambridge, 1987.
[9] Belew, R.K.: Adaptive Information Retrieval: Using a connectionist representation to retrieve and learn about documents. Proceedings of the 12h Annual International ACMSIGIR Conference on Research and Development in Information Retrieval, ACM, New York, 1989, pp. 11-20.
[10] Wildgen, W. and Mottron, L.: Dynamische Sprachtheorie: Sprachbeschreibung und Spracherklärung nach den Prinzipien der Selbstorganisation und der Morphogenese. Bochum, Brockmeyer, 1987.
Timo Honkela's list of publications