Lecture: «Natural language generation, language production and its interaction with vision and attention»

Language is never produced in a vacuum. Typically, humans talk about things, events and places that are available to them perceptually, or through shared experience. Natural Language Generation (NLG) is a field of computational linguistics and Artificial Intelligence that focusses on systems that generate text or speech from non-linguistic data. In a sense, then, NLG too has to deal with the problem of how non-linguistic (perceptual, numerical, and other) input can be mapped to linguistic structure. Does this mean that NLG, beyond its computational and practical aims, can also teach us something about human language production?

This talk will begin with an overview of NLG systems, how they are designed, and what domains they have been successfully deployed in. Central to this discussion will be the question of whether we want NLG systems to produce language like human beings, or whether we prefer such systems to produce language that is understandable or useful to human beings.

Then, Albert Gatt will zoom in on one subtask of NLG as a case study on these issues: Referring Expression Generation (REG) is a subtask of NLG systems concerned with generating referential descriptions (for instance, 'the tall woman in the corner', to refer to and identify one person in a room). Dr. Gatt will use REG to illustrate the problem of generating descriptive language for visual input, with the main argument that one can productively think of REG algorithms as (falsifiable) models of human referential behaviour; alternatively, one can seek experimental and behavioural evidence to improve such algorithms. Given the emphasis on reference in visual domains, some studies comparing the predictions of REG algorithms to those made by cognitive models and models of visual attention will be discussed.

About the speaker

Albert Gatt is a Senior Lecturer and Director of the Institute of Linguistics and Language Technology at the University of Malta and a Research Associate at the Tilburg center for Communication and Cognition (TiCC). His main research interests are in computational linguistics and NLP, particularly Natural Language Generation and the vision-language interface. His research combined machine learning and algorithmic modelling. However, he is also interested in the use of experimental methods to investigate language production from a psycholinguistic perspective.

Dr. Gatt is also actively involved in the creation of language resources and NLP tools for "small" languages, especially Maltese.