Hybrid Neural Symbolic Integration


Stefan Wermter, University of Sunderland, UK

Ron Sun, University of Alabama, USA


Workshop as part of International Conference on Neural Information Processing Systems
December 4 and 5, 1998, Breckenridge, Colorado

Description and motivation

In the past it was very controversial whether neural or symbolic approaches alone will be sufficient to provide a general framework for intelligent processing. In recent years, the field of hybrid neural symbolic processing has seen a remarkable development. The motivation for the integration of symbolic and neural models of cognition and intelligent behavior comes from many different sources.

From the perspective of cognitive neuroscience, a symbolic interpretation of an artificial neural network architecture is desirable, since the brain has a neuronal structure and the capability to perform symbolic processing. This leads to the question how different processing mechanisms can bridge the large gap between, for instance, acoustic or visual input signals and symbolic reasoning for instance for language processing, inferencing, etc.

From the perspective of knowledge-based processing, hybrid neural/symbolic representations are advantageous, since different mutually complementary properties can be integrated. Symbolic representations have advantages with respect to easy interpretation, explicit control, fast initial coding, dynamic variable binding and knowledge abstraction. On the other hand, neural representations show advantages for gradual analog plausibility, learning, robust fault-tolerant processing, and generalization to similar input. Since these advantages are mutually complementary, a hybrid symbolic connectionist architecture can be useful if different processing strategies have to be supported.

Areas of interest

- Integration of symbolic and neural techniques for

    - integrating techniques for language and speech processing
    - integrating different modes of reasoning and inferencing
    - combining different techniques in data mining
    - integration for vision, language, multimedia
    - hybrid techniques in knowledge based systems
    - combining fuzzy/neuro techniques
    - neural/symbolic techniques and applications in engineering

- Exploratory research in
    - emergent symbolic behavior based on neural networks
    - interpretation and explanation of neural networks
    - knowledge extraction from neural networks
    - various forms of interacting knowledge representations
    - dynamic systems and recurrent networks
    - evolutionary techniques for cognitive tasks (language,
      reasoning, etc)

- Autonomous learning systems for cognitive agents
    that utilize both neural and symbolic learning techniques


Talks will be 20 minutes, special invited talks
are marked as *** and are 30 minutes.

Dec. 4 Morning Session:
Structured connectionism, rule representation

Stefan Wermter, Ron Sun
Introduction and welcome to the workshop

***Jerome Feldman, David Bailey
Layered hybrid connectionist models for cognitive science

***Lokendra Shastri
Types and quantifiers in SHRUTI: a connectionist model of rapid
reasoning and relational processing

Steffen Hoelldobler, Yvonne Kalinke, Joerg Wunderlich
A recursive neural network for reflexive reasoning

Rafal Bogacz, Christophe Giraud-Carrier
A novel modular neural architecture for rule-based and
similarity-based reasoning

Nam Seog Park
Addressing knowledge representation issues in connectionist
symbolic rule encoding for general inference

Nelson A. Hallack and Gerson Zaverucha and Valmir C. Barbosa
Towards a hybrid model of first order theory refinement

Panel on "the issues of representation in hybrid models"
Chair: Ron Sun
Panelists: Jerry Feldman, Lee Giles, Risto Miikkulainen, David Waltz

5 min opening statement by each panelist

The focus of the panel is the issue of representation:
how can neural representation contribute to the power
of hybrid models? how can symbolic representation supplement
neural represnetation? how each type of representations
can be developed, acquired, or learned? What are the
principled ways these two types of representation
can be combined, synergistically ?

Dec. 4 Afternoon Session:
Neural language processing, distributed representations

***Marshall R. Mayberry, Risto Miikkulainen
SARDSRN: a neural network shift-reduce parser

***William C. Morris, Garrison W. Cottrell, Jeffrey L. Elman
The empirical acquisition of grammatical relations

Whitney Tabor
Context free grammar representation in neural networks

Curt Burgess, Kevin Lund
The transduction of symbolic environmental input into
high-dimensional distributed representations

Pentti Kanerva
Large patterns make great symbols: an example of learning
from example

Stephen I. Gallant
Context vectors: a step toward a "grand unified representation"

Paolo Frasconi, Marco Gori, Alessandro Sperduti
Integration of graphical-based rules with adaptive learning
of structured information

Stefan C. Kremer and John Kolen
Dynamical Recurrent Networks as Symbolic Processors

Dec. 5 Morning Session:
Neural and hybrid systems for cognitive processing

***David Waltz
The importance of importance

***Noel Sharkey, Tom Ziemke
Life, mind and robots: Biological inspirations and rooted cognition

G.K. Kraetzschmar, S. Sablatnoeg, S. Enderle, G. Palm
Using neurosymbolic integration in modelling robot environments:
a preliminary report

Timo Honkela
Self-organizing maps in symbol processing

Ronan Reilly
Evolution of symbolisation: Signposts to a bridge between
connectionist and symbolic systems

Christos Orovas, James Austin
A cellular neural associative array for symbolic vision

Panel on "hybrid and neural systems for the future"
Chair: Stefan Wermter
Panelists: Jim Austin, Joachim Diederich, Lee Giles, Noel Sharkey,
Hava Siegelman

5 min opening statement by each panelist

The focus of the panel is the impact of hybrid and neural
techniques in the future. How can we develop neural and
hybrid systems for new media? internet communication?
multimedia, web searching, data mining, neurocontrol for
robotics, integrating image/speech/language. What are the
strengths and weaknesses of hybrid neural techniques for
these tasks. Are current principles and methodologies in
neural and hybrid systems useful? How can they be
extende? What will be the impact of hybrid and neural
techniques in the future?

Dec. 5 Afternoon session:
Explanation and composition

***H. Lipson, H. T. Siegelmann
High order shape neurons for data structure decomposition

***Alan Tickle, Frederic Maire, Joachim Diederich
Extracting the knowledge embedded within trained artificial
networks: defining the agenda

Guido Bologna
Symbolic rule extraction form the DIMLP neural network

Gerhard Paass, Joerg Kindermann
Explaining bayesian ensemble classifier models

Peter Tino, Georg Dorffner, Christian Schittenkopf
Understanding state space organization in recurrent neural
networks with iterative function systems dynamics

 M.L. Vaughn, S.J. Cavill, S.J. Taylor, M.A. Foy, A.J.B. Fogg
Direct knowledge extraction and interpretation from a
multilayer perceptron network that performs low back pain

James A. Hammerton, Barry L. Kalman
Holistic computation and the sequential RAAM: an evaluation

Stefan Wermter

More information on the NIPS98 conference can be found at:

If you want to participate and submit please contact:

NIPS Workshop Contact

Professor Stefan Wermter
Research Chair in Intelligent Systems
University of Sunderland
School of Computing & Information Systems
St Peters Way
Sunderland SR6 0DD
United Kingdom

phone: +44 191 515 3279
fax:   +44 191 515 2781
email: stefan.wermter@sunderland.ac.uk