Hybrid Neural Symbolic Integration

 

Stefan Wermter, University of Sunderland, UK

Ron Sun, University of Alabama, USA

 

Workshop as part of International Conference on Neural Information Processing Systems
December 4 and 5, 1998, Breckenridge, Colorado


Description and motivation

In the past it was very controversial whether neural or symbolic approaches alone will be sufficient to provide a general framework for intelligent processing. In recent years, the field of hybrid neural symbolic processing has seen a remarkable development. The motivation for the integration of symbolic and neural models of cognition and intelligent behavior comes from many different sources.

From the perspective of cognitive neuroscience, a symbolic interpretation of an artificial neural network architecture is desirable, since the brain has a neuronal structure and the capability to perform symbolic processing. This leads to the question how different processing mechanisms can bridge the large gap between, for instance, acoustic or visual input signals and symbolic reasoning for instance for language processing, inferencing, etc.

From the perspective of knowledge-based processing, hybrid neural/symbolic representations are advantageous, since different mutually complementary properties can be integrated. Symbolic representations have advantages with respect to easy interpretation, explicit control, fast initial coding, dynamic variable binding and knowledge abstraction. On the other hand, neural representations show advantages for gradual analog plausibility, learning, robust fault-tolerant processing, and generalization to similar input. Since these advantages are mutually complementary, a hybrid symbolic connectionist architecture can be useful if different processing strategies have to be supported.
 

Areas of interest
 

- Integration of symbolic and neural techniques for

    - integrating techniques for language and speech processing
    - integrating different modes of reasoning and inferencing
    - combining different techniques in data mining
    - integration for vision, language, multimedia
    - hybrid techniques in knowledge based systems
    - combining fuzzy/neuro techniques
    - neural/symbolic techniques and applications in engineering

- Exploratory research in
    - emergent symbolic behavior based on neural networks
    - interpretation and explanation of neural networks
    - knowledge extraction from neural networks
    - various forms of interacting knowledge representations
    - dynamic systems and recurrent networks
    - evolutionary techniques for cognitive tasks (language,
      reasoning, etc)

- Autonomous learning systems for cognitive agents
    that utilize both neural and symbolic learning techniques
 

Program

Talks will be 20 minutes, special invited talks
are marked as *** and are 30 minutes.
 

Dec. 4 Morning Session:
Structured connectionism, rule representation
---------------------------------------------

7:30
Stefan Wermter, Ron Sun
Introduction and welcome to the workshop

7:30
***Jerome Feldman, David Bailey
Layered hybrid connectionist models for cognitive science

8:00
***Lokendra Shastri
Types and quantifiers in SHRUTI: a connectionist model of rapid
reasoning and relational processing

8:30
Steffen Hoelldobler, Yvonne Kalinke, Joerg Wunderlich
A recursive neural network for reflexive reasoning

8:50
Rafal Bogacz, Christophe Giraud-Carrier
A novel modular neural architecture for rule-based and
similarity-based reasoning

9:10
Nam Seog Park
Addressing knowledge representation issues in connectionist
symbolic rule encoding for general inference

9:30
Nelson A. Hallack and Gerson Zaverucha and Valmir C. Barbosa
Towards a hybrid model of first order theory refinement

9:50
Panel on "the issues of representation in hybrid models"
Chair: Ron Sun
Panelists: Jerry Feldman, Lee Giles, Risto Miikkulainen, David Waltz

5 min opening statement by each panelist

The focus of the panel is the issue of representation:
how can neural representation contribute to the power
of hybrid models? how can symbolic representation supplement
neural represnetation? how each type of representations
can be developed, acquired, or learned? What are the
principled ways these two types of representation
can be combined, synergistically ?
 
 
 

Dec. 4 Afternoon Session:
Neural language processing, distributed representations
-------------------------------------------------------

4:00
***Marshall R. Mayberry, Risto Miikkulainen
SARDSRN: a neural network shift-reduce parser

4:30
***William C. Morris, Garrison W. Cottrell, Jeffrey L. Elman
The empirical acquisition of grammatical relations

5:00
Whitney Tabor
Context free grammar representation in neural networks

5:20
Curt Burgess, Kevin Lund
The transduction of symbolic environmental input into
high-dimensional distributed representations

5:40
Pentti Kanerva
Large patterns make great symbols: an example of learning
from example

6:00
Stephen I. Gallant
Context vectors: a step toward a "grand unified representation"

6:20
Paolo Frasconi, Marco Gori, Alessandro Sperduti
Integration of graphical-based rules with adaptive learning
of structured information

6:40
Stefan C. Kremer and John Kolen
Dynamical Recurrent Networks as Symbolic Processors
 
 
 
 

Dec. 5 Morning Session:
Neural and hybrid systems for cognitive processing
--------------------------------------------------

7:30
***David Waltz
The importance of importance

8:00
***Noel Sharkey, Tom Ziemke
Life, mind and robots: Biological inspirations and rooted cognition

8:30
G.K. Kraetzschmar, S. Sablatnoeg, S. Enderle, G. Palm
Using neurosymbolic integration in modelling robot environments:
a preliminary report

8:50
Timo Honkela
Self-organizing maps in symbol processing

9:10
Ronan Reilly
Evolution of symbolisation: Signposts to a bridge between
connectionist and symbolic systems

9:30
Christos Orovas, James Austin
A cellular neural associative array for symbolic vision

9:50
Panel on "hybrid and neural systems for the future"
Chair: Stefan Wermter
Panelists: Jim Austin, Joachim Diederich, Lee Giles, Noel Sharkey,
Hava Siegelman

5 min opening statement by each panelist

The focus of the panel is the impact of hybrid and neural
techniques in the future. How can we develop neural and
hybrid systems for new media? internet communication?
multimedia, web searching, data mining, neurocontrol for
robotics, integrating image/speech/language. What are the
strengths and weaknesses of hybrid neural techniques for
these tasks. Are current principles and methodologies in
neural and hybrid systems useful? How can they be
extende? What will be the impact of hybrid and neural
techniques in the future?
 
 
 

Dec. 5 Afternoon session:
Explanation and composition
---------------------------

4:00
***H. Lipson, H. T. Siegelmann
High order shape neurons for data structure decomposition

4:30
***Alan Tickle, Frederic Maire, Joachim Diederich
Extracting the knowledge embedded within trained artificial
networks: defining the agenda

5:00
Guido Bologna
Symbolic rule extraction form the DIMLP neural network

5:20
Gerhard Paass, Joerg Kindermann
Explaining bayesian ensemble classifier models

5:40
Peter Tino, Georg Dorffner, Christian Schittenkopf
Understanding state space organization in recurrent neural
networks with iterative function systems dynamics

6:00
 M.L. Vaughn, S.J. Cavill, S.J. Taylor, M.A. Foy, A.J.B. Fogg
Direct knowledge extraction and interpretation from a
multilayer perceptron network that performs low back pain
classification

6:20
James A. Hammerton, Barry L. Kalman
Holistic computation and the sequential RAAM: an evaluation

6:40
Stefan Wermter
Conclusion
 
 
 

More information on the NIPS98 conference can be found at:
http://www.cs.cmu.edu/Groups/NIPS/index.html

If you want to participate and submit please contact:
 

NIPS Workshop Contact
---------------------

********************************************
Professor Stefan Wermter
Research Chair in Intelligent Systems
University of Sunderland
School of Computing & Information Systems
St Peters Way
Sunderland SR6 0DD
United Kingdom

phone: +44 191 515 3279
fax:   +44 191 515 2781
email: stefan.wermter@sunderland.ac.uk
http://osiris.sunderland.ac.uk/~cs0stw/
********************************************