Hybrid Connectionist Models --- Professor Ron Sun

Connectionist Reasoning and Knowledge Representation

Hybrid Systems Resources

Hybrid Reinforcement Learning

Publications:

R. Sun and X. Zhang, Accounting for a variety of reasoning data within a cognitive architecture. Journal of Experimental and Theoretical Artificial Intelligence, Vol.18, No.2, pp.169-191. 2006.
A. Browne and R. Sun, Connectionist inference models. Neural Networks, Vol.14, No.10, pp.1331-1355, December 2001. [Elsevier-formatted PDF]
A. Browne and R. Sun, Connectionist variable binding. Expert Systems, Vol.16, No.3, pp.189-207. 1999.
R. Sun, Commonsense reasoning with rules, cases, and connectionist models: A paradigmatic comparison. Fuzzy Sets and Systems, Vol.82, pp.187-200, 1996. [Elsevier-formatted PDF]
R. Sun, Hybrid connectionist models. AI Magazine. 17 (2), pp.99-103, Summer 1996.
R. Sun, Robust reasoning: integrating rule-based and similarity-based reasoning. Artifical Intelligence (AIJ). Vol.75, No.2, pp.241-296. June, 1995. [Elsevier-formatted PDF]
R. Sun, A new approach towards modeling causality in commonsense reasoning. International Journal of Intelligent Systems, Vol. 10, No. 3. March, 1995. [formatted PDF ]
R. Sun, Structuring knowledge in vague domains. IEEE Transactions on Knowledge and Data Engineering, Vol. 7, No. 1. pp. 120-136. Feb., 1995.
R. Sun, On schemas, logics, and neural assemblies. Applied Intelligence, Vol. 5, No. 2. pp. 83-102. 1995 (an invited paper for the special issue on high-level connectionist models). [formatted PDF ]
R. Sun, A neural network model of causality. IEEE Transactions on Neural Networks, Vol. 5, No. 4. pp. 604-611. July, 1994. [formatted PDF ]
R. Sun, An efficient feature-based connectionist inheritance scheme. IEEE Transactions on System, Man, and Cybernetics, Vol. 23, No. 1. pp. 23-54. 1993.
R. Sun and L. Bookman, "How do symbols and networks fit together?" Artificial Intelligence magazine. pp. 20-23. Summer, 1993.
R. Sun, On variable binding in connectionist networks. Connection Science, Vol. 4, No. 2. pp. 93-124. 1992. [formatted PDF ]
R. Sun, Beyond associative memories: Logics and variables in connectionist networks. Information Sciences, Special Issue on AI and Neural Networks, Vol. 70, No. 1&2. 1992.
R. Sun, " A connectionist model for commonsense reasoning incorporating rules and similarities." Knowledge Acquisition, Vol. 4. pp. 293-321. 1992.
R. Sun, "Connectionist models of rule-based reasoning." AISB Quarterly, Special Issue on Hybrid Systems, No. 79. pp. 21-24. 1992.

S. Wermter and R. Sun, (eds.) Hybrid Neural Systems. Springer-Verlag, Heidelberg. 2000.
R. Sun and F. Alexandre, (eds.) Connectionist-Symbolic Integration. Lawrence Erlbaum Associates. 1997.
R. Sun, Integrating Rules and Connectionism for Robust Commonsense Reasoning (a monograph). John Wiley and Sons, New York, NY. 1994.
R. Sun & L. Bookman, (eds.), Computational Architectures Integrating Neural and Symbolic Processes. Kluwer Academic Publishers. 1994.

R. Sun, Hybrid systems and connectionist implementationalism. Encyclopedia of Cognitive Science, MacMillan Publishing Company, 2001.
S. Wermter and R. Sun, An overview of hybrid neural systems. (PS) (PDF) In: S. Wermter and R. Sun, (eds.) Hybrid Neural Systems. Springer-Verlag, Heidelberg. 2000.
R. Sun, "Connectionist models of reasoning." In: O. Omidvar and C. Wilson (ed.), Progress in Neural Networks, Vol. 5, Chapter 5. Ablex Publishing, Norwood, NJ. 1997.
R. Sun, Introduction to Connectionist Symbolic Integration. In: R. Sun and F. Alexandre, (eds.) Connectionist-Symbolic Integration. Lawrence Erlbaum Associates. 1997.
R. Sun, On neural networks and symbolic processing. In: R. Sun and L. Bookman, (eds.) Computational Architectures Integrating Neural and Symbolic Processes. Kluwere Academic Publishers. 1994.
R. Sun, "Variables and logics in connectionist models." in V. Honavar and L. Uhr, (eds.) Artificial Intelligence and Neural Networks: Steps towards Principled Integration, Vol. 1. Academic Press, Reading, MA. 1994.
R. Sun, "A two-level hybrid architecture for commonsense reasoning." In: R. Sun and L. Bookman, (eds.) Computational Architectures Integrating Neural and Symbolic Processes. Kluwer Academic Publishers. 1994.
R. Sun, "Connectionist models of commonsense reasoning." in D. Levine et al (eds.), Neural Networks for High Level Knowledge Representation and Inference. pp 241-268. Lawrence Erlbaum Associates. Hillsdale, NJ. 1993.
R. Sun and D. Waltz, "A neurally inspired massively parallel model of rule based reasoning." In: B. Soucek (ed.) Neural and Intelligent Systems Integration. John Wiley and Sons, New York, NY. pp. 341-381. 1992.
R. Sun, "The discrete neuronal model and the probablistic discrete neuronal model." In: B. Soucek (ed.) Neural and Intelligent Systems Integration, John Wiley and Sons, New York, NY. pp. 161-178. 1991.

R. Sun, Beyond simple rule extraction: the extraction of planning knowledge from reinforcement learners. Proceedings of the International Joint Conference on Neural Networks, Como, Italy. July 24-27, 2000. IEEE Press, Piscataway, NJ.
R. Sun, Supplementing neural reinforcement learning with symbolic methods: possibilitiesand challenges. Proceedings of International Joint Conference on Neural Networks, Washington, DC. July 10-15, 1999. IEEE Press, Piscataway, NJ.
R. Sun and C. Sessions, Extracting plans from reinforcement learners. Proceedings of the 1998 International Symposium on Intelligent Data Engineering and Learning, October, 1998. Springer-Verlag.
R. Sun and C. Sessions, Learning to plan probabilistically from neural networks. Proceedings of IEEE International Conference on Neural Networks, Anchorage, Alaska. May 4-9, 1998. IEEE Press, Piscataway, NJ.
R. Sun, A microfeature-based approach toward metaphor interpretation. Proceedings of the International Joint Conference on Artifical Intelligence (IJCAI-95). 1995.
R. Sun, "Connectionist models of rule-based reasoning." Proceedings of the 13th Cognitive Science Conference, Larence Erlbaum Associates, Hillsdale, NJ. pp. 437-442. 1991 (received the 1991 David Marr Award in Cognitive Science).

R. Sun, "A two-level architecture for structuring knowledge for commonsense reasoning." Proceedings of the IEEE International Conference on Neural Networks. Orlando, FL. 1994.
R. Sun, "The CONSYDERR architecture." Proceedings of the International Conference on Fuzzy Logic, Neural Networks and Soft Computing. pp. 153-155. Iizuka, Japan. 1994
R. Sun, "Implementing schemas and logics in connectionist models." Proceedings of the 1st International Symposium on Integrating Knowledge and Neural Heuristics. pp. 32-39. Pensacola Beach, FL. 1994.
R. Sun, "Similarity in cognition: a review of similarity and analogical reasoning." Artificial Intelligence magazine, Vol. 14, No. 4. pp. 81-84. Fall, 1993.
L. Bookman and R. Sun, "Integrating neural and symbolic processes (an editorial)." Connection Science, special issue on integrating neural and symbolic processes, Vol. 5, No. 3-4. 1993.
R. Sun, "On neural networks and symbolic processing." Proceedings of the 1st New Zealand International Conference on Neural Networks and Expert Systems. pp 5-7. ACM Press, New York, NY. 1993.
R. Sun, "Neural schemas and connectionist logics: a synthesis of the symbolic and the subsymbolic." Proceedings of the Workshop on Schema Theory and Neural Networks." Center for Neural Engineering, Los Angeles. 1993.
R. Sun, "Fuzzy evidential logic: a model of causality for commonsense reasoning." Proceedings of the 14th Cognitive Science Society Conference, Lawrence Erlbaum Associates. Hillsdale, NJ. pp. 1134-1139. 1992.
R. Sun, "An efficient connectionist inheritance scheme." Proceedings of the 2nd Pacific Rim International Conference on Artificial Intelligence, Seoul, Korea. 1992.
R. Sun, L. Bookman, and S. Shekhar, (eds.), The Working Notes of the AAAI Workshop on Integrating Neural and Symbolic Processes." American Association for Artificial Intelligence, Menlo Park, CA. 1992.
R. Sun, "Chunking and connectionism." Neural Network Review, Vol. 4, No. 2. pp. 76-78. 1991.
R. Sun, "Neural network models of reasoning." Proceedings of International Joint Conference on Neural Networks, Singapore. November 1991.
R. Sun and D. Waltz, "Neural networks and human intelligence: A review of brain and neural modeling." Journal of Mathematical Physchology, Vol. 34, No. 4. pp. 483-488. 1990.
R. Sun, "The discrete neuronal model and the probabilistic discrete neuronal model." Proceedings of International Neural Network Conference (Paris 1990). pp. 902-907. Kluwer, Netherlands. 1990.
R. Sun, "A discrete neural network model for conceptual representation and reasoning." Proceedings of the 11th Cognitive Science Society Conference. pp. 916-923. Lawrence Erlbaum Associates, Hillsdale, NJ. 1989.

Descriptions:

For the past several years, my research was mainly concerned with everyday commonsense reasoning by agents. This type of reasoning was characterized by a mixture of rule-based and similarity-based processes, exhibiting both rigor and flexibility (as demonstrated in my AIJ paper). To capture such reasoning, I developed a hybrid connectionist architecture (named CONSYDERR) with both localist and distributed components, that unified rule-based and similarity-based processes and accounted for a variety of CSR patterns.

Within the framework, the following issues were also investigated: (1) The connectionist implementations of rules, logics, and schemas, and the variable binding problem in such implementations. They formed the basis for complex reasoning in connectionist models. (2) Inheritance reasoning, which is an integral part of many CSR patterns. Within CONSYDERR, an intensional approach was developed that works in constant time. This work suggests that other similar reasoning patterns may also be handled intensionally. (3) Causality, which is an important commonsense construct. A connectionist account was developed based on CONSYDERR, which extended the existing logic-based account and dealt better with the inexact, cumulative, and subjective nature of commonsense causal reasoning.

Some attempts have also been made to extend the framework to deal with metaphor and analogy. Further work will be done to refine the architecture and to account for human CSR quantitatively.

Go to Prof. Ron Sun's Homepage