A Synaptic Logic Neuron Model

Henry Okora Okoyo

Abstract

Synaptic logic, a theory of quasi-logical integration of neuronal inputs, is advanced as the underlying principle of a novel high-performance constructive incremental-learning neural network architecture. Three new concepts are presented, namely: the synaptic logic neuron (SLN) model; the α-SLNN network architecture; and the design of analogue VLSI sub-SLN activation function microcircuits.

The formulation of the SLN model is motivated by the need for a neural network architecture that is easy to implement as an analogue VLSI microcircuit, and which can resolve an arbitrarily complex feature representation problem using a single layer of adaptive neurons. The model features a hierarchically structured, constructive and regular neuronal architecture. It encodes features by means of super-synaptic structures that implement sigmoid function based logic. The model has a simple analogue electrical circuit representation.

The α-SLNN architecture is a competitive supervised-learning pattern classifying SLN-based network architecture. It supports a network adaptation process that is based on the growth and the parametric optimization of feature encoding units in the SLN cells. The growth of feature encoding units is governed by a competition-driven, error-modulated, and parametrically adjustable constraint. The feature encoding unit optimization rules are local to the SLN cells. A simple pruning scheme is defined for removing superfluous feature encoding units.

Results are presented on the evaluation of the α-SLNN architecture and the NLC-IC2 chip - a 1μm CMOS integrated circuit chip that as designed to support the synaptic logic thesis. The α-SLNN architecture was evaluated on four publicly available neural network benchmarks: the two spirals benchmark, the speaker-independent vowels benchmark, the Landsat image segmentation benchmark, and the letter image recognition benchmark. A fifth benchmark was prepared by corrupting the two spirals benchmark with noise. The performance of the α-SLNN architecture matches, and in some cases or respects, betters the state-of-the-art best neural network performances that have been published for these benchmarks. The results of the evaluation of the NLC-IC2 chip show that synaptic logic activation function microcircuits work in silicon.