Proceedings of Sixth Italian Workshop on Parallel Architectures and Neural Networks.

Vietri sul Mare, Italy, May 12-14, 1993,, World Scientific (E. Caianiello, ed.)

CELLULAR NEURAL NETWORKS: A REVIEW

VALERIO CIMAGALLI and MARCO BALSI

Department of Electronic Engineering, "La Sapienza" University of Rome
via Eudossiana, 18, I-00184, Rome, Italy
tel.: +36-6-44585836 fax: +39-6-4742647 E-mail: [email protected]

ABSTRACT
A unified review of the Cellular Neural Network paradigm is attempted. First of all,
general theoretical framework is stated, followed by description of particular models
proposed in literature and comparison with other Neural Network and parallel computing
paradigms. Theory of such systems, especially the issue of stability, is then resumed by
listing main results available. Applications, design and learning follow. The paper is
concluded by description of proposed and tested hardware realizations.

1. Cellular Neural Networks: spatially defined parallel analog computing for local
and diffusion-solvable problems

Problems defined in space-time, e.g. image processing tasks, partial differential

equations (PDE) systems, and so on, are often characterized by the fact that the
information necessary to solve for the future or steady state of the system at a certain
point is contained (from the start, or from a certain time on) within a finite distance of the
same point. Therefore, these problems are solved by a relaxation and information diffusion
process, which develops at the same time at all points of space domain.
Cellular Neural Network (CNN) is an analog parallel computing paradigm defined in
space, and characterized by locality of connections between processing elements (cells, or
neurons). Such systems are best suited for local and diffusion-solvable problems such as
those considered above.
Two examples may help to give a first glance at CNN operation.
The first problem taken into consideration is halftoning of a grey-scale image. This
kind of processing, used in newspaper photographs, Xerox and fax machines, is used to
convert a continuously shaded image into one made of black dots on white background,
that, when filtered by the eye (spatial low-pass) gives the same impression as the original
image.
When a point of the image is considered, it is apparent that decision whether it
should be black or white depends not only on the grey level of the original in the same
point, but also on neighboring points grey level, and on decisions made for neighboring
points. Therefore, parallel processing is best suited for this problem; however, sequential

1
filtering is generally applied on the scanned image, thereby introducing artefacts in the
halftoned image. A CNN-halftoning system, developed by Crounse, Roska and Chua1,
might do the same job in parallel fashion at the speed of analog circuitry, with greater
accuracy.
An example of halftoning is given in figure 1.
A second example problem,
employing wider diffusion of information,
is connected component detection (CCD),
which can be used as a preprocessing for
pattern recognition. It consists of counting
the number of connected components This figure is missing.
found by scanning an image along a given It was pasted in the original
direction. An example of CCD obtained
by a CNN of an image along three
different directions is shown in figure 2.
This operation may be obtained by making
the image drift towards a border, Figure 1 An example of halftoning
squashing it to 1-pixel width while
preserving 1-pixel wide separation between disconnected parts. This can be done by only
using local information at every instant of time. Evolution of CCD processing of a 1-
dimensional image is depicted in figure 3.

TIME

Figure 3 Time evolution of CCD on a 1-

dimensional image. Grey squares represent
Figure 2 CCD result along three directions non-saturated-state pixels.

The main difference between CNNs and other Neural Network (NN) paradigms is
the fact that information is only exchanged between neighboring neurons. This
characteristic does not prevent the capability of obtaining global processing, as the CCD
example shows. By exploiting locality of connections, electronic IC and optical or electro-
optical implementations become feasible, even for large nets, which is the main advantage
of CNNs over NNs.
In the following section, a general definition of CNN is given, which is particularized
to the different CNN models found in literature. Section 3 discusses stability of CNNs, and
in section 4 a review of applications is given. The question of design and learning is then

2
confronted in section 5; hardware implementations follow in section 6. In the appendix
tables of symbols and acronyms are written for convenient reference.

2. Defining Cellular Neural Networks

2.1 Level 1 (system architecture)

2.1.1 Definitions
A Cellular Neural Network is a system of cells (or neurons) defined on a normed
space (cell grid), which is a discrete subset of ℜ n (generally n≤3), with distance
function d: = ℑ (ℑ
ℑ is the set of integer numbers). Cells are identified by indices defined
in a set One or several neighborhood functions N are defined as follows:
Nr: → α
Nr(i)={j|d(i,j)≤r} (1)
where α depends on r (neighborhood
size) and on space geometry (e.g.
α=(r+1)2 on square 2-dimensional grid).
Figure 4 shows examples of 2-dimensional
CNNs and neighborhoods.
Cells are multiple input - single
output nonlinear processors all described
by one, or one among several different,
parametric functionals. A cell is
characterized by a state variable, that is Figure 4 Examples of neighborhood of size r=1
generally not observable as such outside in CNNs defined on a square or hexagonal grid
the cell itself. Every cell is only connected ⊂ℜ
ℜ 2. Grey cells belong to the neighborhood
to cells within a neighborhood of itself. of black cells.

2.1.2 Comments
When considered as a system, a CNN is a Neural Network characterized by the fact
that connections are only allowed between neighboring neurons. The notion of distance
implies that the network is intrinsically defined in space; generally only 1-, 2- or 3-
dimensional space is considered, so that the CNN can be directly mapped into a physical
realization scheme, that can profit of a dramatically simplified connection layout.
The cell grid can be e.g. a planar array (with rectangular, triangular, hexagonal
geometry), a torus, a 3-dimensional array, generally considered and realized as a stack of
2-dimensional arrays (layers).
Cells may be all identical, or they can belong to a few different types (as is the case
for biological neurons), and more than one connection network may be present, with
different neighborhood size (short range interactions and subsystem connections). It is

3
obvious that, if the neighborhood size were as large as the network itself, we might obtain
a fully connected network. It is understood that we shall not call such a net "cellular", and
that generally the neighborhood shall have small size.
Cells may be very simple, or (moderately) complex. The "moderately" limit might be
described by the fact that CNN dynamics must basically depend on information flow in the
net, rather than on the operation inside the cells, so that we shall e.g. exclude that a
parallel digital computer be a CNN.

2.2 Level 2 (system operation)

2.2.1 Definitions
The CNN is a dynamical system operating in continuous or discrete time. A general
form of the cell dynamical equations may be stated as follows:

(continuous time)
∆ t x j (t ) = g x j (t ) + ∑ F AI
µ j H x j = t − τ, t , yk = t − τ,t ; p j K +
k ∈Nr ( j )

+ ∑ F BI
ν j H x j = t − τ, t , uk = t − τ, t ; p j K + I j ( t )
k ∈Nr ( j )

F I
y j (t ) = xj (2a)
H = t − τ ,t K

(discrete time)
x j ( n + 1) = g x j ( n ) + ∑ Aµ j F x j
H n − m, n
, yk n − m,n ; p jA I +
K
k ∈Nr ( j )

+ ∑ Bν j F x j
H n − m, n
, uk n − m,n ; p B
j
I
K
+ I j (n)
k ∈Nr ( j )

y j ( n) = f F x j I
(2b)
H n − m, n K

In Eqs.2, x,y,u,I denote respectively cell state, output, input, and bias; j and k are
cell indices; g is a local instantaneous feedback function, N is neighborhood function (if
more than one neighborhood is defined, several similar sums are present), pA and pB are
arrays of parameters, notation z|T denotes the restriction of function z(•) to interval T of
its argument (i.e. the set of all its values). In Eqs.2a, t is time, ∆t is a differential operator
(e.g. d/dt), τ is memory duration time, is (one out of several possible, identified by index
µj) neighborhood feedback functional, and in the same way is input functional, is
output functional; in Eqs.2b, A and B are the analogous functions for discrete time, f is
output function, n is time and m memory duration.

4
 Eqs.2 are written so as to be straightforwardly particularized. Therefore, they
involve some general notations that are not strictly necessary. E.g., as feedback, input and
output operators are functionals, it would be sufficient to substitute d/dt for ∆t without
loss of generality, but in this way it is more evident that higher order dynamics is also
allowed.
2.2.2 Comments
Eqs.2 specify at an operational level the way of functioning of cells and connections
(the latter are generally considered just as wires, lumping into the cell any dynamical
behavior of theirs, e.g. transmission delays). Defined as such, cells can be characterized by
a functional block diagram that is typical of neural network theory: figure 5 depicts a
three-stage functional block scheme of a cell, composed of a generalized weighted sum (in
general nonlinear, with memory), (leaky n-th order) integration, output nonlinear
function/functional. It is apparent that this scheme is more general, so that what mostly
distinguishes CNNs from other Neural Networks is the fact that all summations are
performed within a neighborhood.

y ∆ tx x
u y

Figure 5 Functional block scheme of a CNN cell. Symbols are just reminders of more general operation.

Data can be fed to the CNN through two different ports: initial conditions of the
state and proper input u. Bias values I may be used as a third port.

2.3 Level 3 (special cases)

Continuous time models (CT-CNN)

2.3.1 Chua & Yang's CNN (CY-CNN)

Chua & Yang's CNN2 is a first order system, with linear instantaneous connections
and piece-wise linear (PWL) output function (figure 6); Eqs. 2a specialize in this case as
follows:

dx j (t )
dt
= − x j (t ) + ∑ A jk yk (t ) + ∑ B jk uk (t ) + I j
k ∈Nr ( j ) k ∈Nr ( j )
1
y j (t ) = b x(t ) + 1 − x (t ) − 1 g (3)
2

5
 y

Figure 6 Piece-wise linear output function of CY-CNN

2.3.2 Linear Cloning Template CNN (LCT-CNN)

A particular case of Chua & Yang's CNN is obtained by making parameter values
space-invariant2. Without loss of generality, we may consider a square 2-dimensional grid
and use double indices to indicate position in the grid. In this case Eqs. 3 can be written
for the cloning template CNN as:

dxij (t )
dt
= − xij (t ) + ∑ A( i − k )( j − l ) (t ) ykl + ∑ B(i − k )( j − l ) (t )ukl + I
kl ∈Nr ( ij ) kl∈Nr ( ij )
1
yij (t ) = e xij ( t ) + 1 − xij ( t ) − 1 j (4)
2

This means that A, B and I values can be determined by cloning template matrices
that are identically repeated in the neighborhood of every neuron:

L A−1;−1 A−1;0 A−1;1 O L B−1; −1 B−1;0 B−1;1 O

M P M P
A= A
M 0;−1
A0;0 A0;1 P B= B
M 0; −1
B0;0 B0;1 P I (5)
M A A1;0 A1;1 PQ M B B1;0 B1;1 PQ
N 1; −1 N 1;−1

Such CNNs operate a kind of nonlinear convolution with the template.

LCT-CNN is the most widely studied CNN model (see e.g. bibliography of ref. 3),
mainly for image processing applications. Uniformity of the network allows for easier
design and realization.

2.3.3 Full-Range CNN (FR-CNN)

As will be shown in section 3 (theorem 1) CY-CNN state values are bounded by a
limit that may become rather large, which is inconvenient for realization purposes. For this
reason, Rodríguez-Vázquez et al.4 proposed, with respect to CY-CNN, to move the
nonlinear limiting from the output into state dynamics, therefore writing cell equation as
follows:

dx j (t )
dt
= g x j (t ) + ∑ A jk x k (t ) + ∑ B jk uk (t ) + I j
k ∈Nr ( j ) k ∈Nr ( j )

6
 R− m( x + 1) + 1 x < −1
|
g( x ) = lim S − x x <1 (6)
m →∞
| − m ( x − 1) − 1 x >1
T

In this way, state range is confined in the

g(x)
same range as input and output values,
but behavior of the network is not
qualitatively changed. The FR-CNN is
very similar to a Brain-State-in-a-Box x
(BSB5) network, with only local
connections.
In a similar way, a discrete-time FR-
CNN may be defined. Figure 7 Local feedback function of the FR-CNN

2.3.4 Nonlinear and Delay-Type CNN (NL-CNN/D-CNN)

An extension of CY-CNN is obtained by allowing nonlinear and delayed
connections, so that the following terms may be added to dynamical Eqs.3:

∑ 
Aik d y j (t ), yk (t )i + ∑ 
Bik du j (t ), uk (t )i (nonlinear connections) (7)
k ∈Nr ( j ) k ∈Nr ( j )

∑ Aikτ yk (t − τ) + ∑ Bikτ uk (t − τ ) (delayed linear connections) (8)

k ∈Nr ( j ) k ∈Nr ( j )

Nonlinear and delay-type terms may be made space-invariant, therefore defining

nonlinear and delay-type cloning templates (NLCT/DCT-CNN6); more than one delayed
term may be present at once, and delayed nonlinear connections may be also defined. NL-
and D-CNNs have a wider scope of application than CY-CNNs, nonlinearity may allow
e.g. for amplitude-selective connections; delay, besides modelling actual hardware delays,
may be used for temporal processing such as motion detection.

2.3.5 Linear Threshold/Feed-Forward CNN (LT/FF-CNN)

Consider a 3-dimensional square grid CY-CNN written as follows, where we
consider the lower two indices as indices within a (square grid) layer (which is a LCT-
CNN), identified by the upper index, and where inputs are only applied to first layer:

7
dxijm (t )
dt
= − xij (t ) + ∑ A(mn n
i − k )( j − l ) ( t ) ykl + ∑ B(i − k )( j −l ) (t )ukl + I m
kl ∈Nr ( ij ) kl∈Nr ( ij )
1 m
yijm (t ) = m
e xij ( t ) + 1 − xij ( t ) − 1 j (9)
2

A layer is said to belong to the linear threshold class7 if and only if

Aijmm = 0 ∀(i, j ) ≠ ( 0, 0 ) (10)

i.e. if no connections are present between cells of the same layer. The dynamics of cells of
a linear threshold layer depend only upon their own states, the states of neighboring layers,
and external inputs. Nossek, Seiler, Roska and Chua8, add the notion of margin to the
definition of LT-CNN, by imposing on local feedback weight condition A mm jj = 1 + µ j ,
where µj>0 is called margin.
A multilayer CNN is feed-forward7 if and only if Amn=0 for all n>m i.e., if intra-
layer connections are only present from lower to higher index layer (no backward
connections).
If all layers of a feed-forward CNN belong to the linear threshold class, and
mm
A00 = 1 for all layers m>1, the steady state of each layer above the first depends only
upon the input and its initial conditions. Such LT/FF-CNNs resemble a Multi-Layer
Perceptron (MLP) where only local connections exist. In fact, as no feedback loop is
present, cell states evolve monotonically to their steady value, obtained as a weighted sum
of their inputs, just as a physical realization of a MLP would do when transmission delays
and parasitic capacitance are taken into account.

2.3.6 Polynomial CNN (P-CNN)

A polynomial CNN9 is defined by using an odd-degree polynomial as local feedback
function g. General shapes of third and fifth degree polynomials are depicted in figure 8.
Connections are nonlinear (step functions), chosen in a small set of possibilities in order to
shape attractors for a pattern recognition device by synthesis. In fact, these functions allow
directed flow of activation from cells corresponding to recognizable patterns towards cells
corresponding to model (stored) patterns.

g(x) g(x)
x
x

(a) (b)
Figure 8 Polynomial local feedback functions of degree 3 and 5 for the P-CNN

8
 General equations for the P-CNN may be written as follows:

dx j (t )
dt
= gd x j (t ) i + ∑ Aµ jk yb xk (t )g + u j (t )
k ∈Nr ( j )
y j (t ) = x j (t ) (11)

Upon proper scaling of variables, dynamics of a P-CNN cell with third order local
feedback is qualitatively analogous to that of CY-CNN cell (see ref. 2). Higher order
polynomials raise the number of possible distinct equilibria of the cell.

2.3.7 Non Uniform Processor/Multiple Neighborhood Size CNN (NUP/MNS-CNN)

Motivated by neurophysiological evidence about the structure of brain cortex, a
three-layer CNN memory architecture was introduced by Henseler and Braspenning10.
This CNN has cells with different dynamics in the three layers, and neighborhoods have
different sizes too, as shown in figure 9.

Figure 9 Henseler & Braspenning's NUP/MNS-CNN; black cells

belong to the neighborhood of big black cells

Different cells allow for different processing in the three layers, while short- and
long-range interactions make the CNN operate as a network of subsystems.
Other schemes of NUP/MNS CNNs were considered by Roska and Chua6. NUP-
CNNs with two processor types may look as in figure 10, where black cells may be hidden
processors (as proposed by Tan, Hao and Vandewalle11, in order to increase storage
capacity), and/or have different dynamics. Figure 11 shows an example of MNS-CNN,
where fine and coarse grids are present.

9
 Figure 10 Examples of NUP-CNNs
Figure 11 A MNS-CNN

2.3.8 Moving Object Detecting

architecture (MODA)

MODA12 (Cimagalli, Bobbi and

Balsi, 1993) is a NUP-CNN with complex
cells and periodic multiple cloning
template structure. In fact, it is defined on
a square grid with a 3×3 repeating pattern,
as shown in figure 12. Black cells are
called "central", white cells are "off-
center". Off-center cells have two inputs,
coming form two neighboring pixels of an Figure 12 Cells are represented by white and
black boxes. Connections towards a 9-neuron
image; their equations write as follows:
cluster are shown.

dx j (t )
dt
= − x j (t ) + ∑ Aν j yk + Bdu1 j (t − τ ), u2 j (t )i + I
k ∈Nµ j ( j )

y j (t ) = γσ x (t ) (12)

where:

σ( x ) = 0 if x < 0 else σ ( x ) = 1 (unit step)

B( x , y ) = σlσa x − ε f + σ (α − 1) x + y + σ (α + 1) x + y + σ( y − ε ) − βq (13)

α,β,γ and ε are suitable parameters. Feedback coefficients A depend on cell location, as
can be deduced from figure 12.
Central cells have no dynamics: they just process their single input instantaneously:

y j = Bdu j (t − τ ), u j (t )i (14)

10
 With proper choice of parameters, MODA operates as a moving object detector.
Cell state is associated to an element of trajectory (between neighboring pixels); input
function detects permanence of a similar input in successive positions and times (spatio-
temporal coincidence), while first order dynamics acts as a short term memory.

2.3.9 Membrain, Relaxation Oscillator CNN (RO-CNN)

Membrain13 is a CNN with second order dynamics defined on a 2-torus. This means
that borders of the net are connected to their opposites, as shown in figure 13.
For this network, Eqs. (2a) are particularized as follows:

d 2 x j (t ) dx j (t )
dt 2
+α
dt
= ∑ A jk yk (t ) + I j (t )
k ∈Nr ( j )
y j (t ) = x j (t ) (15)

Due to its toroidal structure,

Membrain can perform translation-
invariant pattern recognition. It solves a
discrete-space generalized wave equation,
where inputs, given only as initial states,
act as a perturbation setting up oscillation
of the system, which behaves similar to an
elastic membrane. It is obvious that the
same dynamics can be defined on a plane
grid; in this case the usual reflection Figure 13 Neighborhood of a corner
phenomena of wave theory appear at the pixel in a CNN defined on a 2-torus
borders.
Another model of oscillator network14 employs relaxation oscillators:

dx j (t )
dt
= ∑ Ajk yk (t ) + u j (t )
k ∈Nr ( j )

x j (t ) = f y j (t ) (16)

Output y is defined implicitly through function f, represented in figure 14, where y

can be taken to represent current flowing through a piece-wise linear current-controlled
negative resistor.
This structure exhibited multiple attractor periodic and chaotic behavior, that might
be exploited for Content-Addressable Memory (CAM) realization.

11
 x

y
Figure 14 Implicit output function of the RO-CNN

2.3.10 CNN with Local Logic (CNNL) - Analog Software Universal Machine
CNNs may be expanded3,15 by adding to every cell a few logical components,
thereby making it a parallel "analogic" (analog and logic) universal computing machine,
capable of executing analog software, i.e. applying several cloning templates in succession,
in order to achieve complex tasks. This kind of processing (dual computing), may be
regarded as the analog counterpart of Single-Instruction-Multiple-Data parallel computers.
To do this, a local (analog and) digital memory is added, and a simple control logic, itself
controlled by a global control unit.
A CNN of this type (analog software universal machine3) can be considered as a
time-varying cloning template architecture, and its dynamical equations written as follows
for linear templates:

dxij (t )
dt
= − xij (t ) + ∑ A( i − k )( j − l ) (t ) ykl (t ) + ∑ B( i − k )( j − l ) (t )ukl (t ) + I (t )
kl ∈Nr ( ij ) kl∈N r (ij )
1
yij (t ) = e xij ( t ) + 1 − xij (t ) − 1 j (17)
2

Local memory and logic may also be used to apply simple logical processing to
binary valued outputs, before feeding them back to CNN input16.

2.3.11 Relation with other continuous-time Neural Network models and systems
Among NN paradigms, the continuous Hopfield model (CH-NN17) most closely
resembles CNNs. In fact, it has substantially the same dynamics of CY-CNN, in the limit
of maximum size neighborhood:

dx j (t )
= − αx j + ∑ A jk yk + I j
dt k
yj = f dx j i (18)

where f is a sigmoid function, e.g.

12
 1
f ( x) = (19)
1 − e −βx

The fact that f is not piece-wise linear makes no significant difference in the overall
behavior of the network. Actually, a PWL function can be approximated to any precision
by a continuous function, and sometimes this substitution is done in CNNs because,
besides being more realistic from the point of view of realization, is also useful in
theoretical proofs.
The CH-NN is used as a nearest-neighbor encoding CAM, with the cue given as
initial state; therefore, no input is present.
Theory concerning CH-NNs can obviously be applied to CNNs too, but, as the
restriction of locality of connections endows CNN dynamics with peculiar properties, this
similarity is only seldom used, and most results of CNN theory have been obtained
independently.
It is also interesting to consider the similarity between CNNs and (physical) systems
described by systems of partial differential equations (PDE). In fact, these two kinds of
systems share the property that dynamic behavior only depends on local spatial
interactions2.
In fact, consider divergence (∇) and Laplace (∇ 2) vector differential operators,
which are basic building blocks of fundamental equations of physics (e.g. heat, Poisson,
Navier-Stokes equations ...); in two dimensional rectangular coordinates they are written
as:

∂ ∂
∇= +
∂x ∂y
∂2
2 ∂2
∇ = 2 + 2 (20)
∂x ∂y

If we approximate space derivatives numerically, e.g. by using incremental ratios, it

is apparent that by taking a regular discretization of space with steps hx and hy, the
following cloning template A matrices for LCT-CNN implement the same operators, as
long as all cells operate in the linear region of output function (obviously different
numerical derivation formulas may yield different neighborhood size and coefficients):

L 1 O
L 1 O 0 0
M P
M 0 0 P hy2
2 hy M P
M P
M 1 F 2I 1 P
M −1 1 P 2 (21)
A∇ = 1 A = M G1 − − J P
M 2h 2 hx P ∇2 2
hx2 hy2 K hx2 P
x M hx H
M P
−1 M 1 P
M 0 0 P
2 hy M 0 0 P
M
N P
Q hy2
MN PQ

13
 As a first-order system, CY-CNNs approximate a generalized heat equation; in the
same way, Membrain, a second-order system, implements a generalized wave equation.

Discrete time models

2.3.12 Discrete Time CNN (DT-CNN)

The discrete time version11,18 of CY-CNNs is described by the following equations:

x j ( n + 1) = ∑ A jk yk ( n) + ∑ B jk uk ( n ) + I j
k ∈Nr ( j ) k ∈Nr ( j )

y j ( n) = f x j ( n ) (22)
where f is the sign(•) function:

R1 if x ≥0
f ( x) = S (23)
T0 if x <0

or the usual PWL function as in Eqs. 319 .

DT-CNNs may be used with linear cloning templates18, as well as with nonlinear and
time-varying templates19.

2.3.13 Multi-Valued CNN (MV-CNN)

The multi-valued neuron for discrete time CNN was introduced by Aizemberg and
Aizemberg20. The state and output of such cells is defined in complex space, therefore
dynamical equations for the MV-CNN do not readily fit in the framework of Eqs. 2b.
However, they can be written in similar form:

x j ( n + 1) = ∑ Ak yk + I j
k ∈Nr ( j )
y j = csign( x j ) (24)

Function csign(• ) is defined as follows (i is the imaginary unit):

csign( z ) = e i⋅2 π⋅ j / k if 2 π ⋅ j k ≤ arg( z ) < 2 π ⋅ ( j + 1) k

csign(0 ) = 1 (25)

2.3.14 Relation with other discrete-time neural network models and systems

14
 As discussed for CT-CNNs, also DT-CNNs may be considered as restrictions of
fully connected NNs: Additive Grossberg NNs21 have positive weights, negative local
feedback and step nonlinearity, Discrete Autocorrelators22 also have step nonlinearity,
BSB has PWL nonlinearity, as for the FR-CNN. For these networks, however, the same
comment applies about the fact that theory has just limited interest in CNN context.
It is instead more interesting to compare LCT-(DT-)CNN(L) with two other parallel
computing paradigms, namely Systolic Arrays (SA) and Cellular Automata (CA). Table
123 compares the data type and processing specification for these models:
Table 1 Confronting CA, SA, and CNN
Cellular Automaton Systolic Array CNN
data logic values (1-4 bits) numerical values (8-32 bits) analog values
specification truth table numerical algorithm cloning templates
(analog software)

Analogies and differences between CA and CT-CNNs have been discussed and
exploited by Chua and Shi7.
Operation of the CA consists of two phases: a processing phase and a (local or
global) propagation phase. These two phases (with local propagation) are also present in
DT-CNN realizations18, and can also be replicated in CT-CNNs, even if the latter operate
asynchronously. In fact, Chua and Shi7 showed that a LT-CNN can implement one
iteration of a CA involving only local propagation, and that a LT/FF-CNN can also
implement global propagation operation.
Therefore, it is possible to exploit CA design rules in the design of CNNs, in order
to obtain the same functionality, with the advantage of easier realizability of the analog
CNN chip.
In CA theory, the game of life algorithm has particular importance, because of its
universality; Chua, Roska, Venetianer and Zaràndy19 have discovered LCT-CNN
templates for one step of the algorithm and a DT-CNN(L) analog algorithm to perform the
continuing game, thereby proving universality of CNNLs.

3. Stability

Main theoretical results concerning CNN dynamics are hereafter reviewed. For
proofs of theorems, please refer to quoted papers.

Theorem 1 (boundedness of state for CY/NL/D-CNNs)

All states x of a CY/NL/D-CNN are bounded for all time by

15
 xmax = 1 + max M
L
∑
j ∈ M k ∈N ( j )
e A jk + B jk + max 
A jk ( y1 , y2 ) +
y1 , y2 ∈ −1,1
N r

+ max 
B jk ( y1 , y2 ) + Aτjk + B τjk + I j J P
IO
(26)
y1 , y2 ∈ −1,1 KQ

provided that nonlinear connection functions and initial conditions are bounded by 1.
Moreover, ω-limit points of x ( x j a∞ f ) are bounded by xmax-1.2,6,24,25

Theorem 2 (stability of CY-CNNs with symmetric connections)

A CY-CNN with symmetric connections (i.e. A jk = Akj ∀j , k ∈ k ∈Nr ( j )) is
asymptotically stable2.

Theorem 3 (stability of positive cell-linking CY/NL-CNNs)

A NL-CNN satisfying the following condition (positive cell-linking - PCL -
condition), is asymptotically stable, except possibly for a set of initial conditions of zero
measure.


∂A jk d y j , yk i

a) ∀j , k ∈ k ∈ Nr ( j ): > 0 or A jk ≡ 0;
∂yk
b) ∀j , k ∈ there exists a path in li0 
= j , i1, ..., in = kq Aim im +1 ≠ 0 (27)

This is true in particular of PCL CY/LCT/NLCT-CNNs. For CY/LCT-CNNs,

condition a) means that weights must be non-negative6,26.

Theorem 4 (stability of opposite-sign LCT-CNNs)

A one-dimensional LCT-CNN with cloning template matrix A = (a-1 a0 a+1), with
a-1a+1<0, a0>1 is asymptotically stable if and only if |a-1|<a0-1 or |a+1|<a0-1 27,28.

Theorem 5 (stability of positive strictly sign-symmetric CY-CNNs)

A CNN is positive if A jk ≥ 0 ∀j , k ∈ j ≠ k ; it is strictly sign-symmetric if

whenever A jk ≠ 0: A jk Akj > 0 . A positive strictly sign-symmetric CNN is stable almost

everywhere29.

Theorem 6 (stability of acyclic CY-CNNs)

A CNN is acyclic if it has no cycles, i.e. if there exists no set of indices {i1,i2,...,in},
all distinct except in=i1, such that ∀j = 1, 2,..., n − 1: Ai j i j +1 ≠ 0 .

16
An acyclic CNN with A jj > 1 ∀j ∈ is asymptotically stable29.

Remark 1 (state space transformations extending stability results for CY-CNNs)

Consider a state space transformation of the form x'=Jx, with

k p 
J = diage( −1)n1 , ( −1)n2 , ..., ( −1)nk j and ni ∈ 0, 1 . Let A = ed A jk ij. If the transformed
 ′ = J −1AJ
CNN, with state x', and weights obtained from A  , is stable, so is the original
CNN. Chua and Wu29 specify possible different transformations for LCT-CNNs.

Theorem 7 (binary output property of CY-CNNs)

In a CY-CNN, if A jj > 1 ∀j ∈ , then any stable equilibrium state must have all
components with magnitude greater than 1, i.e. x j a∞ f > 1 ∀j ∈ 2.

Theorem 8 (global asymptotic stability of low-feedback CY-CNNs)

A CY-CNN is globally asymptotically stable (i.e. it has a single asymptotically stable
equilibrium point) if it is asymptotically stable and A jj < 2 ∀j ∈ 30.

Theorem 9 (global asymptotic stability of low-self-feedback CY-CNNs)

A CY-CNN is globally asymptotically stable if
1
A jj + ∑ e A jk + Akj j < 1 ∀j ∈ 31.
2 k ∈Nr ( j )

Theorem 10 (stability of D-CNNs)

  
In a D-CNN, let A = ed A jk ij, and A τ = ee A τjk jj . If: A is non-negative in the off-
  
diagonal locations; A τ has only non-negative elements; A + A τ is an irreducible matrix;
and the set of equilibrium points is finite, then the network is almost everywhere
asymptotically stable24.

Theorem 11 (stability of D-CNNs)

  
Let P= -1+ A + A τ (1 is the identity matrix). If A τ is invertible, P is symmetric, and
2
Aτ < (where • denotes the Euclidean norm), the corresponding D-CNN is
3τ
asymptotically stable32.

17
Theorem 12 (global asymptotic stability of D-CNNs)

If a D-CNN is such that: A has only non-negative off-diagonal elements, A τ has 
 
non-negative elements, and −e A + A τ j is row-sum dominant, i.e.

∑ − e A jk + Aτjk j ≥ 0 ∀j ∈ , it is globally asymptotically stable25.

k ∈Nr ( j )

Theorem 13 (global asymptotic stability of NL/D-CNNs)

  
In a NL/D-CNN, let A nl b f1 a y fg = ee Ajk d y j , yk ijj. If A nl satisfies a Lipschitz

 
condition with constant L: A nl by1 g − A nl b y 2 g ≤ L y1 − y 2 ∀y1 , y 2 , and L + A τ < 1 
,
then the NL/D-CNN is globally asymptotically stable . 25

Theorem 14 (stability of eigendominant DT-CNNs)

In a DT-CNN, let b j = min ∑ Bjk uk + I j . If ∀j: A jj ≥ 0, and

u k ∈N ( j )
r

A jj + b j > ∑ A jk , the network converges in a maximum number of time steps

k ∈N r ( j )
which is equal to the number of cells in the network18.

Theorem 15 (stability of symmetric DT-CNNs)

A DT-CNN with symmetric feedback coefficients A jk = Akj either converges, or
oscillates with a period of two18.

4. Applications

Since their introduction, a wide range of applications has been proposed and tested
for CNNs. In this section, only a synthetic review of these applications can be given, for
reason of space; detailed description can obviously be found in quoted papers.

4.1 Applications of LCT-CNNs

LCT-CNNs have found most applications in image processing, also profiting of filter
theory (e.g. ref. 33, 34). Besides local processing, global effect are obtained by
information propagation, also by properly setting both input and initial state (e.g. CCD,
hole-filler). CNNs may be used as preprocessors for pattern recognition tasks, also in
connection with other kinds of NNs and with logical computers (e.g. feature detection,
character recognition). Table 2 summarizes the most significant contributions.

18
 Table 2 LCT-CNN applications
Application Reference
line, edge and corner extraction 33
noise removal (low-pass filtering) 33
connected component detection 35
hole filling 35
shadow detection 35
image thinning/thicking 35, 36
small object isolating (for counting) 37
halftoning and information compression 1,38
character recognition 35, 39, 40
Table 2 (continued)
printed circuit layout error detection 41
vehicle guiding 42
Radon transforming 7

4.2 Applications of NLCT-CNNs

NLCT-CNNs have been applied to image processing (grey-scale image contour
detection43); simulation of physical and biological systems (lattice equations, CNN-retina).
Table 3 lists more contributions.

Table 3 Applications of NLCT-CNNs

Application Reference
Grey-scale image contour extraction 44
image thicking 44
convex corner detection 45
simulation of lattice equations 46
CNN-retina 47
local Boolean function evaluation 48

4.3 Applications of other CT-CNNs

Delays have been exploited in order to detect moving parts of images, by

confronting subsequent snapshots (D-CNNs, MODA); analog software for CNNLs has
been developed for the solution of complex tasks, among them the game of life, which
proves universality of CNNL machines. Globally asymptotically stable CY-CNNs may be
applied to approximation of mappings between continuous or discrete vector spaces.
Other applications are listed in table 4.

19
 Table 4 Applications of other CT-CNNs
Application Reference
motion detection 12, 49
mapping approximation 31, 50
game of life 19
artificial vision (stereopsis) 51
object counting 19
translation-invariant pattern recognition 13
oscillators, chaos generators 52

4.4 Applications of discrete time CNNs

Several authors have discussed use of discrete time CNNs as CAM; concerning
image processing, DT-CNNs cloning templates have been found for many tasks, including
some new functionalities in which speed of processing can be usefully controlled. A list of
these applications is given in table 5.

Table 5 Applications of discrete time CNNs

Application Reference
content-addressable memory (CAM) 11, 53, 54, 55
CCD, concentric contouring, minimal distance testing,
18
image thinning and thicking
shadow detector 56

5. Design and learning

As stated in section 1, CNN dynamics is substantially governed by local interactions

between cells, that just perform a quite simple processing. Therefore, as is usual in NN
theory, CNN design involves as a basic step the choice of (generally a lot of) parameters
(connection weights). It is to be noticed that cloning template CNNs have a very peculiar
characteristic among NNs; in fact, they can be described by a small number of weights
(e.g. a square LCT-CNN with r=1 has 19 independent weights).
The choice of suitable parameters is configured as an optimization task, that can be
accomplished in two extreme ways53: (1) given a complete and precise formulation of the
problem to be solved, synthesis is performed by choosing systematically a nominal set of
optimal parameters; (2) given a vague description of the task through a set of examples,
learning is accomplished by presenting examples to the network, adjusting parameters
with a suitable algorithm, that converges to an (at least local) optimum set. Proper
learning involves generalization of functionality from a limited training set; however, we
shall consider as learning also pattern storage (i.e. all examples given) and design by
examples, where the task is also defined in terms of examples, but design is done by a
systematic (non-adaptive) direct choice of parameters.

20
 The first steps of CNN theory were done by obtaining useful LCT-CNN cloning
templates by a sort of empirical synthesis, based on experience in other fields (e.g.
filtering) and on trial-and-error. Most design theory for CNNs has been developed quite
recently, and has concentrated in particular to cloning template CNNs. A brief review of
the various methods proposed is given below.
Analog software, recently rigorously defined by Roska and Chua15, opens the way
to analog ("analogic") programming, which can be as an extension of CNN synthesis
beyond the hardware level.

5.1 Design by synthesis

Synthesis methods for LCT-CNNs have been proposed by Chua and Thiran57 and
Osuna and Moschytz58. Both algorithms involve building a system of inequalities
describing solution space, and solving them by a standard method (e.g. relaxation). The
first paper gives sufficient conditions on template parameters that guarantee correct
functioning; applying these conditions, however, is only feasible for rather simple problems
and small number of free parameters. The second method quoted, aimed at finding
symmetric templates satisfying the binary output condition (theorem 7) consists of a set of
inequalities to be imposed exhaustively on desired and forbidden input/final output
couples, which can be considered as restricted to neighborhood size. Slot59 distinguishes
two aspects of the feature detection problem: recognition of presence of the feature in
input pattern, and propagation of information about this recognition. Cloning template
matrices B and A are then designed so as to satisfy two sets of inequalities describing the
two mentioned tasks. Chua and Shi7 exploited analogy with CAs to design a LCT-CNN
implementing the Radon transform; Crounse, Roska and Chua38 designed halftoning
templates by adapting parameters of known filters.
Synthesis of CY-CNN weights was considered by Seiler, Schuler and Nossek54.
They gave a rigorous mathematical framework to the problem of obtaining robust solution
to pattern storage problems, by stating sufficient conditions on nominal and actual values
of parameters, written in terms of inequalities to be solved by a standard method, e.g. the
simplex algorithm. They also give practical guidelines to apply the theory in the design of a
CAM with given stable equilibria.
Realization of locally-defined Boolean functions by time-varying cloning template
DT-CNNs is solved by Galias49: simple rules are given to set the weights from a product-
of-sums or sum-of-products definition of the function.
For P-CNNs, used as CAM, an explicit synthesis procedure is given to choose
nonlinear connection functions to store given patterns9. MODA, a dedicated architecture
is also designed by explicit synthesis, implementing constraints of the problem12.

5.2 Learning

LCT-CNNs have much less parameters than general CNNs and NNs. Therefore,
learning for those networks has been accomplished with several optimization algorithms
that are generally impractical for other models.

21
 Zou, Schwarz and Nossek60 obtain straightforwardly from examples a system of
linear inequalities, to be solved by a relaxation method; the cost function is identified as
the sum of distances of the current solution from separation planes defined by these
inequalities. The same system is solved by Szolgay and Kozek61 by the simplex algorithm.
Schuler, Nachbar, Nossek and Chua62 taught a LCT-CNN state space trajectories by
using backpropagation-through-time algorithm with conjugate gradient descent. Cost
function is taken as an integral of distance between actual and desired solution over the
entire trajectory of state evolution.
A different approach is used by Kozek, Roska and Chua63, who employed a genetic
algorithm, which can be easily applied to complex tasks too, due to the simple way of
giving constraints and optimization criteria to be enforced.
CY-CNNs can be trained by recurrent back-propagation, which is a generalization of
the well-known back-propagation algorithm to non-feed-forward networks. Operation of
this algorithm is time-consuming when simulated, but it might be directly realized in
hardware30,31,64. Hansen65 employed the Boltzmann machine algorithm, maximizing a
posteriori probability of correct output by gradient decent.
Concerning CNN-CAMs, Tan, Hao and Vandewalle11 applied Hebb's rule and
computed capacity of the network. Mizutani56 also used Hebb-like storage. Aizemberg
and Aizemberg20 adapt weights for the MV-CNN in an a iterative manner, resembling a
gradient descent, which finds a solution in a finite number of steps.
Learning for DT-CNNs was considered by Harrer, Nossek and Zou57, who
employed a relaxation method, as described for CT-CNNs, and by Magnussen and
Nossek66. In the latter paper, a new algorithm called LASTNERD is defined, that employs
line searches in order to minimize a cost function obtained as mean square distance from
training examples. Training examples, and direction of line searches are chosen at random,
so as to introduce a noise component into the process, that enhances robustness and
reliability of the solution.

6. Hardware implementation

CY-CNN equations (3) are readily translated in a circuit scheme by interpreting cell
state as voltage across an RC dipole realizing first-order dynamics. Connections are
realized by voltage-controlled current sources (VCCS), so that summations are performed
by injecting currents into the same node.
Based on this scheme (figure 15a), several implementations were proposed
employing operational transconductance amplifiers (OTA) as basic blocks. Weighting of
signals is performed at the output instead of input of cells, together with nonlinear output
function. Therefore, the cell has multiple outputs and a single input instead of single
output and multiple inputs (figure 15b).

22
 u x y
+- ... ... f

Bu Ax
(a)

to neighbors from neighbors

u x y

+- ... ... f

Bu Ax

(b)

Figure 15 Alternative models of CNN cell: (a) Chua & Yang's model, with bias
represented as voltage instead of current; (b) dual model with weighting at the output

To do this, OTAs are used both in

their linear operation range and in
saturation, exploiting their transfer +
function, that approximates a PWL OTA R
from -
sigmoid very well. In this way, Cruz and neighbors
67
Chua designed a cell composed of three x
OTAs and two capacitors (figure 16). In
+
this circuit, Cx is state capacitor, Cu is Cx
OTA A to neighbors
loaded with cell input, OTA A implements
-
all current sources controlled by state
voltage, and PWL output function. OTAs
B and R work in their linear range; the u
+
first implements all current sources Cu
OTA B
controlled by input voltage, the latter -
works as state resistor (with voltage bias).
A 6×6 prototype was realized in 2µm Figure 16 Cruz & Chua's CNN cell
CMOS technology.
Each cell contains about 50 transistors and occupies an area of 31,000µm2 (32 cells per
mm2). Settling time of the circuit is of order 1µs.
Halonen, Porra, Roska and Chua16 realized in similar way a CNN containing local
digital memory and logic. Weights are programmable in a discrete set, and biases
continuously; logic operations may be performed on steady outputs, and they can be fed
back to cell input. In a 4×4 prototype, realized in 2µm CMOS technology, each cell
contains about 500 transistor and occupies 1mm2. Settling time is about 3µs.

23
 Nossek, Seiler, Roska and Chua8 propose a fully-programmable scheme based on
operational amplifiers (op amp) and variable conductance blocks. The inner cell circuit is
first transformed by adding an op amp, so that currents are drawn from virtual ground
node instead of inner node, thereby stabilizing its voltage against loading effects (figure
17).

x x
ΣI load
8
+ -
ΣI

Figure 17 Equivalent inner circuit structures

The circuit is then transformed to a balanced structure (figure 18) that employs variable
conductance blocks, that can be realized with four transistors, as in figure 19.

Cx -x G +y
+ - + -
Gx Gf
- + - +
Cx +x G -y

Figure 18 Balanced inner circuit structure

Almost the same structure was used

control by Harrer, Nossek and Stelzl68 to design a
DT-CNN. A 4×4 prototype on hexagonal
grid was fabricated in 1.5µm CMOS
technology, with 12 cells per mm2. The
circuit operated correctly at 1MHz clock
G 2
1 2 1 frequency.
A different approach was taken by
Rodríguez-Vázquez et al.4, who associate
all variables to currents. The reason to
abandon voltage variables is that
Figure 19 Balanced variable conductance block combination of voltage and current
variables complicates design for the
necessity of scaling signals to compensate nonlinearities, and requires high impedance
internal nodes, that cause large time constants to appear. Besides, an efficient way of
realizing input is by means of photosensors, that give current output.
Their design is based on the FR-CNN model, obtaining simplified design, good
speed/power ratio and low cell complexity. Basic building blocks for this realization are
current mirrors, which are used for weighted state replication for connections, and to

24
realize lossy integration and delay operators required to generate continuous- or discrete-
time dynamics (figure 20)

Φ
1 1 1 1

(a) (b)
Figure 20 (a) lossy integrator; (b) half-clock delay in current mode

The quoted paper also reports about realization and testing in 1.6µm CMOS
technology of several CT and DT prototypes. 9×9 and 1×16 CT prototypes have less than
20 transistors per cell, so that 60 to 160 cells per mm2 density is achieved. These circuits
settle in about 0.25 to 1.5µs. A DT-CNN programmable 9×9 network with local logic was
successfully tested at 5 MHz clock frequency. Due to programming and testing circuitry
and lines, cells are large: 500µm×500µm.
Important issues to be confronted in practical realizations are those of control and
initialization. Accessing all cells at once is generally impossible by means of electronic
signals because of the excessive number of lines required. Multiplexed accessing (e.g. by
rows) is therefore necessary, together with analog storage, that may be done by capacitors
connected to voltage followers. Even bigger difficulty is involved in weight programming.
The easiest case is when cloning templates are used, and programming is only allowed in
discrete values, that can be selected by a few global lines and some local logic and
memory. A promising alternative, especially for image processing purposes, is using on-
chip photosensors as input devices.
Realization of DT-CNNs can was also attempted by use of optical devices69. Main
advantages are speed-of-light computing in the forward path, and possibility of large
neighborhood; however, bottlenecks occur in electronic addressing of cells for input (but
optical addressing might also be implemented) and in electronic feedback of intermediate
results.
Main building blocks for the optical realization are liquid crystal devices (Spatial
Light Modulators - SLM), and lenses. SLMs are used to perform analog multiplication
between images, by applying variable attenuation to an image transmitted through the
panel; their nonlinear sigmoid-like transparency vs. voltage characteristic is also used to
implement output nonlinearity. Lenses are used to realize cross-correlation with cloning
template. In fact, they realize a Fourier transform, which can be followed by SLM
multiplication by a hologram representing cloning template. Inverse transform is obtained
by another lens, after observing that a second direct transform yields a mirror image of the
desired inverse transform. Complete optical CNN block scheme is depicted in figure 21.

25
 f f f f
input input output output
signal spectrum spectrum signal

CCD camera

input lens filter lens

display

input ...
bias
Figure 21 Optical realization of a DT-CNN.

In order to minimize speed limitations of optical CNNs, design should concentrate

on minimizing the number of time steps necessary to obtain desired task, by maximizing
derivatives of state components and enlarging neighborhood size70.

7. Appendix

Table of symbols

symbol meaning defin. ℑ set of natural numbers

A,B feedback and input 2.2.1 m discrete time delay 2.1.1
coefficients or functions µj,νj indices of feedback and input 2.1.1
nonlinear feedback and input 2.3.3
A , B functionals
functions n discrete time 2.1.1
A τ , B τ functions
nonlinear feedback and input 2.3.3 Nr(•) neighborhood function 2.1.1
pA, pB parameters of feedback and 2.1.1
A,B cloning template matrices 2.2.2 input functional
feedback and input 3 r neighborhood size 2.1.1
A , B
coefficient matrices ℜ set of real numbers
, feedback, input functionals 2.2.1 σ unit step function 2.3.7
cell grid 2.1.1 t continuous time 2.1.1
d distance function defined 2.1.1 τ continuous time delay 2.1.1
over u cell input 2.1.1
x cell state 2.1.1
∆t generic differential operator 2.2.1
y cell output 2.1.1
in time domain
f cell output function 2.1.1
cell output functional 2.1.1
g local feedback function 2.1.1
i imaginary unit
I cell bias
domain of indices 2.1.1

26
 Table of acronyms

acronym meaning defin. MODA Moving Object Detecting 2.3.8

BSB Brain-State-in-a-Box Architecture
CA Cellular Automaton MV-CNN Multi-Valued CNN 2.3.13
CAM Content-Addressable NL-CNN Non-Linear CNN 2.3.4
Memory NLCT-CNN Non-Linear Cloning 2.3.4
CCD Connected Component 1 Template CNN
Detector NN Neural Network
CH-NN Continuous Hopfield NN NUP-CNN Non-Uniform Processor 2.3.7
CNN Cellular Neural Network 2.1; 2.2 CNN
CNNL CNN with Local Logic 2.3.10 OTA Operational
CT-CNN Continuous Time CNN 2.3 Transconductance
CY-CNN Chua & Yang's CNN 2.3.1 Amplifier
D-CNN Delay CNN 2.3.4 PCL Positive Cell-Linking 3
DCT-CNN Delay-type Cloning 2.3.4 P-CNN Polynomial CNN 2.3.6
Template CNN PDE Partial Differential
DT-CNN Discrete Time CNN 2.3.12 Equation
FF-CNN Feed-Forward CNN 2.3.5 PWL Piece-Wise Linear
FR-CNN Full Range CNN 2.3.3 RO-CNN Relaxation Oscillator 2.3.9
LCT-CNN Linear Cloning 2.3.2 CNN
Template CNN SA Systolic Array
LT-CNN Linear Threshold CNN 2.3.5 SLM Spatial Light Modulator 6
MLP Multi-Layer Perceptron
MNS-CNN Multiple Neighborhood 2.3.7
Size CNN

27
8. References

In this section, the following abbreviations are used:

CNNA-90: Proc. of IEEE Int. Workshop on Cellular Neural Networks and their
Applications (CNNA-90), Budapest, Hungary, Dec. 16-19, 1990
CNNA-92: Proc. of Second IEEE Int. Workshop on Cellular Neural Networks and their
Applications (CNNA-92), Munich, Germany, October 14-16, 1992
Only most recent versions of papers on the same topic have been quoted.

1. K.R. Crounse, T. Roska, L.O. Chua, "Image Halftoning with CNNs", IEEE Trans. on
Circ. and Syst. CAS-II-40(3) (1993)
2. L.O. Chua, L. Yang, "Cellular Neural Networks: Theory", IEEE Trans. on Circ. and
Syst. CAS-35(10), 1257 (1988)
3. L.O. Chua, T. Roska, "The CNN Universal Machine - Part 1: The Architecture",
CNNA-92, 1
4. A. Rodríguez-Vázquez, S. Espejo, R. Domínguez-Castro, J.L. Huertas, E. Sánchez-
Sinencio, "Current-Mode Techniques for the Implementation of Continuous and
Discrete-Time Cellular Neural Networks", IEEE Trans. on Circ. and Syst. CAS-II-
40(3) (1993)
5. J. Anderson, J. Silverstein, S. Ritz, R. Jones, "Distinctive Features, Categorical
Perception, and Probability Learning: Some Applications of a Neural Model",
Psychological Review, 84, 413 (1977)
6. T. Roska, L.O. Chua, "Cellular Neural Networks with Non-linear and Delay-Type
Template Elements and Non-Uniform Grids", Int. j. c. th. appl. 20, 469 (1992)
7. L.O. Chua, B. Shi, "Exploiting Cellular Automata in the Design of Cellular Neural
Networks for Binary Image Processing", University of California at Berkeley,
memorandum no. UCB/ERL M89/130 (1989)
8. J.A. Nossek, G. Seiler, T. Roska, L.O. Chua, "Cellular Neural Networks: Theory and
Circuit Design", Int. j. c. th. appl. 20, 533 (1992)
9. A. Barone, M. Balsi, V. Cimagalli, "Polynomial Cellular Neural Network: A New
Dynamical Circuit for Pattern Recognition", Proc. of Int. Specialist Workshop on
"Nonlinear Dynamics of Electronic Systems", Dresden, Germany, July 23-24, 1993
10. J. Henseler, P.J. Braspenning, "A Cortex-like Architecture of a Cellular Neural
Network", CNNA-90, 244
11. S. Tan, J. Hao, J. Vandewalle, "Cellular Neural Networks as a Model of Associative
Memories", CNNA-90, 27
12. V. Cimagalli, M. Bobbi, M. Balsi, "MODA: Moving Object Detecting Architecture",
IEEE Trans. on Circ. and Syst. CAS-II-40(3) (1993)
13 J. Henseler, P.J. Braspenning, "Membrain: A Cellular Neural Network Model Based
on a Vibrating Membrane", Int. j. c. th. appl. 20, 483 (1992)
14. A. Barone, M. Balsi, V. Cimagalli, "Cellular Networks of Oscillators", CNNA-92, 246
15. T. Roska, L.O. Chua, "The Dual CNN Analog Software", Hungarian Academy of
Sciences, Budapest, Hungary, report no. DNS-2-1992

28
16. K. Halonen, V. Porra, T. Roska, L.O. Chua, "Programmable Analog VLSI Chip with
Local Digital Logic", Int. j. circ. th. appl., 20, 573 (1992)
17. J. Hopfield, "Neurons with Graded Response Have Collective Computational
Properties like Those of Two-State Neurons", Proc. Nat. Acad. Sci, 81, 3088 (1984)
18. H. Harrer, J.A. Nossek, "Discrete-time Cellular Neural Networks", Int. j. c. th. appl.
20, 453 (1992)
19. L.O Chua, T. Roska, P.L. Venetianer, A. Zaràndy, "Some Novel Capabilities of CNN:
Game of Life and Examples of Multipath Algorithms", Hungarian Academy of
Sciences, Budapest, Hungary, report no. DNS-3-1992
20. N.N. Aizemberg, I.N. Aizemberg, "CNN Based on Multi-Valued Neuron as a Model
of Associative Memory for Grey-Scale Images", CNNA-92, 37
21. S. Grossberg, "Some Nonlinear Networks Capable of Learning a Spatial Pattern of
Arbitrary Complexity", Proc. Nat. Acad. Sci., 59, 368 (1968)
22. S.-I. Amari, "Learning Patterns and Pattern Sequences by Self-Organizing Nets of
Threshold Elements", IEEE Trans. on Computers, C-21, 1197 (1972)
23. T. Roska, L.O. Chua, "Cellular Neural Networks with Non-linear and Delay-Type
Template Elements", CNNA-90, 12
24. T. Roska, C.W. Wu, M. Balsi, L.O. Chua, "Stability and Dynamics of Delay-Type
General and Cellular Neural Networks", IEEE Trans. on Circ. and Syst. CAS-I-
39(6), 487 (1992)
25. T. Roska, C.W. Wu, M. Balsi, L.O. Chua, "Stability of CNNs with Dominant
Nonlinear and Delay-Type Templates", IEEE Trans. on Circ. and Syst. CAS-I-40(3)
(1993)
26. L.O Chua, T. Roska, "Stability of a Class of Nonreciprocal Cellular Neural
Networks", IEEE Trans. on Circ. and Syst. CAS-37(12), 1520 (1990)
27. F. Zou and J.A. Nossek, "Stability of Cellular Neural Networks with Opposite-Sign
Templates", IEEE Trans. on Circ. and Syst. CAS-38, 675 (1991)
28. M. Balsi, "Stability of Cellular Neural Networks with One-Dimensional Templates",
Int. j. circ. th. appl., in press
29. L.O. Chua, C.W. Wu, "On the Universe of Stable Cellular Neural Networks", Int. j. c.
th. appl. 20, 497 (1992)
30. M. Balsi, "Recurrent Back-Propagation for Cellular Neural Networks", Proc. of
European Conference on Circuit Theory and Design (ECCTD'93), Davos,
Switzerland, Aug.30-Sep. 3, 1993 (Elsevier)
31. M. Balsi, "Generalized CNN: Potentials of a CNN with Non-Uniform Weights",
CNNA-92, 129
32. P.P. Civalleri, M. Gilli, "Some Stability Properties of CNNs with Delay", CNNA-92,
94
33. L.O. Chua, L. Yang, "Cellular Neural Networks: Applications", IEEE Trans. on Circ.
and Syst. CAS-35(10), 1272 (1988)
34. A. Dzielinski, S. Skoneczny, R. Zbikowski, S. Kuklinski, "Cellular Neural Network
Application to Moiré Pattern Filtering", CNNA-90, 139

29
35. T: Matsumoto, T. Yokohama, H. Suzuki, R. Furukawa, A. Oshimoto, T. Shimmi, Y.
Matsushita, T. Seo, L.O. Chua, "Several Image Processing Examples by CNN",
CNNA-90, 100
36. D.-H. Yu, C.-Y. Ho, X. Yu, S. Mori, "On the Application of Cellular Automata to
Image Thinning with Cellular Neural Networks", CNNA-92, 210
37. G. Seiler, "Small Object Counting with Cellular Neural Networks", CNNA-90, 114
38. M. Tanaka, K.R. Crounse, T. Roska, "Template Synthesis of Cellular Neural
Networks for Information Coding and Decoding", CNNA-92, 29
39. T. Szirànyi, T. Roska, P. Szolgay, "A Cellular Neural Network Embedded in a a Dual
Computing Structure (CNND) and its Use for Character Recognition", CNNA-90, 92
40. K. Nakayama, Y. Chigawa, "Japanese Kanji Character Recognition Using Cellular
Neural Networks and Modified Self-Organizing Feature Map", CNNA-92, 191
41. P. Szolgay, I. Kispàl, T. Kozek, "An Experimental System for Optical Detection of
Layout Errors of Printed Circuit Boards Using Learned CNN Templates", CNNA-92,
203
42. Gy. Eröss, T. Boros, À. Kiss, A. Radvànyi, T. Roska, J. Bitò, J. Vass, "Optical
Tracking System for Automatic Guided Vehicles Using Cellular Neural Networks",
CNNA-92, 216
43. T. Boros, K. Lotz, A. Radvànyi, T. Roska, "Some Useful, New, Nonlinear and Delay-
Type Templates", Hungarian Academy of Sciences, Budapest, Hungary, report no.
DNS-1-1991
44. S. Jankowski, R. Wanczuk, "Nonlinear CNN Cloning Templates for Image Thicking",
CNNA-92, 197
45. P. Nachbar, A.J. Schuler, T. Füssl, J.A. Nossek, L.O. Chua, "Robustness of Attractor
Networks and an Improved Convex Corner Detector", CNNA-92, 55
46. S. Paul, K. Hüper, J.A. Nossek, L.O. Chua, "Mapping Nonlinear Lattice Equations
onto CNNs", IEEE Trans. on Circ. and Syst. CAS-I-40(3) (1993)
47. T. Roska, K. Lotz, J. Hàmori, E. Làbos, J. Takàcs, "The CNN Model in the Visual
Pathway - Part I: the CNN-Retina and some Direction- and Length-Selective
Mechanisms", Hungarian Academy of Sciences, Budapest, Hungary, report no. DNS-
8-1991
48. Z. Galias, "Designing Discrete-Time Cellular Neural Networks for the Evaluation of
Local Boolean Functions", CNNA-92, 23
49. T. Roska, T. Boros, A. Radvànyi, P. Thiran, L.O. Chua, "Detecting Moving and
Standing Objects Using Cellular Neural Networks", Int. j. circ. th. appl., 20(5), 613
(1992)
50. C. Güzelis, "Supervised Learning of the Steady-State Outputs in Generalized Cellular
Neural Networks", CNNA-92, 74
51. A. Radvànyi, "A Dual CNN Model of Cyclopean Perception and its Application
Potentials in Artificial Stereopsis", CNNA-92, 222
52. Y. Andreyev, Y. Belsky, A. Dmitriev and D. Kuminov, "Inhomogeneous Cellular
Neural Networks: Possibility of Functional Device Design", CNNA-92, 135

30
53. G. Seiler, A.J. Schuler, J.A. Nossek, "Design of Robust Cellular Neural Networks",
Technische Universität München, Munich, Germany, report no. TUM-LNS-TR-91-13
(1991)
54. G. Martinelli, R. Perfetti, "Associative Memory Design Using Space-Varying Cellular
Neural Networks", CNNA-92, 117
55. H. Mizutani, "The Effective Weighting Method of CNN as a Recall", CNNA-92, 86
56. H. Harrer, J.A. Nossek, F. Zou, "A Learning Algorithm for Discrete-Time Cellular
Neural Networks", Technische Universität München, Munich, Germany, report no.
TUM-LNS-TR-91-1 (1991)
57. L.O. Chua, P. Thiran, "An Analythic Method for Designing Simple Cellular Neural
Networks", IEEE Trans. on Circ. and Syst. CAS-38(11), 1332 (1991)
58. J.A. Osuna, G.S. Moschytz, "Exact Design of Reciprocal Cellular Neural Networks",
CNNA-92, 11
59. K. Slot, "Cellular Neural Networks Design for Solving Specific Image-Processing
Problems", Int. j. circ. th. appl. 20, 629 (1992)
60. F. Zou, S. Schwarz, J.A. Nossek, "Cellular Neural Network Design Using a Learning
Algorithm"; CNNA-90, 73
61. P. Szolgay, T. Kozek, "Optical Detection of Layout Errors of Printed Circuit Boards
Using Learned CNN Templates", Hungarian Academy of Sciences, Budapest,
Hungary, report no. DNS-10-1991
62. A.J. Schuler, P. Nachbar, J.A. Nossek, L.O. Chua, "Learning State Space Trajectories
in Cellular Neural Networks", CNNA-92, 68
63. T. Kozek, T. Roska, L.O. Chua, "Genetic Algorithm for CNN Template Design",
IEEE Trans. on Circ. and Syst. CAS-I-40(3) (1993)
64. C. Güzelis, "Supervised Learning of the Steady-State Outputs in Generalized Cellular
Neural Networks", CNNA-92, 74
65. L.K. Hansen, "Bolzmann Learning of Parameters in Cellular Neural Networks",
CNNA-92, 62
66. H. Magnussen, J.A. Nossek, "Towards a Learning Algorithm for Discrete-Time
Cellular Neural Networks", CNNA-92, 80
67. J.M. Cruz, L.O. Chua, "Design of High-Speed, High-Density CNNs in CMOS
Technology", Int, j. circ. th. appl., 20, 555 (1992
68. H. Harrer, J.A. Nossek, R. Stelzl, "An Analog Implementation of Discrete-Time
Cellular Neural Networks", IEEE Trans. on Neural Networks NN-3(3), 466 (1992)
69. E. Lüder, N. Frühauf, "Optical Signal Processing for CNN's", CNNA-92, 45
70. K. Slot, "Optically Realized Feedback Cellular Neural Networks", CNNA-92, 175

31