Table of Contents

Part 1 Threshold gates: threshold gates; computational capabilities of polynomial threshold gates; general position and the function counting theorem; minimal PTG realization of arbitrary switching functions; ambiguity and generalization. Part 2 Computational capabilities of artificial neural networks: some preliminary results on neural network mapping capabilities; necessary lower bounds on the size of LTG networks; approximation capabilities of feedforward neural networks for continuous functions; computational effectiveness of neural networks; summary. Part 3 Learning rules: supervised learning in a single-unit setting; reinforcement learning; competitive learning; self-organizing feature maps - topology-preserving competitive learning; summary. Part 4 Mathematical theory of neural learning: learning as a search/approximation mechanism; mathematical theory of learning in a single-unit setting; characterization of additional learning rules; principal-component analysis (PCA); theory of reinforcement learning; theory of simple competitive learning; theory of feature mapping; generalization; complexity of learning; summary. Part 5 Adaptive multilayer neural networks I: learning rule for multilayer feedforward neural networks; backprop enhancements and variations; applications; extensions of backprop for temporal learning; summary. Part 6 Adaptive multilayer networks II: radial basis function (RBF) networks; cerebellar model articulation controller (CMAC); unit-allocating adaptive networks; clustering networks; summary. Part 7 Associative neural memories: basic associative memory models; DAM capacity and retrieval dynamics; characteristics of high-performance DAMs; other DAM models; the DAM as a gradient net and its application to combinatorial optimization; summary. Part 8 Global search methods for neural networks: local versus global search; simulated annealing-based global search; simulated annealing for stochastic neural networks; mean-field annealing and deterministic Boltzmann machines; genetic algorithms in neural network optimization; genetic algorithm-assisted supervised learning; summary.