|
|
|
“Algorithms/Applications/Architectures of Artificial Neural Nets”
by Jenq-Neng Hwang
December 1988
In the research field of artificial neural nets (ANNs), important contributions have come from cognitive science, neurobiology, physics, computer science, control, analog VLSI, and optics. This dissertation is intended to broaden the present scope so as to achieve an in-depth understanding of neural computing from the perspectives of parallel algorithm design, digital VLSI array architectures, digital signal processing, and numerical analysis. Therefore, in a broader sense, the aim of this dissertation has been to accomplish truly cross-disciplinary research.
A variety of neural nets are considered, including single layer feedback neural nets (e.g., Hopfield neural nets, Rumelhart memory/learning modules, and Boltzmann machines), multilayer feed-forward nets (e.g., multilayer Perceptrons), and hidden Markov models (HMMs). Algorithmic studies based on algebraic projection (AP) analysis are presented to deal with two critical and important issues in back propagation (BP) learning: the selection of the optimal number of hidden units and the optimal learning rate. Potential applications of ANNs to two promising areas, computer vision and robotic processing, are discussed. A systematic algorithm mapping methodology is proposed for mapping neural algorithms in VLSI array architectures. This methodology derives array architectures for the algorithms via a two-stage design, i.e., dependence graph (DG) design and array processor design. Programmable ring systolic ANNs (linear and cascaded) are developed, which can exploit the strength of VLSI and offer intensive and pipelined computing. Both the retrieving and learning phases are integrated in the design. The proposed architectures are more versatile than other existing ANNs; therefore, they can accommodate the most widely used neural nets. A unifying viewpoint is proposed for multilayer Perceptrons and HMMs, and the ring systolic array architectures can further be extended to the application domain of implementing both the scoring and learning phases of HMMs. Finally, some mathematical aspects for ANNs worthy of further pursuit are also presented, which include expressibility and discrimination capabilities, generalization capability, convergence in the retrieving and learning phases, and merging of multilayer Perceptrons and HMMs.