Exploring architecture variations in constructive cascade networks

Abstract

Constructive neural networks employing a cascade architecture face a number of problems. These include large propagation delays, high fan-in and irregular network connections. These problems are especially relevant with regards to VLSI implementation of these algorithms. This work explores the effect of limiting the depth of the cascades created by the CasPer algorithm, a constructive network algorithm. Instead of a single cascade of hidden neurons, a series of cascade towers are built. The maximum size of each tower is set prior to training, thus limiting maximum network depth, creating regular connections and enabling a reduction in maximum fan-in. The networks created in this manner are shown to maintain or better network generalization over a number of different tower sizes. Show more