A new Stochastic Dynamic Adaptive Local Search algorithm for Elman Neural Network

Elman Neural Network (ENN) has found numerous applications in such as time series prediction, system identification and adaptive control since it has powerful dynamic memories. However, the local minima problem usually occurs in the process of the learning. In this paper, we firstly propose a new adaptive local search (ALS) method for the ENN by instead of traditional Back-Propagation (BP) algorithm. Based on this algorithm, we further propose a Stochastic Dynamic Adaptive Local Search (SDALS) algorithm for the ENN which introduces stochastic dynamics into the ALS algorithm in order to avoid the possible local minima. The proposed learning algorithm maintains some trends of quick descent to either global minimum or a local minimum, and at the same time has some chance of escaping from the local minima by permitting temporary error increases during learning. Thus, the proposed algorithm may eventually reach the global minimum state or its best approximation with very high probability. Simulation results show that the proposed algorithm has the superior abilities to find the global minimum than other algorithms.