The real-time recurrent network (RTRN) is characterized as containing hidden neurons and allowing arbitrary dynamics with a fully connected network structure. The RTRN is especially capable of dealing with time-varying input or output through its own temporal operation and has been applied to speech recognition.
The RTRN has M external inputs, N concatenated nodes, and K outputs. Figure 5 shows the schematic diagram of RTRN. An external input vector of size M is applied to the network at a discrete time t. Let y(t) denote the corresponding vector of size N of individual neuron outputs produced one step later at time t. The input vector and the one-step delayed output vector are concatenated to form vectors of size (M + N). In total, an N by (M + N) recurrent weight matrix is formed.
The net internal activity of neuron j at time t is as follows:
where v(t) is x(t) ifj denotes the external input, and y(t — 1) if j denotes the neuron for outputs. The term wj{t) indicates the weight between the input and the hidden layers. At the next time step (t + 1), the output of neuron j is computed by passing v(t) through the nonlinearity (e.g., logistic function), resulting in the following:
The real-time recurrent learning handles weight feedback in the real-time process and allows faster convergence in recurrent learning. The detailed algorithm could be referred to Williams and Zisper.
Was this article helpful?
Start Saving On Your Electricity Bills Using The Power of the Sun And Other Natural Resources!
Post a comment