Artificial Neural Networks

ANNs are computer programs designed for inducing problem solutions (models, knowledge) from complex data by means of principles of information processing similar to biological neurons in the human brain. A biological neuron consists of three major components: the cell body, dendrites, and the axon (Figure 3 a).

Cell body

Dendrites

Hillock zone

Hillock zone

Synapses

Cell body

Synapses

Dendrites

Axon

Axon

(b) Input Input Weighted units weights inputs

Summation of total weighted input

Sigmoid input-output transfer function

Neural activation

Output

Figure 3 Conceptual structures of biological and artificial neurons.

Figure 3 Conceptual structures of biological and artificial neurons.

Connections between neurons are formed at synapses. Information is represented and transmitted by chemically generated electrical activity within the cell. Both excitatory and inhibitory inputs to the neuron enter through synaptic connections with other neurons. Input potentials are summed up within the cell body. If the total input potential is sufficient (e.g., meets a certain threshold value), the neuron acts. Ultimately an action potential is generated and propagated down the axon toward the synaptic junctions with other nerve cells.

The design of ANNs (Figure 3b) has been inspired by the structure and functioning of biological neurons. The dendrites which are acting as input receptors were represented by input units. The cell body that acts as information accumulator was represented by activation units adjusting and summing up the weights of inputs, and the input-output transfer function. The axon that acts as the biological output channel was represented as the output.

ANNs gain adaptive capability by undergoing training similar to neural learning where two basic training modes are distinguished: supervised and nonsupervised. The supervised training aims at the optimal approximation of the calculated output Yc to the observed (desired) output Yo. An iterative adjustment of input weights takes place in order to minimize the error (Yo — Yc). After training, the generalization of the supervised ANN is assessed by feeding it only with input values, not observed output values, and testing how close calculated outputs match observed outputs. The two most common methods for assessing generalization are the 'split-sample validation' and the 'cross-validation'. The 'split-sample validation' means that part of the data is reserved as a test set, which must not be used in any way during training. The test set must be representative for the problem to be modeled by the ANN. After training, the ANN is run on the test set, and the error on the test set provides an estimate of the generalization error usually expressed by the root mean square error (RMSE) or the correlation coefficient r . The disadvantage of split-sample validation is that it reduces the amount of data available for both training and validation. By contrast, 'cross-validation' allows use of all the data for training. In ¿-fold cross-validation, the data is divided into k equal-sized subsets. The net is trained k times, each time leaving out one of the subsets from training, but using only the omitted subset to compute the generalization error. If k equals the sample size, this is called 'leave-one-out' cross-validation. The disadvantage of cross-validation is that the ANNs need to be retrained many times.

Depending on using external inputs only or feedback inputs as well, supervised ANNs are differentiated into feedforward or feedback ANNs (see Figures 4a and 4b). By contrast, nonsupervised ANNs process external inputs only without adjusting calculated outputs to known outputs (Figure 4c).

The supervised feedforward ANN proves to be a universal approximator of multivariate nonlinear functions and is usually implemented as multilayer perceptron with back-propagation training (see Multilayer Perceptron). The multilayer perceptron represents input units as input layer, adjusted and accumulated input weights as hidden layer(s), and outputs as output layer. The back-propagation algorithm performs the iterative adjustment of input weights (activation units) in order to minimize the approximation error (Yo — Yc).

Supervised feedforward ANNs are widely applied in ecology either using cross-sectional data to predict discrete ecosystem states or using time-series data to predict continuous ecosystem behavior. Successful applications by means of cross-sectional data have been demonstrated for fish communities in streams, macroinvertebrate communities in streams, river salinity, primary productivity in esturaries, chlorophyll a concentrations in lakes, coastal vegetation, and bird populations.

Successful applications by means of time-series data have been demonstrated for marine fish and zooplankton communities, river hydrology, macroinvertebrate communities in streams, freshwater phyto- and zooplankton communities.

The majority of the supervised feedforward ANNs documented achieved forecasting results that were superior to conventional modeling techniques such as multiple linear regression. Even though supervised ANNs do not provide explicit mathematical representations of the underlying ANN model, most of the authors have conducted sensitivity analyses in order to identify inputs as key driving forces of the predictive ANN.

Supervised feedback or recurrent ANNs are designed to use not only external inputs for training but also activation levels of the previous training iteration which are constantly fed back (see Figure 4b). Their functioning can be compared with ordinary differential equations that calculate the current system state Z(t) by taking into account current external inputs Xe(t) and the system state Z(t — 1) of the time step before:

where P represents constant parameters.

Supervised feedback ANNs prove to be very powerful for modeling time-series data where the fed-back activation levels provide extra training information on the system state of the time step before (see Artificial Neural Networks: Temporal Networks).

External inputs Xe

Activation units Z

Observed outputs Vo

Feedback inputs Z (t- 1) |
1 N | ||||||||||||||||||||||||||||

Calculated | |||||||||||||||||||||||||||||

outputs Yc |
/ | ||||||||||||||||||||||||||||

4 L, | |||||||||||||||||||||||||||||

A7 |
/ |
/ | |||||||||||||||||||||||||||

N | |||||||||||||||||||||||||||||

Error |
External | ||||||||||||||||||||||||||||

(Yo - Yc) ¿L. pi. |
/ |
inputs Xe(t) |
Activation units Z(t) Calculated outputs Yc(t) Observed outputs Yo(t)
Activation units Z Calculated outputs Yo Figure 4 Basic types of ANNs: (a) supervised feedforward ANN; (b) supervised feedback ANN; (c) nonsupervised ANN. Figure 5 Supervised feedback ANN for 4-day-ahead forecasting of population densities of Microcystis aeruginosa and Stephanodiscus hantzschii in River Nakdong (South Korea). Modified from Jeong K-S, Recknagel F, and Joo G-J (2006) Prediction and elucidation of population dynamics of the blue-green algae Microcystis aeruginosa and the diatom Stephanodiscus hantzschii in the Nakdong River-Reservoir System (South Korea) by a recurrent artificial neural network. In: Recknagel F (ed.) Ecological Informatics: Scope, Techniques and Applications, 2nd edn., pp. 255-273. Berlin: Springer. Figure 5 Supervised feedback ANN for 4-day-ahead forecasting of population densities of Microcystis aeruginosa and Stephanodiscus hantzschii in River Nakdong (South Korea). Modified from Jeong K-S, Recknagel F, and Joo G-J (2006) Prediction and elucidation of population dynamics of the blue-green algae Microcystis aeruginosa and the diatom Stephanodiscus hantzschii in the Nakdong River-Reservoir System (South Korea) by a recurrent artificial neural network. In: Recknagel F (ed.) Ecological Informatics: Scope, Techniques and Applications, 2nd edn., pp. 255-273. Berlin: Springer. Figure 5 shows an example for a supervised feedback ANN that has successfully been trained and tested by split-sample validation for the forecasting of the algal populations Microcystis and Stephanodiscus in River Nakdong in South Korea. The weekly measured limno-logical data of the river study site were interpolated to daily values. The interpolated data from 1995 to 1998 were used as training set, and the interpolated data of 1994 were used as testing set. In order to achieve a 4-day-ahead forecasting, a 4 days time lag was imposed between the measured inputs and the measured outputs of the training data set. The design of the feedback ANN considered the following 18 external input variables: irra-diance, precipitation, discharge, evaporation, water is 1 160 140 120 100 80 60 40 20 0 160 140 120 100 80 60 40 20 0 Jan. Feb. Mar. Apr. May Jun. Jul. Aug. Sep. Oct. Nov. Dec. Jan. 1994 Figure 6 Four-day-ahead forecasting of population densities of M. aeruginosa and S. hantzschii in River Nakdong (South Korea) by means of a supervised feedback ANN. Modified from Jeong K-S, Recknagel F, and Joo G-J (2006) Prediction and elucidation of population dynamics of the blue-green algae Microcystis aeruginosa and the diatom Stephanodiscus hantzschii in the Nakdong RiverReservoir System (South Korea) by a recurrent artificial neural network. In: Recknagel F (ed.) Ecological Informatics: Scope, Techniques and Applications, 2nd edn., pp. 255-273. Berlin: Springer. Jan. Feb. Mar. Apr. May Jun. Jul. Aug. Sep. Oct. Nov. Dec. Jan. 1994 Figure 6 Four-day-ahead forecasting of population densities of M. aeruginosa and S. hantzschii in River Nakdong (South Korea) by means of a supervised feedback ANN. Modified from Jeong K-S, Recknagel F, and Joo G-J (2006) Prediction and elucidation of population dynamics of the blue-green algae Microcystis aeruginosa and the diatom Stephanodiscus hantzschii in the Nakdong RiverReservoir System (South Korea) by a recurrent artificial neural network. In: Recknagel F (ed.) Ecological Informatics: Scope, Techniques and Applications, 2nd edn., pp. 255-273. Berlin: Springer. temperature, Secchi depth, turbidity, pH, DO, nitrate, ammonia, phosphate, silica, rotifera, caldocera, copepoda (see also for input sensitivity in Figure 5), 21 hidden activation units, and the two output variables: Microcystis aeruginosa and Stephanodiscus hantzschii. After 2100 training iterations, an RMSE of 0.0017 was achieved and the generalization of the trained ANN was tested based on testing data of 1994. Figure 6 shows the visual comparison between the observed and the 4-day-ahead predicted data for M. aeruginosa (r2 = 0.68) and S. hantzschii (r = 0.73). The results indicate a high degree of accuracy in the forecasting regarding both the timing and the magnitudes of populations dynamics of the two algal species, which have their distinctive seasonal patterns. This application has demonstrated that supervised feedback ANNs achieve a high generalization degree and forecasting accuracy after training by time-lagged time-series data. The typical rapid growth and blooming of the blue-green algae Microcystis under warm and calm conditions in mid- and late summer as observed in River Nakdong in 1994 was well reflected by the predicted data in Figure 6a. By contrast diatoms tend to be abundant at moderate temperatures and turbulent conditions. Both observed and predicated data for S. hantzschii in River Nakdong correspond well by showing population densities in the highest range, in spring and autumn (Figure 6b). This case study has also convincingly demonstrated the benefits of sensitivity analyses in order to gain insights and test hypotheses regarding ecological relationships between input and output variables. Results in Figure 7 compare the input sensitivities of the two different algal populations that have been interpreted in great detail by Jeong et al. The most obvious differences between M. aeruginosa and S. hantzschii can be seen in their preferred water temperature, pH, and silica levels that comply with ecological theory. Successful applications have also been demonstrated for time-series modeling of macroinvertebrate communities in streams and phytoplankton communities in freshwater lakes and rivers. Nonsupervised ANN Nonsupervised ANNs are designed to identify unknown input patterns based on similarities between inputs. 240 320 400 480 560 240 320 400 480 560 2000 4000 2000 4000 30 45 60 75 90 105 120 30 40 50 60 70 80
0 30 60 90 120 150 6000 30 35 40 45 50 55 60 30 45 60 75 90 105 120 30 40 50 60 70 80 0 30 60 90 120 150 30 35 40 45 50 55 60
-M. aeruginosa Figure 7 Input sensitivity curves for the population densities of M. aeruginosa (solid lines) and S. hantzschii (dotted lines) in River Nakdong (South Korea) by means of a supervised feedback ANN. Modified from Jeong K-S, Recknagel F, and JooG-J (2006) Prediction and elucidation of population dynamics of the blue-green algae Microcystis aeruginosa and the diatom Stephanodiscus hantzschii in the Nakdong River-Reservoir System (South Korea) by a recurrent artificial neural network. In: Recknagel F (ed.) Ecological Informatics: Scope, Techniques and Applications, 2nd edn., pp. 255-273. Berlin: Springer. Input layer Water temperature Secchi depth Algal species i-1 Algal species i Ordination and clustering of inputs Mapping of clustered inputs Figure 8 Conceptual diagram of the structure and functioning of nonsupervised ANN. So-called self-organizing maps (SOMs) developed by Kohonen are the most popular nonsupervised ANNs which can be applied to ordination, clustering, and mapping of complex nonlinear data (see Animal Defense Strategies). The principal approach ofnonsupervised ANNs according to Kohonen is represented in a simplified manner in Figure 8. It shows that the neurons of the nonsupervised ANN learn to distinguish between similar and dissimilar features of the normalized input data, which are mapped as clustered inputs. The term nonsupervised in this context means that the learning algorithm is not guided by known output patterns but learns the patterns from features of the inputs. Those features can be expressed by Euclidean distances, which are calculated between the inputs and weights. Similarities between inputs in terms of Euclidean distances can be visualized and partitioned by the unified distance matrix (U-matrix) and the K-means map. In order to illustrate opportunities of applications of nonsupervised ANN to ecological time-series data, Figures 9-12 show results of a case study carried out for limnological data of Lake Kasumigaura in Japan. Figure 9 represents seasonal clusters for Lake Kasumigaura as mapped by the U-matrix and K-means partitioning using the SOM Toolbox of MATLAB 5.3. The U-matrix map in Figure 9a visualizes the relative distances between neighboring data of the input data space as shades of gray. The light areas in the U-matrix visualize neighboring data with distances in the shortest range belonging to a region or cluster. The black colors represent the distances in the longest range between neighboring data and denote borders between clusters. The K-means algorithm partitions the input data space into a specified number of clusters based on the U-matrix. Figure 9b represents the corresponding partitioned map for five seasons. U-matrix U-matrix 0.33 0.33 0.060 8 Figure 9 Ordination and clustering of seasons of Lake Kasumigaura by means of nonsupervised ANN visualized as unified distance matrix map (U-matrix) (a), and as partitioned map (K-means) (b); the seasons were defined as follows: winter from 1 December, spring from 15 March, early summer from 1 June, late summer from 1 August, autumn from 1 October. Figure 10 visualizes seasonal distributions of abundances of the blue-green algae Microcystis and Oscillatoria in Lake Kasumigaura based on data of the years 1984-86 (left column) and 1987-89 (right column). Figure 11 represents the seasonal distributions of concentrations of NO3-N and PO4-P in Lake Kasumigaura in correspondence with the time periods differentiated in Figure 9. Figure 10 highlights that while Microcystis declines in cell numbers by more than 50% between 1984-86 and Early summer Early summer Autumn Autumn 1984-86 Microcystis cells ml- 1984-86 Microcystis cells ml- 440000 20 000 440000 20 000 1987-89 Microcystis cells ml-1 1987-89 Microcystis cells ml-1 186 000 51 700 186 000 ## 51 700Figure 10 Component planes for seasonal abundances of Microcystis and Oscillatoria populations in Lake Kasumigaura for the years 1984-86 (left column) and 1987-89 (right column). 1984-86 1984-86 XGU- .; rrrriYrr 1987-89 NO3-N mg l-1 1987-89 NO3-N mg l-1 P04-P mg P P04-P mg P 1106 TOWM rmrrr m 1390 143.5 Figure 11 Component planes for seasonal concentrations of PO4-P and NO3-N in Lake Kasumigaura for the years 1984-86 (left column) and 1987-89 (right column). 1987-89, Oscillatoria doubles in cell numbers. It also shows that seasonal dominance of two algal populations for the early and the late 1980s shifted for Microcystis from late summer to autumn, and for Oscillatoria from early summer to late summer. Takamura etal. pointed at changes of NO3-N/PO4-P ratios as possible explanations for the succession of the two blue-green algal populations during the 1980s in Lake Kasumigaura, that are indicated by the component planes in Figure 10. From the early to the late 1980s, the NO3-N concentrations increased by 50% while PO4-P concentrations dropped to 50%, causing a significant change of the NO3-N/PO4-P ratios (from 8.5 to 32). A combination of input sensitivity curves by supervised feedback ANN with component planes by Microcystis cells ml CTOCC Cyclotella cells ml Microcystis cells ml Cyclotella cells ml 25 000 20 000 lls 15 000 e cys 10 000 oc 5 000 20 000 lls 15 000 e cys 10 000 oc 5 000 4000 3000 2000 lote Cyclo 1000 14 16 18 20 22 Water temperature (°C) Figure 12 Component planes for water temperature preferences of Microcystis and Cyclotella populations (top) and water temperature sensitivity curves for Microcystis and Cyclotella populations (bottom) in Lake Kasumigaura for the years 1984-93. 5000 4000 3000 2000 lote Cyclo 1000 14 16 18 20 22 Water temperature (°C) Figure 12 Component planes for water temperature preferences of Microcystis and Cyclotella populations (top) and water temperature sensitivity curves for Microcystis and Cyclotella populations (bottom) in Lake Kasumigaura for the years 1984-93. nonsupervised ANN proves to be an informative approach (e.g., for hypothesis testing). While component planes allow to map nonlinear relationships of output variables with predefined input ranges in a qualitative manner (see Figure 12, top), input sensitivity curves draw numerical relationships of output variables over the whole range of input variables as learnt from training data. Both the component planes and sensitivity curves in Figure 12 confirm theoretical assumptions that the diatoms Cyclotella have a preference of low to medium water temperatures typically occurring in spring and autumn, while the population growth of Microcystis reaches rates in the highest range at high water temperatures in mid- and late summer. Successful applications of nonsupervised ANNs have been demonstrated for cross-sectional data of macroinvertebrate communities in streams and vegetation types. Successful applications of nonsupervised ANNs have been demonstrated for time-series data of plankton communities in lakes and rivers. |

Was this article helpful?

Get All The Support And Guidance You Need To Be A Success At Helping Save The Earth. This Book Is One Of The Most Valuable Resources In The World When It Comes To How To Recycle to Create a Better Future for Our Children.

## Post a comment