Model Evaluation

Issues or questions related to uncertainty, prediction error, sensitivity, and robustness are becoming more important because models are increasingly used in forest management planning and decision making. Thus, it is important to provide policymakers with quantifiable methods to evaluate the extent to which these issues may affect their judgment or decisions. Models are imperfect representations of reality, which means that their

Figure 2 Basic schematic diagram that illustrates the integrated structure of LANDIS to simulate forest succession and disturbance over landscapes. Reproduced from Mladenoff DJ (2004) LANDIS and forest landscape models. Ecological Modelling 180: 7-19, with permission from Elsevier.

Figure 2 Basic schematic diagram that illustrates the integrated structure of LANDIS to simulate forest succession and disturbance over landscapes. Reproduced from Mladenoff DJ (2004) LANDIS and forest landscape models. Ecological Modelling 180: 7-19, with permission from Elsevier.

outputs or predictions contain errors or have a degree of uncertainty associated with them. Uncertainty may be caused by model structure, that is, the lack of understanding of biological processes or incorrect mathematical representation, data and parameter estimates, natural variation, and scaling.

Uncertainty in the predictions, sensitivity, and robustness have often been evaluated with validation methods. Model validation (the term validation is frequently used interchangeably with verification in the literature, which contributed to maintaining the controversy on what model validation consists of) may include the comparison of model outputs with observations from an independent data set and the examination of the consistency of its logical structure. For instance, the comparison of predictions with observations using historical data is a common validation method. Historical data for forest ecosystems may consist of tree or site variables (e.g., dbh, stem height, foliage, and soil nutrients) measured on the same sample plots at different periods. An essential rule is to conduct the comparison of a model's outputs with data that have not been previously used for its calibration. However, some may argue that this method is simply a statistical validation, and does not allow one to conclude if a model is biologically consistent or realistic. Biological consistency consists in examining the behavior of a model by varying systematically its inputs to represent the variation in widely different initial conditions. This is important for evaluating if the predictions indicate a logically consistent pattern, determining if a model conforms to basic laws of biological growth, or detecting illogical or improper formulation or representation of the underlying processes. For instance, a forest growth model that would predict that the diameter growth of individual trees within an even-aged forest ecosystem increases with increase in stand density (i.e., with the intensity of competition) would be inconsistent, but the statistical fit could be significant. A biological consistency analysis can be conducted simultaneously with a sensitivity analysis, which consists in examining the degree to which the outputs of a model change with variation in input variables or parameters. When the variation in input variables or parameters is extended to their extremes, it allows the modeler to verify the reaction of the model when there is extrapolation.

When a statistical method is used to estimate the parameters of a model, the coefficient of determination (which expresses the proportion of variability in the dependent variable accounted for by the regression on the independent variable(s)) and the mean square error (a measure of dispersion) are usually computed. For more complex models that contain several relationships, other statistics computed from predicted and observed values have been suggested in the forestry literature as quantitative measures to evaluate the performance of growth and yield models in terms of goodness of fit:

Mean residual or prediction error = y ^ (y -y¡)/n [10] Root mean square error = y¡ -y¡ )2/n - 1 -p [11]

where n is the number of samples in the independent data set, y, are observations, y, predictions, and p the number of independent variables. Several statistical tests exist to validate models, such as the Kolmogorov-Smirnov, \ -test, or /-test of a regression.

Solar Power

Solar Power

Start Saving On Your Electricity Bills Using The Power of the Sun And Other Natural Resources!

Get My Free Ebook


Post a comment