Testing your methods with simulated data and then some of the real data

The analysis of models is pretty much a science, but the development of models is an art. There has grown up a large literature concerning ''model validation,'' which generally intends that one tests or validates the model by comparison with the data. Here, I offer three suggestions about testing a model that you have developed (also see Mangel et a/. (2001)). First, always try to test the assumptions that go into the model independent of the predictions of the model. Second, always test your model or method with simulated data in which you know exactly what is happening. If the method does not work on simulated data, it is almost surely not going to work on real data. Third, set aside some of your data at the beginning of an analysis. Estimate parameters with the remainder and then use the set-aside data as a means of testing the predictions of the method. Hilborn and Mangel (1997) discuss model size and model validation in more detail.

0 0

Post a comment