## Abstract

A method for evaluating predictive models is developed by giving a precise and statistically meaningful interpretation to the statement that a model is accurate within a factor of k. This method is applicable to any model for which there is a set {y_{1},…, y_{n}} of observations corresponding to a set {x_{1}, …, x_{n}} of model predictions. We define a geometrically intuitive measure of model reliability k_{g} in terms of the ratios y_{i}/x_{i} and a statistically rigorous measure k_{s} in terms of ln (y_{i}/x_{i}). For reasonably accurate models, k_{g} and k_{s} are in virtual agreement and thus can be used interchangeably as a reliability index. The index k_{s} estimates a model parameter of the form exp[(V_{1}+ V_{2})^{1/2}], where V_{1} describes an observational variance and V_{2} is related to an uncertainty associated with the model itself. The computed value for k_{s} is not unique but depends on the sample of observations. The probability distribution of k_{s} can be characterized provided the observational distributions are lognormal, independent, and satisfy a homoscedasticity condition. These requirements are often satisfied by quantities of interest in radiation risk analyses. The reliability indices k_{g} and k_{s} may be applied even if the underlying observational distributions are not lognormal, although the probability distribution of k_{s} cannot be characterized in this case. .

Original language | English |
---|---|

Pages (from-to) | 85-95 |

Number of pages | 11 |

Journal | Health Physics |

Volume | 46 |

Issue number | 1 |

DOIs | |

State | Published - Jan 1984 |