Constructive contrasts between modeled and measured climate responses over a regional scale

Reducing uncertainty in predictions of regional-scale models depends on meaningful contrasts with field measurements. This paper introduces a two-stage process that works from the premise that an appropriate goal for regional models is to produce reasonable behavior over dominant environmental gradients. We demonstrate two techniques for contrasting models with data, one based on the shape of modeled relationships (functional contrasts) and the other based on an examination of the residuals (residual contrasts) between the model and an empirically derived surface fit to field data. Functional contrasts evaluated the differences between the response of simulated net primary production (NPP) to climate variables and the response observed in field measurements of NPP. Residual contrasts compared deviations of NPP from the empirical surface to identify groupings (for example, vegetation classes, geographic regions) with model deviations different from those of the field data. In all model-data contrasts, we assigned sample weights to field meaSurementS to ensure unbiased representation of the region, and we included both constructive comparisons and formal statistical tests. In general, we learned more from constructive methods designed to reveal structure or pattern in discrepancy than we did from statistical tests designed to falsify models. Although our constructive methods were more subjective and less concise, they succeeded in revealing gaps in our understanding of regional-scale processes that can guide future efforts to reduce scientific uncertainty. This was best illustrated by NPP predictions from the Biome-BGC model, which showed a stronger response to precipitation than apparently operates in the field. In another case, differences revealed in savanna and dry woodlands had insufficient field-data support, suggesting a need for future field studies to improve understanding in this, and other, poorly studied ecosystems.