Methods and guidelines for effective model calibration

A
-3
-2
-1
0
1
2
3
-2
-1
0
1
2
Standard norm ally distributed 
num ber
W
e
ighted residua
l
head
flow
prior
B
-3
-2
-1
0
1
2
3
-2
0
2
Standard norm ally distributed 
num ber
Independent 
norma
l number
C
-3
-2
-1
0
1
2
3
-2
0
2
Standard norm ally distributed 
num ber
C
o
rre
la
te
d 
normal number

---

62
not reflect all aspects of system uncertainty, and, conservatively, they might be best thought of as 
indicating the least amount of uncertainty. That is, actual uncertainty might be larger than indicated 
by the confidence intervals. If prediction intervals are dominated by the measurement error term, 
they are less likely to be prone to error. Unfortunately, in many circumstances the confidence in-
tervals are of more interest because they reflect model uncertainty most clearly. Cooley (1997) pro-
vides additional analysis of nonlinear confidence intervals.
Guideline 14: Formally reconsider the model calibration from the perspective of 
the desired predictions
It is important to evaluate the model relative to the desired predictions throughout model 
calibration, as discussed in the beginning of the section “Guidelines for Effective Model Calibra-
tions”. For reasonably accurate models, it also is useful to consider the predictions more formally, 
as described below. In this work it is suggested that formal analysis using uncalibrated models is 
likely to produce misleading results, given the nonlinearity of the models considered. It can be dif-
ficult to determine when a model is sufficiently accurate, but at the very least the obvious errors in 
system representation and the relation of the observations to simulated equivalents need to be re-
solved, and weighted residuals need to be approximately random. The analysis is divided into two 
approaches.
First, predictions and linear confidence intervals on the predictions can be calculated for all 
reasonably accurate models to evaluate how different sets of observations and conceptual models 
are likely to affect both the simulated predictions and their likely precision. Linear confidence in-
tervals are suggested instead of nonlinear confidence intervals or either kind of prediction interval 
because linear confidence intervals can be calculated quickly and represent the prediction uncer-
tainty contributed by the model and the parameter estimates.
Second, the model parameters and the simulated predictions can be evaluated to determine 
which parameters and what system features are likely to be most important to prediction accuracy. 
This is accomplished using sensitivities related to the regression observations and the predictions, 
and statistics calculated from these sensitivities, and can be used to guide subsequent field and 
model calibration efforts. The procedure for such an analysis is outlined in figure 16. 

---

63
1. Acceptable means that this parameter is estimated well compared to other parameters, from the perspec-
tive of simulating predictions, or is unimportant to the predictions of interest. Improved estimation of this 
parameter and improved representation of the system features with which this paramter is associated are 
likely to be less important to improving prediction accuracy than for other parameters
2. Improved estimation of this parameter and improved representation of the system features with which it 
is associated probably are important to improved prediction accuracy.
3. The parameter correlation coefficients needed for this analysis are calculated using unestimated as well 
as estimated parameters, and include only the observations and prior information used in the calibration.
4. The prediction correlation coefficients needed for this analysis are as in 3, but include predictions as 
well as the observations and prior information used in the calibration.
Figure 16: Classification of the need for improved estimation of a parameter and, perhaps, associ-
ated system features. The classification is based on statistics which indicate the impor-
tance of parameters to predictions of interest and (A) the precision of parameter 
estimates or (B) the uniqueness with which parameters are estimated by the regression.