Methods and guidelines for effective model calibration

Diagnostic and Inferential Statistics
A powerful aspect of using nonlinear regression is the useful statistics that can be generat-
ed. The statistics presented here can be used diagnostically to measure the amount of information 
provided by the data and to identify model error (bias), or to infer the uncertainty with which values 
are calculated. The statistics also can be used to determine what aspects of the model are important 
to predictions of interest. Difficulties common to nonlinear regression make diagnostic statistics 
invaluable to its success, and the diagnostic use of statistics is stressed in this work. 
The sections below show how the statistics are calculated. Use of the statistics during re-
gression is discussed in the following section "Guidelines for Effective Model Calibration," and 
example figures are provided there as listed in the following section 
“
Example Figures.
”
Statistics for Sensitivity Analysis
Dimesionless Scaled Sensitivities and Composite Scaled Sensitivities
When a diagonal weight matrix is used, the scaled sensitivities, 
are calculated as in 
Hill (1992):
(8)
where
is the simulated value associated with the ith observation;
is the jth estimated parameter;
is the sensitivity of the simulated value associated with the ith observation with respect to the 
ss
ij
ss
ij
b
j
∂
∂
y’
i






b
j
ω
ii
1 2
⁄
=
y’
i
b
j
b
j
∂
∂
y’
i

---

15
jth parameter, and is evaluated at ;
is a vector which contains the parameter values at which the sensitivities are evaluated. Because 
the problem is nonlinear with respect to many parameters of interest, the value of a sen-
sitivity will be different for different values in ; and
is the weight of the ith observation.
Similar scaling was employed by Harvey and others (1996). These scaled sensitivities are dimen-
sionless quantities that can be used to compare the importance of different observations to the es-
timation of a single parameter or the importance of different parameters to the calculation of a 
simulated value. In both cases, greater absolute values are associated with greater importance.
For the full weight matrix that can be used in MODFLOWP, the two indices on the weight 
matrix need to be different, and scaled sensitivities are calculated as:
(9)
Composite scaled sensitivities are calculated for each parameter using the scaled sensitivi-
ties for all observations, and indicate the total amount of information provided by the observations 
for the estimation of one parameter. The composite scaled sensitivity for the jth parameter, 
, 
is calculated as (Hill, 1992; Anderman and others, 1996; Hill and others, 1998):
(10)
where 
is the number of observations being used in the regression and the quantity in paren-
theses equals the scaled sensitivities of equation 8 or 9. The composite scaled sensitivity was de-
rived from a similar statistic used by R.L. Cooley (U.S. Geological Survey, written commun., 
1988), equals a scaled version of the square root of the diagonal of the Fisher information matrix 
(X
T
ω
X), and is similar in form and function to the CTB statistic of Sun and Yeh (1990), but is 
scaled differently. The composite scaled sensitivity is independent of the observed values and, 
therefore, model fit.