Methods and guidelines for effective model calibration

Modified Gauss-Newton Optimization
The Gauss-Newton optimization method is an iterative form of standard linear regression, 
and works well only if modified by the addition of, for example, a damping parameter and a Mar-
quardt parameter, as described below. The modified Gauss-Newton method presented here closely 
follows that of Cooley and Naff (1990, ch. 3), which is similar to methods presented by Seber and 
Wild (1989), and other texts on nonlinear regression.
Normal Equations and the Marquardt Parameter
Parameter values that minimize the objective function are calculated using normal equa-
tions. One of the differences between linear regression and nonlinear regression is that in linear re-
gression parameter values are estimated by solving the normal equations once, while nonlinear 
regression is iterative in that a sequence of parameter updates is calculated, solving linearized nor-
y
y’ b
( )
–
[
]
T
ω
y
y’ b
( )
–
[
]
e
T
ω
e
=
ω
y
y’ b
( )
e
ω
1 2
⁄
y y’ b
( )
–
[
]
ω
y
y’
–
(
)
T
ω
y
y’
–
(
)
+
ω

---

8
mal equations once for each update. Thus, in nonlinear regression there are parameter-estimation 
iterations. The normal equations and the iterative process for the modified Gauss-Newton optimi-
zation method used in UCODE and MODFLOWP can be expressed as:
(C
T
X
T
r
ω
X
r
C + Im
r
)C
-1
d
r
= C
T
X
T
r
ω
(y - y’(b
r
))
(4a)
b
r+1 
= 
ρ
r
d
r 
+ b
r 
(4b)
where 
r is the parameter-estimation iteration number; 
X
r
is the sensitivity matrix evaluated at parameter estimates b
r
, with elements equal to 
(calcu-
lated by the sensitivity equation method in MODFLOWP and using forward or central 
differences in UCODE); 
is the weight matrix (can be a full matrix in MODFLOWP); 
(X
T
ω
X)is a symmetric, square matrix of dimension NP by NP that is an estimate of the Fisher in-
formation matrix, and which is used to calculate statistics described in the section "Pa-
rameter Statistics";
C is a diagonal scaling matrix with element c
jj
equal to [(X
T
ω
X)
jj
]
-1/2
,
which produces a scaled 
matrix with the smallest possible condition number (Forsythe and Strauss, 1955; Hill, 
1990); 
d
r 
is a vector with the number of elements equal to the number of estimated parameters. It is used 
in eq. 4b to update the parameter estimates; 
I is an NP dimensional identity matrix;
m
r 
is the Marquardt parameter (Marquardt, 1963); and 
ρ
r
is a damping parameter. 
Figure 1C shows the paths that this modified Gauss-Newton method followed from two sets of
starting parameter values to the minimum of the objective-function surface of the simple example 
problem.
A quasi-Newton term can be added to the matrix on the left-hand side of equation 4a, as 
described in Appendix B, to aid convergence of the modified Gauss-Newton equations in some cir-
cumstances. The modified Gauss-Newton method used in this work also could be termed a Leven-
berg-Marquardt method.
The Marquardt parameter is used to improve regression performance for ill-posed problems 
(Theil, 1963; Seber and Wild, 1989). Initially m
r
=0 for each parameter-estimation iteration r. For