The McGraw-Hill Series Economics essentials of economics brue, McConnell, and Flynn Essentials of Economics

324
Part Two
Relaxing the Assumptions of the Classical Model
10.2
Estimation in the Presence of Perfect Multicollinearity
It was stated previously that in the case of perfect multicollinearity the regression coeffi-
cients remain indeterminate and their standard errors are infinite. This fact can be demon-
strated readily in terms of the three-variable regression model. Using the deviation form,
where all the variables are expressed as deviations from their sample means, we can write
the three-variable regression model as
y
i
= ˆ
β
2
x
2
i
+ ˆ
β
3
x
3
i
+ ˆ
u
i
(10.2.1)
Now from Chapter 7 we obtain
ˆ
β
2
=
y
i
x
2
i
x
2
3
i
−
y
i
x
3
i
x
2
i
x
3
i
x
2
2
i
x
2
3
i
−
x
2
i
x
3
i
2
(7.4.7)
ˆ
β
3
=
y
i
x
3
i
x
2
2
i
−
y
i
x
2
i
x
2
i
x
3
i
x
2
2
i
x
2
3
i
−
x
2
i
x
3
i
2
(7.4.8)
Assume that 
X
3
i
=
λ
X
2
i
, where 
λ
is a nonzero constant (e.g., 2, 4, 1.8, etc.). Substituting
this into Eq. (7.4.7), we obtain
ˆ
β
2
=
y
i
x
2
i
λ
2
x
2
2
i
−
λ
y
i
x
2
i
λ
x
2
2
i
x
2
2
i
λ
2
x
2
2
i
−
λ
2
x
2
2
i
2
=
0
0
(10.2.2)
which is an indeterminate expression. The reader can verify that 
ˆ
β
3
is also indeterminate.
8
Why do we obtain the result shown in Eq. (10.2.2)? Recall the meaning of 
ˆ
β
2
: It gives
the rate of change in the average value of 
Y
as 
X
2
changes by a unit, holding 
X
3
constant.
But if 
X
3
and 
X
2
are perfectly collinear, there is no way 
X
3
can be kept constant: As 
X
2
changes, so does 
X
3
by the factor 
λ
. What it means, then, is that there is no way of disen-
tangling the separate influences of 
X
2
and 
X
3
from the given sample: For practical purposes
X
2
and 
X
3
are indistinguishable. In applied econometrics this problem is most damaging
since the entire intent is to separate the partial effects of each 
X
upon the dependent
variable.
To see this differently, let us substitute 
X
3
i
=
λ
X
2
i
into Eq. (10.2.1) and obtain the
following [see also Eq. (7.1.12)]:
y
i
= ˆ
β
2
x
2
i
+ ˆ
β
3
(
λ
x
2
i
)
+ ˆ
u
i
=
(
ˆ
β
2
+
λ
ˆ
β
3
)
x
2
i
+ ˆ
u
i
(10.2.3)
= ˆ
α
x
2
i
+ ˆ
u
i
where
ˆ
α
=
(
ˆ
β
2
+
λ
ˆ
β
3
)
(10.2.4)
8
Another way of seeing this is as follows: By definition, the coefficient of correlation between 
X
2
and 
X
3
,
r
2 3
, is 
x
2
i
x
3
i
/
x
2
2
i
x
2
3
i
. If 
r
2
2 3
=
1, i.e., perfect collinearity between 
X
2
and 
X
3
, the denominator of
Eq. (7.4.7) will be zero, making estimation of 
β
2
(or of 
β
3
) impossible.
guj75772_ch10.qxd 12/08/2008 02:44 PM Page 324

---

Chapter 10
Multicollinearity: What Happens If the Regressors Are Correlated?
325
Applying the usual OLS formula to Eq. (10.2.3), we get
ˆ
α
=
(
ˆ
β
2
+
λ
ˆ
β
3
)
=
x
2
i
y
i
x
2
2
i
(10.2.5)
Therefore, although we can estimate
α
uniquely, there is no way to estimate
β
2
and
β
3
uniquely;
mathematically
ˆ
α
= ˆ
β
2
+
λ
ˆ
β
3
(10.2.6)
gives us only one equation in two unknowns (note 
λ
is given) and there is an infinity of
solutions to Eq. (10.2.6) for given values of 
ˆ
α
and 
λ
. To put this idea in concrete terms, let
ˆ
α
=
0
.
8 and 
λ
=
2
.
Then we have
0
.
8
= ˆ
β
2
+
2
ˆ
β
3
(10.2.7)
or
ˆ
β
2
=
0
.
8
−
2
ˆ
β
3
(10.2.8)
Now choose a value of 
ˆ
β
3
arbitrarily, and we will have a solution for 
ˆ
β
2
. Choose another
value for 
ˆ
β
3
, and we will have another solution for 
ˆ
β
2
. No matter how hard we try, there is
no unique value for 
ˆ
β
2
.
The upshot of the preceding discussion is that in the case of perfect multicollinearity one
cannot get a unique solution for the individual regression coefficients. But notice that one
can get a unique solution for linear combinations of these coefficients. The linear combi-
nation (
β
2
+
λβ
3
) is uniquely estimated by 
α
, given the value of 
λ
.
9
In passing, note that in the case of perfect multicollinearity the variances and standard
errors of 
ˆ
β
2
and 
ˆ
β
3
individually are infinite. (See Exercise 10.21.)
10.3
Estimation in the Presence of “High”
but “Imperfect” Multicollinearity
The perfect multicollinearity situation is a pathological extreme. Generally, there is no
exact linear relationship among the 
X
variables, especially in data involving economic time
series. Thus, turning to the three-variable model in the deviation form given in Eq. (10.2.1),
instead of exact multicollinearity, we may have
x
3
i
=
λ
x
2
i
+
v
i
(10.3.1)
where 
λ
=
0 and where 
v
i
is a stochastic error term such that 
x
2
i
v
i
=
0. (Why?)
Incidentally, the Ballentines shown in Figure 10.1
b
to 10.1
e
represent cases of imperfect
collinearity.
In this case, estimation of regression coefficients 
β
2
and 
β
3
may be possible. For exam-
ple, substituting Eq. (10.3.1) into Eq. (7.4.7), we obtain
ˆ
β
2
=
(
y
i
x
2
i
)
λ
2
x
2
2
i
+
v
2
i
−
λ
y
i
x
2
i
+
y
i
v
i
λ
x
2
2
i
x
2
2
i
λ
2
x
2
2
i
+
v
2
i
−
λ
x
2
2
i
2
(10.3.2)
where use is made of 
x
2
i
v
i
=
0
.
A similar expression can be derived for 
ˆ
β
3
.
9
In econometric literature, a function such as (
β
2
+
λβ
3
) is known as an
estimable function.
guj75772_ch10.qxd 12/08/2008 02:44 PM Page 325

---

326
Part Two
Relaxing the Assumptions of the Classical Model
Now, unlike Eq. (10.2.2), there is no reason to believe a priori that Eq. (10.3.2) cannot
be estimated. Of course, if 
v
i
is sufficiently small, say, very close to zero, Eq. (10.3.1) will
indicate almost perfect collinearity and we shall be back to the indeterminate case of 
Eq. (10.2.2).
10.4
Multicollinearity: Much Ado about Nothing? 
Theoretical Consequences of Multicollinearity
Recall that if the assumptions of the classical model are satisfied, the OLS estimators of the
regression estimators are BLUE (or BUE, if the normality assumption is added). Now it
can be shown that even if multicollinearity is very high, as in the case of 
near multi-
collinearity,
the OLS estimators still retain the property of BLUE.
10
Then what is the mul-
ticollinearity fuss all about? As Christopher Achen remarks (note also the Leamer quote at
the beginning of this chapter):
Beginning students of methodology occasionally worry that their independent variables are
correlated—the so-called multicollinearity problem. But multicollinearity violates no regres-
sion assumptions. Unbiased, consistent estimates will occur, and their standard errors will be
correctly estimated. The only effect of multicollinearity is to make it hard to get coefficient
estimates with small standard error. But having a small number of observations also has that
effect, as does having independent variables with small variances. (In fact, at a theoretical level,
multicollinearity, few observations and small variances on the independent variables are essen-
tially all the same problem.) Thus “What should I do about multicollinearity?” is a question like
“What should I do if I don’t have many observations?” No statistical answer can be given.
11
To drive home the importance of sample size, Goldberger coined the term