Natural conjugate priors for the instrumental variables regression model applied to the Angrist–Krueger data
Introduction
Endogeneity is a fundamental property of many economic series which explains the long tradition of analysing the instrumental variables (IV) regression model. The founding articles on the econometric analysis of the IV regression model therefore belong to some of the earliest contributions to the econometrics literature. Given the prevalence of maximum likelihood in the statistics literature in the 1930s and 1940s, maximum likelihood was the first inferential procedure that was developed for a statistical analysis of the IV regression model, see Anderson and Rubin (1949) and Hood and Koopmans (1953). Later Theil (1953) and Basmann (1957) added two stage least squares (2SLS) as another inferential procedure for the IV regression model. The initial contributions to the literature on Bayesian analysis of the IV regression model considerably lagged the previously referred to classical ones. Drèze (1976) initialized the Bayesian analysis of the IV regression model. For a long time, the literature on Bayesian analysis of the IV regression model was primarily concerned with the numerical computation of integrals, see e.g. Kloek and Van Dijk (1978), Bauwens (1984), Steel (1991), Geweke (1996) and Bauwens and Van Dijk (1989). Inspired by the resurgence of the discussion about classical inference in the IV regression model due to the appearance of ‘weak instruments’, see e.g. Nelson and Startz (1990) and Staiger and Stock (1997), a further development of the Bayesian analysis of the IV regression model studied identification and prior specification. Because the priors specified in the earlier Bayesian literature did not incorporate the local non-identification of the structural parameter, pathologies in the resulting marginal posteriors arise, see e.g. Kleibergen and Van Dijk (1998) and Chao and Phillips (1998). This is not the only peculiar property of these marginal posteriors since the tails of the posterior of the structural parameter also become thinner when (possibly superfluous) instruments are added to the model, see e.g. Maddala (1976) and Kleibergen and Zivot (2003). These properties made Kleibergen and Zivot (2003) conclude that the earlier Bayesian analyses of the IV regression model had more in common with 2SLS than with limited information maximum likelihood (LIML). Thus, Kleibergen and Zivot (2003) state, following Chao and Phillips, 1998, Chao and Phillips, 2002 for the exact identified IV case, that the Jeffreys prior is the Bayesian counterpart of LIML since it leads to a marginal posterior of the structural parameters that has the same functional expression as the sampling density of the LIML estimator.
Drèze and Richard (1983) construct a class of informative priors for usage in the IV regression model. These priors are based on Drèze (1976) and thus also lead to the previously mentioned pathologies in the marginal posteriors. We therefore propose a class of natural conjugate priors that lead to the Jeffreys prior for a specific setting of the prior parameters. The priors are based on the property of the IV regression model that it results from a reduced rank restriction on the parameter matrix of the encompassing unrestricted reduced form (URF). We specify a natural conjugate prior on the parameters of the URF and impose rank reduction on its parameter matrix. The rank reduction restriction satisfies an invariance principle to avoid statistical paradoxes, see Kleibergen (2004). A straightforward algorithm allows us to compute the resulting marginal prior for the structural parameter in the common case of one endogenous variable. By computing the marginal prior of the structural parameter, we can specify the prior parameters in such a manner that they adequately reflect our a priori information. The same algorithm can be used to compute the marginal posterior of the structural parameter since it has the same functional form as the prior. Hence, the appellation of the prior as a natural conjugate one. The functional expressions of the marginal prior and posterior of the structural parameters also directly reveal the update from prior to posterior.
We use our natural conjugate priors to analyse the data from Angrist and Krueger (1991). For a specific setting of the model used by Angrist and Krueger (1991) that uses both the interactions of year-of-birth and quarter-of-birth and state-of-birth and quarter-of-birth as instruments, we show that the posterior results on the return to education using the Jeffreys prior for the US are completely determined by those from the Southern region. The marginal posterior of the return on education in the US is almost an overlay of the marginal posterior in the Southern region when we use the Jeffreys prior. This results since the Jeffreys prior, identical to LIML, focusses on the strongest available instruments. The marginal posteriors for the other regions, the Northeast, Midwest and West, are rather distinct from that of the South and have little with it in common. We therefore compute the natural conjugate prior needed for the other regions to make their posteriors coincide with that from the South which turn out to be rather informative.
The paper is organized as follows. In the second section, we briefly discuss the IV regression model. The third section discusses the priors advocated by Drèze (1976) and the Jeffreys prior. The fourth section discusses the natural conjugate prior for the IV regression model. It first briefly discusses the natural conjugate prior for the linear regression model and then proceeds with discussing the reduced rank restriction, the construction of the (marginal) prior and an algorithm to compute the marginal prior of the structural parameter. The fifth section updates the prior with the likelihood to obtain the posterior. The sixth section applies the natural conjugate priors to the Angrist and Krueger (1991) data. Finally, the seventh section concludes.
Throughout the paper we use the notation: for the column vectorization of the matrix A such that for , , is the identity matrix, and for a full rank dimensional matrix X, is the expectation operator.
Section snippets
Instrumental variables regression model
The structural form (SF) of the linear IV regression model can be represented as a limited information simultaneous equations model, see e.g. Hausman (1983),The vector y contains observations on the endogenous variable that is to be explained by the IV model; the matrix contains the explanatory endogenous variables; Z is a matrix of instruments. The vector consists of structural errors and is a matrix of reduced form errors with The
Drèze's (1976) approach
In one of the earliest Bayesian analyses of the IV regression model, Drèze (1976) specifies the diffuse prior:where the subscript SF denotes that the prior is on the parameters of the structural form. The primary motivation of Drèze's approach is that it has an invariance property such that the prior on the SF implies the same kind of prior on the parameters of the RRF (which is proportional to ).
The marginal posteriors of and resulting
Specification of informative priors
The diffuse prior advocated by Drèze (1976) has been extended to incorporate prior information, see Drèze and Richard (1983). Drèze and Richard (1983) refer to the resulting class of priors as extended natural-conjugate priors.
In linear regression models, diffuse priors lead to marginal posteriors of the regression parameters that have similar functional expressions as the sampling density of the maximum likelihood or least squares estimator. The extensions of these diffuse priors that allow
Posteriors
The assumption of independent normal disturbances implies that the posterior results from the natural conjugate prior (23) in a straightforward manner. Corollary 3 When we specify the natural conjugate prior (23) on , the posterior of readswith , .
For the previously discussed standard specifications of
Data and model
We illustrate the prior specification framework using data from Angrist and Krueger (1991). Angrist and Krueger (1991) analyse the return on education by regressing the (logarithm of) income on the education spell and some additional control variables. Because of the endogeneity of both the education spell and income, Angrist and Krueger use instruments that are obtained from the quarter of birth. It is hard to find instruments that are correlated with education but uncorrelated with unobserved
Conclusions
Using the property that the RRF of the IV regression model results from a reduced rank restriction on the parameter matrix of the URF, we construct a natural conjugate prior for the parameters of the IV regression model. The prior is proportional to a natural conjugate prior for the parameters of the URF with the reduced rank restriction imposed on them. In the case of one included endogenous variable, we provide a straightforward algorithm for computing the marginal posterior of the structural
Acknowledgements
We thank two anonymous referees for helpful comments and suggestions. The second author's research documented in this article has been partially funded by the NWO research grant “Empirical Comparison of Economic Models”.
References (34)
- et al.
Posterior distributions in limited information analysis of the simultaneous equations models using the Jeffreys’ prior
Journal of Econometrics
(1998) - et al.
Jeffreys prior analysis of the simultaneous equations model in the case with endogenous variables
Journal of Econometrics
(2002) - et al.
Bayesian analysis of simultaneous equations systems
Bayesian reduced rank regression in econometrics
Journal of Econometrics
(1996)Invariant Bayesian inference in regression models that is robust against the Jeffreys–Lindleys paradox
Journal of Econometrics
(2004)- et al.
Priors, posteriors and Bayes factors for a Bayesian analysis of cointegration
Journal of Econometrics
(2002) - et al.
Bayesian and classical approaches to instrumental variable regression
Journal of Econometrics
(2003) Exact small sample theory in the simultaneous equations model
- et al.
Bayesian specification analysis and estimation of simultaneous equation models using Monte Carlo methods
Journal of Econometrics
(1988) - et al.
Estimators of the parameters of a single equation in a complete set of stochastic equations
Annals of Mathematical Statistics
(1949)