Panel data

Analysis

A panel has the form

${\displaystyle X_{it},i=1,\dots ,N,t=1,\dots ,T,}$

where ${\displaystyle i}$ is the individual dimension and ${\displaystyle t}$ is the time dimension. A general panel data regression model is written as ${\displaystyle y_{it}=\alpha +{\beta }^{\prime }{X}_{it}+u_{it}}$. Different assumptions can be made on the precise structure of this general model. Two important models are the fixed effects model and the random effects model.

Consider a generic panel data model:

${\displaystyle y_{it}=\alpha +{\beta }^{\prime }{X}_{it}+u_{it},}$

${\displaystyle u_{it}={\mu }_i+v_{it}.}$

${\displaystyle {\mu }_i}$ are individual-specific, time-invariant effects (e.g., in a panel of countries this could include geography, climate, etc.) which are fixed over time, whereas ${\displaystyle v_{it}}$ is a time-varying random component.

If ${\displaystyle {\mu }_i}$ is unobserved, and correlated with at least one of the independent variables, then it will cause omitted variable bias in a standard OLS regression. However, panel data methods, such as the fixed effects estimator or alternatively, the first-difference estimator can be used to control for it.

If ${\displaystyle {\mu }_i}$ is not correlated with any of the independent variables, ordinary least squares linear regression methods can be used to yield unbiased and consistent estimates of the regression parameters. However, because ${\displaystyle {\mu }_i}$ is fixed over time, it will induce serial correlation in the error term of the regression. This means that more efficient estimation techniques are available. Random effects is one such method: it is a special case of feasible generalized least squares which controls for the structure of the serial correlation induced by ${\displaystyle {\mu }_i}$.

Dynamic panel data

Dynamic panel data describes the case where a lag of the dependent variable is used as regressor:

${\displaystyle y_{it}=\alpha +{\beta }^{\prime }{X}_{it}+\gamma y_{it-1}+u_{it}.}$

The presence of the lagged dependent variable violates strict exogeneity, that is, endogeneity may occur. The fixed effect estimator and the first differences estimator both rely on the assumption of strict exogeneity. Hence, if ${\displaystyle u_i}$ is believed to be correlated with one of the independent variables, an alternative estimation technique must be used. Instrumental variables or GMM techniques are commonly used in this situation, such as the Arellano–Bond estimator. While estimating this we should have the proper information about the instrumental variables.

Data sets which have a panel design

• German Socio-Economic Panel (SOEP)
• Household, Income and Labour Dynamics in Australia Survey (HILDA)
• British Household Panel Survey (BHPS)
• Survey of Income and Program Participation (SIPP)
• Lifelong Labour Market Database (LLMDB)
• Panel Study of Income Dynamics (PSID)
• China Family Panel Studies (CFPS)
• National Longitudinal Surveys (NLSY)
• Labour Force Survey (LFS)

Data sets which have a multi-dimensional panel design