COVARIANCE NOTATION
An easier method for performing calculations with covariances (as compared to summation notation) is as follows. Let X, Y, W, and Z be random variables. Let α and β be parameters.
ERROR COVARIANCE MATRICES
An error covariance matrix shows the structure of covariances among different error terms. By definition, error covariance matrices are symmetric (the number in the ith row and jth column is the same as the number in the jth row and ith column). The number, or “element”, in the ith row and jth column is the covariance between the ith and jth errors.
Let a simple time series data set be comprised of T time periods. Let ut be the error term at time t. The error covariance matrix, Ω, is a T x T matrix that takes the form:
OLS Assumptions
Under the standard OLS assumptions, the covariance of error terms across time periods is zero and the variance of the error term at a point in time is constant across time. This means that:
This implies that the error covariance matrix takes the form:
Heteroskedasticity
In the presence of heteroskedasticity, the variance of the error term is no longer constant across time. This means that:
This implies that the error covariance matrix takes the form:
Serial Correlation
In the presence of serial correlation, the covariance of the error term across different time periods is no longer zero. The form (AR or MA) and order (1, 2, etc.) of the serial correlation describes the pattern of the covariances.
Serial Correlation: AR(1) error
AR(1) errors take the form where . This implies that:
and that the error covariance matrix takes the form:
Serial Correlation: AR(2) error
AR(2) errors take the form where . This implies that:
and that the error covariance matrix takes the form:
Serial Correlation: MA(1) error
MA(1) errors take the form where . This implies that:
(note that an MA(1) process is the same as an AR(∞) process). The error covariance matrix takes the form:
Combined Heteroskedasticity and AR(1) Serial Correlation
In the presence of heteroskedasticity and an order 1 autoregressive error, we have: where .
The error covariance matrix takes the form:
Panel Data
Let a panel data set be comprised of S cross-sections and T time periods where the data is arranged first by cross-section then by time period. The error covariance matrix, Ω, is an ST x ST matrix that takes the form:
where each submatrix is a T x T matrix of error covariances for cross-sections s1 and s2 across all combinations of time periods. For example, let us,t be the error term associated with cross-section s at time t. The submatrix is constructed as:
Expanding Ω, we have:
Note that there are two possible types of heteroskedasticity: within individuals and between individuals. Within individuals heteroskedasticity manifests as non-constant diagonal elements of Ω. Between individuals heteroskedasticity manifests as non-constant diagonal elements of the Σ. For each type of heteroskedasticity, the heteroskedasticity can be common or individual specific. Common heteroskedasticity is heteroskedasticity that is the same for all individuals. For example:
Notice that common within-individuals heteroskedasticity is identical to heteroskedasticity as seen in a time series data set with the exception that the same heteroskedasticity structure is repeated for each individual.
Similarly, serial correlation can occur within-individuals or between-individuals and can be common or individual specific.