Newey-West Covariance Matrix (Part 1)

Univariate Case

A well behaved random variable.

Let $X$ represent a time-varying variable that doesn’t exhibit heteroskedasticity nor autocorrelation:

$$\begin{align} Var(\bar X) &= Var\left[\frac{1}{T}(X_1 + X_2 + \cdots + X_T \right] \\ &= \frac{1}{T^2}Var(X_1 + X_2 + \cdots + X_T) \\ &= \frac{1}{T^2}\sum_{i=1}^TVar(X) \\ &= \frac{Var(X)}{T} \end{align}$$

Thus,

$$\begin{align} Var(\bar X) = \frac{1}{T}Var(X) = \frac{1}{T}\sum_{i=1}^T\frac{(X_i-\bar X)^2}{T} \end{align}$$

A random variable that is heteroskedastic and autocorrelated.

If $x$ exhibits heteroskedasticity and autocorrealtion, then a more appropriate approximation is:

$$\begin{align} Var(\bar X) &= Var \left[ \frac{1}{T}(X_1 + X_2 + \cdots + X_T \right] \\ &=\frac{1}{T^2} Var(X_1 + X_2 + \cdots + X_T) \\ &= \frac{1}{T^2} \sum_{i=1}^T \sum_{j=1}^T Cov(X_i, X_j) \end{align}$$

We can break this expression into what should be reminiscent of “rows”.

$$\begin{align} Var(\bar X) = \frac{1}{T^2}\huge[\ & \underbrace{Cov(X_1, X_1 + X_2 + \cdots +X_T)}_\text{Row 1} \\ & \underbrace{Cov(X_2, X_1 + X_2 + \cdots +X_T)}_\text{Row 2} \\ & \qquad \qquad \qquad \vdots \\ & \underbrace{Cov(X_N, X_1 + X_2 + \cdots +X_T)}_\text{Row N}\ \huge] \end{align}$$

Imposing an assumption

Let’s assume that as time between observations increases, they become less correlated. Precisely, let’s assume that when the time between observations is greater than $q$ that covariance goes to zero. In math, $|t-s|> q \implies Cov(x_t, x_s) = 0$.

Imposing this assumption:

$$\begin{align} Var(\bar X) = \frac{1}{T^2}\huge[\ & \underbrace{\sum_{t=1}^T Var(X_t)}_\text{Diagonal} \\ & \underbrace{\sum_{t=2}^T Cov(X_t, X_{t-1}) + Cov(X_t, X_{t+1})}_\text{Off-1 Diagonal} \\ & \qquad \qquad \qquad \vdots \\ & \underbrace{\sum^T_{t=q+1} Cov(X_t, X_{t-q}) + Cov(X_t, X_{t+q})}_\text{Off-q Diagonal}\ \huge] \end{align}$$

Notice that if we only have one observation at each point in time for a random variable then,

$$\begin{align} Cov(X_t, X_{t-q}) \approx \widehat{Cov}(X_t, X_{t-q}) = (X_t - \bar X)(X_{t-q}-\bar X) \end{align}$$

What happened to dividing by $T$? In this case, we’re technically summing over a set containing a single variable so we’re dividing by $1$.

Though our long formula for $Var(\bar X)$ is easy to visualize with the matrix analogy, we don’t have a concise way of writing this down yet. We define a couple variables to help us rewrite our verbose formula in a more compact form.

$$\begin{align} \hat \gamma_v = (1/T) \sum_{t=v+1}^T (X_t - \bar X)(X_{t-v} - \bar X) \end{align}$$

See that $v$ determines how far off the diagonal we’re summing up the elements, and that the numbers from $v+1$ to $T$ count the elements. This formula would get one side of the off-diagonal while the following formula will get us the other side.

$$\begin{align} \hat \gamma_{-v} = (1/T) \sum_{t=v+1}^T (X_{t-v} - \bar X)(X_t - \bar X) \end{align}$$

Note that the special case $v=0 \iff \hat \gamma_0$ will sum up all the diagonal elements (the variance terms).

Using our large expression for $Var(\bar X)$ and our definition for $\hat \gamma$ we can finally rewrite $Var(\bar X)$.

$$\begin{align} \widehat {Var}(\bar X) = (1/T) \left[\hat \gamma_0 + \sum_{v=1}^q (\hat\gamma_{v} + \hat \gamma_{-v})\right] \end{align}$$

Correcting for undesirable small sample problems

Though our formula for $\hat \sigma_{\bar X}^2$ is consistent (converges to the truth as $n \to \infty$), it is not always positive for small sample sizes. The following formula preserves the asymptotic properties and allows variance to always be positive

$$\begin{align} \hat \sigma_{\bar X}^2 = (1/T)\left(\hat \gamma_0 + \sum_{v=1}^q \left[ 1 - \frac{v}{q+1}\right](\hat \gamma_v + \hat \gamma_{-v})\right) \end{align}$$