3.1 Random walk

The simplest model of a variable with a stochastic trend is the random walk, which depends on past values of itself and Gaussian white noise errors,

\[\begin{eqnarray} y_{t}=y_{t-1}+\varepsilon_{t} \;\;\; \text{where } \; \varepsilon_{t}\sim \mathsf{i.i.d.} \mathcal{N}\left(0,\sigma^{2}\right) \tag{3.1} \end{eqnarray}\]

These random walk processes have a number of interesting features, where the best forecast of \(y_{t+1}\) at time \(t\), is given by

\[\begin{eqnarray}\nonumber \mathbb{E}\left[y_{t+1}|\;\;y_{t}\right] =y_{t} \end{eqnarray}\]

The mean squared forecast error (MSFE) of this process, which describes the forecast error variance, would grow with the forecast horizon,

\[\begin{eqnarray}\nonumber \mathbb{E}_t \left[ \sigma^f_{t+h} \right] = \mathsf{var}\left( y_{t+1}-\mathbb{E}\left[ y_{t+1}| y_{t}\right] \right) =\sigma^{2}h \end{eqnarray}\]

Note that in this instance, the forecasting horizon may be used to denote the progression of time, where the difference between a two and a one step-ahead forecast is one period of time. With the aid of this expression, we would suggest that the variance depends on time, which would imply that it is nonstationary. This may be confirmed with the aid of a recursive substitution exercise.

For the random walk model, \(y_{t}=y_{t-1}+\varepsilon_{t}\), we may use substitute recursive lag values of \(y_t\) to describe the evolution of the process,

\[\begin{eqnarray}\nonumber y_{t} & =& y_{t-1}+\varepsilon_{t}\\ \nonumber & =& y_{t-2}+\varepsilon_{t-1}+\varepsilon_{t}\\ \nonumber & =& y_{t-3}+\varepsilon_{t-2}+\varepsilon_{t-1}+\varepsilon_{t}\\ \nonumber & \vdots & \\ \nonumber y_{t} & =& \overset{t-1}{\underset{j=0}{\sum}}\varepsilon_{t-j}+y_{0} \end{eqnarray}\]

Therefore each shock, \(\varepsilon_{t-j}\), will influence subsequent values of \(y_{t}\). This would imply that a shock to a random walk has a permanent effect on the time series variable. Alternatively, we may infer that these variables have infinite memory. If we assume that \(y_{0}\) is equal to zero, without any loss of generality, we can write the random walk as,

\[\begin{eqnarray} \nonumber y_{t}=\overset{t-1}{\underset{j=0}{\sum}}\varepsilon_{t-j} \end{eqnarray}\]

This would allow us to write the mean and variance of the random walk as

\[\begin{eqnarray}\nonumber \mathbb{E}\left[y_{t}\right]=0 \;\;\; \text{and } \;\; \mathsf{var}\left( y_{t}\right) =\sigma^{2}t \end{eqnarray}\]

The covariance, \(\gamma_{t-j}\), between \(y_t\) and \(y_{t-j}\) with \(y_0=0\), would then yield the following result,

\[\begin{eqnarray} \nonumber \mathbb{E}\big[(y_t - y_0)(y_{t-j}-y_0)\big] & = & \mathbb{E} \big[(\varepsilon_t + \varepsilon_{t-1} + \ldots + \varepsilon_1) \ldots \\ \nonumber & & (\varepsilon_{t-j} + \varepsilon_{t-j-1} + \ldots + \varepsilon_{1})\big]\\ \nonumber & = & \mathbb{E}\big[(\varepsilon_{t-j})^2 + (\varepsilon_{t-j-1})^2 + \ldots + (\varepsilon_1)^2\big]\\ \nonumber & = & (t-j)\sigma^2 \end{eqnarray}\]

which also depends on time. Hence, since the variance and covariance of the process depend on time, the random walk is certainly nonstationary. With such a process, the effect of a change in the error term in \(t-j\) will continue to effect \(y_{t}\), and the roots of the linear difference equation would contain a unitary element, which infers that such a process is a unit root.

These processes are also termed difference-stationary (DS), as the first difference of the random walk would may be expressed as,

\[\begin{eqnarray}\nonumber y_{t} & =&y_{t-1} + \varepsilon_{t}\\ \nonumber \Delta\ y_{t} & =&\varepsilon_{t} \end{eqnarray}\]

where \(\Delta y_{t}\) clearly is stationary as the expected mean and variance of the white noise error are stationary. If a variable, \(y_{t}\), could be made stationary after differencing it once, it is integrated of the first order. We use the notation \(I(1)\), to describe such a process. Stationary random variables, such as \(\Delta y_{t}\) are thus integrated of order zero (i.e. \(\Delta y_{t}\) is \(I(0)\)). If it is necessary to take the second difference to achieve stationarity, where \(\Delta^2 y_t\) is \(I(0)\). Such a process is integrated of the second order, where we would use the notation, \(I(2)\).

3.2 Random walk with drift

Adding a constant term to the random walk model in equation (3.1) results in a random walk with drift, which may be expressed as,

\[\begin{eqnarray}\nonumber y_{t}= \mu + y_{t-1}+\varepsilon_{t} \end{eqnarray}\]

Using recursive substitution we can show that the random walk with drift can be written as a function of a deterministic trend and stochastic term,

\[\begin{eqnarray} \nonumber y_{t} & =&\mu+y_{t-1}+\varepsilon_{t}\\ & =&\mu+(y_{t-2}+\mu+\varepsilon_{t-1})+\varepsilon_{t}\nonumber\\ & =&2\mu+(y_{t-3}+\mu+\varepsilon_{t-2})+\varepsilon_{t-1}+\varepsilon_{t}\nonumber\\ & \vdots & \nonumber\\ \ y_{t} & =&\mu \cdot t+\overset{t-1}{\underset{j=0}{\sum}}\varepsilon_{t-j} \tag{3.2} \end{eqnarray}\]

where we again assume that the starting value, \(y_{0}\), is equal to zero. In contrast with the random walk model in equation (3.1), a random walk with drift now also contains a deterministic trend, which results from the inclusion of the constant term, \(\mu\), that influences the slope of the of the deterministic trend. However, in contrast with the trend stationary model, the deviations from the deterministic trend are not stationary. This would imply that each \(\varepsilon_{t-j}\) will influence the value of \(y_{t}\), even after removing the deterministic trend from the variable.

4.1 The autocorrelation function

As has been noted previously, the autocorrelation function may be used to describe the persistence in a process. When we are modelling a stationary AR(1) process, the first correlation coefficient, \(\rho_1\), is equivalent to the coefficient in the AR(1) model, \(\phi\). Similarly, the second correlation coefficient, \(\rho_2\), is equivalent to \(\phi^2\).

The subsequent values of the correlation coefficient, \(\rho_j\), may be derived from the more general expression that considers the value of the covariance function, which is divided by the product of the standard deviation of \(y_t\) and the standard deviation of \(y_{t-j}\). Hence, for a random walk the standard deviation of \(y_t\) may be derived from \(\sqrt{\mathsf{var}(y_t)} = \sqrt{ T\sigma^2}\), where \(T\) is the sample size. In addition, the standard deviation of \(y_{t-j}\) may be similarly derived from, \(\sqrt{\mathsf{var}(y_{t-j})} = \sqrt{(T-j)\sigma^2}\). Given these conditions, the general form of the autocorrelation coefficient, for lag \(j\), may then be derived from the following expression.

\[\begin{eqnarray} \nonumber \rho_j & = & (T-j)\sigma^2 / \sqrt{(T-j)\sigma^2} \sqrt{(T)\sigma^2} \\ \nonumber & = & (T-j) / \sqrt{(T-j)T}\\ \nonumber & = & \sqrt{(T-j) / T} \;\;\;\; < 1 \end{eqnarray}\]

In most cases, where the sample size is large, when compared with the value for \(j\), the ratio \((T-j)/T\) is approximately equal to unity. However, it will in all instances be less than \(1\). This is rather unfortunate as it would infer that we are not able to use the autocorrelation function to distinguish between a process that has a unit root and an AR(1) process that is stationary, but has a high degree of persistence. As such, a slowly decaying autocorrelation function indicates that the process has a large characteristic root, where the process may possibly include a unit root, a deterministic trend, or both of these features. In addition, such a slowly decaying autocorrelation function could also suggest that the process is stationary, but somewhat persistent. Furthermore, as we previously noted that the value of \(\rho_1\) is equivalent to the \(\hat{\phi}\) coefficient estimate in the AR(1) model, this would imply that the parameter estimate would be biased, as it will be less than the true value of unity.

Examples of these processes are included in Figure 4, where we note that it would be difficult to use the autocorrelation function to distinguish between the various processes. This exercise should motivate for the use of formal tests that would allow for the identification of a variable that contains a deterministic or a stochastic trend. In addition, we would also need to be able to identify those variables that contain both of these features, as well as those cases where the variable may indeed be stationary.

5.1 Dickey-Fuller test

The most widely used test for a presence of a unit root was originally proposed by Dickey and Fuller (1979), which tests the null hypothesis of whether a series is a random walk against the alternative that it is stationary. To perform this test, we assume that we have an AR(1) process,

\[\begin{eqnarray} \nonumber y_{t}=\phi y_{t-1}+\varepsilon_{t} \;\;\; \text{where } \; \varepsilon_{t}\sim\mathsf{i.i.d.} \mathcal{N}\left(0,\sigma^{2}\right) \end{eqnarray}\]

With the use of this equation, we would want to determine whether \(|\phi|=1\), against the alternative that \(|\phi|<1\). If \(|\phi|=1\) then the above model would represent a random walk process, while if \(|\phi|<1\), the above process is stationary. As noted above, the value of the autocorrelation coefficient, \(\rho_1\) and the estimated value of \(\hat{\phi}\) would be biased towards a value that is less than one, when the underlying data generating process contains a unit root.⁴

To provide a formal test for the presence of a unit root, Dickey and Fuller (1979) make use of the following extension, where \(\pi=\hat{\phi}-1\). Therefore, the test for whether or not \(\phi=1\), would equivalent to the test for whether or not \(\pi=0\). In their model, Dickey and Fuller (1979) make use of the following transformation of the above AR(1) model,

\[\begin{eqnarray} \nonumber y_{t}&=&\phi y_{t-1}+\varepsilon_{t}\\ \nonumber y_{t} - y_{t-1} &=&\phi y_{t-1} - y_{t-1}+\varepsilon_{t}\\ \nonumber \Delta y_{t}&=& (\phi -1) y_{t-1}+\varepsilon_{t}\\ \Delta y_{t}&=&\pi y_{t-1}+\varepsilon_{t} \tag{5.1} \end{eqnarray}\]

Such a statistic would take the form of a traditional \(t\)-test, where we are primarily interested in determining the degree of certainty with which this coefficient has been estimated (i.e. is it significantly different from zero). Thus, using equation (5.1), the test for a unit root would make use of the following null hypothesis:

\[\begin{eqnarray} \nonumber H_{0}\; :\pi=0 \end{eqnarray}\]

If we are unable to reject the null hypothesis, it would imply that \(y_{t}\) is integrated of order one, such that \(y_{t}\sim I(1)\). The alternative hypothesis would then take the form,

\[\begin{eqnarray} \nonumber H_{1}\; :\pi<0 \end{eqnarray}\]

which implies that \(y_{t}\) is stationary, such that \(y_{t}\sim I(0)\). This procedure would imply that we would need to calculate an appropriate value for the \(t\)-statistic that is associated with the \(\pi\) parameter, which considers whether or not this parameter is significantly different from zero. Therefore, the test for the null hypothesis, \(H_{0}\) may be expressed as,

\[\begin{eqnarray} \nonumber \hat{t}_{DF}=\frac{\hat {\pi}}{SE\left(\hat{\pi}\right)} =\frac{{\phi}-1}{SE\left({\phi}\right)} \end{eqnarray}\]

where \(SE\) denotes the standard error that is associated with the coefficient estimate. The Dickey-Fuller test is a one-sided test, since the relative alternative to the null hypothesis is that \(y_{t}\) is stationary (i.e. \(\phi \ne 1\)).⁵ Note however, that the asymptotic distribution for this \(t\)-statistic is non-Gaussian, owing to the possible inclusion of bias in the parameter estimate, as was noted when we discussed the properties of the autocorrelation coefficient. This would imply that we cannot use the critical values from the standard \(t\)-distribution. The relevant critical values are included in the work of Dickey and Fuller (1979), Dickey and Fuller (1981) and MacKinnon (1991).⁶

5.2 Augmented Dickey-Fuller test

The above test describes the procedure for investigating whether the null hypothesis assumes a unit root, while the alternative hypothesis is that of stationarity. The use of such a null and alternative hypothesis would be appropriate for time series variables that do not drift systematically in any direction. However, if the time series contains evidence of a trend and is either increasing or decreasing over the sample, we would like to include a deterministic trend in the alternative hypothesis.⁷ Therefore, where there is evidence of a potential deterministic trend in the variable, such a testing procedure would consider the use of the regression model,

\[\begin{eqnarray} \nonumber y_{t}=\beta_1 + \beta_2 t+\phi y_{t-1}+\varepsilon_{t} \end{eqnarray}\]

which can be rewritten as,

\[\begin{eqnarray} \Delta y_{t}=\beta_1 + \beta_2 t+\pi y_{t-1}+\varepsilon_{t} \tag{5.2} \end{eqnarray}\]

where \(\pi=\hat{\phi}-1\), once again. This test for a unit root would still consider whether or not \(\pi=0\), with the aid of the null hypothesis,

\[\begin{eqnarray} \nonumber H_{0}\; :\;\; \pi=0 \end{eqnarray}\]

which implies that \(y_{t}\sim I(1)\), but in this case it would suggest that the variable is represented by a random walk with drift. The alternative hypothesis would then be given as,

\[\begin{eqnarray} \nonumber H_{1}\; :\;\; \pi<0 \end{eqnarray}\]

which implies that \(y_{t}\sim I(0)\), after the deterministic time trend has been removed (i.e. the process is trend-stationary). Note that the properties of the asymptotic distribution of the \(t\)-statistic will change if either a constant or a time trend are included in the estimated regression model. As such, the critical values would differ to those that are provided in the previous case.

Since the alternative hypotheses in both of the above tests do not allow for any persistence in the underlying process, the residuals may be autocorrelated. This lead to the development of the augmented Dickey-Fuller (ADF) test, which is describe in Dickey and Fuller (1981). It controls for residual autocorrelation by including lagged values of \(\Delta y_{t}\), which are allowed to follow a higher order AR(\(p\)) process. To see how this works, consider an AR(2) representation,

\[\begin{eqnarray} \nonumber y_{t}=\beta_1 + \beta_2 t+\phi_{1}y_{t-1}+\phi_{2}y_{t-2}+\varepsilon_{t} \end{eqnarray}\]

which is the same as,

\[\begin{eqnarray} \nonumber y_{t}=\beta_1 + \beta_2 t+(\phi_{1}+\phi_{2})y_{t-1}-\phi_{2}(y_{t-1}-y_{t-2})+\varepsilon_{t} \end{eqnarray}\]

Subtracting \(y_{t-1}\) from both sides provides the expression

\[\begin{eqnarray} \nonumber \Delta y_{t}=\beta_1+\beta_2 t+\pi y_{t-1}+\gamma_{1}\Delta y_{t-1}+\varepsilon_{t} \end{eqnarray}\]

where we have defined \(\pi=\phi_{1}+\phi_{2}-1\) and \(\gamma_{1}=-\phi_{2}\). Hence, if we allowed for \(p\) lags in the autoregressive process, we would have

\[\begin{eqnarray}\nonumber \Delta y_{t}=\beta_1 +\beta_2 t+\pi y_{t-1}+\overset{p}{\underset{j=1}{\sum}}\gamma_{j}\Delta y_{t-j}+\varepsilon_{t} \end{eqnarray}\]

where \(\pi=\sum_{j=1}^{p}\phi_{j}-1\) and \(\gamma_{j}=\sum_{k=j+1}^{p}\phi_{k}\), for \(j=\{1,2,3,\ldots, p\}\). To identify an appropriate value for \(p\), the number of autoregressive lags, we can make use of information criteria, such as the AIC or BIC. This would allow us to isolate the persistence from other stationary components and this particular test may also be used to isolate the effects of intercepts and linear time trends, where we essentially make use of three test equations,

\[\begin{eqnarray} \Delta y_t = \pi y_{t-1} + \sum_{i=2}^{p}\gamma_i \Delta y_{t-i+1} + \varepsilon_t \tag{5.3} \\ \Delta y_t = \beta_1 + \pi y_{t-1} + \sum_{i=2}^{p}\gamma_i \Delta y_{t-i+1} + \varepsilon_t \tag{5.4} \\ \Delta y_t = \beta_1 + \beta_2 t + \pi y_{t-1} + \sum_{i=2}^{p}\gamma_i \Delta y_{t-i+1} + \varepsilon_t \tag{5.5} \end{eqnarray}\]

The difference between these test equations concerns the inclusion of \(\beta_1\) and \(\beta_2\) coefficients, where under the null hypothesis, equation (5.3) refers to a pure random walk model, equation (5.4) includes an intercept or drift term and equation (5.5) includes both a drift and linear time trend. In each case, the parameter of interest is \(\pi\), where if \(\pi =0\) then the process \(y_t\) contains a unit root. Comparing the calculated \(t\)-statistic with the critical values from the Dickey-Fuller tables determines whether or not we should reject the null hypothesis, \(H_{0}: \; \pi =0\).

Although the procedure is essentially the same, regardless of which test equation is used, the critical values of the \(t\)-statistics depend on whether the intercept or time trend is included, as noted previously. In addition, the critical values also depend upon the sample size.

Dickey and Fuller (1981) include three additional \(F\)-statistics, which we denote \(\varphi_1 , \varphi_2\) and \(\varphi_3\). These statistics are used to test joint hypotheses relating the respective coefficient values and may be used to determine whether (5.3), (5.4) or (5.5) are appropriate for the underlying data generating process.

• The null hypothesis for equation (5.4), where \(\pi = \beta_1 = 0\) is tested using \(\varphi_1\), to determine whether the process could possibly include a drift term.
• The most general null hypothesis for equation (5.5), where \(\pi = \beta_1 = \beta_2 = 0\) is tested using \(\varphi_2\), to determine whether the process could possibly include a drift and a linear time trend.
• And lastly, the joint hypothesis \(\pi = \beta_2 = 0\) may be tested with the aid of \(\varphi_3\), which seeks to determine whether the process has a deterministic time trend.

The values for the \(\varphi_1 , \varphi_2\) and \(\varphi_3\) statistics are constructed as if they were \(F\)-tests,

\[\begin{eqnarray} \nonumber \varphi_i = \frac{[RSS(restricted) - RSS(unrestricted)] / r}{RSS(unrestricted) / (T-k)} \end{eqnarray}\]

where \(RSS(restricted)\) and \(RSS(unrestricted)\) are the sum of the squared residuals for the two variants of the model, \(r\) refers to the number of restrictions, \(T\) is the number of usable observations, and \(k\) is determined by the number of estimated parameters in the unrestricted model.

When comparing the calculated value of \(\varphi_i\) to the values in Augmented Dicky-Fuller tables, we need to determine the significance level at which the restriction is binding, to test the null hypothesis that the data is generated by the restricted model. In this case the alternative hypothesis is that the data is generated by the unrestricted model.

If the restriction is not binding, \(RSS(restricted)\) should be close to the value for \(RSS(unrestricted)\), and \(\varphi_i\) will be small. This would imply that large values of \(\varphi_i\) suggest that the restriction is binding, which would result in a rejection of the null hypothesis.

5.3 Implementing an ADF test

When implementing the augmented Dickey-Fuller test, it has been suggested that one should employ a general-to-specific approach, where the first step is to make use of the test equation that includes a constant and time trend, as provided in (5.5). If we find that we are unable to reject the null of a unit root (i.e. \(\pi = 0\)), we would then need to consider the value of \(\varphi_3\) test statistic. If we are again unable to reject the joint null hypothesis, which in this case if given by \(\pi = \beta_2 = 0\), then the process does not include a time trend and we would need to estimate the test equation that is provided in (5.4). To evaluate the results of this test equation, we would firstly consider whether or not we are able to reject the null hypothesis of a unit root. If we are unable to reject the null hypothesis, then we would consider the joint null hypothesis \(\pi = \beta_1 = 0\) and compare it to the critical values for \(\varphi_1\) test statistic. If we are once again unable to reject the joint null hypothesis then we would need to make use of the test equation in (5.3), where we are only interested in whether or not we are able to reject the null hypothesis of a unit root.

The full details of this testing procedure are provided in Figure 5.

Note that if we suspect that the process is integrated of the second order, we would need to perform Dickey-Fuller tests on successive differences of \(y_t\). For example, if we want to test whether \(y_t \sim I(2)\) then we would estimate the equation,

\[\begin{eqnarray}\nonumber \Delta^2 y_t = \mu + \xi_1 \Delta y_{t-1} + \varepsilon_t \end{eqnarray}\]

where we cannot reject the null that \(\xi_1 = 0\), we would conclude that \(y_t\) is \(I(2)\).

6.1 Testing for a unit root with a known breakpoint

In a much cited paper, Perron (1989) showed that the ADF test has little power to discriminate between a stochastic and deterministic trend when the data is subject to structural break. This would imply that in the presence of a structural break, the various ADF tests are biased towards the non-rejection of a unit root.

For example, consider the moving average representation of an autoregressive model, \(y_t = S_t + 0.5 \sum \varepsilon_t\). This time series has been simulated for 500 observations, where the level shift is described by \(S_t\), where for the first half of the sample, \(S_{1-249} = 0\) and for the second half, \(S_{250-500} = 10\). This time series is depicted in Figure 6.

If we were to fit a AR(1) model to this process, the coefficient would be biased towards unity, since low values are followed by other low values during the first half of the sample, and high values are followed by other high values during the later part of the sample. In addition, since a unit root process has infinite memory, the effect of the structural break at the mid-point of the sample would be present in the remainder of the time series. Hence, if we misrepresent the structural break as a shock that does not dissipate, then the ADF tests may suggest that this process follows a random walk plus drift, where it is clearly just a stationary time series with a structural break.

Perron (1989) includes a formal procedure for testing unit roots in the presence of a structural change, where the parameter \(\tau\), is used to denote the position of the structural break, which in the above example would occur at position \(250\). This test could take one of the following three forms.

6.2 Testing for a unit root with an unknown breakpoint

While this technique is highly intuitive, Christiano (1992) and a number of other researchers criticised the Perron approach on the basis that it required prior knowledge about the exact date of the break point, which is not always available. This lead to the development of a number of different methods that treat the break point as unknown (prior to testing). Examples of these procedures are contained in the work of Perron and Vogelsang (1992), Banerjee, Lumsdaine, and Stock (1992), Perron (1997), and Vogelsang and Perron (1998).

While most of these studies provide interesting insights, the technique that is described in Zivot and Andrews (2002) is the most popular procedure for identifying a unit root with an unknown endogenous structural break. This procedure makes use of an optimisation routine that identifies the date of the most likely structural break, which is the point that gives the least favourable result for the null hypothesis of a random walk with drift.

Therefore, these test statistics are formulated as,

\[\begin{eqnarray} \nonumber \Delta y_t = \mu + \pi y_{t-1} + \alpha t + \beta_2 D_L \hat{\lambda} + \sum_{i=1}^{k} \gamma_i \Delta y_{t-i} + \varepsilon_t \\ \nonumber \Delta y_t = \mu + \pi y_{t-1} + \alpha t + \beta_3 D_T \hat{\lambda} + \sum_{i=1}^{k} \gamma_i \Delta y_{t-i} + \varepsilon_t \\ \nonumber \Delta y_t = \mu + \pi y_{t-1} + \alpha t + \beta_2 D_L \hat{\lambda} + \beta_3 D_T \hat{\lambda} + \sum_{i=1}^{k} \gamma_i \Delta y_{t-i} + \varepsilon_t \end{eqnarray}\]

where \(\lambda\) is the estimated date for the structural break and we are essentially interested in the value for \(\pi=\phi-1\). Critical values for this technique are provided in Zivot and Andrews (2002).

10.1 Monte Carlo simulations for the bias in a unit root

In the tutorial we constructed a number of simulation exercises, where we noted that after generating a random walk process 10,000 times, the estimated coefficients for \(\hat{\phi}\), were biased to values below 1. The results of this simulation exercise are contained in Figure 9.

To make use of a Monte Carlo simulation for such a data generating process (DGP) that may have been generated for a particular model, we need to specify information relating to:

• Model structure
• Error term’s distribution
• Parameter values
• Sample size

Therefore, if we assume that the DGP is generated by an AR(1) model that does not have a constant, such as:

\[\begin{eqnarray} \nonumber y_t = \phi y_{t -1} + \epsilon_t, \;\;\; \text{for } t = 1, . . . , T \;\;\; \text{and } \epsilon_t \sim \mathsf{i.i.d.} \mathcal{N}(0, \sigma^2 ) \end{eqnarray}\]

Then we would need to specify values for the follow terms, where by way of example, \[ y_0 = 0, \phi = 1, \sigma = 1 \; \text{and } T = 100 \].

We would then be able to generate values for the variables with the aid a some form of simulatio, where the number of simulations would need to take on a defined value, e.g. \(N = 10,000\). Thereafter, we could estimate an AR(1) model for each of these simulated time series, which could be used to investigate the bias in the estimated value of \(\hat{\phi}\). Hence,

• For \(i = 1, \dots , N\):
  • Generate \(y_t\) for \(t = 1, \dots , T\) from the AR(1) model for given \(\phi\) and \(\sigma\)
  • Estimate an AR(1) model for the simulated \(y_t\) to obtain estimated values for \(\hat{\phi}\) and \(\hat{\sigma}\)
  • Compute the \(t\)-statistics that is associated with \(\hat{\phi}\)
  • Store the \(\hat{\phi}\) estimates and the associated \(t\)-statistics in a vector that has a length of \(N\)
• To compute the average bias calculate:

\[\begin{eqnarray}\nonumber \text{Average Bias } = \frac{1}{N} \sum_{i=1}^{N} (\hat{\phi}_i - \phi) \end{eqnarray}\]

• One could then consider different values of $T , , $ and \(\sigma\) to investigate the relation between the bias and \(T\), \(\phi\), and \(\sigma\)
• To obtain critical values from the sampled \(t\)-statistics, sort the absolute values of the \(t\)-statistics from smallest to largest. Then as we are seeking to perform a one-sided test, where you want to apply a 95% confidence interval, sort the sample from smallest to largest and take the \((0.95 \times N)\) \(t\)-statistic from the sorted sample

10.2 Power studies

The power of a test is the probability of rejecting the null hypothesis given that the null hypothesis is not true (that is, one minus type II error).

For example consider the power of the Dickey-Fuller test, where we assume that you know the \(5\%\) critical value of the one-sided \(t\)-test for \(\phi = 1\) denoted by \(\tau_{0.05}\). To ascertain the power of the Dickey-Fuller test for \(\phi \ne 1\), which is where the test suggests that the series contains a unit root (when you know it doesn’t).

• Fix \(T , N,\) and assume the errors have, \(\sigma = 1\)
• Fix \(\phi\) at a different value to the null hypothesis, for example \(\phi = 0.95\)
• Then setup the loop, where for \(j = 1, \dots , N\):
  • Generate \(y_t\) for \(t = 1, \dots , T\) from the AR(1) model for \(\phi=0.95\) and \(\sigma=1\)
  • Estimate an AR(1) model for the simulated \(y_t\) to obtain \(\hat{\phi}\)
  • Compute the \(t\)-statistic that is associated with \(\hat{\phi}\)
  • Store the \(\hat{\phi}\) estimates and the associated \(t\)-statistics in a vector that has a length of \(N\)

To obtain sample of estimated \(t\)-statistics:

• Count the relative number of times that the \(t\)-statistics for \(\hat{\phi}\) are larger than \(\tau_{0.05}\), where you have incorrectly assumed that \(y_t\) contains a unit root
• This is the power of the Dickey-Fuller test for \(\phi = 0.95\)

We could then consider different values of \(\phi\) to investigate the relation between power and \(\phi\), which may be used to draw a power function. These studies suggest that the power of the Dickey-Fuller test is relatively low. For example, when making use of a simulation exercise for a stationary time series process that has a long memory, where \(\phi = 0.95\), we noted that the Dickey-Fuller test was only able to reject the null of a unit root 4.3% of the time (when using the critical values at the 95% level).