Chapter 4 in The Essentials of Time Series Modeling: An Applied Treatment withEmphasis on Topics Relevant to Financial Analysis © by Houston H. Stokes 200x.
18 September 2013draft. All rights reserved. Preliminary Draft
Chapter 4
Stationary Time Series Models
4.0 Overview and Definitions
4.1 Identifying a Time Series Filter
4.2 Frequency Representation
4.3 Examples
Stationary Time Series Models
4.0Overview and Definitions
Economics works with discrete rather than continuous models. An important issue is how to form aggregates? For example, assume we have weekly money data. When building the monthly money series do we seasonally adjust the weekly series and aggregate or seasonally adjust the aggregated monthly series? An alternative is not to seasonally adjust at all. (Stokes 1980). These and other related questions can be addressed using filter theory.
White noise => for all k>0 and s>0
(4.0-1)
Given a white noise process a MA model can be developed
Here is a weighted average of past shocks. Mann and Wald (1943)
Assume
since .
An ARMA(p,q) model combines a linear difference equationAR(p) model with a MA(q) model.
(4.0-2)
4.1 Identifying a Time Series Filter
The basic idea behind a time series filter is to be able to capture the structure of the series. Once series is filtered to white noise the filter can be reversed to forecast. The goal is to go from the covariances of the process to initial estimates of the filter coefficients. The next step is to estimate the filter using a ML procedure. Building a filter is an iterative process.
Steps:
1. Make Series Stationary and calculate ACF and PACF: Estimate autocorrelations and partial autocorrelations of raw series. If series is non stationary (ACF does not die out) difference the series to obtain stationarity and calculate the ACF and PACF.
2. Preliminary Estimates Step: Using below listed table to obtain a preliminary idea of the form of the model. See Stokes (1997, 159) or Enders (1995, 85)(2010, 68) or
ModelACF PatternPACF Pattern
MA(q)Spikes as lags 1-qTail off
Then cut off
AR(p)Tail off Spikes at lags
1-p then cut off
ARMA(p,q)Irregular pattern 1-qTail off
then tail off.
______
3. Estimation Step: Estimate the tentative model using ML and calculate ACF and PACF of the residuals. If ACF and PACF indicate white noise in the residuals, model is complete and can be used in forecasting etc. If ACF and PACF indicate that there still is information in the series, adjust the model and repeat estimation step.
4. Forecasting Step. Use coefficients estimated in step 3 to forecast out of sample.
The ACF and PACF coefficients and model coefficients are related. From inspection of the ACF and PACF the preliminary model is selected which is then tested to see if white noise residuals are obtained. To prove why this works, we prove the reverse link from the model coefficients to a specific pattern in the ACF and PACF coefficients.
If the process is stationary, then the variances and covariances can be defined. If the ACF dies out => have stationarity.
Assume a series . There are n expectations, and (n2 +n)/2 variances () and covariances ().
It is not possible to estimate (n2 + n)/2 values with n data points.
Simplifications that form basis for stationarity are for any m:
(4.1-1)
Define the autocorrelation
(4.1-2)
where
(4.1-3)
For a sample we have
(4.1-4)
note that we use the “large sample” devisor (1/T) so that high order autocorrelations are small. Also note that the complete sample mean is used. Bartlett showed that in theory
(4.1-5)
If we assume the autocorrelations will have died out at lags > q then the variance of the estimated autocorrelation is
(4.1-6)
If all correlations at lags less that k are zero, we have the asymptotic value of N-1
while in other cases we have to adjust the SE upward. The SE is adjusted up so that we do not try to model spurious spikes in the estimated ACF.[1]
4.2 Frequency Representation
The ACF can be transformed to the frequency domain. While variance is measured at all frequencies, it is often useful to decompose the variance by frequency. Stokes (1997, chapter 15) shows how this can be done. A brief overview is given here.
The periodogram measures the amplitude of a sine component buried in a series. The amplitude can be estimated using OLS as the explained sum of squares of a model of the series as a function of sine and cosine vectors. Assume a model of the form
(4.2-1)
The periodogram consists (for odd values of N) of the q=(N-1)/2 values
(4.2-2)
The periodogram can be weighted to form the spectrum and other measures of association between two series. These will be discussed later.
The spectrum can be used to distinguish the difference between an AR(1) model having a parameter of .9 from one with a parameter of -.9. The former model is characterized as having a low frequency component while the latter model has a high frequency component.
4.3 Examples
A sample job (genarma from matrix.mac) listed next illustrates the above discussion.
Triangular weights are used and 1000 data points are calculated. Experiment with alternative numbers of observations. The ACF is also calculated and listed.
b34sexec matrix;
ar=array(:.9);
n=1000; start=array(:.1);
testar1a=genarma(ar,ma,1.0,start,.1,n);
ar=array(:-.9);
testar1b=genarma(ar,ma,1.0,start,.1,n);
acf1a=acf(testar1a,20,se,pacf1a);
acf1b=acf(testar1b,20,se,pacf1b);
call spectral(testar1a,sinx,cosx,px,sx1a,freq:1 2 3 2 1);
call spectral(testar1b,sinx,cosx,px,sx1b,freq:1 2 3 2 1);
call graph(freq,sx1a:heading 'Spectrum of ar(.9)'
:plottype xyplot);
call graph(freq,sx1b:heading 'Spectrum of ar(-.9)'
:plottype xyplot);
call tabulate(acf1a,acf1b,pacf1a,pacf1b);
b34srun;
From (3.1-8), a linear stochastic process can be written in MA form as
(4.3-1)
If the MA terms converge such that , then and
(4.3-2)
since . Because of the white noise assumption, all the cross product terms are zero since . For we have
(4.3-3)
From (4.2-2) and the more general (4.3-3) we have
(4.3-4)
where and . For a model we have . This is illustrated by the setup:
b34sexec matrix;
call free(ar); ma=array(:.5);
n=1000;
start=array(:.1);
test=genarma(ar,ma,1.0,start,.1,n,100);
acf1=acf(test,30,se,pacf1);
call tabulate(acf1);
b34srun;
which when run gives:
=> CALL FREE(AR)$
=> MA=ARRAY(:.5)$
=> N=1000$
=> START=ARRAY(:.1)$
=> TEST=GENARMA(AR,MA,1.0,START,.1,N,100)$
=> ACF1=ACF(TEST,30,SE,PACF1)$
=> CALL TABULATE(ACF1)$
Obs ACF1
1 -0.4080
2 -0.1071E-01
3 0.1301E-01
4 -0.2512E-01
5 -0.1861E-02
6 -0.2843E-02
7 -0.2952E-02
8 -0.1634E-01
9 0.5529E-01
10 -0.6841E-01
11 0.3059E-01
12 0.1808E-01
13 -0.8914E-01
14 0.1150
15 -0.9589E-01
16 0.1051
17 -0.5873E-01
18 0.7364E-01
19 -0.7742E-01
20 0.7816E-02
21 0.2816E-01
22 -0.1771E-01
23 -0.3248E-01
24 0.6770E-01
25 -0.2819E-01
26 0.4016E-01
27 -0.8653E-01
28 0.9583E-01
29 -0.2587E-01
30 -0.2669E-01
For the MA(2) model we have
The importance of the (4.3-4) is that is shows how to go directly from the MA parameters to the ACF.
Assume a white noise series is generated. Define . From (4:14) . The following matrix commands illustrate this for n=2000 and n=20000.
b34sexec matrix;
n=2000;
nn2=200000;
xx=rn(array(n:));
xx2=rn(array(nn2:));
call print('Dif. of White Noise has acf(1)=-.5':);
call tabulate(acf(xx,20),acf(dif(xx),20),acf(dif(xx2),20));
b34srun;
Output from this job is:
=> CALL PRINT('Dif. of White Noise has acf(1)=-.5':)$
Dif. of White Noise has acf(1)=-.5
=> CALL TABULATE(ACF(XX,20),ACF(DIF(XX),20),ACF(DIF(XX2),20))$
Obs ##30 ##29 ##26
1 -0.2548E-01 -0.5072 -0.4992
2 -0.1005E-01 0.1017E-01 -0.7046E-03
3 -0.1405E-01 -0.5274E-02 -0.1312E-02
4 -0.8723E-02 0.1508E-02 -0.3324E-04
5 -0.6717E-02 -0.1301E-02 0.2949E-02
6 -0.9782E-03 -0.3407E-02 -0.1061E-02
7 0.1186E-01 0.1142E-01 -0.1425E-02
8 0.1421E-02 -0.6090E-02 0.1468E-02
9 0.3201E-02 0.1280E-01 -0.2184E-02
10 -0.2237E-01 -0.1869E-01 0.1331E-03
11 -0.9187E-02 -0.5120E-02 0.2836E-02
12 0.1459E-01 0.3096E-01 -0.2290E-03
13 -0.2590E-01 -0.4461E-01 -0.2587E-02
14 0.2510E-01 0.3470E-01 0.1810E-02
15 0.5161E-02 -0.3971E-01 -0.1026E-02
16 0.6663E-01 0.6050E-01 0.9570E-03
17 0.4892E-02 -0.2832E-01 0.7650E-03
18 0.1879E-02 0.1762E-01 -0.3377E-02
19 -0.3839E-01 -0.5294E-01 0.3976E-02
20 0.2850E-01 0.4667E-01 -0.2339E-02
Differencing has added structure to the generated white noise series. This may later have to be removed.
Seasonal and Seasonal and first differencing effects can be tested with the commands:
call print('Seasonal Differencing effects':);
call tabulate(acf(xx,20),acf(dif(xx,1,12),20),
acf(dif(xx2,1,12),20));
call print('Seasonal and First Difference Effects':);
call tabulate(acf(dif(dif(xx ,1,12)),20) ,
acf(dif(dif(xx2,1,12)),20));
The results of seasonal and first differencing a white noise series are:
Seasonal and First Difference Effects
=> CALL TABULATE(ACF(DIF(DIF(XX ,1,12)),20) ,
=> ACF(DIF(DIF(XX2,1,12)),20))$
Obs ##60 ##55
1 -0.4991 -0.4992
2 0.3506E-02 -0.1672E-02
3 0.7818E-02 0.2972E-03
4 -0.2625E-01 -0.1272E-02
5 0.6978E-02 0.3294E-02
6 -0.1160E-01 0.1157E-02
7 0.4090E-01 -0.4894E-02
8 -0.3046E-01 0.2666E-02
9 0.2022E-01 -0.2679E-02
10 -0.2056E-01 0.2951E-02
11 0.2584 0.2495
12 -0.4928 -0.4979
13 0.2226 0.2457
14 0.2511E-01 0.2928E-02
15 -0.1523E-01 -0.1280E-02
16 0.1591E-01 0.2021E-02
17 0.9557E-03 -0.2702E-02
18 0.2471E-01 0.2802E-03
19 -0.6616E-01 0.2576E-02
20 0.4168E-01 -0.3231E-02
which show induced spikes at lags 1, 11, 12 and 13.
bjest_5 will illustrate the use of (4.3-4) for various MA models.
b34sexec data noob=30002 maxlag=2$
build ma1 ma2 norm$
gen norm=rn()$
gen ma1= lp(0,2,norm) ma(1. -.5 ) $
gen ma2= lp(0,3,norm) ma(1. -.5 -.11) $
b34srun$
b34sexec bjiden$
var ma1$
seriesn var=ma1$
rauto ma1$ b34srun$
b34sexec bjiden$
var ma2$
seriesn var=ma2$
rauto ma2$ b34srun$
b34sexec bjest nac=24$
model ma1$
modeln q=(1)$
b34seend$
b34sexec bjest nac=24$
model ma2$
modeln q=(1,2)$
b34seend$
The above job produces edited output
Variable Mean Standard Deviation Variance Maximum Minimum
MA1 1 0.27860970E-02 1.1139618 1.2409110 4.2203918 -4.5228799
MA2 2 0.21822486E-02 1.1201630 1.2547651 4.3767500 -4.5912190
NORM 3 0.54974293E-02 0.99646227 0.99293705 4.0471005 -4.1740627
CONSTANT 4 1.0000000 0.0000000 0.0000000 1.0000000 1.0000000
Data file contains 30000 observations on 4 variables. Current missing value code is 0.1000000000000000E+32
Autocorrelation Function
Data - VAR=MA1 SERIES 1 30000 Observations
Original Series
Mean of the Series = 0.27860970E-02
St. Dev. of Series = 1.1139433
Number of observations = 30000
S. E. of mean = 0.6431E-02
T value of mean (against zero) = 0.4332
1- 12 -0.40 -0.01 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 4715.6 4717.0 4717.6 4723.7 4725.0 4729.3 4732.3 4732.3 4732.6 4732.8 4733.1 4734.3
13- 24 0.01 0.00 0.00 0.00 0.00 0.00 -0.01 0.02 -0.01 0.00 0.00 0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 4735.1 4735.5 4735.5 4736.1 4736.1 4736.2 4741.5 4748.9 4751.8 4751.8 4751.9 4752.9
Mean divided by St. Error (using N in S. D. ) = 0.43321
Q Statistic 4752.4 DF 24 Prob. 1.0000
Modified Q Statistic 4752.9 DF 24 Prob. 1.0000
NOTE: In some cases degrees of freedom for Q and Modified Q Statistics may have to be adjusted.
Partial Autocorrelations
Data - VAR=MA1 SERIES 1 30000 Observations
Original Series
Mean of the Series = 0.27860970E-02
St. Dev. of Series = 1.1139433
Number of observations = 30000
S. E. of mean = 0.6431E-02
T value of mean (against zero) = 0.4332
1- 12 -0.40 -0.19 -0.10 -0.04 -0.02 -0.03 -0.01 0.00 0.00 0.00 0.00 -0.01
13- 24 0.00 0.00 0.00 0.00 0.00 0.01 -0.01 0.01 0.00 0.00 -0.01 0.00
Autocorrelation Function
Data - VAR=MA2 SERIES 1 30000 Observations
Original Series
Mean of the Series = 0.21822486E-02
St. Dev. of Series = 1.1201443
Number of observations = 30000
S. E. of mean = 0.6467E-02
T value of mean (against zero) = 0.3374
1- 12 -0.35 -0.09 0.00 0.02 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 3645.6 3915.7 3916.1 3923.3 3924.7 3929.8 3933.1 3933.3 3933.8 3933.9 3934.2 3935.1
13- 24 0.00 0.00 0.00 0.00 0.00 0.00 -0.01 0.02 -0.01 0.00 0.00 0.00
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 3935.9 3936.2 3936.2 3936.9 3937.0 3937.0 3941.7 3948.7 3950.9 3951.0 3951.0 3951.7
Mean divided by St. Error (using N in S. D. ) = 0.33744
Q Statistic 3951.3 DF 24 Prob. 1.0000
Modified Q Statistic 3951.7 DF 24 Prob. 1.0000
NOTE: In some cases degrees of freedom for Q and Modified Q Statistics may have to be adjusted.
Partial Autocorrelations
Data - VAR=MA2 SERIES 1 30000 Observations
Original Series
Mean of the Series = 0.21822486E-02
St. Dev. of Series = 1.1201443
Number of observations = 30000
S. E. of mean = 0.6467E-02
T value of mean (against zero) = 0.3374
1- 12 -0.35 -0.25 -0.16 -0.09 -0.06 -0.06 -0.03 -0.02 -0.01 -0.01 -0.01 -0.01
13- 24 0.00 -0.01 -0.01 0.00 0.00 0.00 -0.01 0.01 0.00 -0.01 -0.01 0.00
Note: Initial value for Estimated Parameter 1 was zero. Reset to .10
Time Series Parameter Estimation for Model 1
Data - Z = VAR=MA1
Observations 30000
Differencing on Z - None
Transformations Examined - None.
Univariate Model Parameters.
Parameter Beginning
# Type Order Value
1 Mean 0.000
2 Moving average 1 1 0.1000
Output at each iteration has been suppressed.
Residual output has been suppressed.
Initial sum of Squares = 0.3497E+05
Iteration stops - Relative parameter change < 0.4000E-02
Correlation Matrix of the Parameters. 1/Cond = 0.39587718
1 2
1 1.0000
2 -0.0001 1.0000
End of Estimation for Model 1
Summary of Model 1
Data - Z = VAR=MA1
Observations 30000
Differencing on Z - None
Univariate Model Parameters.
Parameter Estimated 95 Per Cent
# Type Order Value Lower Limit t Upper Limit Std. Error
1 Mean 0.2747E-02 -0.2965E-02 0.9618 0.8460E-02 0.2856E-02
2 Moving average 1 1 0.5035 0.4936 101.3 0.5135 0.4972E-02
Other Information and results.
Residual Sum of Squares 29787.429 29998 D.F.
Residual Mean Square 0.99298050
Number of Residuals 30000
Residual Standard Error 0.99648407
Backforecasting was suppressed in Parameter Estimation.
Autocorrelation Function
Data - THE ESTIMATED RESIDUALS - MODEL 1 30000 Observations
Original Series
Mean of the Series = 0.24156879E-04
St. Dev. of Series = 0.99645085
Number of observations = 30000
S. E. of mean = 0.5753E-02
T value of mean (against zero) = 0.4199E-02
1- 12 0.00 -0.01 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 0.4 1.7 2.0 4.9 6.7 11.6 12.6 12.7 13.2 13.6 13.6 14.5
13- 24 0.00 0.00 0.00 0.01 0.00 0.00 -0.01 0.01 -0.01 0.00 0.00 0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 14.7 14.8 14.8 15.7 15.8 16.0 19.6 21.6 23.4 23.7 23.8 25.1
Mean divided by St. Error (using N in S. D. ) = 0.41990E-02
Q Statistic 25.117 DF 22 Prob. 0.70850
Modified Q Statistic 25.129 DF 22 Prob. 0.70904
NOTE: In some cases degrees of freedom for Q and Modified Q Statistics may have to be adjusted.
Partial Autocorrelations
Data - THE ESTIMATED RESIDUALS - MODEL 1 30000 Observations
Original Series
Mean of the Series = 0.24156879E-04
St. Dev. of Series = 0.99645085
Number of observations = 30000
S. E. of mean = 0.5753E-02
T value of mean (against zero) = 0.4199E-02
1- 12 0.00 -0.01 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
13- 24 0.00 0.00 0.00 0.01 0.00 0.00 -0.01 0.01 -0.01 0.00 0.00 0.01
Note: Initial value for Estimated Parameter 1 was zero. Reset to .10
Time Series Parameter Estimation for Model 1
Data - Z = VAR=MA2
Observations 30000
Differencing on Z - None
Transformations Examined - None.
Univariate Model Parameters.
Parameter Beginning
# Type Order Value
1 Mean 0.000
2 Moving average 1 1 0.1000
3 Moving average 1 2 0.1000
Output at each iteration has been suppressed.
Residual output has been suppressed.
Initial sum of Squares = 0.3516E+05
Iteration stops - Relative parameter change < 0.4000E-02
Correlation Matrix of the Parameters. 1/Cond = 0.97610223E-01
1 2 3
1 1.0000
2 -0.0001 1.0000
3 -0.0001 -0.5712 1.0000
End of Estimation for Model 1
Summary of Model 1
Data - Z = VAR=MA2
Observations 30000
Differencing on Z - None
Univariate Model Parameters.
Parameter Estimated 95 Per Cent
# Type Order Value Lower Limit t Upper Limit Std. Error
1 Mean 0.2138E-02 -0.2269E-02 0.9703 0.6544E-02 0.2203E-02
2 Moving average 1 1 0.4997 0.4882 87.18 0.5111 0.5732E-02
3 Moving average 1 2 0.1174 0.1059 20.40 0.1289 0.5755E-02
Other Information and results.
Residual Sum of Squares 29785.247 29997 D.F.
Residual Mean Square 0.99294087
Number of Residuals 30000
Residual Standard Error 0.99646418
Backforecasting was suppressed in Parameter Estimation.
Autocorrelation Function
Data - THE ESTIMATED RESIDUALS - MODEL 1 30000 Observations
Original Series
Mean of the Series = 0.31760130E-04
St. Dev. of Series = 0.99641436
Number of observations = 30000
S. E. of mean = 0.5753E-02
T value of mean (against zero) = 0.5521E-02
1- 12 0.00 0.00 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 0.0 0.0 0.0 4.3 5.4 9.6 10.8 10.9 11.3 11.7 11.7 12.6
13- 24 0.00 0.00 0.00 0.01 0.00 0.00 -0.01 0.01 -0.01 0.00 0.00 0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 12.8 12.9 13.0 13.9 13.9 14.1 17.8 19.7 21.6 21.9 21.9 23.3
Mean divided by St. Error (using N in S. D. ) = 0.55208E-02
Q Statistic 23.298 DF 21 Prob. 0.67172
Modified Q Statistic 23.310 DF 21 Prob. 0.67230
NOTE: In some cases degrees of freedom for Q and Modified Q Statistics may have to be adjusted.
Partial Autocorrelations
Data - THE ESTIMATED RESIDUALS - MODEL 1 30000 Observations
Original Series
Mean of the Series = 0.31760130E-04
St. Dev. of Series = 0.99641436
Number of observations = 30000
S. E. of mean = 0.5753E-02
T value of mean (against zero) = 0.5521E-02
1- 12 0.00 0.00 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
13- 24 0.00 0.00 0.00 0.01 0.00 0.00 -0.01 0.01 -0.01 0.00 0.00 0.01
Note that the generated series have the ACF that would be predicted by theory and that after estimation of the model it is possible to filter the series back. This demonstration shows the link between the ACF and the MA model. The reader is advised to try other models to gain experience on how the ACF is linked to the generated model. This is done next for a number of interesting cases.
Consider (4.3-1) but make the assumption that . The mean is . From (3.1-11) we know that . Using (4.3-2) and (4.3-3) we obtain for this case
If then we have random walk or a non stationary model . Since does not converge, the variance and the covariance cannot in theory be calculated. For the stationary case , which implies that the ACF trails off. An when we realize that:
Since all the parameters in the MA form of the model are related, it better to estimate the model as an AR(1) rather than a high order MA model. This is illustrated using the bjest_2 sample program listed next:
/$ Shows generation of models for testing
/$ As setup we look at ACF and estimate a AR(1) model
/$
/$ Note how parameters converge
/$
b34sexec data noob=3002 maxlag=2$
build ar1 ar2 ma1 ma2 arma norm$
gen norm=rn()$
gen ma1= lp(0,2,norm) ma(1. .7 ) $
gen ma2= lp(0,3,norm) ma(1. .7 .45) $
gen ar1= lp(1,1,norm) ar(.8) ma(1.0) values(.01);
gen ar2= lp(2,1,norm) ar(.8 .2) ma(1.0) values(.01 .01);
gen arma=lp(2,2,norm) ar(.6, -.2)
ma(1.0 .2) values(.01 .01);
b34srun$
b34sexec bjiden$
var ar1$
seriesn var=ar1$
rauto ar1$ b34srun$
b34sexec bjest nac=24$
model ar1$
modeln p=(1)$
b34seend$
b34sexec matrix;
call loaddata;
call graph(acf(ma1,24 ) :heading 'ma1 acf' :plottype hist2d);
call graph(acf(ar1,24 ) :heading 'ar1 acf' :plottype hist2d );
call graph(acf(arma,24) :heading 'arma acf' :plottype hist2d );
b34srun;
Three models are estimated and their ACF plotted. The ACF and PACF of the AR(1) model having parameter .8 is shown and the model estimated to test the parameters. Edited output is shown next. Readers are encouraged to experiment with this setup to try to estimate the other types of models. If graphics are available, the matrix command can be used to display the ACF in a variety of ways. As an option you can calculate and plot the spectrum.
Autocorrelation Function
Data - VAR=AR1 SERIES 1 30000 Observations
Original Series
Mean of the Series = 0.27265848E-01
St. Dev. of Series = 1.6455559
Number of observations = 30000
S. E. of mean = 0.9501E-02
T value of mean (against zero) = 2.870
1- 12 0.80 0.63 0.50 0.40 0.31 0.25 0.20 0.16 0.12 0.10 0.08 0.06
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 19000.2 30957.7 38524.4 43327.2 46280.7 48106.9 49282.9 50018.2 50465.0 50738.0 50908.0 51012.8
13- 24 0.05 0.04 0.03 0.03 0.02 0.01 0.01 0.01 0.01 0.01 0.02 0.02
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 51082.9 51128.5 51160.4 51182.8 51196.0 51202.7 51206.1 51210.4 51213.5 51218.3 51226.9 51241.6
Mean divided by St. Error (using N in S. D. ) = 2.8699
Q Statistic 51234. DF 24 Prob. 1.0000
Modified Q Statistic 51242. DF 24 Prob. 1.0000
NOTE: In some cases degrees of freedom for Q and Modified Q Statistics may have to be adjusted.
Partial Autocorrelations
Data - VAR=AR1 SERIES 1 30000 Observations
Original Series
Mean of the Series = 0.27265848E-01
St. Dev. of Series = 1.6455559
Number of observations = 30000
S. E. of mean = 0.9501E-02
T value of mean (against zero) = 2.870
1- 12 0.80 -0.01 0.00 0.00 -0.01 0.00 0.01 -0.01 0.00 0.00 0.00 0.00
13- 24 0.01 0.00 0.00 0.00 -0.01 0.00 0.00 0.01 -0.01 0.01 0.01 0.01
Note: Initial value for Estimated Parameter 1 was zero. Reset to .10
Time Series Parameter Estimation for Model 1
Data - Z = VAR=AR1
Observations 30000
Differencing on Z - None
Transformations Examined - None.
Univariate Model Parameters.
Parameter Beginning
# Type Order Value
1 Mean 0.000
2 Autoregressive 1 1 0.1000
Output at each iteration has been suppressed.
Residual output has been suppressed.
Initial sum of Squares = 0.6925E+05
Iteration stops - Relative parameter change < 0.4000E-02
Correlation Matrix of the Parameters. 1/Cond = 0.15625316E-01
1 2
1 1.0000
2 0.0001 1.0000
End of Estimation for Model 1
Summary of Model 1
Data - Z = VAR=AR1
Observations 30000
Differencing on Z - None
Univariate Model Parameters.
Parameter Estimated 95 Per Cent
# Type Order Value Lower Limit t Upper Limit Std. Error
1 Mean 0.2736E-01 -0.2900E-01 0.9710 0.8372E-01 0.2818E-01
2 Autoregressive 1 1 0.7958 0.7889 227.6 0.8028 0.3496E-02
Other Information and results.
Residual Sum of Squares 29785.198 29997 D.F.
Residual Mean Square 0.99293922
Number of Residuals 29999
Residual Standard Error 0.99646335
Backforecasting was suppressed in Parameter Estimation.
Autocorrelation Function
Data - THE ESTIMATED RESIDUALS - MODEL 1 29999 Observations
Original Series
Mean of the Series = -0.52506805E-12
St. Dev. of Series = 0.99643014
Number of observations = 29999
S. E. of mean = 0.5753E-02
T value of mean (against zero) = -0.9127E-10
1- 12 0.00 -0.01 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 0.6 1.3 1.4 5.2 6.4 10.5 11.8 12.1 12.4 12.7 12.7 13.5
13- 24 0.00 0.00 0.00 0.01 0.00 0.00 -0.01 0.01 -0.01 0.00 0.00 0.01
St.E. 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
Mod. Q 13.7 13.8 13.9 14.8 14.9 15.1 18.7 20.7 22.5 22.8 22.8 24.2
Mean divided by St. Error (using N in S. D. ) = 0.91269E-10
Q Statistic 24.169 DF 22 Prob. 0.66156
Modified Q Statistic 24.180 DF 22 Prob. 0.66214
NOTE: In some cases degrees of freedom for Q and Modified Q Statistics may have to be adjusted.
Partial Autocorrelations
Data - THE ESTIMATED RESIDUALS - MODEL 1 29999 Observations
Original Series
Mean of the Series = -0.52506805E-12
St. Dev. of Series = 0.99643014
Number of observations = 29999
S. E. of mean = 0.5753E-02
T value of mean (against zero) = -0.9127E-10
1- 12 0.00 -0.01 0.00 0.01 -0.01 -0.01 0.01 0.00 0.00 0.00 0.00 -0.01
13- 24 0.00 0.00 0.00 0.01 0.00 0.00 -0.01 0.01 -0.01 0.00 0.00 0.01
Partial autocorrelation function. Given the AR(p) model
(4.3-5)
we multiply by , for k > 0, take expectations (if the mean of the series is zero) and divide through by after noting that =0 since future shocks cannot affect past values.
(4.3-6)
By substituting k=1, k=2 ... in (4.3-6) we get the Yule-Walker equations that solve the AR parameters in terms of the theoretical autocorrelations . From (4.3-6) when k=0
(4.3-7)
since because only the most recent part of will be correlated with . Note that After solving (4.3-7) the variance of from the AR(p) model becomes
(4.3-8)
which can be solved for
(4.3-9)
Equation (4.3-9) shows how the variance of is related to the AR parameters and the autocorrelations. (4.3-6) shows how for an AR(p) model how the autocorrelations and AR parameters are related. For the pure MA model, (4.3-4) shows how the MA parameters and the autocorrelations are related. These two equations form the basis for the Box-Jenkins identification strategy that goes from inspection of the autocorrelations and partial autocorrelations to a preliminary guess about the underlying model.
From (4.3-6) we get for k=1, p
(4.3-10)
A quick example will show how to use the Yule-Walker equation (4.3-10). Assume that the estimated ACF values for the first three terms were .9, .8 and .7. At issue is what is the AR coefficients that are associated with these values; The below listed code will solve equation (4.3-10) for estimates of and test the model.
b34sexec matrix;
/;
/; Going from correlation to an estimate of ar(3).
/; Testing to see how close we get to answers.
call echooff;
call load(rtest);
left=vector(:.9 .8 .7);
x=matrix(3,3:
1.0000 0.9000 0.8000
0.9000 1.0000 0.9000
0.8000 0.9000 1.0000 );
ar=inv(x)*left;
call print(left,x,ar);
n=100000; ndrop=500; start=array(:.1,.1,.1);
test=genarma(ar,ma,1.0,start,.1,n,ndrop);
call autobj(test :print :nac 24 :npac 24
:ar index(1 2 3) );
/;
/; Optional residual analysis
/;
/; call rtest(%res,test,48);
b34srun;
As will be shown when this code is run, the AR coefficients, should be0.944444 0.444089E-15 and -0.555556E-01. Using 100,000 observations, the estimated coefficients turned out to be .9480, -.004142 and -.05518 respectively with t scores of 300.2, .9513 and -17.48. Detailed answers are listed next:
B34SI Matrix Command. d/m/y 29/ 9/11. h:m:s 9:23: 2.
=> CALL ECHOOFF$
LEFT = Vector of 3 elements
0.900000 0.800000 0.700000
X = Matrix of 3 by 3 elements
1 2 3
1 1.00000 0.900000 0.800000
2 0.900000 1.00000 0.900000
3 0.800000 0.900000 1.00000
AR = Vector of 3 elements
0.944444 0.444089E-15 -0.555556E-01
Time Series Parameter Estimation for Model 1
Data - Z = VAR= TEST
Observations 100000
Differencing on Z - None
Transformations Examined - None.