#LyX 1.1 created this file. For more info see http://www.lyx.org/ \lyxformat 218 \textclass article \language english \inputencoding auto \fontscheme pslatex \graphics default \paperfontsize default \spacing single \papersize Default \paperpackage a4 \use_geometry 0 \use_amsmath 0 \paperorientation portrait \secnumdepth 3 \tocdepth 3 \paragraph_separation indent \defskip medskip \quotes_language english \quotes_times 2 \papercolumns 1 \papersides 1 \paperpagestyle default \layout Title Quick Notes on Time Series \layout Standard Suppose you have a series of observations, \begin_inset Formula \( y_{0},y_{1},y_{2}...y_{T} \) \end_inset . You want to know if there is a pattern, or the observations come from some kind of underlying order. Suppose you say they are random: \layout Standard \begin_inset Formula \( y_{t} \) \end_inset \begin_inset Formula \( \sim f(y) \) \end_inset \layout Standard This means that \begin_inset Formula \( y_{t} \) \end_inset is drawn from some distribution f(y). (You specify what...). \layout Standard Now, suppose somebody else comes along and says \begin_inset Quotes eld \end_inset wait \begin_inset Quotes erd \end_inset . If you were correct, then we would say \layout Standard \begin_inset Formula \( y_{t}=\varepsilon _{t} \) \end_inset and \begin_inset Formula \( \varepsilon _{t}\sim f(\varepsilon ) \) \end_inset \layout Standard and that's wrong. Instead, y results from an averaging together of previous input values. So we really need this model: \layout Standard \begin_inset Formula \( y_{t}=\varepsilon _{t}+\theta _{1}\varepsilon _{t-1}+\theta _{2}\varepsilon _{t-2} \) \end_inset (you can add as many lagged terms as you want) \layout Standard This is an MA(2) process. The number of MA terms is q in these models. \layout Standard . \layout Standard Now, another person says \begin_inset Quotes eld \end_inset hold the phone \begin_inset Quotes erd \end_inset . y does not reflect an average of random inputs. It is way too stable for that. Instead, it represents an accumulation of its own past values, as in \layout Standard \begin_inset Formula \( y_{t}=\phi _{1}y_{t-1}+\phi _{2}y_{t-2}+\phi _{3}y_{t-3} \) \end_inset (add as many as you want). \layout Standard This is an AR(3) process. \layout Standard . \layout Standard You get an ARMA process if you add the MA and the AR things together, as in: \layout Standard \begin_inset Formula \( y_{t}=\phi _{1}y_{t-1}+\phi _{2}y_{t-2}+\phi _{3}y_{t-3}+\varepsilon _{t}+\theta _{1}\varepsilon _{t-1}+\theta _{2}\varepsilon _{t-2} \) \end_inset \layout Standard This is ARMA(3,2). \layout Standard . \layout Standard Much of the theory of time series analysis is about the problem that trends make it difficult to see what is going on in a time series. It is only meaningful to estimate all these coefficients, according to Box and Jenkins, if the series is \series bold stationary \series default , meaning it has the same expected value across the whole time line. There are several different \begin_inset Quotes eld \end_inset twists \begin_inset Quotes erd \end_inset on the idea. \layout Section* Quick Notation Note \layout Standard You have to get a little patient with notation! \layout Standard L is the backshift operator. \layout Standard \begin_inset Formula \[ y_{t-1}=Ly_{t}\] \end_inset \layout Standard \begin_inset Formula \[ y_{t-2}=L^{2}y_{t}\] \end_inset \layout Standard And so forth. \layout Standard As a result, the above ARMA model is represented thus: \layout Standard \begin_inset Formula \( y_{t}=\phi _{1}Ly_{t}+\phi _{2}L^{2}y_{t}+\phi _{3}L^{3}y_{t}+\varepsilon _{t}+\theta _{1}L^{1}\varepsilon _{t}+\theta _{2}L^{2}\varepsilon _{t} \) \end_inset \layout Standard and you might as well write: \layout Standard \begin_inset Formula \( y_{t}-\phi _{1}Ly_{t}-\phi _{2}L^{2}y_{t}-\phi _{3}L^{3}y_{t}=\varepsilon _{t}+\theta _{1}L^{1}\varepsilon _{t}+\theta _{2}L^{2}\varepsilon _{t} \) \end_inset \layout Standard or \layout Standard \begin_inset Formula \( (1-\phi _{1}L-\phi _{2}L^{2}-\phi _{3}L^{3})y_{t}=(1+\theta _{1}L^{1}+\theta _{2}L^{2})\varepsilon _{t} \) \end_inset \layout Standard And you can let the things in parentheses, the polynomials in the lags and coefficients, fall out of view by referring to them as \begin_inset Formula \( \phi (L) \) \end_inset and \begin_inset Formula \( \theta (L), \) \end_inset so the whole ugly mess becomes: \layout Standard \begin_inset Formula \[ \phi (L)y_{t}=\theta (L)\varepsilon _{t}\] \end_inset \layout Standard or, to be really succinct about it, (with stability assumed), write \layout Standard \begin_inset Formula \[ y_{t}=\frac{\theta (L)}{\phi (L)}\varepsilon _{t}\] \end_inset \layout Standard There's the old cliche, AR(1)=MA( \begin_inset Formula \( \infty \) \end_inset ). \layout Standard . \layout Section* What's that I part about? \layout Standard Suppose y is not stationary, it has a visible trend, for example. Then the theory goes out the window, and, frankly, it is hard for me to understand why they bothered to work out such a complicated theory of ARMA processes when data is so typically not stationary. \layout Standard You can make a series stationary by differencing it, i.e., \layout Standard \begin_inset Formula \[ \Delta y_{t}=y_{t}-y_{t-1}\] \end_inset \layout Standard A \begin_inset Quotes eld \end_inset unit root \begin_inset Quotes erd \end_inset process, also known as a random walk with drift, is given as \layout Standard \begin_inset Formula \[ y_{t}=y_{t-1}+\delta +e_{t}\] \end_inset \layout Standard That time series y is not stationary. But if you difference \begin_inset Formula \( y_{t} \) \end_inset , you can make it stationary. If that resulting series is not stationary, you can difference again: \layout Standard \begin_inset Formula \( \Delta ^{2}y_{t}=(y_{t}-y_{t-1})-(y_{t-1}-y_{t-2}) \) \end_inset \layout Standard Note that \begin_inset Formula \( \Delta =1-L, \) \end_inset so above we could write \layout Standard \begin_inset Formula \[ (1-L)^{d}y_{t}=\frac{\theta (L)}{\phi (L)}\varepsilon _{t}\] \end_inset \layout Standard d is the order of the differencing required to get a stationary series. \layout Standard Fractional Itegration is the (in my opinion) hard to understand notion that d might not have values of 0, 1, or 2, as Box and Jenkins originally assumed, but rather it can be any real number, 0 or greater. \the_end