#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass article
\begin_preamble
\usepackage{ragged2e}
\RaggedRight
\setlength{\parindent}{1 em}
\end_preamble
\language english
\inputencoding auto
\fontscheme pslatex
\graphics default
\paperfontsize 12
\spacing single 
\papersize Default
\paperpackage a4
\use_geometry 1
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\leftmargin 1in
\topmargin 1in
\rightmargin 1in
\bottommargin 1in
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default

\layout Title

Kalman Filter
\layout Standard

Paul E.
 Johnson
\layout Standard

August 18, 2004
\layout Section

The problem behind the Kalman filter
\layout Standard

We have a 
\begin_inset Quotes eld
\end_inset 

messy time series
\begin_inset Quotes erd
\end_inset 

 and we want to get rid of the 
\begin_inset Quotes eld
\end_inset 

noise
\begin_inset Quotes erd
\end_inset 

.
 Make a picture with a messy time series, then imagine a smooth line instead.
\layout Section

How do you get from here to there?
\layout Standard

The structure of the model is the famous part.
 It is the state-space model.
 
\layout Standard

The observed data is assumed to reflect two parts:
\layout Enumerate

The 
\begin_inset Quotes eld
\end_inset 

true
\begin_inset Quotes erd
\end_inset 

 state of the system, and
\layout Enumerate

A random error.
\layout Standard

The true state of the system is never observed directly, but rather it is
 assumed to exist and it follows a dynamical process that we assume.
 
\layout Standard

It seems to me that no two authors use the same notation for these models,
 but generally speaking, the underlying state follows an equation like this:
\layout Standard


\begin_inset Formula \[
z_{t+1}=Fz_{t}+Ge_{t+1}\]

\end_inset 


\layout Standard

z: the state
\layout Standard

F: coefficients for past values of the state
\layout Standard

G: weight for random error input.
\layout Standard

e: inputs.
 typically assumed normal, white noise
\layout Standard

This model allows the possibility that z is a vector of many variables,
 and it generates a VAR in that case.
 Dont worry.
 Many models assume that the state vector z actually includes the input
 variables x within it, some do not.
 Sometimes that's confusing.
 Its confusing because sometimes it seems like there are 
\begin_inset Quotes eld
\end_inset 

no inputs
\begin_inset Quotes erd
\end_inset 

 into a system, but that's just because somebody labels them as part of
 the state.
\layout Standard

The state is not directly observed, but rather we only see it after some
 random crap and input variables are 
\begin_inset Quotes eld
\end_inset 

thrown in
\begin_inset Quotes erd
\end_inset 

, as in
\layout Standard


\begin_inset Formula \[
y_{t}=Az_{t}+Bx_{t}+w_{t}\]

\end_inset 


\layout Standard

w: another random error
\layout Standard

x: input variables (vector of)
\layout Standard

B: coefficients
\layout Standard

A: coefficients
\layout Standard

y: observed output.
\layout Section

Iterative processing through the Kalman filter.
\layout Standard

Begin with a prior belief about the error terms and we have as information
 all the past values of the observed series.
\layout Standard

Suppose at step k your estimate (guess) of the state is 
\begin_inset Formula $\hat{z}_{k}^{-}$
\end_inset 

.
 That estimate takes into account all information you have from stages 0
 through k.
 And then when you take the measurement of the observed value is 
\begin_inset Formula $y_{k}$
\end_inset 

 and from that you estimate the state is 
\begin_inset Formula $\hat{z}_{k}$
\end_inset 

.
\layout Standard

Then the a priori error is
\layout Standard


\begin_inset Formula $u_{k}^{-}=z_{k}-\hat{z}_{_{k}^{-}}$
\end_inset 


\layout Standard

and the a posteriori error is
\layout Standard


\begin_inset Formula $u_{k}=z_{k}-\hat{z}_{k}$
\end_inset 


\layout Standard

The Kalman filter calculates you posterior estimate by putting together
 your a priori estimate with a formula that takes input from the data.
 Something like:
\layout Standard


\begin_inset Formula \[
\hat{z}_{k}=\hat{z}_{k}^{-}+something(y_{k})\]

\end_inset 


\layout Standard

And the magic of it is that Kalman figured out that the 
\begin_inset Quotes eld
\end_inset 

something
\begin_inset Quotes erd
\end_inset 

 should be like this:
\layout Standard


\begin_inset Formula \[
\hat{z}_{k}=\hat{z}_{k}^{-}+K(y_{k}-H\hat{z}_{k}^{-})\]

\end_inset 


\layout Standard

The difference in parentheses is the 
\begin_inset Quotes eld
\end_inset 

innovation
\begin_inset Quotes erd
\end_inset 

, or 
\begin_inset Quotes eld
\end_inset 

residual
\begin_inset Quotes erd
\end_inset 

, because it 
\begin_inset Formula $H\hat{z}_{k}^{-}$
\end_inset 

 is the predicted measurement.
 The matrix K is the 
\begin_inset Quotes eld
\end_inset 

gain
\begin_inset Quotes erd
\end_inset 

 or 
\begin_inset Quotes eld
\end_inset 

blending
\begin_inset Quotes erd
\end_inset 

 factor.
 It is chosen to minimize the a posteriori estimate of the error covariance,
\layout Standard


\begin_inset Formula \[
P_{k}=E(u_{k}*u_{k}^{T})\]

\end_inset 


\layout Standard

[Note I'm stopping here for lack of time
\backslash 

\backslash 
]
\the_end