#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass article
\begin_preamble
\usepackage{ragged2e}
\RaggedRight
\setlength{\parindent}{1 em}
\end_preamble
\language english
\inputencoding auto
\fontscheme palatino
\graphics default
\paperfontsize 12
\spacing single
\papersize Default
\paperpackage a4
\use_geometry 1
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\leftmargin 1in
\topmargin 1in
\rightmargin 1.1in
\bottommargin 1in
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default
\layout Title
Linear operators
\layout Author
Paul E.
Johnson
\layout Section
Linear Operator
\layout Standard
\begin_inset Quotes eld
\end_inset
Linear operator
\begin_inset Quotes erd
\end_inset
sounds jargonish.
But it is a pretty important term.
In just about any problem in statistics, the linearity of an operator will
make for massive simplification.
\layout Subsection
Definition
\layout Standard
A linear operator is one that can
\begin_inset Quotes eld
\end_inset
take in
\begin_inset Quotes erd
\end_inset
a sum and give back a result in the form of a sum of the applied operators.
That is, it is like
\begin_inset Formula $F()$
\end_inset
in an expression like this
\begin_inset Formula \[
F(x+y+z)=F(x)+F(y)+F(z)\]
\end_inset
\newline
Here you see that a linear operator has a distributive quality.
\layout Standard
In my usual poetic style,
\layout Quote
The operator over a sum is the sum of the applications of the operator.
\layout Standard
If you throw in constants
\begin_inset Formula $a$
\end_inset
,
\begin_inset Formula $b$
\end_inset
, and
\begin_inset Formula $c$
\end_inset
, then linearity also means that
\begin_inset Formula \[
F(ax+by+cz)=aF(x)+bF(y)+cF(z)\]
\end_inset
\layout Subsection
Linear Operators you already know and love.
\layout Standard
You already know many linear operators.
\layout Subsubsection
Summation.
\layout Standard
This is obvious, isn't it? This is just the principle of addition.
\begin_inset Formula \[
\sum(x+y)=\sum(x)+\sum(y).\]
\end_inset
Given two variables,
\begin_inset Formula $x=\{ x_{1},x_{2},x_{3},...,x_{N}\}$
\end_inset
and
\begin_inset Formula $y=\{ y_{1},y_{2},y_{3},...,y_{N}\}$
\end_inset
\begin_inset Formula \[
\sum_{i=1}^{N}(x_{i}+y_{i})=\sum_{i=1}^{N}x_{i}+\sum_{i=1}^{N}y_{i}\]
\end_inset
\layout Subsubsection
Expected Value.
\layout Standard
The Expected Value operator is linear.
Recall, for a discrete variable with
\begin_inset Formula $m$
\end_inset
possible different values,
\begin_inset Formula $\{ x_{1},x_{2},...,x_{m}\}$
\end_inset
, the expected value is defined as:
\begin_inset Formula \[
E(x)=\sum_{i=1}^{m}f(x_{i})\cdot x{}_{i}\]
\end_inset
\newline
What's
\begin_inset Formula $E(x+y)$
\end_inset
? Doesn't it depend on the probability distributions for
\begin_inset Formula $x$
\end_inset
and
\begin_inset Formula $y$
\end_inset
? NO.
\begin_inset Formula $E(x)$
\end_inset
is a linear operator, so
\begin_inset Formula \[
E(x+y)=E(x)+E(y)\]
\end_inset
Since
\begin_inset Formula $E()$
\end_inset
is a linear operator, it radically simplifies many calculations in statistics.
\layout Subsection
Operators that you love which are not linear
\layout Standard
Don't make the mistake of thinking everything that is good is also linear.
Recall, for example:
\begin_inset Formula \[
V(x+y)=V(x)+V(y)+2Cov(x,y)\]
\end_inset
Even so, we often try to
\begin_inset Quotes eld
\end_inset
cheat
\begin_inset Quotes erd
\end_inset
and make
\begin_inset Formula $V()$
\end_inset
act as if it were linear.
How many times do you assume away the covariance term? Almost all the time
in intermediate regression.
\layout Standard
I mention this to remind you that, if you want to apply the property of
linearity,
\shape smallcaps
you can only do so when you have some evidence that the operator really
is linear.
\layout Section
The Derivative is a linear operator
\layout Standard
You might be asked to find the derivative of a sum of functions, such as
\begin_inset Formula \begin{eqnarray*}
\frac{\partial}{\partial x}\left(f_{1}(x)+f_{2}(x)\right) & = & ?\end{eqnarray*}
\end_inset
The derivative is a linear operator, apply the
\begin_inset Quotes eld
\end_inset
derivative operator
\begin_inset Quotes erd
\end_inset
\begin_inset Formula $\frac{\partial}{\partial x}$
\end_inset
to the individual terms, to find:
\begin_inset Formula \[
\frac{\partial f_{1}(x)}{\partial x}+\frac{\partial f_{2}(x)}{\partial x}\]
\end_inset
\layout Standard
The beauty of this is that you can solve a series of small problems and
add up the solutions, rather than solving one giant confusing problem.
If
\begin_inset Formula \[
\frac{\partial f(x)}{\partial x}=7x\]
\end_inset
and
\begin_inset Formula \[
\frac{\partial g(x)}{\partial x}=3x^{2}\]
\end_inset
then
\begin_inset Formula \[
\frac{\partial}{\partial x}\left(f(x)+g(x)\right)=7x+3x^{2}\]
\end_inset
\layout Section
The Integral is also a linear operator.
\layout Standard
One of the really handy rules is that the integral of a sum is the sum of
the integrals.
\begin_inset Formula \[
\int\{ f(x)+g(x)\} dx=\int f(x)dx+\int g(x)dx\]
\end_inset
\the_end