#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass article
\begin_preamble
\usepackage{ragged2e}
\RaggedRight
\setlength{\parindent}{1 em}
\end_preamble
\language english
\inputencoding auto
\fontscheme palatino
\graphics default
\paperfontsize 12
\spacing single 
\papersize Default
\paperpackage a4
\use_geometry 1
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\leftmargin 1in
\topmargin 1in
\rightmargin 1.1in
\bottommargin 1in
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default

\layout Title

Linear operators
\layout Author

Paul E.
 Johnson
\layout Section

Linear Operator
\layout Standard


\begin_inset Quotes eld
\end_inset 

Linear operator
\begin_inset Quotes erd
\end_inset 

 sounds jargonish.
 But it is a pretty important term.
 In just about any problem in statistics, the linearity of an operator will
 make for massive simplification.
\layout Subsection

Definition
\layout Standard

A linear operator is one that can 
\begin_inset Quotes eld
\end_inset 

take in
\begin_inset Quotes erd
\end_inset 

 a sum and give back a result in the form of a sum of the applied operators.
 That is, it is like 
\begin_inset Formula $F()$
\end_inset 

 in an expression like this
\begin_inset Formula \[
F(x+y+z)=F(x)+F(y)+F(z)\]

\end_inset 


\newline 
Here you see that a linear operator has a distributive quality.
 
\layout Standard

In my usual poetic style, 
\layout Quote

The operator over a sum is the sum of the applications of the operator.
\layout Standard

If you throw in constants 
\begin_inset Formula $a$
\end_inset 

, 
\begin_inset Formula $b$
\end_inset 

, and 
\begin_inset Formula $c$
\end_inset 

, then linearity also means that
\begin_inset Formula \[
F(ax+by+cz)=aF(x)+bF(y)+cF(z)\]

\end_inset 


\layout Subsection

Linear Operators you already know and love.
\layout Standard

You already know many linear operators.
 
\layout Subsubsection

Summation.
\layout Standard

This is obvious, isn't it? This is just the principle of addition.
\begin_inset Formula \[
\sum(x+y)=\sum(x)+\sum(y).\]

\end_inset 

Given two variables, 
\begin_inset Formula $x=\{ x_{1},x_{2},x_{3},...,x_{N}\}$
\end_inset 

 and 
\begin_inset Formula $y=\{ y_{1},y_{2},y_{3},...,y_{N}\}$
\end_inset 


\begin_inset Formula \[
\sum_{i=1}^{N}(x_{i}+y_{i})=\sum_{i=1}^{N}x_{i}+\sum_{i=1}^{N}y_{i}\]

\end_inset 


\layout Subsubsection

Expected Value.
\layout Standard

The Expected Value operator is linear.
 Recall, for a discrete variable with 
\begin_inset Formula $m$
\end_inset 

 possible different values, 
\begin_inset Formula $\{ x_{1},x_{2},...,x_{m}\}$
\end_inset 

, the expected value is defined as:
\begin_inset Formula \[
E(x)=\sum_{i=1}^{m}f(x_{i})\cdot x{}_{i}\]

\end_inset 


\newline 
What's 
\begin_inset Formula $E(x+y)$
\end_inset 

? Doesn't it depend on the probability distributions for 
\begin_inset Formula $x$
\end_inset 

 and 
\begin_inset Formula $y$
\end_inset 

? NO.
 
\begin_inset Formula $E(x)$
\end_inset 

 is a linear operator, so
\begin_inset Formula \[
E(x+y)=E(x)+E(y)\]

\end_inset 

 Since 
\begin_inset Formula $E()$
\end_inset 

 is a linear operator, it radically simplifies many calculations in statistics.
\layout Subsection

Operators that you love which are not linear
\layout Standard

Don't make the mistake of thinking everything that is good is also linear.
 Recall, for example:
\begin_inset Formula \[
V(x+y)=V(x)+V(y)+2Cov(x,y)\]

\end_inset 

 Even so, we often try to 
\begin_inset Quotes eld
\end_inset 

cheat
\begin_inset Quotes erd
\end_inset 

 and make 
\begin_inset Formula $V()$
\end_inset 

 act as if it were linear.
 How many times do you assume away the covariance term? Almost all the time
 in intermediate regression.
\layout Standard

I mention this to remind you that, if you want to apply the property of
 linearity, 
\shape smallcaps 
you can only do so when you have some evidence that the operator really
 is linear.
 
\layout Section

The Derivative is a linear operator
\layout Standard

You might be asked to find the derivative of a sum of functions, such as
 
\begin_inset Formula \begin{eqnarray*}
\frac{\partial}{\partial x}\left(f_{1}(x)+f_{2}(x)\right) & = & ?\end{eqnarray*}

\end_inset 

The derivative is a linear operator, apply the 
\begin_inset Quotes eld
\end_inset 

derivative operator
\begin_inset Quotes erd
\end_inset 

 
\begin_inset Formula $\frac{\partial}{\partial x}$
\end_inset 

 to the individual terms, to find:
\begin_inset Formula \[
\frac{\partial f_{1}(x)}{\partial x}+\frac{\partial f_{2}(x)}{\partial x}\]

\end_inset 


\layout Standard

The beauty of this is that you can solve a series of small problems and
 add up the solutions, rather than solving one giant confusing problem.
 If
\begin_inset Formula \[
\frac{\partial f(x)}{\partial x}=7x\]

\end_inset 

 and
\begin_inset Formula \[
\frac{\partial g(x)}{\partial x}=3x^{2}\]

\end_inset 

 then
\begin_inset Formula \[
\frac{\partial}{\partial x}\left(f(x)+g(x)\right)=7x+3x^{2}\]

\end_inset 


\layout Section

The Integral is also a linear operator.
\layout Standard

One of the really handy rules is that the integral of a sum is the sum of
 the integrals.
\begin_inset Formula \[
\int\{ f(x)+g(x)\} dx=\int f(x)dx+\int g(x)dx\]

\end_inset 


\the_end