#LyX 1.3 created this file. For more info see http://www.lyx.org/ \lyxformat 221 \textclass literate-article \begin_preamble \usepackage{ragged2e} \RaggedRight \setlength{\parindent}{1 em} \usepackage{lmodern} \end_preamble \language english \inputencoding default \fontscheme times \graphics default \paperfontsize 12 \spacing single \papersize Default \paperpackage a4 \use_geometry 1 \use_amsmath 0 \use_natbib 0 \use_numerical_citations 0 \paperorientation portrait \leftmargin 1in \topmargin 1in \rightmargin 1in \bottommargin 1in \secnumdepth 3 \tocdepth 3 \paragraph_separation indent \defskip medskip \quotes_language english \quotes_times 2 \papercolumns 1 \papersides 1 \paperpagestyle default \layout Title Normal Distribution \layout Author Paul E. Johnson \layout Section Mathematical Description \layout Standard The Normal distribution describes a continuous variable that takes on values in the real number line. The formula for the Normal has two \series bold parameter \series default s, \begin_inset Formula $\mu$ \end_inset and \begin_inset Formula $\sigma^{2}$ \end_inset . The parameter \begin_inset Formula $\mu$ \end_inset is a \begin_inset Quotes eld \end_inset location \begin_inset Quotes erd \end_inset parameter and \begin_inset Formula $\sigma^{2}$ \end_inset is a \begin_inset Quotes eld \end_inset scale \begin_inset Quotes erd \end_inset parameter. The probability density function is often written as \begin_inset Formula \[ p(x)=\frac{1}{\sqrt{2\pi\sigma^{2}}}\, e^{-\left(\frac{(x-\mu)^{2}}{2\sigma^{2}}\right)}\] \end_inset \layout Standard I think it looks a little nicer if rearranged. \begin_inset Formula \[ p(x)=\frac{1}{\sqrt{2\pi}\sigma}\, e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^{2}}\] \end_inset \layout Standard If you want to make sense out of that, it is vital for you to have a mental image of the function \begin_inset Formula $e^{-z^{2}}$ \end_inset on the interval between 0 and 5: \layout Standard \begin_inset ERT status Open \layout Standard <>= \newline asequence<- seq(from=0,to=5,by=0.1) \newline expnegx2 <- exp(-asequence^2) \layout Standard plot(asequence,expnegx2,type="l",ylab=expression(exp(-z^2)),xlab="z") \newline @ \end_inset \layout Standard Note well that \begin_inset Quotes eld \end_inset all of the action \begin_inset Quotes erd \end_inset in the Normal probability formula is in the \begin_inset Formula $e^{-z^{2}}$ \end_inset part, not in the \begin_inset Quotes eld \end_inset normalizing constant \begin_inset Quotes erd \end_inset \begin_inset Formula $\frac{1}{\sqrt{2\pi\sigma^{2}}}$ \end_inset . \layout Standard In maximum likelihood, we usually end up trying to maximize the log of the likelihood function, and when the Normal is logged, then it simplifies quite dramatically: \layout Standard \begin_inset Formula \begin{eqnarray*} ln(p(x)) & = & ln\left[\frac{1}{\sqrt{2\pi\sigma^{2}}}\right]+ln\left[e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^{2}}\right]\\ & = & \left[ln(1)-ln(\sqrt{2\pi\sigma^{2}})\right]-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^{2}\\ & & 0-ln((2\pi\sigma^{2})^{1/2})-\frac{2}{2\sigma^{2}}(x-\mu)^{2}\\ & = & -\frac{1}{2}ln(2\pi\sigma^{2})-\frac{1}{2\sigma^{2}}(x-\mu)^{2}\\ & & -\frac{1}{2}ln(2)-\frac{1}{2}ln(\pi)-ln(\sigma)-\frac{1}{2\sigma^{2}}(x-\mu)^{2}\end{eqnarray*} \end_inset \newline \layout Section Illustrations \layout Standard The probability density function of a Normal distribution with \begin_inset Formula $\mu=3$ \end_inset and \begin_inset Formula $\sigma=5$ \end_inset is shown in Figure \begin_inset LatexCommand \ref{cap:Normal-Distribution} \end_inset . The R code which produces that figure is: \layout LyX-Code \begin_inset ERT status Open \layout Standard <>= \newline mu <- 3 \newline sigma <- 5 \layout Standard xrange<- seq(from=mu-3*sigma,to=mu+3*sigma,by=0.2) \layout Standard mainlabel<- expression(paste("Normal Distribution, N(",mu,",",sigma,"^2",")",sep="")) \layout Standard xprob <- dnorm(xrange, mean=mu, sd=sigma, log=F) \layout Standard plot(xrange, xprob, type="l",main=mainlabel,xlab="possible values of x",ylab="probability of x") \layout Standard @ \end_inset \layout Standard How would one describe that? Well, off the top of my head, I'd say the most outstanding characteristics are that it is: \layout Enumerate Unimodal \layout Enumerate Symmetric \layout LyX-Code \begin_inset Float figure wide false collapsed false \layout Caption Normal Distribution \begin_inset LatexCommand \label{cap:Normal-Distribution} \end_inset \layout Standard \begin_inset ERT status Open \layout Standard <>= \layout Standard <> \layout Standard @ \end_inset \end_inset \layout Standard How does this distribution change in appearance if \begin_inset Formula $\mu$ \end_inset and \begin_inset Formula $\sigma^{2}$ \end_inset are changed? Let's do some experimentation. The following R code creates an array of figures with 4 rows and 2 columns with various values of \begin_inset Formula $\mu$ \end_inset and \begin_inset Formula $\sigma$ \end_inset . \layout Standard \begin_inset ERT status Open \layout Standard <>= \newline par(mfrow=c(4,2)) \layout Standard for ( i in 1:4){ \layout Standard for ( j in 1:2) { \layout Standard mu=3*j \newline sigma<-2*i \newline xrange<-seq(from=mu-3*sigma,to=mu+3*sigma,by=0.2) \newline mainlabel<- paste("N(",mu,",",sigma*sigma,",)",sep="") \newline xprob <- dnorm(xrange, mean=mu, sd=sigma, log=F) \newline plot(xrange, xprob, type="l", main=mainlabel, xlab="possible values of x", ylab="probability of x") \newline } \newline } \newline @ \end_inset \layout Standard \begin_inset LatexCommand \ref{cap:Variety-of-Normals} \end_inset \begin_inset Float figure wide false collapsed false \layout Caption Variety of Normals \begin_inset LatexCommand \label{cap:Variety-of-Normals} \end_inset \layout Standard \begin_inset ERT status Open \layout Standard <>= \newline <> \newline @ \end_inset \end_inset \layout Standard While Figure \begin_inset LatexCommand \ref{cap:Variety-of-Normals} \end_inset is quite boring and repetitious, it does convey one very important attribute of the Normal distribution: \series bold it always keeps the same shape \series default . At least for these parameter values, it is unimodal and symmetric. These graphs look the same because the X axis is allowed to re-scale itself to use up the allocated space. \layout Standard If we restrict the display so that the axes of all of the figures are kept the same--in a position that suits the largest set of values--then the impact of changing the parameters is a bit more apparent. The code only needs to be modified very slightly by a specification of the xlim option in the plot statement. \layout Standard \begin_inset ERT status Open \layout Standard <>= \newline par(mfrow=c(4,2)) \layout Standard for ( i in 1:4){ \layout Standard for ( j in 1:2) { \layout Standard mu=3*j \newline sigma<-2*i \newline xrange<-seq(from=mu-3*sigma,to=mu+3*sigma,by=0.2) \newline mainlabel<- paste("N(",mu,",",sigma*sigma,",)",sep="") \newline xprob <- dnorm(xrange, mean=mu, sd=sigma, log=F) \newline plot(xrange, xprob, type="l", main=mainlabel, xlab="possible values of x", ylab="probability of x", xlim=c(-20,30)) \newline } \newline } \newline @ \end_inset \layout Standard The result is to be seen in Figure \begin_inset LatexCommand \ref{cap:Normals-(again)} \end_inset . \begin_inset Float figure wide false collapsed false \layout Caption Normals (again) \begin_inset LatexCommand \label{cap:Normals-(again)} \end_inset \layout Standard \begin_inset ERT status Open \layout Standard <>= \newline <> \layout Standard @ \end_inset \end_inset \layout Section What about the parameters \begin_inset Formula $\mu$ \end_inset and \begin_inset Formula $\sigma^{2}$ \end_inset ? \layout Standard As already mentioned, the parameter \begin_inset Formula $\mu$ \end_inset indicates the \begin_inset Quotes eld \end_inset location \begin_inset Quotes erd \end_inset , the left-right position of the distribution. The parameter \begin_inset Formula $\sigma$ \end_inset is a \begin_inset Quotes eld \end_inset scale \begin_inset Quotes erd \end_inset parameter, determining how far it reaches from left to right. \layout Standard Most students are already aware of the fact that \begin_inset Formula $\mu$ \end_inset and \begin_inset Formula $\sigma$ \end_inset actually play a more familiar role. In fact, many students tell me that the mean of anything is represented by a parameter \begin_inset Formula $\mu$ \end_inset and standard deviation is \begin_inset Formula $\sigma.$ \end_inset But it is not always so. \layout Standard Now, in case you are not one of those students, here's some background. The \series bold expected value \series default of a distribution is defined as the \begin_inset Quotes eld \end_inset probability weighted sum \begin_inset Quotes erd \end_inset of outcomes. For \begin_inset Formula $x\sim N(\mu,\sigma^{2})$ \end_inset , \begin_inset Formula \[ E(x)=\int_{-\infty}^{+\infty}p(x)\cdot x\, dx\] \end_inset \newline and, through the magic of mathematics, it turns out that \begin_inset Formula \[ E(x)=\mu\] \end_inset \layout Standard How do you find that? Well, honestly, I think most of us just look up the answer in a book! If you really want to calculate it, get a book on mathematica l statistics and find out how to use a \begin_inset Quotes eld \end_inset moment generating function \begin_inset Quotes erd \end_inset to calculate the expected values of distributions. \layout Standard The \series bold variance \series default of a distribution is the \begin_inset Quotes eld \end_inset probability weighted sum \begin_inset Quotes erd \end_inset of the squared differences between outcomes and their expected values. \begin_inset Formula \[ Var(x)=\int_{-\infty}^{+\infty}p(x)\cdot\left[x-E(x)\right]^{2}\, dx\] \end_inset which can be rearranged as \begin_inset Formula \[ Var(x)=\int_{-\infty}^{+\infty}p(x)\cdot x^{2}\, dx-E(x)^{2}=E(x^{2})-E(x)^{2}\] \end_inset \newline Repeat out loud: \begin_inset Quotes eld \end_inset The Variance of x equals the Expected value of \begin_inset Formula $x$ \end_inset squared minus the Expected value of \begin_inset Formula $x$ \end_inset , squared. \begin_inset Quotes erd \end_inset \layout Standard Again through the magic of mathematics, one would find \begin_inset Formula \[ Var(x)=\sigma^{2}\] \end_inset \layout Standard I think my point here is that it is a completely fortuitous thing that the parameters \begin_inset Formula $\mu$ \end_inset and \begin_inset Formula $\sigma^{2}$ \end_inset are simply equal to the expected value and variance (respectively). This is quite rare in the gamut of statistical distributions. The expected value and variance are almost always equal to some more complicate d formulae which combine the parameters. \the_end