#LyX 1.5.3 created this file. For more info see http://www.lyx.org/ \lyxformat 276 \begin_document \begin_header \textclass literate-article \begin_preamble \usepackage{ragged2e} \RaggedRight \setlength{\parindent}{1 em} \usepackage{lmodern} \end_preamble \language english \inputencoding default \font_roman times \font_sans default \font_typewriter default \font_default_family default \font_sc false \font_osf false \font_sf_scale 100 \font_tt_scale 100 \graphics default \paperfontsize 12 \spacing single \papersize default \use_geometry true \use_amsmath 1 \use_esint 0 \cite_engine basic \use_bibtopic false \paperorientation portrait \leftmargin 1in \topmargin 1in \rightmargin 1in \bottommargin 1in \secnumdepth 3 \tocdepth 3 \paragraph_separation indent \defskip medskip \quotes_language english \papercolumns 1 \papersides 1 \paperpagestyle default \tracking_changes false \output_changes false \author "" \end_header \begin_body \begin_layout Title Wishart \end_layout \begin_layout Author Paul Johnson & James W. Stoutenborough \end_layout \begin_layout Section Introduction. \end_layout \begin_layout Standard Suppose you have drawn an observation from a Wishart distribution. What do you have? \end_layout \begin_layout Standard You have a matrix! A square set of numbers: \begin_inset Formula \[ \left[\begin{array}{cccc} w_{11} & w_{12} & w_{13} & w_{14}\\ w_{21} & w_{22} & w_{23} & w_{24}\\ w_{31} & w_{32} & w_{33} & w_{44}\\ w_{41} & w_{42} & w_{43} & w_{45}\end{array}\right]\] \end_inset \end_layout \begin_layout Standard Would you like a sample? Run this in R: \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Standard <>= \end_layout \begin_layout Standard library(mvtnorm) \end_layout \begin_layout Standard meanV <- c(0,0,0,0) \end_layout \begin_layout Standard varV <- 3*diag(4) \end_layout \begin_layout Standard X<- rmvnorm(10, mean=meanV,sigma=varV) \end_layout \begin_layout Standard CPX <- t(X) %*% X \end_layout \begin_layout Standard CPX \end_layout \begin_layout Standard @ \end_layout \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Standard <>= \end_layout \begin_layout Standard <> \end_layout \begin_layout Standard @ \end_layout \end_inset \end_layout \begin_layout Standard How about another draw from my Wishart? \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Standard <>= \end_layout \begin_layout Standard <> \end_layout \begin_layout Standard @ \end_layout \end_inset \end_layout \begin_layout Standard There's more where that came from: \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Standard <>= \end_layout \begin_layout Standard <> \end_layout \begin_layout Standard @ \end_layout \end_inset \end_layout \begin_layout Standard Are you getting the idea? The outcomes are blocks of numbers. \end_layout \begin_layout Standard Is there any restriction on these values? Yes. They are sums of squares and cross products. \end_layout \begin_layout Standard If you note the R code, you see that I've just told R to collect a sample of 10 draws from a multivariate Normal \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Standard <>= \end_layout \begin_layout Standard X \end_layout \begin_layout Standard @ \end_layout \end_inset \newline and then create the cross product matrix, \begin_inset Formula $X'X$ \end_inset . \end_layout \begin_layout Section Mathematical Description \end_layout \begin_layout Standard Please review the writeup on the Chi-Square distribution. Recall a Chi-Square variable can be thought of as the result of sampling from a standard normal distribution and then squaring the elements and summing them. In vectors, think of \begin_inset Formula $x$ \end_inset as a column vector. Then the sum of squares is: \begin_inset Formula \[ x'\cdot x=\left[\begin{array}{cccc} x_{1} & x_{2} & x_{3} & x_{4}\end{array}\right]\left[\begin{array}{c} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\end{array}\right]\] \end_inset \end_layout \begin_layout Standard If you draw a set of observations from this multivariate Normal distribution, then when you multiply, you'd have \begin_inset Formula \[ X'X\] \end_inset \newline The number of rows in \begin_inset Formula $X$ \end_inset is the number of random samples drawn, and it is called the \begin_inset Quotes eld \end_inset degrees of freedom. \begin_inset Quotes erd \end_inset The number of columns is equal to the desired number of rows (= columns) of the Wishart output. \end_layout \begin_layout Standard The Wishart distribution is the multivariate analog to the chi-square, in the sense that the Wishart describes the variation you would observe if you repeatedly drew samples and calculated \begin_inset Formula $X'X$ \end_inset . It is related to the multivariate normal in the same way that the chi-square is related to the univariate normal. \end_layout \begin_layout Standard The Wishart draw \begin_inset Formula $w_{i}$ \end_inset might be thought of as a reflection of the PRECISION of an MVN distribution. If the covariance matrix of the MVN is called \begin_inset Formula $\Sigma$ \end_inset , then its inverse, \begin_inset Formula $\Sigma^{-1}$ \end_inset , is its precision. \end_layout \begin_layout Section The Probability Denisty \end_layout \begin_layout Standard The Wishart distribution can be characterized by its probability density fuction, as follows. \end_layout \begin_layout Standard Let \begin_inset Formula $W_{i}$ \end_inset be a \begin_inset Formula $m\times m$ \end_inset symmetric matrix which a randomly drawn value according to a Wishart distributi on. The PDF of \begin_inset Formula $W_{i}$ \end_inset is: \begin_inset Formula \[ prob(W_{i}|v,S)\propto|W_{i}|^{(v-m-1)/2}exp(-\frac{1}{2}trace(S\cdot W_{i}))\] \end_inset \newline The \begin_inset Quotes eld \end_inset trace \begin_inset Quotes erd \end_inset is the sum of all diagonal elements. \end_layout \begin_layout Standard By construction, \begin_inset Formula $W_{i}$ \end_inset is \begin_inset Quotes eld \end_inset positive definite \begin_inset Quotes erd \end_inset , so it can serve as a precision matrix in a MVN distribution. The parameter \begin_inset Formula $v$ \end_inset is \begin_inset Quotes eld \end_inset degrees of freedom \begin_inset Quotes erd \end_inset and \begin_inset Formula $S$ \end_inset is a \begin_inset Quotes eld \end_inset cross product \begin_inset Quotes erd \end_inset matrix. See Lancaster, p. 179. \end_layout \begin_layout Section Moments \end_layout \begin_layout Standard The expected value of a Wishart is \begin_inset Formula \[ E(W_{i})=v\, S^{-1}\] \end_inset \end_layout \begin_layout Section About the example code. \end_layout \begin_layout Standard The multivariate Normal is used here, and I chose a particularly simple covariance matrix (mostly because I was lazy). \begin_inset Formula \[ \Sigma_{4}=3*\left[\begin{array}{cccc} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\end{array}\right]\] \end_inset \end_layout \begin_layout Standard The inverse of that very simple matrix is (obviously) \end_layout \begin_layout Standard \begin_inset Formula \[ 1/3*\left[\begin{array}{cccc} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\end{array}\right]\] \end_inset \end_layout \begin_layout Standard This plays the role of \begin_inset Formula $S^{-1}$ \end_inset in the model. \end_layout \end_body \end_document