2.1 The two step estimation procedure

When estimating structural equation models in Mplus, the estimators known as “MLMV”, “WLSMV”, and “ULSMV” include a feature to create a model summary snapshot file (the so-called DIFFTEST file). The syntax of the SAVEDATA command is:

SAVEDATA:
    DIFFTEST = m0.dif;

This saves a model snapshot that we have chosen to name m0.dif (for clarity). A second model is then estimated, and the input file (say, m1.inp) includes an ANALYSIS stanza that requests a comparison against m0.dif by a DIFFTEST statement:

ANALYSIS:
    ESTIMATOR = WLSMV;
    DIFFTEST = m0.dif

In this nested model comparison, following Asparouhov and Muthén ( 2006) terminology, the first model is called the parent (or the \(H_1\)) model. The dif file for that model (m0.dif) is necessary to calculate the DIFFTEST which compares the parent against the second, nested model (\(H_0\)). In order to calculate the \(T_3\) estimate with R, we need to save the dif snapshot files for both \(H_1\) and \(H_0\) models.

3.1 Step 1 - Extracting model information from an Mplus DIFFTEST File

An Mplus DIFFTEST file contains a single column vector. It combines, without punctuation or labels, the values of 4 scalars, \(T\), \(g\), \(s\) and \(p\), along with consecutive sets of values that must be separated and rearranged into 3 matrices, \(\mathbf{\Delta}_i\), \(\mathbf{P}_i\), and \(\mathbf{V}_i\).

1. \(T\) is the minimized value of the fit function; this matches the value reported for the last iteration in the TECH5 output.

2. The number of groups (\(g\)).

3. The number of sample statistics (\(s\)).

4. The number of model parameters (\(p\)).

5. \(\mathbf{\Delta}_i\), an \(s \cdot g \times p\) matrix to be reconstructed from the next \(s \cdot g \cdot p\) elements in the dif file (there is one stanza for each group to be constructed).

6. \(\mathbf{P}_i\), a symmetric \(p \times p\) matrix, reconstructed from the next \(p(p + 1)/2\) elements in the dif file.

7. \(\mathbf{V}_i\), a symmetric \(p \times p\) matrix, reconstructed from the final \(p(p + 1)/2\) elements in the dif file.

Because the DIFFTEST file provides the one long column of numbers, some trial and error was necessary to separate the values and place them into \(\mathbf{\Delta}_i\), \(\mathbf{P}_i\) and \(\mathbf{V}_i\). The individual values are placed into the matrices row wise. The matrix \(\mathbf{\Delta}_i\) has separate stanzas for each of \(g\) groups. Because \(\mathbf{P}\) and \(\mathbf{V}\) are symmetric matrices, the dif file provides only \(p(p+1)/2\) distinct values for each one. The matrix is reconstructed by filling in a lower triangle row-wise (e.g., insert \(\mathbf{P}[1,1]\), then \(\mathbf{P}[2,1]\), \(\mathbf{P}[2,2]\), \(\mathbf{P}[3,1]\) and so forth). The fact that these are symmetric is relied upon to complete their reconstruction.

3.2 Step 2 - Calculating \(T.chisq\)

T.chisq is computed by first taking the difference between the \(T\) values (the minimized fit function),

\[ T_\Delta = T_{nested} - T_{parent}, \tag{2} \]

and then rescaling \(T_\Delta\) by multiplying \(N*2\), where N is the total sample size:

\[ T.chisq = T_\Delta*N*2. \tag{3} \]

The value of \(T.chisq\) cannot be reconstructed from the output of the two fitted models. In particular, T.chisq is not based on the model \(\chi^2\) values that are reported in Mplus output. When a model is estimated with “WLSMV”, the chi-square value reported is a mean and variance adjusted version of \(T\). In the \(T_3\) test, T.chisq incorporates unscaled values of \(T_{nested}\) and \(T_{parent}\).

3.3 Step 3 - Calculating the \(\mathbf{H}\) Matrix

The calculation of \(T_3\) will require the construction of several separate matrices that will be used to derive the scaling and shifting parameters. Here we are concerned with \(\mathbf{H}\). This matrix is mentioned in Asparouhov and Muthén (2006), where the details of its calculation are left as an exercise for the reader.

For that reason, we do not know how this is calculated in Mplus, but the following will prove to be correct:

\[ H = (\Delta'_1\Delta_1)^{-1}\Delta_1'\Delta_0. \tag{4} \]

This method is suggested by Yves Rosseel in the source code of the R package lavaan (Rosseel, 2017). Note that to obtain the two matrices, \(\mathbf{\Delta}_0\) and \(\mathbf{\Delta}_1\), it is necessary parse both of the model dif files.

3.4 Step 4 - Calculating The \(\mathbf{M}\) Matrix

\(\mathbf{M}\) is an ingredient in the equations for the scaling factor and shift parameter.

As shown by Asparouhov and Muthen (2006), \(\mathbf{M}\) can be calculated as:

\[ M = (P_1 - P_1(H(H'P_1H)^{-1}H'P_1)V_1 \tag{5} \]

using the matrices constructed previously.

3.5 Step 5 - Calculating the Scaling Factor and the Shift Parameter

The scaling factor \(a\) is computed with the following formula:

\[ a = \sqrt{\frac{D}{Tr(M*M)}}. \tag{6} \]

Here \(D\) is the degrees of freedom, calculated as the difference between the numbers of parameters estimated in the two models.

The shift parameter \(b\) is computed as

\[ b = D - \sqrt{\frac{D*Tr(M)^2}{Tr(M*M)}}. \tag{7} \]

In these formulas, \(Tr\) is the trace operator, the sum of the elements along the main diagonal.

3.6 Step 6 - Compute the Scaled \(T_3\) Statistic

Once \(a\) and \(b\) have been computed, they are used in Equation 1 along with \(T.chisq\) to compute the \(T_3\) chi-square difference statistic. Asparouhov and Muthén (2010) suggest that the \(T_3\) statistic follows a chi-square distribution with \(D\) degrees of freedom.

3.7 Using the crmda_difftest Function

The crmda_difftest function takes the file names of the Mplus output files for two nested models as input. For example:

crmda_difftest("parent.out", "nested.out")

We use functions provided by the MplusAutomation package (Hallquist and Wiley, 2017) for R to parse the Mplus output files.

The function detects the location of the companion dif files, parent.dif and nested.dif. In our examples, these are available in the same folder as the output files. The dif files are parsed by an auxiliary function named diffReader. If there are multiple groups, the user must also specify the total sample size as a third argument to crmda_difftest. The function returns the \(T_3\) value calculated by the aforementioned formulas, along with the estimate of \(T_3\) retrieved from the Mplus output file.

4.1 The demonstration data

A subset of the HBSC data (Health Behavior in School-Aged Children; Iannotti, 2005-2006) was used to obtain the demonstration results. The HBSC data contains responses from students in grades 6, 7, 8, 9, and 10. The subset data being used here contains responses to ordinal measures (five-point Likert-type) on self-reported depression level, physical well-being, bullying experiences (got bullied by others and bullied others), and alcohol use. A total of 25 nested model comparison scenarios were devised. The Mplus estimator “WLSMV” was used for all of the model estimations.

4.2 Example models

The following models were fitted to the HBSC data.

4.3 Nested model comparisons

In the following table, we summarize the \(T_3\) values calculated with crmda_difftest, based on the dif files from Mplus, and compare against the values of the \(T_3\) estimates reported in the Mplus output files. These results show that across a variety of model comparison scenarios the crmda_difftest function reproduces the test statistic provided by Mplus exactly to three decimal places.

5.1 R Session Info

R version 3.4.3 (2017-11-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.3

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] psych_1.7.8         MplusAutomation_0.7 xtable_1.8-2       
[4] crmda_0.49          kutils_1.34        

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.15       knitr_1.18         magrittr_1.5      
 [4] mnormt_1.5-5       pbivnorm_0.6.0     lattice_0.20-35   
 [7] quadprog_1.5-5     stringr_1.2.0      plyr_1.8.4        
[10] tools_3.4.3        parallel_3.4.3     grid_3.4.3        
[13] nlme_3.1-131       texreg_1.36.23     coda_0.19-1       
[16] htmltools_0.3.6    yaml_2.1.16        rprojroot_1.3-2   
[19] digest_0.6.15      lavaan_0.5-23.1097 codetools_0.2-15  
[22] evaluate_0.10.1    gsubfn_0.6-6       rmarkdown_1.8     
[25] openxlsx_4.0.17    stringi_1.1.6      pander_0.6.1      
[28] compiler_3.4.3     backports_1.1.2    boot_1.3-20       
[31] stats4_3.4.3       foreign_0.8-69     proto_1.0.0       

Available under Created Commons license 3.0