Power of the score test

SEM programs such as LISREL, EQS, and Mplus give statistics called the Modification Index for each parameter that was constrained to a value. You would get MI’s for parameters set to zero such as loadings in Confirmatory Factor Analysis, but also for parameters that are constrained to be equal, as is done in invariance testing in cross-national research.

Consider for example, a simple two factor model of two types of “political trust”: trust in authorities and trust in institutions, as measured by four variables.

This model has six correlations and five parameters. It therefore has one degree of freedom (df). This one restriction could be used to test, for example, a cross-loading of trust in the parliament on trust in institutions. In the model this parameter is restricted to equal zero, but it might be non-zero.

Modification indices in SEM are also known as ‘score tests’ in econometrics. Each modification index (MI) is the expected decrease in the Likelihood Ratio (chi-square) test statistic if the constrained parameter were to be freed. Under the null hypothesis, each MI (as the sample size increases) has a chi-square distribution with one degree of freedom. The Expected Parameter Change is the corresponding increase in the parameter.

Thus, in the example above, if the cross-loading of trust in the parliament on trust in institutions is to be tested, the MI would be the expected improvement in the chi-square test if that parameter were to be freed, while the EPC would be the value the parameter could be expected to take. (in this rather trivial example with only one restriction in total the MI will equal the chi square test as the sample size increases.)

There is, however, a problem with this approach. If the loading is large but the sample size is small, it may not be detected by the MI (score) test. On the other hand, if the sample size is very large, even a very small and substantively unimportant loading would be enough to reject the score test. Both would lead to an unsatisfactory model. So we have to consider the probability that the MI test’s hypothesis of no effect will be rejected, given that it is false. This probability is called the power of the test.

Given the power of the test, the MI, and the EPC, Saris, Satorra, and Van der Veld suggested the following decision rules:

.	.	Power
.	.	High	Low
MI	Significant	Inspect EPC	Misspecified
.	Insignificant	Not misspecified	Inconclusive

These rules are implemented in the program JRule.