A block diagonal covariance matrix test and discriminant analysis of high-dimensional data

Abstract

In this dissertation, a new test statistic for testing for a block diagonal covariance matrix structure with a multivariate normal population where the number of variables p exceeds the number of observations n is proposed. Whereas classical approaches such as the likelihood ratio test cannot be applied when p n  , the proposed test statistic is based on the ratio of the estimators of 2 tr and 2 trD , where  is the population covariance matrix and D is the population covariance matrix under the null hypothesis. Furthermore, the asymptotic distribution of the proposed test statistic under the null hypothesis is standard normal. The performance of proposed test statistic was assessed using a simulation study, in which empirical type I error values and the empirical power were used to show its performance. The empirical type I error values were close to the significance level and the empirical power values were closed to 1 in all cases. Moreover, the performance of the proposed test was compared with another previously reported test statistic, and the empirical power values of the proposed test statistic were shown to be higher than those of the comparative test statistic in some cases

Two new discriminant methods for high-dimensional data under the multivariate normal population with a block diagonal covariance matrix structure are also proposed. For the first method, the sample covariance matrix is approximated as a singular matrix based on the idea of reducing the dimensionality of the observations and using a well-conditioned covariance matrix. For the second method, a sample block diagonal covariance matrix is used instead. The performance of these two methods were compared with some of the previously reported methods via a simulation study, the results of which show that both proposed methods outperformed the other comparative methods under various conditions. In addition, the proposed test for testing block diagonal covariance matrix structure and the two new proposed methods for discriminant analysis were applied to a real-life dataset