Department of Electrical & Computer Engineering 

Discriminant analysis via neural network reduction of the class separation

22. M. Aladjem, (1998) "Nonparametric discriminant analysis via recursive optimization of Patrick-Fisher distance", IEEE Trans. on Syst. ,Man, Cybern, vol. 28B, No 2, pp. 292-299.

23. M. Aladjem, (1998) "Supervised learning of a neural network for classification via successive modification of the training data - an experimental study " in A.P. del Pobil, J.Mira and M.Ali (eds.), Lecture Notes in Artificial Intelligence-11th Int. Conf. on Industrial & Engineering Applications of the Artificial Intelligence & Expert Systems (IEA/AIE-98), Vol. II, Benicassim, Castellon (Spain), June 1-4, 1998, Springer, 593-602.

24. M. Aladjem, (1998) "Training of an ML neural network for classification via recursive reduction of the class separation", 14th Int. Conf. on Pattern Recognition, Brisbane, Queensland, Australia, August 17-20, 1998, IEEE Computer Society Press, (in press). 
25. M. Aladjem, (1998) "Linear discriminant analysis for two classes via recursive neural network reduction of the class separation ", in A.Amin and P.Pudil (eds.), Lecture Notes in Computer Science- 2nd Int. Workshop on Statistical Techniques in Pattern Recognition, Sydney, Australia, August 11-13, 1998, Springer, (in press).

In conference paper [25] we discuss discriminant analysis of two classes which is carried out by a linear mapping, which maximizes the Patrick-Fisher (PF) distance. The PF distance is a highly nonlinear function with respect to the mapping, and has more than one maximum. In [22] we proposed a recursive method which searches for several large local maxima of the PF distance via successive “reduction of the class separation”. In this work we generalize this method. We propose a neural network (NN) implementation of the procedure for “reduction of the class separation”, which increases its efficacy. We use an auto-associative multi-layer network having non-linear activation functions instead of a linear transformation performed in our previous work [22]. This increases the computational complexity, which is the price we pay in order to gain the following advantages:
1. The NN implementation improves the preservation of the training data: Our method, proposed in [22], exactly preserves the data in the subspace orthogonal to the vector which is the object of the reduction of the class separation. The NN implementation, by performing highly non-linear data transformation, increases the range of data preservation, which is demonstrated by the experiments explained in the conference paper [25]. 
2. The NN implementation can be applied for reduction of the class separation of the non-linear classification functions: Actually, by using the auto-associative network, we overcome the use of an orthonormal linear transformation . This makes it possible to apply NN implementation for the non-linear classification functions. Using this feature of the NN implementation, we proposed a method for recursive training of a multi-layer (ML) neural network for classification (conference papers [23,24]). We have compared our method and conventional training with random initialization of the weights using a synthetic data set and the data set of an OCR problem for discrimination the upper-case handwritten letters “M” and the lower-case handwritten letters “m”. The results obtained confirm the efficacy of our method which finds solutions with lower misclassification errors than does conventional training.

More Synopsis of Research

Novel discriminant criteria

Interactive system for exploratory data analysis

Method for estimating the significance of the control parameters of projection procedures

Multiclass discriminant projections

New methods for successive optimization of the discriminant criteria

Comparative study of neural networks for multivariate data projection

Discriminant analysis via neural network reduction of the class separation

Back to main home page