Artificial Classifier Generation for Multi-expert System Evaluation

The evaluation of combination methods for multi-classifier systems is a difficult problem. In many cases multi-classifier combination methods are too complex to be formally studied and the experimental approach is the unique possible strategy. Of course, in order to simulate a multitude of real working conditions, sets of artificial classifiers with diverse characteristics must be generated. This paper presents an effective technique for generating sets of artificial classifiers with different characteristics both at the individual-level (i.e. recognition performance) and at the collective-level (i.e. degree of similarity). In the experimental tests, sets of artificial classifiers simulating different working conditions are generated and the performances of abstract-level combination methods are estimated. The results points out the effectiveness of the new technique for generating sets of artificial classifiers with different characteristics and their usefulness in estimating the performances of combination methods.