A research on classification performance of fuzzy classifiers based on fuzzy set theory

Abstract

Due to the complexities of objects and the vagueness of the human mind, it has attracted considerable attention from researchers studying fuzzy classification algorithms. In this paper, we propose a concept of fuzzy relative entropy to measure the divergence between two fuzzy sets. Applying fuzzy relative entropy, we prove the conclusion that patterns with high fuzziness are close to the classification boundary. Thus, it plays a great role in classification problems that patterns with high fuzziness are classified correctly. Meanwhile, we draw a conclusion that the fuzziness of a pattern and the uncertainty of its class label are equivalent. As is well known, entropy not only measures the uncertainty of random variable, but also represents the amount of information carried by the variable. Hence, a fuzzy classifier with high fuzziness would carry much information about training set. Therefore, in addition to some assessment criteria such as classification accuracy, we could study the classification performance from the perspective of the fuzziness of classifier. In order to try to ensure the objectivity in dealing with unseen patterns, we should make full use of information of the known pattern set and do not make too much subjective assumptions in the process of learning. Consequently, for problems with rather complex decision boundaries especially, under the condition that a certain training accuracy threshold is maintained, we demonstrate that a fuzzy classifier with high fuzziness would have a well generalization performance.