Fusion of Explainable Deep Learning Features Using Fuzzy Integral in Computer Vision

Fuzzy integral fusion has been shown as an effective tool for enhancing classification accuracy while also achieving explainability. With the deep learning boom in the past decade, many researchers have investigated the advantages of fusing various deep neural networks (DNNs) with fuzzy integral techniques in computer vision. However, recent studies focus on only the explainable fusion process. Thus, features learned by each DNN are difficult to understand. Moreover, DNNs are usually trained on the ImageNet dataset, whereas the effectiveness of applying the fuzzy integral methods to this dataset is yet to be investigated. This is the gap that motivates our research study. To address this issue, we explore fuzzy integral fusion classification models that make both the fusion process and extracted features explainable. Specifically, we use two well-known fuzzy integral fusion methods [i.e., Sugeno integral (SI) and Choquet integral (ChI)] to combine three explainable deep learning features (i.e., shape, texture, and color) in a manner that mimics the human visual recognition process. The originality of our work includes the emphasis on complete explainability in the classification process, the investigation of applying fuzzy integral methods to the ImageNet dataset, and extensive experimental validation of the effectiveness of fuzzy integral. Computational experiments show that fuzzy integral fusion can improve classification accuracy by 14.6% compared with an individual DNN on subsets derived from the ImageNet dataset. Furthermore, fuzzy integral fusion helps understand contributions, relationships, and interactions among the three features (shape, texture, and color) for each class, providing convincing evidence for the final classification result. Consequently, the proposed models not only achieve impressive performance, but also provide a thorough understanding of how these models work. © 1993-2012 IEEE.