Towards Explainable Deep Learning via Successive Subspace Learning

In this tutorial, I will first describe a new interpretable feedforward (FF) design with the LeNet-5 as an example. The FF-designed CNN (FF-CNN) is a data-centric approach that determines network parameters based on training data statistics layer by layer in a one-pass feedforward manner. To build the convolutional layers, we adopt the Saab (Subspace approximation with adjusted bias) transform. The bias in filter weights is chosen to annihilate nonlinearity of the activation function. To build the fully-connected (FC) layers, we adopt a label-guided linear least squared regression (LSR) method. To generalize the FF-CNN, we present the notion of “successive subspace learning (SSL)” and present a couple of concrete examples for image and point cloud classification. The SSL methodology consists of three modules: 1) unsupervised feature extraction, 2) supervised feature selection and feature space partitioning, and 3) ensemble classification. For the feature extraction module, we exploit the weak correlation property among spectral coefficients and decompose a joint-spatial-spectral tensor into multiple joint-spatial-single-spectral tensors. The purpose is to reduce the dimension of vectorized tensors. For the feature selection and feature space partitioning, we introduce the cost function and would like to reduce the cost function as much as possible.  Similarities and differences between SSL and deep learning (DL) are compared.

C.-C. Jay Kuo received his Ph.D. degree from the Massachusetts Institute of Technology in 1987. He is now with the University of Southern California (USC) as Director of the Media Communications Laboratory and Distinguished Professor of Electrical Engineering and Computer Science. Dr. Kuo received the 2016 IEEE Computer Society Taylor L. Booth Education Award, the 2016 IEEE Circuits and Systems Society John Choma Education Award, the 2016 IS&T Raymond C. Bowman Award, the 2017 IEEE Leon K. Kirchmayer Graduate Teaching Award, the 2017 IEEE Signal Processing Society Carl Friedrich Gauss Education Award, the 2019 IEEE Computer Society Edward J. McCluskey Technical Achievement Award, the 2019 IEEE Signal Processing Society Claude Shannon-Harry Nyquist Technical Achievement Award and the 2020 IEEE TCMC Impact Award.