Prof. En-hui Yang
University of Waterloo, Canada
Hosted by Department of Information Engineering, CUHK
Date: 5 October 2023 (Thursday)
Time: 10:00am – 11:00am
Venue: Room 801, Ho Sin Hang Engineering Building, CUHK
Abstract:
In 1943, Shannon, father of information theory, and Turing, father of computer science, met at teatime in the Bell Labs cafeteria. During that time, both were involved in wartime research. No one knew what they really discussed except that they impressed each other with their own intellectual depth. 80 years later, when deep learning-based artificial intelligence pushes our information age to a new high, one may wonder what they would discuss when these two giants would meet again.
Aiming to provide an educated guess, in this talk we will introduce the concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) to measure the concentration and separation performance of a classification deep neural network (DNN) in the output probability distribution space of the DNN, where CMI and the ratio between CMI and NCMI represent the intra-class concentration and inter-class separation of the DNN, respectively. By using NCMI to evaluate popular DNNs pretrained over ImageNet in the literature, it is shown that NCMI and error rate have essentially a positive linear relationship with their correlation > 0.99. Based on this observation, the standard deep learning (DL) framework is further modified to minimize the standard cross entropy function subject to an NCMI constraint, yielding CMI constrained deep learning (CMIC-DL). A novel alternating learning algorithm is proposed to solve such a constrained optimization problem. Extensive experiment results show that DNNs trained within CMIC-DL outperform the state-of-the-art models trained within the standard DL and other loss functions in the literature in terms of both accuracy and robustness against adversarial attacks. In addition, visualizing the evolution of learning process through the lens of CMI and NCMI will also be advocated.
Biography:
After graduation from the Dept. of Mathematics at Nankai University with his PhD degree in 1991, En-hui Yang stayed on at Nankai University as a faculty member. Since 1997, he has been with the Dept. of Electrical and Computer Engineering, University of Waterloo, Ontario, Canada, where he is now a Professor and Canada Research Chair in information theory and applications. He is the founding director of the Leitch-University of Waterloo multimedia communications lab, a co-founder of SlipStream Data Inc. (now a subsidiary of BlackBerry), and the founder of BicDroid Inc. He also serves as an Executive Council Member of China Overseas Friendship Association, an Expert Advisor for the Overseas Chinese Affairs Office of the State Council of China, an Overseas Advisor for the Overseas Chinese Affairs Office of the City of Shanghai, a Board Trustees of Huaqiao University, China, and a member of IEEE Eric E. Sumner Award Committee.
He served, inter alia, as a member of Presidential Nominating Committee of the Academy of Science of Royal Society of Canada; a Board Governor of the University of Waterloo; a member of IEEE Founders Medal Committee; a review panel member for the International Council for Science; an Evaluator for the 2017 Japan Prize; an Associate Editor for IEEE Transactions on Information Theory; a general co-chair of the 2008 IEEE International Symposium on Information Theory; a technical program vice-chair of the 2006 IEEE International Conference on Multimedia & Expo (ICME); the chair of the award committee for the 2004 Canadian Award in Telecommunications; a co-editor of the 2004 Special Issue of the IEEE Transactions on Information Theory; a co-chair of the 2003 US National Science Foundation (NSF) workshop on the interface of Information Theory and Computer Science; and a co-chair of the 2003 Canadian Workshop on Information Theory.
Dr. Yang is a Fellow of IEEE, a Fellow of the Canadian Academy of Engineering, and a Fellow of the Royal Society of Canada. He is also a recipient of several awards, including the 2023 Canadian Award for Telecommunications Research; the 2021 IEEE Eric E. Sumner Award; the prestigious Inaugural Ontario Premier’s Catalyst Award in 2007 for the Innovator of the Yearthe 2007 Ernest C. Manning Award of Distinction, one of the Canada’s most prestigious innovation prizes; the 2013 CPAC Professional Achievement Award; the 2014 IEEE Information Theory Society Padovani Lecture Award; and the 2014 FCCP Education Foundation Award of Merit. Products based on his early inventions and commercialized by his previous company, SlipStream, received the 2006 Ontario Global Traders Provincial Award. His research work has benefited people over 170 countries through commercialized products, video coding open sources, and video coding standards. In 2011, he was selected for inclusion in Canadian Who’s Who.
Announcement: 
View
Slide: View
Seminar Video: View
Illustrative Video: View