Generalized Operational Neural Networks

Recent advances in machine learning led to remarkable solutions in many research problems, notably in fields of computer vision, natural language processing and games. This is due to new parallel computing capabilities for scientific computing (Graphical Processing Units–GPUs and distributed computing), the availability of enormous sets of annotated data, and methodological contributions in a family of models called artificial neural networks, and commonly referred to as Deep Learning. Such state-of-the-art models are commonly formed by an enormous number (in the order of hundreds of thousand, or even millions) of parameters which are jointly tuned in an end-to-end optimization process to fit the training data. However, most of research and practical implementations of these powerful learning models exploit a simplistic model for the artificial neuron, while the topologies used for different tasks are manually optimized based on laborious experimentation requiring expert knowledge (either human expert or expert systems) leading to extremely high computational optimization processes.

Very recently, the Generalized Operational Perceptron (GOP) was proposed leading to a generalized family of artificial neural networks. GOPs are feedforward neural network architectures that exploit more sophisticated types of artificial neurons for efficiently capturing different types of data transformations and nonlinearities appearing in different problems. A number of methodologies have been also proposed for automatically determining the network’s topology, as well as the type of the individual neurons forming each layer of the network. Extensions using skip (or memory) connections and weighted optimization schemes for handling imbalanced classes have been also very recently proposed. Convergence analysis of GOPs and empirical evidence on a wide range of problems indicate that GOPs can achieve competitive (or higher) performance compared to conventional feedforward neural networks, while being more compact and efficient. Finally, the most recent ANN model, Operational Neural Networks (ONNs) that are derived from GOPs have achieved remarkable image processing capabilities such as image denoising, syntheses and transformation, that cannot be otherwise achieved by the conventional Convolutional Neural Nets (CNNs) without a “deep” architecture.

Dat Thanh Tran received the B.A.Sc. degree in Automation Engineering from Hameen University of Applied Sciences, and M.Sc. degree in Data Engineering and Machine Learning from Tampere University in 2017 and 2019 respectively. He is currently working as a Doctoral Researcher in the Signal Analysis and Machine Intelligence Group at Tampere University led by Professor Moncef Gabbouj. His research interests include statistical pattern recognition and machine learning, especially machine learning models with efficient computation and novel neural architectures, finding applications in computer vision as well as financial time-series analysis.

Alexandros Iosifidis (SM’16) is an Associate Professor of Machine Learning in Aarhus University, Denmark. He has (co-)authored 60 journal papers, 83 conference papers and 4 abstracts, 4 book chapters and 1 patent in topics of his expertise. Dr. Iosifidis is a Senior Member of IEEE, he is a member of the EURASIP Technical Area Committee on Visual Information Processing, and he served as an Officer of the Finnish IEEE Signal Processing/Circuits and Systems Chapter from 2016 to 2018. He is currently serving as an Associate Editor for Neurocomputing, IEEE Access, and BMC Bioinformatics journals, as an Area Editor for Signal Processing: Image Communication journal, and as an Academic Editor for PLOS One journal. He was an Area Chair in IEEE ICIP 2018, 2019 & 2020, and a Technical Committee Chair in IEEE ICASSP 2019. His work received several awards, including the H.C. Ørsted Forskerspirer 2018 prize for research excellence and the Academy of Finland Postdoc Fellowship.

Serkan Kiranyaz (SM’13) is a Professor in Qatar University, Doha, Qatar. He published 2 books, 5 book chapters, more than 50 journal papers in high impact journals, and more than 100 papers in international conferences. He made contributions on evolutionary optimization, machine learning, bio-signal analysis, computer vision with applications to recognition, classification, and signal processing.

Prof. Kiranyaz has co-authored the papers which have nominated or received the “Best Paper Award” in ICIP 2013, ICPR 2014, ICIP 2015 and IEEE TSP 2018. He had the most-popular articles in the years 2010 and 2016, and most-cited article in 2018 in IEEE Transactions on Biomedical Engineering. During 2010-2015 he authored the 4th most-cited article of the Neural Networks journal. His research team has won the 2nd and 1st places in PhysioNet Grand Challenges 2016 and 2017, among 48 and 75 international teams, respectively.

Moncef Gabbouj (F’11) is a well-established world expert in the field of image processing, and held the prestigious post of Academy of Finland Professor during 2011-2015. He has been leading the Multimedia Research Group for nearly 25 years and managed successfully a large number of projects in excess of 18M Euro. He has supervised 45 PhD theses and over 50 MSc theses. He is the author of several books and over 700 papers. His research interests include Big Data analytics, multimedia content-based analysis, indexing and retrieval, artificial intelligence, machine learning, pattern recognition, nonlinear signal and image processing and analysis, voice conversion, and video processing and coding. Dr. Gabbouj is a Fellow of the IEEE and member of the Academia Europaea and the Finnish Academy of Science and Letters. He is the past Chairman of the IEEE CAS TC on DSP and committee member of the IEEE Fourier Award for Signal Processing. He served as associate editor and guest editor of many IEEE, and international journals.

Junaid Malik received his Bachelor’s degree in Electrical Engineering from National University of Science and Technology, Pakistan in 2013. He completed his MSc in Information Technology with majors in Signal Processing from Tampere University of Technology in 2017. He is currently pursuing a PhD degree from Tampere University, Finland and works in SAMI research group as a Doctoral Researcher. His research interests include salient object segmentation, discriminative learning-based image restoration and operational neural networks.