Spring 2018: Deep Learning and Its Applications to Signal and Image Processing and Analysis, 361-21120

Tammy Riklin Raviv

Time Monday, 10:00-13:00
Location Building 90, Room 145


About the course

The class will cover a diverse set of topics in signal processing, image analysis and computer vision and various Neural Network architectures. It will be an interactive course where we will discuss interesting topics on demand and latest research buzz. The goal of the class is to learn about different domains of the field, understand, identify and analyze the main challenges, what works and what doesn't, as well as to identify interesting new directions for future research.
Starting from the last week of November (the second month of the semester) we will read 1 to 4 papers every week. The papers will be presented by the students in a seminar format. The success of the discussion in class will thus be due to how prepared the students come to class. Each student is expected to read at least one paper for each class (and prepare a question or a comment for a discussion). In addition, each student will present a paper in one of the classes. There will be also talks by invited speakers and a lab. class (Nov. 14, room 330, building 33) on Tensor Flow. The Last class will be dedicated to final project presentation (about ten minutes for each project). Details on the final project will be published soon. The final project does not have to be necessarily related to the paper presented in class.

Schedule, links and lecture notes

March 5, 2018   Course overview

March 12, 2018     Optimination, Stochastic gradient descent, Backpropagation, Regularization, Loss functions

March 19, 2018     Regularization for Deep Learning

March 26, 2018   Convolutional Neural Networks (CNN)

                        Tensor Flow (by Assaf Arbelle)
                         location: Electrical Engineering Computer Laboratory, room 330, building 33

April 9, 2018   New Matlab tools for Deep Learning
                       Guest Lecture, Roy Fahn, MATLAB and Simulink

                        Looking into Deep Neural Networks

April 16, 2018   Training deep networks learning based on noisy data
                       Guest Lecture, Prof. Jacob Goldberger, Electrical Engineering, Bar-Ilan University

                       On the importance of single directions for generalization
                       Authors:Ari S. Morcos, David G.T. Barrett, Neil C. Rabinowitz, Matthew Botvinick
                       Presenters: Shir Cohen, Harel Gazit, Or Shwartzman

April 23, 2018   ICA and Information Maximization Approach to Unsupervised Learning in Feedforward and Recurrent Neural Networks
                       Guest Lecture , Dr. Oren Shriki, Cognitive and Brain Sciences at Ben-Gurion University

April 30, 2018   You Only Look Once: Unified, Real-Time Object Detection
                       Authors: Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi
                       Presenters: Alon Matzafi and Gilad Rosenthal

                       U-Net: Convolutional Networks for Biomedical Image Segmentation
                       Authors: Olaf Ronneberger, Philipp Fischer, Thomas Brox
                       Presenters: Liali Ali and Eliya Ben-Avraham

May 7, 2018   Deep Residual Learning for Image Recognition
                       Authors: Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
                       Pesenters: Aharon Kalantar, Ran Bezen, Maor Gaon

                       Where are they looking?
                       Authors: Adria Recasens, Aditya Khosla, Carl Vondrick, Antonio Torralba
                       Presenters: Snir Bar and Ruby Simply

                       Get to the Point: Summarization with Pointer-Generator Networks
                       Authors: Abigail See, Peter J. Liu, Christopher D. Manning
                       Presenters: Matan Eyal

May 21, 2018     A First Look at Music Composition using LSTM Recurrent Neural Networks
                       Douglas Eck and Jurgen Schmidhuber
                       Presenters: Danit Itzkovich and Mor Yemini

                       Spatial Transformer Networks
                       Authors: Max Jaderberg, Karen Simonyan, Andrew Zisserman and Koray Kavukcuoglu
                       Presenter: Asher Fredman

                       A Neural Algorithm of Artistic Style
                       Leon A. Gatys, Alexander S. Ecker, Matthias Bethge
                       Presenter: Mingyu Xia

May 28, 2018   Cost-Effective Active Learning for Deep Image classification
                       Keze Wang, Dongyu Zhang, Ya Li, Ruimao Zhang, and Liang Lin
                       Presenters: Anton Poznov, Yifat Shemesh and Ofir Aktm

                       FaceNet: A Unified Embedding for Face Recognition and Clustering
                       Florian Schroff, Dmitry Kalenichenko, James Philbin
                       Presenters: Coral Sharoni and Tal Sheffer

                       Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
                       Mateusz Malinowski, Marcus Rohrbach and Mario Fritz
                       Presenter: Hadar Szostak

June 4, 2018   Deep Diffeomorphic Transformer Networks
                       Guest Lecture, Dr. Oren Freifeld, Computer Science, Ben-Gurion University

                       Optical Flow Estimation using a Spatial Pyramid Network
                       Anurag Ranjan and Michael J. Black
                       Presenters: Irit Chelly and Roy Uziel

June 11, 2018   Generative Adversarial Networks
                       Ian J. Goodfellow, Jean Pouget-Abadie ,Mehdi Mirza, Bing Xu, David Warde-Farley,Sherjil Ozair, Aaron Courville, Yoshua Bengio
                       Presenters: Ziv Alperovich and Eliav Elul

                       Building High-level Features Using Large Scale Unsupervised Learning
                       Q.V. Le, M.A. Ranzato, R. Monga, M. Devin, K. Chen, G.S. Corrado, J. Dean, A.Y. Ng
                       Presenters: Din Malachi and Tomer Gafni

June 18, 2018   Weakly Supervised Learning
                       Guest Lecture, Dr. Rami Ben-Ari, IBM

                       Auto-Encoding Variational Bayes
                       Diederik P Kingma, Max Welling
                       Presenters: Or Dinari and Zalman Ibragimov

June 27, 2018   Deep Learning and the Information Bottleneck Principle
                       Naftali Tishby,Noga Zaslavsky
                       Presenters: Moshe Bensimon and Yossi Eni

                       Playing Atari with Deep Reinforcement Learning
                       Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller
                       Presenters: Moshe Abuhasira, Andrey Gurevich and Shachar Schnapp

                       A Survey of FPGA Based Neural Network Accelerator
                       Kaiyuan Guo, Shulin Zeng, Jincheng Yu, Yu Wang, Huazhong Yang
                       Presenters: Erez Manor and Dror Mataraso

Links

Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016). Deep Learning. MIT Press Online

Convolutional Neural Networks for Visual Recognition, Stanford Course cs231n, see also Lectures

Neural Networks for Machine Learning, Coursera course by Jeff Hinton

ConvNet Arithmetic

ConvNet Demo by Stanford

Tensor flow course

Tensor flow official site

Tensor flow code

Tensor flow playground

Student Presentations

Each student will need to present a paper in class. Please refer to the list of papers below and note time schedule to be presented. Ideas for papers not in the list need approval. The presentation should be clear and practiced and the student should read the assigned paper and related work in enough detail to be able to lead a discussion and answer questions. A presentation should be roughly 30 minutes long including time for questions and a discussion. Typically this is about 25 to 30 slides. In the presentation provide the citation to the paper you present and to any other related work you reference.
Deadline:The presentation should be handed in one day before the class (or before if you want feedback).
Structure of presentation: High-level overview with contributions, Main motivation, Clear statement of the problem, Overview of the technical approach, Strengths/weaknesses of the approach, Overview of the experimental evaluation, Strengths/weaknesses of evaluation, Discussion: future directions, links to other works.

List of papers

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe and Christian Szegedy

Layer Normalization
Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton

Dropout: A Simple Way to Prevent Neural Networks from Overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
Yarin Gal and Zoubin Ghahramani

Rich feature hierarchies for accurate object detection and semantic segmentation
L-C. Chen, G. Papandreou, I. Kokkinos, Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik

U-net: Convolutional networks for biomedical image segmentation
Ronneberger, O., Fischer, P., Brox, T.

Spectral Representations for Convolutional Neural Networks
Oren Rippel, Jasper Snoek, Ryan P. Adams

You Only Look Once: Unified, Real-Time Object Detection
Joseph Redmon , Santosh Divvala, Ross Girshick , Ali Farhad

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun

Very Deep Convolutional Networks for large-scale image recognition
Karen Simonyan & Andrew Zisserman

Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, Alexander A. Alemi

Recurrent Neural Networks and Long Short Term Memory (LSTM) Networks

Unsupervised Learning of Video Representations using LSTMs
Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov

DRAW: A Recurrent Neural Network For Image Generation
Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende and Daan Wierstra

A First Look at Music Composition using LSTM Recurrent Neural Networks
Douglas Eck and Jurgen Schmidhuber

Generative Adversarial Networks
Ian J. Goodfellow, Jean Pouget-Abadie ,Mehdi Mirza, Bing Xu, David Warde-Farley,Sherjil Ozair, Aaron Courville, Yoshua Bengio

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
Alec Radford, Luke Metz, Soumith Chintala

A Neural Algorithm of Artistic Style
Leon A. Gatys, Alexander S. Ecker, Matthias Bethge

Multimodal learning with Deep Boltzman Machines
Nitish Srivastava and Ruslan Salakhutdinov

Auto-Encoding Variational Bayes
Diederik P Kingma,Max Welling

Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
Pascal Vincent, Hugo Larochelle , Isabelle Lajoie , Yoshua Bengio , Pierre-Antoine Manzagol

Building High-level Features Using Large Scale Unsupervised Learning
Quoc V. Le, Marc Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S. Corrado ,Jeff Dean, Andrew Y. Ng

Where Are They Looking?
Adria Recasens, Aditya Khosla, Carl Vondrick, Antonio Torralba

FaceNet: A Unified Embedding for Face Recognition and Clustering
Florian Schroff, Dmitry Kalenichenko, James Philbin

Learning to Compare Image Patches via Convolutional Neural Networks
Sergey Zagoruyko, Nikos Komodakis

Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
Jure Zbontar, Yann LeCun

Automatic Segmentation of MR Brain Images With a Convolutional Neural Network
Pim Moeskops, Max A. Viergever, Adrienne M. Mendrik, Linda S. de Vries, Manon J. N. L. Benders, and Ivana Isgum

Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks
Ozan Oktay, Wenjia Bai, Matthew Lee, Ricardo Guerrero,Konstantinos Kamnitsas, Jose Caballero, Antonio de Marvao,Stuart Cook, Declan ORegan
and Daniel Rueckert



Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia,Pierre Sermanet,Scott Reed,Dragomir Anguelov,Dumitru Erhan,Vincent Vanhoucke,Andrew Rabinovich

ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky, Ilya Sutskever,Geoffrey E. Hinton


Monocular Object Instance Segmentation and Depth Ordering with CNNs
Ziyu Zhang, Alex Schwing, Sanja Fidler, Raquel Urtasun

Instance-Level Segmentation with Deep Densely Connected MRFs
Ziyu Zhang, Sanja Fidler, Raquel Urtasun

DRAW: A Recurrent Neural Network For Image Generation
Karol Gregor,Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, Daan Wierstra

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
Alec Radford, Luke Metz, Soumith Chintala


The Multiverse Loss for Robust Transfer Learning
Etai Littwin, Lior Wolf

Simultaneous Deep Transfer Across Domains and Tasks
Eric Tzeng, Judy Hoffman, Trevor Darrell

Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
Mateusz Malinowski, Marcus Rohrbach, Mario Fritz

Auto-Encoding Variational Bayes
Diederik P Kingma, Max Welling

Tutorial: Bayesian Reasoning and Deep Learning
Shakir Mohamed

Deep learning and the information bottleneck principle
Naftali Tishby,Noga Zaslavsky

Unsupervised Learning of Video Representations using LSTMs
Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov

Explaining and Harnessing Adversarial Examples
Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy

Recurrent Models of Visual Attention
Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu

Generative Adversarial Nets
Ian J. Goodfellow,Jean Pouget-Abadie,Mehdi Mirza, Bing Xu, David Warde-Farley,Sherjil Ozair,Aaron Courville, Yoshua Bengio

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
S. M. Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, Koray Kavukcuoglu, Geoffrey E. Hinton

DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Yaniv Taigman, Ming Yang, Marc' Aurelio Ranzato, Lior Wolf

FaceNet: A Unified Embedding for Face Recognition and Clustering
Florian Schroff, Dmitry Kalenichenko, James Philbin

Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learning
Oren Z. Kraus, Lei Jimmy Ba, Brendan Frey

Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks
Ozan Oktay, Wenjia Bai, Matthew Lee, Ricardo Guerrero, Konstantinos Kamnitsas, Jose Caballero, Antonio de Marvao, Stuart Cook, Declan ORegan, and Daniel Rueckert

Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller

We Are Humor Beings: Understanding and Predicting Visual Humor
Arjun Chandrasekaran, Ashwin K Vijayakumar, Stanislaw Antol, Mohit Bansal, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh

A Neural Algorithm of Artistic Style
Leon A. Gatys, Alexander S. Ecker, Matthias Bethge

A First Look at Music Composition using LSTM Recurrent Neural Networks
Douglas Eck, Jurgen Schmidhuber


Homework assignments

Homework assignments should be submitted via moodle using a single zip file: "YourName-XX.zip" where XX is the assignment number.

Assignment #1 due to April 5, 2018