Fall 2016: Deep Learning and Its
Applications to Signal and Image Processing and Analysis, 361-21120
Tammy Riklin Raviv
Time Monday, 10:00-13:00
Location Building 72, Room 122
About the course
The class will cover a diverse set of topics in signal processing,
image analysis and computer vision and various Neural Network architectures. It will be an interactive course where we will discuss interesting topics on demand and latest research buzz.
The goal of the class is to learn about different domains of the
field, understand, identify and analyze the main challenges, what
works and what doesn't, as well as to identify interesting new
directions for future research.
Starting from the last week of November (the second month of the semester) we will read
1 to 4 papers every week. The
papers will be presented by the students in a seminar format. The
success of the discussion in class will thus be due to how prepared
the students come to class. Each student is expected to read at least
one paper for each class (and prepare a question or a comment for a discussion). In
addition, each student will present a paper in one
of the classes. There will be also talks by invited speakers and a
lab. class (Nov. 14, room 330, building 33) on Tensor Flow. The Last
class will be dedicated to final project presentation (about ten
minutes for each project). Details on the final project will be
published soon. The final project does not have to be necessarily
related to the paper presented in class.
Tentative schedule and lecture notes
Oct. 31, 2016 Course
overview
Nov. 7, 2016 Optimination, Stochastic
gradient descent, Backpropagation, Regularization, Loss functions
Nov. 14, 2016 1. Convolutional Neural
Networks (CNN)
2. Tensor Flow (by Ohad Shitrit)
location: Electrical Engineering
Computer Laboratory, room 330, building 33
Nov. 21, 2016 Guest Lecture , Dr. Oren
Shriki, Cognitive and Brain Sciences at Ben-Gurion University
ICA and Information Maximization Approach to
Unsupervised Learning in Feedforward and Recurrent Neural Networks
Nov. 28, 2016 1. Dropout and mini-batch optimization
Guest Lecture, Prof. Jacob Goldberger, Electrical
Engineering, Bar-Ilan University
2. Regularization for Deep Learning
3. Semantic Image Segmentation with Deep Convolutional Nets and Fully
Connected CRFs
L-C. Chen, G. Papandreou, I. Kokkinos,
K. Murphy, A. L Yuille
Presenter: Topaz Gilad
Dec. 5, 2016 Rich feature hierarchies for accurate object detection and semantic segmentation
L-C. Chen, G. Papandreou, I. Kokkinos,
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik
Presenter: Ilia Iofedov
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun
Presenter: Assaf Livne
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian
Sun
Presenter: Felix Vilensky
Dec. 12 2016
Learning to Compare Image Patches via Convolutional Neural Networks
Sergey Zagoruyko, Nikos Komodakis
Presenter: Ilan Schvartzman
Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
Jure Zbontar, Yann LeCun
Presenter: Shahar Bar
Automatic Segmentation of MR Brain Images With
a Convolutional Neural Network
Pim Moeskops, Max A. Viergever, Adrienne M. Mendrik, Linda S. de Vries, Manon J. N. L.
Benders, and Ivana Isgum
Presenter: Dr. Shiri Gordon
Dec. 19 2016
Multimodal learning with Deep Boltzman Machines
Nitish Srivastava and Ruslan Salakhutdinov
Presenter: Nir Halay
De-noising of Contrast-Enhanced MRI Sequences by an Ensemble of Expert Deep Neural Networks
Ariel Benou, Ronel Veksler, Alon
Friedman and Tammy Riklin Raviv
Presenter: Ariel Benou
A Sensor Fusion Method Based on an Integrated Neural Network and Kalman Filter for Vehicle Roll Angle Estimation
Vargas-Melendez, Leandro and Boada, Beatriz L. and Boada, María Jesus L. and Gauchia, Antonio and Diaz, Vicente
Presenter: Alon Baruch
Dec. 26 2016
Recurrent Neural Networks and Long Short Term Memory (LSTM) Networks
Presenter: Assaf Arbelle
Unsupervised Learning of Video Representations using LSTMs
Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov
Presenter: Boris Kodner
Convolutional Neural Networks for No-Reference Image Quality Assessment
Le Kang, Peng Ye, Yi Li, David
Doermann
Presenter: Tsachi Hershkovitch
Jan. 2, 2017
Where Are They Looking?
Adria Recasens, Aditya Khosla, Carl Vondrick, Antonio Torralba
Presenter: Nir Regev
FaceNet: A Unified Embedding for Face Recognition and Clustering
Florian Schroff, Dmitry Kalenichenko, James Philbin
Presenter: Omri Bar
Guest Lecture Yaron Gurovich, FDNA
Deep Learning for Facial Recognition
Jan. 9, 2017, Building 33, room 330 --
Note the unusual location
Generative Adversarial Networks
Ian J. Goodfellow, Jean Pouget-Abadie
,Mehdi Mirza, Bing Xu, David Warde-Farley,Sherjil Ozair, Aaron Courville, Yoshua Bengio
Presenter: Nimrod Shenor
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
Alec Radford, Luke Metz, Soumith Chintala
Presenters: Shay Ben-Sasson and Gideon Rosenthal
A Practical Guide to Applying Echo State Networks
Mantas Lukosevicius
Presenter:Eyal Zakkay
Guest Lecture: Nir Lotan and Itamar Ben-Ari,
Electronics Ltd. Intel
Fast and easy deep learning solutions on CPU
Jan. 16, 2017
Deep Knowledge Tracing
Chris Piech, Jonathan
Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas and Jascha Sohl-Dickstein
Presenter: Avi Segal
A Neural Algorithm of Artistic Style
Leon A. Gatys, Alexander S. Ecker,
Matthias Bethge
Presenter: Amit Diamant
Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI
Ran Mnor and Amir Geva
Presenter: Yehu Sapir
DRAW: A Recurrent Neural Network For Image Generation
Karol Gregor, Ivo Danihelka, Alex
Graves, Danilo Jimenez Rezende and Daan Wierstra
Presenter: Itay Benou
EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation
Suwicha Jirayucharoensak,1,2 Setha Pan-Ngum,1 and Pasin Israsena2
Presenter: Dror Haor
Jan. 23, 2017
Looking into Deep Neural Networks
A First Look at Music Composition using LSTM Recurrent Neural Networks
Douglas Eck and Jurgen Schmidhuber
Presenter: Tuvy Lemberg
Caffeinated FPGAs: FPGA Framework For
Convolutional Neural Networks
Roberto DiCecco, Griffin Lacey, Jasmina Vasiljevic, Paul Chow, Graham Taylor and Shawki Areibi
Presenter: Nir Hasidim
Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks
Ozan Oktay, Wenjia Bai, Matthew
Lee, Ricardo Guerrero,Konstantinos Kamnitsas, Jose Caballero, Antonio
de Marvao,Stuart Cook, Declan ORegan
and Daniel Rueckert
Presenter: Lena Shulmanovich
Links
Ian Goodfellow, Yoshua Bengio, and Aaron Courville (2016). Deep Learning. MIT Press
Online
Convolutional Neural Networks for Visual Recognition, Stanford Course
cs231n, see also
Lectures
Neural Networks for Machine Learning, Coursera course by Jeff Hinton
ConvNet Arithmetic
ConvNet Demo by Stanford
Tensor flow course
Tensor flow official site
Tensor flow code
Tensor flow playground
Student Presentations
Each student will need to present a paper in
class. Please refer to the list of papers below and note
time schedule to be presented. Ideas for
papers not in the list need approval. The presentation should be clear and practiced
and the student should read the assigned paper and related work in enough detail to be able to lead a discussion and answer questions.
A presentation should be roughly 30 minutes long
including time for questions and a discussion. Typically this is about
25 to 30 slides. In the presentation provide the citation to the paper you present and to any other related work you reference.
Deadline:The presentation should be handed in one day before
the class (or before if you want feedback).
Structure of presentation:
High-level overview with contributions,
Main motivation,
Clear statement of the problem,
Overview of the technical approach,
Strengths/weaknesses of the approach,
Overview of the experimental evaluation,
Strengths/weaknesses of evaluation,
Discussion: future directions, links to other works.
List of papers
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia,Pierre Sermanet,Scott Reed,Dragomir Anguelov,Dumitru Erhan,Vincent Vanhoucke,Andrew Rabinovich
ImageNet Classification with Deep Convolutional
Neural Networks
Alex Krizhevsky, Ilya Sutskever,Geoffrey
E. Hinton
Monocular Object Instance Segmentation and Depth Ordering with CNNs
Ziyu Zhang, Alex Schwing, Sanja Fidler, Raquel Urtasun
Instance-Level Segmentation with Deep Densely Connected MRFs
Ziyu Zhang, Sanja Fidler, Raquel Urtasun
DRAW: A Recurrent Neural Network For Image Generation
Karol Gregor,Ivo Danihelka, Alex Graves,
Danilo Jimenez Rezende, Daan Wierstra
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
Alec Radford, Luke Metz, Soumith Chintala
The Multiverse Loss for Robust Transfer Learning
Etai Littwin, Lior Wolf
Simultaneous Deep Transfer Across Domains and Tasks
Eric Tzeng, Judy Hoffman, Trevor Darrell
Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
Mateusz Malinowski, Marcus Rohrbach, Mario Fritz
Auto-Encoding Variational Bayes
Diederik P Kingma, Max Welling
Tutorial: Bayesian Reasoning and Deep Learning
Shakir Mohamed
Deep learning and the information bottleneck principle
Naftali Tishby,Noga Zaslavsky
Unsupervised Learning of Video Representations using LSTMs
Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov
Explaining and Harnessing Adversarial Examples
Ian J. Goodfellow, Jonathon Shlens, Christian
Szegedy
Recurrent Models of Visual Attention
Volodymyr Mnih, Nicolas Heess, Alex Graves, Koray Kavukcuoglu
Where Are They Looking?
Adria Recasens, Aditya Khosla, Carl Vondrick, Antonio Torralba
Generative Adversarial Nets
Ian J. Goodfellow,Jean Pouget-Abadie,Mehdi Mirza, Bing Xu, David Warde-Farley,Sherjil Ozair,Aaron Courville, Yoshua Bengio
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
S. M. Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, Koray Kavukcuoglu, Geoffrey E. Hinton
DeepFace: Closing
the Gap to Human-Level Performance in Face Verification
Yaniv Taigman, Ming Yang, Marc' Aurelio Ranzato, Lior Wolf
FaceNet: A Unified Embedding
for Face Recognition and Clustering
Florian Schroff, Dmitry Kalenichenko, James Philbin
Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learning
Oren Z. Kraus, Lei Jimmy Ba, Brendan Frey
Multi-input Cardiac Image Super-Resolution
Using Convolutional Neural Networks
Ozan Oktay, Wenjia Bai, Matthew Lee, Ricardo Guerrero,
Konstantinos Kamnitsas, Jose Caballero, Antonio de Marvao, Stuart Cook,
Declan ORegan, and Daniel Rueckert
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller
We Are Humor Beings: Understanding and Predicting Visual Humor
Arjun Chandrasekaran, Ashwin K Vijayakumar,
Stanislaw Antol, Mohit Bansal, Dhruv Batra, C. Lawrence Zitnick, Devi
Parikh
A Neural Algorithm of Artistic Style
Leon A. Gatys, Alexander S. Ecker, Matthias
Bethge
A First Look at Music Composition using LSTM Recurrent Neural Networks
Douglas Eck, Jurgen Schmidhuber
Homework assignments
Homework assignments should be submitted via moodle using a single zip
file: "YourName-XX.zip" where XX is the assignment number.
Assignment #1 due to Dec. 5, 2016