This module instructs students on the basics of deep learning as well as building better and faster deep network classifiers for sensor data. The module is strongly project-based, with two main phases. In the first phase, students will learn the basics of deep learning and Computer Vision, e.g. stochastic gradient descent, multi-layer perceptron, convolutional neural networks, filtering, and corner detection. At the end of this first phase, students should be ready to run simple networks in Keras and implement basic computer vision methods in Python. In the second phase, students will be divided into teams of 2 or 3. Each team will tackle a problem of their choosing, from fields such as computer vision, pattern recognition, distributed computing. Example projects include face recognition and emotion recognition.

Instructor: PD. Dr. Shih-Chii Liu
Language of Instruction: English
Prerequisites Basic programming (e.g. Python, C++), Basics in Linear Algebra
TA: Yuhuang Hu, Jithendar Anumula, and Stefan Braun

All materials are available at: https://github.com/PnS2018

Schedule

Session: Thursdays 1-5pm
Starting date: March 1st, 2018
Place: Institute of Neuroinformatics, Dungeon Lab, Y35 E 30, University Zurich Irchel Campus
Module ends: May 31th, 2018

Session 01

Basics of Linear Algebra (vector, matrix, tensors, etc), Introduction to Python, numpy basics, symbolic computation basics.

Session 02

Brief introduction to Linear Regression, Logistic Regression, Stochastic Gradient Descent and its variants.

Session 03

Brief introduction to Multi-layer Perceptron and Convolutional Neural Networks.

Session 04

Basic techniques of Computer Vision using OpenCV, such as thresholding, edge detection, etc.

Session 05

Advanced techniques of Computer Vision such as filtering, corner detection, keypoints, etc.

Projects

Baby Jarvis: Implement a face recognition system using Keras, OpenCV, and Raspberry Pi. The device will voice the name of the face it sees. (Difficulty: 5)
Baby Jarvis II: Distinguish between happy and sad faces using Keras, OpenCV and Raspberry Pi. (Difficulty: 5)
Baby Jarvis III: Proof-of-concept driver monitoring system by detecting if the user is sleeping, checking the phone or doing other reckless activities while driving. (Difficulty: 6)
Where’s Waldo?: Find Waldo in the original picture given a query using OpenCV. (Difficulty: 6)
Are You Human?: Break a CAPTCHA that uses simple math as the question. (Difficulty: 6)
Which station is this?: Take a picture of the station name, recognize the station and find the position from Google Map! (Difficulty: 7)
Implementing backpropagation: Implement the backpropagation method in basic python without relying on any high level APIs like Keras or Tensorflow. (Difficulty: 7)
(Advanced) Paper Piano: Print the piano key on a paper, use CV techniques to identify which key is pressed and play the sound. (Difficulty: 8)
(Advanced) Raspberry that talks: Analysis an image via image captioning, then broadcast the information to a speaker. (Difficulty: 8)
(Advanced) Object Tracking: Develop a model-free object tracker for VOT Challenge! (Difficulty: 9)
Or… your ideas!

Remark: Difficulty scale: 1 (so easy) – 10 (way too hard)

Timeline

Week 5 (March 29th): Project Briefing
Week 6 (April 12th): Assign project based on the team’s decision. Assign team name based on the team’s decision. Setup project, first group discussion on the plan to complete the project.
Week 7-9 (April 19th - May 3rd): Collecting data, writing utility functions, first prototype by end of the 9th week
Week 10-11 (May 10th, May 17th): Refine the model and implementation so that the system can run as fast as possible
Week 12-13 (May 24th, May 31st): Project write up and presentation

Acknowledgment

This module uses many online resources on Deep Learning and Raspberry Pi. We would like to acknowledge the following resources:

P&S Module 2018

Deep Learning on Raspberry Pi