Content:

INF5860 - Machine Learning for Image Analysis

What should the student learn:

-You have good knowledge about the theory behind central classification algorithms, logistic regression, and neural nets and how these are applied to images.

-You are familiar with central mathematical method applied in the algorithms.

-You can discuss and evaluate how different feature extraction methods affect the classification error rate.

-You have knowledge about overtraining, generalization, and validation.

-You know how convolutional networks work and how they can be adapted to various applications.

-You have experience in using tools for deep learning likeTensorflow.

Date / Theme / Sections / Weekly problems / Links
17.1 /
  1. Introduction (Anne)
/ What is machine learning?
Motivation:
-Why do we care about machine learning?
-What can we do with machine learning?
-Brief history of computer vision
-What will the students learn in this course?
Prerequisite
-What should you know before taking the class
Topics:
-Classification vs regression
-Unsupervised vs supervised models
-Generative and Discriminative models
-Data driven vs feature driven classifiers / Basic python 2/3
Learnnumpy
24.1 /
  1. Traditional classifiers: (Anne)
/ Classifiers:
-KNN, Bayesian classifier, classification, regression (linear, poly), SVM
Cost/loss functions
-L1, L2, cross entropy, Softmax, Sigmoid,
Understand the relationship between classifier/loss function and the complexity of decision boundaries.
Hyperparameters (number of features) / Curse of dimensionality / KNN and Bayesian classifier
Oblig 1: cs231n “Assignment 1” (del 1, KNN, SVM)
31.1 /
  1. Loss functions and optimization (Anne)
/ Gradient descent (and stochastic)
Data splitting:
-Training data, test data, validation data
7.2 /
  1. Tensorflow (Tollef)
/ Introduction to Tensorflow
-Building a tensorflow graph
-Tf.Variables
-Tf.session
-Tf.placeholder / Install tensorflow and read documentation – see tensorflow videos (youtube)
14.2 /
  1. Neural network classifier (Ole)
/ Multi-layer perceptron’s (and their decision boundaries)
Intro to optimization and backprop
Challenges with image classification:
-Illumination
-View angle variations
-Deformation
-Occlusion
-Background Clutter
-Interclass variation / Introduksjon til Tensorflow:
Linear classification of MNIST data (using numpy and using Tensorflow)
21.2 /
  1. Convolution neural networks (Tollef)
/ -What is convolution
-Stride/field of view/pooling
-Feature extraction
-Hierarchical learning
-Reuse of features
-Data driven / Classification of the MNIST data using CNNs (tensorflow)
28.2 /
  1. Training a neural network (Anne)
/ The effect of different activation functions
-Exploding/vanishing gradients, exploding values
How can we reduce this challenge?
-Learning rate
-Initialization
-Batch normalization
Target may not be learnable / Training deep networks and deep learning with small data (cifar)
What to watch, when training.
7.3 /
  1. Generalization (Tollef)
/ Generalization:
-Dropouts
-Transfer learning
-Multitask learning
-Early stopping
-Regularization
-Learning rate
-Data augmentation
-Data preprocessing (feature scaling)
-batch size
-Capacity (hypotheses space / Vapnik-Chervonenkis VC dimension)
-Data spilt: Train/test/validation
-Bias/variance / Cont.
14.3 /
  1. Architectures for convolutional nets.(Ole)
/ Brief history of network architectures
-Alexnet
-ResNet
-Inception
Details:
-Kernel size (field of view, memory, nonlinearity)
-Resnet module
-Separable convolutions / channelwise
-Dilated convolutions
Effect:
-Abstraction level
-Memory demand
-Network capacity
-Gradient flow (why is my network not training)
-Model ensemble (snapshot ensembles)
21.3 /
  1. Localization
Detection
Segmentation (Ole) / Localization:
-Localization and classification
Detection:
-Faster R-CNN
-YOLO
Semantic segmentation:
-Fully convolutional
-Convolution transposed
-U-net
Instance segmentation: / Ex1. Localization and classification of objects
Ex2. Segmentation of objects
4.4 /
  1. Visualization and Understanding (Anne)
/ Understand the features learned by the neural network
-Visualize filters
-Guided backprop
-GradCam
-Occlusion experiments
-Saliency maps
-Visualizing CNN features by gradient ascent
-Adversarial examples (fooling the network)
-Feature Inversion
-Neural Style transfer / Ex. Show them how guided backprop works
11.4 /
  1. Recurrent Neural Networks (Tollef)
/ Neural networks with feedback
Vanilla RNN (sequence processing)
-Unfolding the RNN
-One-to-one, one-to-many, many-to-one, many-to-many
-Truncated backprop through time (TBTT)
-Exampels:
-Language model (as example)
-Consider image captioning (with and without attention)
-Gradient flow
-LSTM
-GRU
-Video classification / Ex: language model
18.4 /
  1. Semi supervised learning / Unsupervised learning (Ole)
/ Siamese networks (semi supervised)
Adversarial domain adaption
t-SNE / PCA
Autoencoders
Variational autoencoders (VAE)
Abnormality detection / Ex: Variationalautoencoder?
25.4 /
  1. Generative Models (Ole)
/ Generative Adversarial Networks
-Generator network
-Discriminator network
2.5 /
  1. Deep Reinforcement Learning (Tollef)
/ Introduction to reinforcement leering
-Policy leering
-Q-learning
-Actor-Critic / Ex: atari game?
Architectures for convolutional nets. / Segmentering, modeller for kunstig oppmerksomhet. Konvolusjonsnettverk for video. / Hvordan kan man tilpasse slike nettverk til forskellige formål?
Inspeksjon og visualsiering av konvolusjonsnettverk / Inspeksjon og visualsiering av konvolusjonsnettverk
Backprop på bildet (Visualisering, deepdream, “Adverserialfooling”) / Prøv å lage “adverserialexamples”.

Spatial transformer network

Soft attention

Hard attention