Introduction to Deep Learning

Welcome

Welcome to the Introduction to Deep Learning course website. Here you will find all the necessary information related to our class.

Presentation of the module

Description of the module

Class Schedule

Classes are held Tuesdays at 16:15. Beware of possible changes of rooms.

• Class 1 23/09 16:15 - 18:15, room P122: Introduction, general principle, datasets, simple example
  Slides MNIST dataset Example neural network using tensorflow
• Class 2 30/09 16:15 - 18:15, room S206: Problem class 1
  Problem Sheet 1 Example script as solution suggestion
• Class 3 07/10 16:15 - 17:45, room P122: Gradient descent and Backpropagation
  Slides Demo linear regression
• Class 4 14/10 16:15 - 18:45, room P122: Quiz 1, Problem Class 2
  Problem Sheet 2 Diabetes dataset Example python classes implementing dense and relu layers without tensorflow Example script as solution suggestion
• Class 5 21/10 16:15 - 18:45, room P122: Convolutional nerual networks and autoencoders
  Slides Convolution intuition Example Convolutional neural network for MNIST Example Convolutional neural network with Batch Norm for MNIST Example Denoising autoencoder
• Class 6 04/11 16:15 - 18:45, room P122: Quiz 2, Problem Class 3
  Problem Sheet 3 Example script as solution suggestion
• Class 7 18/11 16:15 - 18:15, room E08: More advanced topics
  Slides Transformer architecture
• Class 8 25/11 16:15 - 18:45, room P122: Project work
• Class 9 09/12 16:15 - 18:45, room P122: Project discussions, Quiz 3

  Upload project file here

Mini-projects

Datsets (select one)

• The "Fashion MNIST" dataset is a collection of 60,000 training and 10,000 test grayscale images of 10 different fashion categories, each 28x28 pixels in size. Designed as a more challenging alternative to the classic MNIST dataset of handwritten digits, it includes images of items like t-shirts, trousers, pullovers, dresses, and shoes.
• The "Flickr-Faces-HQ (FFHQ)" dataset consists of 70,000 high-quality PNG images (for our purposes, use the 128x128 pixel thumbnails) of human faces, each at a resolution of 1024x1024 pixels, sourced from Flickr. Designed to provide a diverse set of real-world face images, the dataset spans a wide range of ages, ethnicities, and image backgrounds.
• The DeepSat (SAT-4) and (SAT-6) Airborne Image Classification datasets each comprise about half a million 28x28 pixel multispectral satellite images, representing four/six distinct land cover types. Sourced from airborne sensors, these datasets offeri a rich collection of images to facilitate the development and benchmarking of classification models for aerial earth observation tasks.
• The CIFAR-10 dataset consists of 60,000 32x32 color images spanning 10 different classes, with 6,000 images per class. The classes include objects like airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks.

Tasks (select one, but beware: not all tasks are compatible with all data sets)

• Train a convolutional neural network model to accurately categorize images into predefined classes. The model will learn distinctive features and patterns for each category. Try to optimize for classification accuracy.
• Design and train a neural network that determines the orientation of images, categorizing them as upright, sideways, or upside-down. To create the dataset, start with upright images and programmatically rotate them to various orientations, ensuring each image has a corresponding labeled orientation.
• Design and train a neural network model that can accurately predict the correct order of a set of four subimages, which when arranged correctly, create a coherent larger image. The dataset can be created by taking complete images and segmenting them into a 2x2 grid of subimages. After shuffling the order of these subimages, the correct sequence will be recorded. To represent the 24 possible orderings, a one-hot encoding scheme may be employed.
• Train a denoising autoencoder, a specialized type of neural network, to remove noise from digital images. By training on pairs of noisy and clean images, the model learns to reconstruct high-quality, noise-free visuals. The outcome will be a powerful tool that enhances image clarity, surpassing traditional denoising techniques. You will need to artificially add noise in order to create the noisy of the database.
• Implement a variational autoencoder (VAE), a specialized type of deep learning model, to learn compressed, latent representations of images. By encoding high-dimensional input data into a lower-dimensional latent space and subsequently decoding it back, the VAE will be trained to both compress and reconstruct images with minimal loss of information. The outcome will be a model that enables image generation.

• Fraud detection dataset: Dataset

Grades

• 3 multi-choice style tests (75%)
• Participation in the project discussions (25%)

• Please share your feedback about the module anonymously: Feedback form

Textbooks

• Simon J.D. Prince "Understanding Deep Learning", MIT Press (2023)
• Ian Goodfellow and Yoshua Bengio and Aaron Courville: "Deep Learning", MIT Press (2016)
• François Chollet: "Deep Learning with Python", Second Edition, Manning (2021)

Important Papers

• Perceptron (Rosenblatt, 1957)
• Dense Neural Network (LeCun, 1989)
• Convolutional Neural Network (LeCun et al., 1998)
• Variational autoencoder (Kingma & Welling, 2013)
• Adam optimizer (Kingma & Ba, 2014)
• Weight initialization (He et al., 2015)
• Batch normalization (Ioffe & Szegedy, 2015)
• Residual connections (He et al., 2015)
• Layer normalization (Ba et al., 2016)
• Transformer architecture (Vaswani et al., 2017)

Contact

Daniel FORSTER
Email: daniel.forster[at]univ-orleans.fr