• Justify the development state-of-the-art deep learning algorithms.
• Make design choices regarding the construction of deep learning algorithms.
• Implement, optimize and tune state-of-the-art deep neural network architectures.
• Identify and address the security aspects of state-of-the-art deep learning algorithms.
• Examine open research problems in deep learning and propose approaches in the literature to tackle them.

Module 1: Introduction to Deep Feedforward Networks

• • Gradient-based learning
  • Sigmoidal output units
  • Back propagation

Module 2: Regularization for Deep Learning

• • Regularization strategies
  • Noise injection
  • Ensemble methods
  • Dropout

Module 3: Optimization for Training Deep Models

• • Optimization algorithms: Gradient, Hessian-Free, Newton
  • Momentum
  • Batch normalization

Module 4: Convolutional Neural Networks

• • Convolutional kernels
  • Downsampled convolution
  • Zero padding
  • Backpropagating convolution

Module 5: Recurrent Neural Networks

• • Recurrence relationship & recurrent networks
  • Long short-term memory (LSTM)
  • Back propagation through time (BPTT)
  • Gated and simple recurrent units
  • Neural Turing machine (NTM)