Course Description
This course provides a comprehensive introduction to Deep Learning. Beginning with the history of neural networks and the fundamental Perceptron, we move to modern Feedforward Neural Networks, training methodologies, and optimization techniques. The course covers specialized architectures including Convolutional Neural Networks (CNNs) for computer vision, Recurrent Neural Networks (RNNs) for sequence modeling, and Transformers for natural language processing. We conclude with advanced generative models such as GANs and Diffusion models.
Grading Scheme & Key Dates
| Component | Weight | Date/Deadlines |
|---|---|---|
| Quiz 1 (MCQ) | 20% | February 18, 2026 |
| Quiz 2 (MCQ) | 20% | March 25, 2026 |
| Final Exam (Handwritten) | 40% | May 8, 2026 |
| Assignments (4 × 5%) | 20% | See Assignment Schedule |
Detailed Course Content
Module 1: Introduction & Fundamentals
- Overview of Deep Learning: History, evolution, and success stories
- Neural Fundamentals: McCulloch-Pitts Neuron, Threshold Logic
- Perceptrons: Architecture, Learning Algorithm, and Limitations (XOR problem)
- Multilayer Perceptrons (MLPs): Hidden layers, non-linearity, and Representation Power
Module 2: Feedforward Networks & Optimization
- Computations: Sigmoid neurons, Automatic differentiation, Backpropagation derivation
- Optimization: Gradient Descent (GD), Stochastic GD, Momentum, Nesterov, AdaGrad, RMSProp, and Adam
- Initialization: Xavier and He initialization
Module 3: Regularization & Training Methodology
- Bias-Variance Tradeoff, L2 Regularization, Early Stopping
- Dataset Augmentation, Dropout, Injecting Noise
- Practical Issues: Handling Class Imbalance, Domain Generalization
Module 4: Convolutional Neural Networks (CNNs)
- Fundamentals: Image Processing, Convolution, stride, padding, pooling
- Architectures: LeNet, AlexNet, VGGNet, GoogLeNet, ResNet
- Advanced Training: Batch Normalization, Improved Activation Functions
- Applications: Object Detection (R-CNN family, YOLO, SSD), Semantic Segmentation (U-Net)
Module 5: NLP & Sequence Models
- Seq2Seq Tasks, Need for a different architecture
- Embeddings: Word2Vec (Skip-gram/CBOW), Vector representations
- RNNs: Backpropagation Through Time (BPTT), Vanishing/Exploding gradients
- Gated Units: LSTMs and GRUs
- Seq2Seq: Encoder-Decoder models (Overview), Introduction to Attention mechanisms
Module 6: Transformers & Generative Models
- Transformers: Self-Attention (Multi-headed), Cross Attention, Positional Encoding
- Modern Tokenization (BPE), Vision Transformers (ViT)
- Generative Models: Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs)
- Diffusion Models & Introduction to Large Language Models (LLMs)
Lecture Materials
| Module # | Topic | Materials | Additional Resources |
|---|---|---|---|
| 1A | Introduction to Deep Learning and Artificial Neuron Models | Slides | |
| 1B | Modern Neuron & MLPs | Slides | |
| 2A | Backpropagation in MLPs | Slides | |
| 2B | Gradient Descent | Slides | |
| T1 | Practice Problems Set-I | Tutorial | |
| Q1 | Quiz-I | Set-A Set-B Set-C | |
| 2C | Adaptive Optimizers | Slides | |
| 3A | Introduction to Convolutional Neural Networks | Slides | |
| Q2 | Quiz-II | Set-A Set-B Set-C | |
| 4A | Sequence Modelling | Slides | |
| 4B | BPTT | Slides | |
| 5A | Machine Translation with RNN | Slides | |
| 5B | Attention & Transformers - 1 | Slides | |
| 5C | Attention & Transformers - 2 | Slides | |
| ES | End Semester | SET-A SET-B SET-C Key |
Programming Assignments
Assignments will be implemented in Python (NumPy/PyTorch). They are designed to complement the theoretical material without conflicting with quiz weeks.
| Assignment | Topic | Description | Release Date | Due Date |
|---|---|---|---|---|
| Assignment 1 | Multi-Layer Perceptron for Image Classification | Implement a MLP and backpropagation from scratch using Numpy to classify MNIST digits. | Feb 09, 2026 10:00 AM | Mar 08, 2026 23:59 PM |
| Assignment 2 | Building a Complete Visual Perception Pipeline | Implement a multi-task CNN in PyTorch to classify, bound, and segment pet images. | Mar 21, 2026 00:00 AM | Apr 12, 2026 23:59 PM |
| Assignment 3 | Implementing a Transformer for Machine Translation | Implement a Transformer model for machine translation task (German to English). | Apr 24, 2026 10:00 AM | May 19, 2026 23:59 PM |
References
- Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville, MIT Press
- Dive into Deep Learning, Zhang et al. (Available online: d2l.ai)