COS 432/COS 532 : Deep Learning
Fall 2020

Professor:  Bruce MacLeod
    Room 222 Science Bldg.                                                     
    email : macleod at                                            
    Tel: 780-4285                                                                       
    Office Hours: M  12:45 PM - 1:30 PM
                           W 1:00 PM - 2:30 PM  or by appointment

Course Objectives:

This class provides a practical introduction to deep learning, including theoretical motivations and how to implement it in practice. As part of the course we will cover multilayer perceptrons, backpropagation, automatic differentiation, and stochastic gradient descent. After some background material, we start by introducing convolutional networks for image processing, starting from the simple LeNet to more recent architectures such as ResNet. Secondly, we discuss sequence models and recurrent networks, such as LSTMs, GRU, and the attention mechanism.  The goal of the course is to provide both a good understanding and good ability to build modern nonparametric estimators. The course loosely follows Dive into Deep Learning in terms of notebooks, slides and assignments


Assignments and a final project are a fundamental part of this course. Two assignments will be group projects which will involve a short (ten minute) presentation to the class. The final project will involve ongoing consideration throughout the semester and will be due on the day the final is scheduled.


Dive Into Deep Learning :

Any assignments handed in late will incur a 5% a day reduction in the assignment grade. Any students which choose to work together on an assignment must make a note of the fact on the top of the assignment. Failure to do so will result in no credit for the assignment and possible failure of the course.

Teamwork on Programs It is acceptable to work with other students in the analysis, design, and debugging phases of your programming assignments. However, unless it is a team project, do not write code as a team. Do not copy your code from or for another student. . Please list the students with whom you collaborated, and the type of help you gave or received in a comment at the top of your program.

Links to related material

Reading, assignment, and exam schedule

 8/31  Course Overview :
    Reading : Chapter 1, Introduction
 Lab :  Jupyter Notebooks and installing software on your own machine
 9/2   Data Manipulation, Linear Algebra
     Reading : Chapter 2 : Sections 2.1-2.3
 Labor Day
  Preliminaries : Calculus, Probability, and Automatic Differentiation
     Reading : Finish Chapter 2
   Problem Set #1 Due
  Linear Regression
      Reading : Chapter 3 : Sections 3.1-3.3
  Lab : Linear Regression
 Softmax Regression
     Reading : Finish Chapter 3
 Multilayer Perceptrons
      Reading : Chapter 4 : Sections 4,.1-4.4
Lab : Underfit, Overfit lab
          Extra credit/Graduate Students : Do exercises.
 Multilayer Perceptrons
      Reading : Finish Chapter 4
 Problem Set #2 Due
  Deep Learning Computation
       Reading : Chapter 5.1, 5.2, 5.4-5.6
 Lab : Follow instructions in Section 4.10 and make a submission to Kaggle. See how well you can do ... will have small class contest
 Convolutional Neural Networks
        Reading : Chapter 6
 Modern Convolutional Networks
       Reading : Chapter 7 : Sections 7.1-7.4
Lab : Convolutional Networks
  Modern Convolutional Networks
       Reading : Finish Chapter 7
Problem Set #3 Due
 Fall break
 First Exam
 10/19   Recurrent Neural Networks
       Reading : Chapter 8, Sections 8.1-8.4  Lab : Recurrent Neural Networks
Lab : Recurrent Neural Networks or Fastai
 Recurrent Neural Networks
       Reading : Finish Chapter 8
 10/26  Midterm Project Report and Presentation
 Lab : Recurrent Neural Networks and/or Fastai
 Modern Recurrent Neural Networks
       Reading : Chapter 9, Sections 9.1-9.4
 Modern Recurrent Neural Networks
       Reading : Finish Chapter 9
  Lab : Modern Recurrent Networks
Problem Set #3 Due
  Optimization Algorithms
      Reading : Chapter 11, Sections 11.1-11.3
 11/9  Optimization Algorithms
     Reading : Chapter 11, Sections 11.4-11.6
     Graduate students/Extra Credit : finish Chapter 11, provide summary
 11/11   Veterans Day
 Computer Vision
      Reading : Sections 13.1-13.4
 Lab :
 Bias, Deployment issues
 Problem Set #4 Due
 11/23   Second Exam
 Thanksgiving break
 Project time ! Optional class, discussions about project topics
Project time ! Optional class, discussions about project topics 
No formal lab (available for project help)
 Final Project Presentations
 Final Project Presentations
Final Project Due :