Syllabus
COS 432/COS 532 : Deep Learning
Fall 2020

Professor:  Bruce MacLeod
    Room 222 Science Bldg.                                                     
    email : macleod at maine.edu                                            
    Tel: 780-4285                                                                       
    Office Hours: M  12:45 PM - 1:30 PM
                           W 1:00 PM - 2:30 PM  or by appointment

Course Objectives:

This class provides a practical introduction to deep learning, including theoretical motivations and how to implement it in practice. As part of the course we will cover multilayer perceptrons, backpropagation, automatic differentiation, and stochastic gradient descent. After some background material, we start by introducing convolutional networks for image processing, starting from the simple LeNet to more recent architectures such as ResNet. Secondly, we discuss sequence models and recurrent networks, such as LSTMs, GRU, and the attention mechanism.  The goal of the course is to provide both a good understanding and good ability to build modern nonparametric estimators. The course loosely follows Dive into Deep Learning in terms of notebooks, slides and assignments

Assignments:

Assignments and a final project are a fundamental part of this course. Two assignments will be group projects which will involve a short (ten minute) presentation to the class. The final project will involve ongoing consideration throughout the semester and will be due on the day the final is scheduled.

Text:

Dive Into Deep Learning : https://d2l.ai/

Grading:
Any assignments handed in late will incur a 5% a day reduction in the assignment grade. Any students which choose to work together on an assignment must make a note of the fact on the top of the assignment. Failure to do so will result in no credit for the assignment and possible failure of the course.

Teamwork on Programs It is acceptable to work with other students in the analysis, design, and debugging phases of your programming assignments. However, unless it is a team project, do not write code as a team. Do not copy your code from or for another student. . Please list the students with whom you collaborated, and the type of help you gave or received in a comment at the top of your program.

Links to related material



















Reading, assignment, and exam schedule

  August/September
 8/31  Course Overview :
    Reading : Chapter 1, Introduction
 Lab :  Jupyter Notebooks and installing software on your own machine
 9/2   Data Manipulation, Linear Algebra
     Reading : Chapter 2 : Sections 2.1-2.3
 9/7
 Labor Day
 9/9
  Preliminaries : Calculus, Probability, and Automatic Differentiation
     Reading : Finish Chapter 2
   Problem Set #1 Due
 9/14
  Linear Regression
      Reading : Chapter 3 : Sections 3.1-3.3
  Lab : Linear Regression
 9/16
 Softmax Regression
     Reading : Finish Chapter 3
 9/21
 Multilayer Perceptrons
      Reading : Chapter 4 : Sections 4,.1-4.4
Lab : Underfit, Overfit lab
          Extra credit/Graduate Students : Do exercises.
 9/23
 Multilayer Perceptrons
      Reading : Finish Chapter 4
 Problem Set #2 Due
 9/28
  Deep Learning Computation
       Reading : Chapter 5.1, 5.2, 5.4-5.6
 Lab : Follow instructions in Section 4.10 and make a submission to Kaggle. See how well you can do ... will have small class contest
 9/30
 Convolutional Neural Networks
        Reading : Chapter 6
October
 10/5
 Modern Convolutional Networks
       Reading : Chapter 7 : Sections 7.1-7.4
Lab : Convolutional Networks
 10/7
  Modern Convolutional Networks
       Reading : Finish Chapter 7
Problem Set #3 Due
 10/12
 Fall break
 10/14
 First Exam
 10/19   Recurrent Neural Networks
       Reading : Chapter 8, Sections 8.1-8.4  Lab : Recurrent Neural Networks
Lab : Recurrent Neural Networks or Fastai
 10/21
 Recurrent Neural Networks
       Reading : Finish Chapter 8
 10/26  Midterm Project Report and Presentation
 Lab : Recurrent Neural Networks and/or Fastai
 10/28
 Modern Recurrent Neural Networks
       Reading : Chapter 9, Sections 9.1-9.4
November
 11/2
 Modern Recurrent Neural Networks
       Reading : Finish Chapter 9
  Lab : Modern Recurrent Networks
Problem Set #3 Due
 11/4
  Optimization Algorithms
      Reading : Chapter 11, Sections 11.1-11.3
 11/9  Optimization Algorithms
     Reading : Chapter 11, Sections 11.4-11.6
     Graduate students/Extra Credit : finish Chapter 11, provide summary
 Lab:
 11/11   Veterans Day
 11/16
 Computer Vision
      Reading : Sections 13.1-13.4
 Lab :
 11/18
 Bias, Deployment issues
 Problem Set #4 Due
 11/23   Second Exam
 11/25
 Thanksgiving break
December
 12/2
 Project time ! Optional class, discussions about project topics
 12/7
Project time ! Optional class, discussions about project topics 
No formal lab (available for project help)
 12/9
 Final Project Presentations
 12/11
 Final Project Presentations
 12/?
Final Project Due :