Applied Mathematics for Deep Learning/딥러닝수학

Course Information 교과목

Course ID 학수번호Course Title 교과목명Classification 과목구분학점-강의-실습선수과목수업학기
AUE8070Applied Mathematics for Deep Learning
딥러닝수학
Elective
전공선택
3-3-0없음1학기

Lecturer 교강사

Affilication 소속Name 성명Contact 연락처E-MAIL
Department of Automotive Engineering
미래자동차공학과
Seungjae Min
민승재
(02) 2220-0457seungjae@hanyang.ac.kr

Course Description 교과목 개요

Deep Learning theory is a field that intersects linear algebra, probability theory and statistics, multivariate calculus, and algorithms, complex optimizations arising from learning iteratively from data and finding hidden insights which can be used to build intelligent applications. Despite the immense possibilities of Machine and Deep Learning, a thorough mathematical understanding of many of these techniques is necessary for a good grasp of the inner workings of the algorithms and getting good results.

딥러닝 이론은 반복적으로 데이터를 학습하고 지능형 애플리케이션을 구축하는 데 사용될 수있는 숨겨진 인사이트를 발견하기 위한 선형대수학, 확률론 및 통계, 다변수 미적분학, 알고리즘 및 복잡도 최적화 등이 교차하는 분야이다. 머신러닝 및 딥러닝의 엄청난 가능성에도 불구하고, 알고리즘의 내부 작동을 잘 파악하고 좋은 결과를 얻으려면 이러한 기술 중 많은 부분을 수학적으로 완전히 이해해야한다.

Course Objective 수업 목표

The goal of the course is to understand applied mathematics identifies the key quantities in Deep Learning.

(1) 정확도, 트레이닝 시간, 모델 복잡도, 파라미터 수 및 피쳐 (features) 수에 대한 고려를 포함하는 올바른 알고리즘 선택, (2) 파라미터 설정과 검증 (validation) 전략 선택, (3) 편향 분산 (bias-variance)의 트레이드오프의 이해를 기반으로한 언더피팅 (underfitting)과 오버피팅 (overfitting)의 식별, (4) 올바른 신뢰 구간과 불확실성 추정을 위하여 필요한 수학을 학습하고 기술을 이해한다.

Textbook 교재

G. Strang, Linear Algebra and Learning from Data, Wellesley-Cambridge Press, 2019 (Book website: https://math.mit.edu/~gs/learningfromdata/)

References 부교재

N/A

Evaluation 평가방법

Midterm
중간
Final
기말
Attendance
출석
Homework
과제
Participation
수업참여도
Project
프로젝트
Total
404002000100

Schedule 강의계획

Week 주Topics 주제
1Linear Algebra: Matrix, Space
2Linear Algebra: SVD, PCA
3Linear Algebra: Least Squares
4Probability and Statistics
5Optimization: Minimum Problems
6Optimization: Linear Programming
7Optimization: Stochastic Gradient Descent
8Midterm Exam
9Learning from Data: Structure of Neural Nets for Deep Learning
10Learning from Data: Convolutional Neural Network (CNN)
11Learning from Data: Distance Matrices, Clustering
12Linear Regression Model
13Logistics Regression Model: Two-value Classification
14Logistics Regression Model: Multi-value Classification
15Deep Learning Model
16 Final Exam
WeekDateLectureTextbookNoteHomework
ftp://cdl.hanyang.ac.kr
Due
13/16Overview, Linear Algebra in a Nutshelloverview
Linear Algebra in a Nutshell
MATLAB Tutorial
3/19[Applied Linear Algebra]
Four fundamental subspaces
I.1~I.3in-class note
23/23Summary of Previous Lecture
Orthogonal matrices
I.4~I.5review (Part I-1)
in-class note
3/25Eigenvalues and Eigenvectors
Positive Definite and Semidefinite Matrices
I.6~I.7in-class noteHW#1 Problems
4/1
33/30Positive Definite and Semidefinite Matrices
Singular Value Decomposition (SVD)
I.7~I.8review (Part I-2)
in-class note
4/1Singular Value Decomposition (SVD)
Principal Components
Norms of Vectors and Matrices
I.8~I.9
I.11
in-class noteHW#2 Problems
4/8
44/6Norms
Principal Components
I.11, I.9in-class note
4/8Principal Components
Ax=b
review (Part I-3) / review-note
in-class note
54/13[Large Matrices]
Least Squares
II.2in-class noteHW#3 Problems

4/20
4/15Election Day (no class)
64/20Computing Eigenvalues and Singular ValuesII.1, II.2in-class note
4/22Randomized Linear AlgebraII.4review (Part II) / review (Part II) note
in-class note
74/27[Low Rank]
Low Rank Change in A and Its Inverse
III.1in-class note
WolframAlpha: Computational Intelligence
4/29Derivatives of Inverse, Eigenvalues and Singular ValuesIII.2in-class note
85/4Interlacing Eigenvalues and Low Rank Signals
Rapidly Decaying Singular Values
III.2, III.3in-class note
5/6Rapidly Decaying Singular Values
III.3in-class note
95/11[Statistics]
Mean, Variance, Covariance
Optimization
V.1, V.3in-class note
5/13[Optimization]
Gradient Descent
VI.4
in-class note
Related Link: Gradient Descent
105/18Accelerating Gradient Descent
Linear Programming(LP)
VI.4
VI.3
in-class note
Related Link: Momentum
5/20Stochastic Gradient Descent(SGD)VI.5in-class note
Related Link: Stochastic Gradient Descent
HW#4 Problems

HW#5 Computations
5/27
6/3
115/25[Learning from Data]
Construction of Deep Neural Network
VI.5
VII.1
An overview of gradient descent optimization algorithms
Neural Network Playground / Problems
in-class note
5/27Construction of Deep Neural NetworkVII.1
The Functions of Deep Learning
Deep Learning: An Introduction for Applied Mathematicians
in-class note
HW#6 Problems
6/3
126/1Backpropagation and the Chain RuleVII.3Calculus on Computational Graphs: Backpropagation
in-class note
6/3Loss FunctionTeaching Calculus to a Deep Learner
in-class note
136/8ConvolutionImageNet classification with deep convolutional neural networks
in-class note
6/10Convolution
Circulant Matrix, Fourier Matrix
ImageNet classification with deep convolutional neural networks
in-class note
146/15CNN, ConvNetVII.2Convolutional Neural Network
6/17Final Exam (10:30~12:00 @ Rm 102)
156/22Training Neural NetworksTraining Neural Network

PAST EXAM

2020
Midterm
Finalexam

[DATA]

MNIST database of handwritten digits (Yann LeCun, Courant Institute, NYU)

DELVE (University of Toronto, Data for Evaluating Learning in Valid Experiments)

UC Irvine, Machine Learning Repository

[Codes for Machine Learning]

Caffe: Convolutional Architecture for Fast Feature Embedding (2014)

MatConvNet: CNN for MATLAB

Torch

Keras

Theano: A Python framework for fast computation of mathematical expression (2016)

TensorFlow