Machine Learning 101

Machine Learning 은, 데이터를 통해 패턴을 학습하여 일부의 데이터만으로 예측하는 알고리즘의 집합

Kind of Machine Learning Decided by Data

• Supervised Learning
• Unsupervised Learning

• Reinforcement Learning

  

Machine Learning Pipe Line

Supervised Learning

---

---

Regression

• Linear and Non-linear Regression
  • Partial Least Squares (PLS)
    • 작성 예정
• Gradient Descent
  • Learning Rate
  • LRS (Learning Rate Scheduler)
  • Optima (Local minima problem)
    • Adam, RMSProp, SGD …
  • Deep Learning - Loss Funcition
  • Cost Funcition
• Bias and Variance Trade-off
  • Loss Funcition
• Over and Under-fitting
• Weight Regularization

Classification

• Logistic and Soft-max Regression
• Support Vector Machine (SVM)
• Dicison Tree
• Linear Discriminant Analysis (LDA, Supervised)

Ensemble (Complex, Super/Unsupervised)

• Bagging
• Boosting

Regression vs. Classification

Regression (회귀)

• Input (Feature) : Real number (실수형), Discrete value(범주형) etc..,
• Output (Predict) : Real number (실수형, 이산값)
• Model shape : normal function shape (eg. $y = w_{1}x + w_{0}$)

Classification (분류)

• Input (Feature) : Real number (실수형), Discrete value(범주형) etc..
• Output (Predict) : Discrete value (범주형)
• Essential Function for last node
  • Binary classification (이진 분류) : Sigmoid function
  • Multiple Classification (다중 분류) : Soft-max function

Unsupervised Learning

---

---

Dimension Reduction (차원 축소; Data pre-processing)

Kind of Feature Extraction in DR

• Singular Value Decompostion (SVD)
• Principal Component Analysis (PCA)
• Linear Discriminant Analysis (LDA)
• t-SNE
• UMAP

Clustering

Parametric

• Gaussian Mixture Model

Non-Parametric

• K-Means
• Mean Shift
• density-based-spatial-clustering-of-applications-with-noise

Hierarchial Clustering

Parameter vs. Hyper-parameter

---

---

Parameter (weight)

• learnable parameter within model
• ex) $w_{0}, w_{1}, … w_{D}$

Weight Regularization (L1 / L2 Regul’n)

Weight Regularization

• L1 Lasso Regularization, L2 Ridge Regularization, Elastic Net

Hyper-parameter

• adjust by administer
• ex) Learning rate, Disposition size(배치 크기)

Data structure

---

---

• Feature (attribute) : information = X
• Label : results = y (predict : y_hat)

Data for Machine Learning

All Relative Documents

---

---

Linear and Non-linear Regression

Gradient Descent

Learning Rate

LRS (Learning Rate Scheduler)

Optima (Local minima problem)

Bias and Variance Trade-off

Logistic and Soft-max Regression

Support Vector Machine (SVM)

Dicison Tree

Linear Discriminant Analysis (LDA)

Ensemble (Complex)

Bagging

Boosting

Dimension Reduction

Singular Value Decompostion (SVD)

Principal Component Analysis (PCA)

Unsupervised Linear Discriminant Analysis (LDA)

t-SNE

UMAP

Clustering

Gaussian Mixture Model

K-Means

Mean Shift

DBSCAN

Hierarchial Clustering

Weight Regularization

Over and Under-fitting

Data for Machine Learning