Skip to content

Lecture #1

Author: Vinay Kumar (@imflash217) | Date: 30/January/2021

  • AI v/s ML v/s DL (Venn Diagram)
  • One-hot-encoding
  • Loss Function
  • Training v/s Evaluation Error
  • Model Selection
  • Hyperparams
  • Overfitting v/s Underfitting
  • Generalization Gap
  • Model Capacity
  • K-fold Cross Validation
  • Leave-one-out Cross Validation
What is Machine Learning?

It is a field that aims to extract relationships and structures in the data. Example: How to map data to annotations?

Loss Function

We need a measure to see how well our system is doing at learning. This measure is called Loss Function

  • Sum-of-Squared-Error (SSE): \(2^2\) \(\sum_{i}\normalize{(y_i - f(x_i)}_2^2\)
Training

The process of teaching our system to minimize errors is called as Training.

Evaluation

The process of determining the performance of our trained system over an unseen dataset is called as Evaluation.

Unsupervised Learning
  • Generative Models (GAN, AE, RBM)
  • Latent Variable Modeling (PCA, AE)
  • Clustering
[special case of] Cross Validation

If there are many point on the graph of CV(\(\theta\)) with similar values near the minimum; we choose the most parsimonious model that has a CV value within the standard deviation from the best model \(\theta^*\).

In other words; we pick the first \(\theta\) for which the CV value satisfies \(CV(\theta) < CV(\theta^*) + std(CV(\theta^*))\)

Benefits of this process: It decreases the possibility of choose an underfit or slightly ovefit model than what is required. Provides better Guarantees.