Paste_Image.png
Each observation is represented by a set of numbers(features).
Paste_Image.png
Paste_Image.png
\
Formally, given training set (xi,yi) for i=1…n, we want to create a classification model f that can predict label y for a new x.
Paste_Image.png
The machine learning algorithm will create the function f for you.
The predicted y for a new x is the sigh of f(x).
Loss Functions For Classificaiton
How do we measure classification error?
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Statistical Learning Theory For Supervised Learning
Statistical Learning Theory
- Ockham's Razor: The best models are simple models that fit the data well.
- William of Ockham,English frier and philosopher (1287-1347) said that among hypotheses that predict equally well, we should choose the one with the fewest assumptions.
Paste_Image.png
We need a balance between accuracy and simplicity.
Most common machine learning methods choose f to minimize training error and complexity.
Aims to thwart the "curse" of dimensionality.
Paste_Image.png
Basic Outline for ML
- step 1: Split data randomly into training and test sets.
- step 2: Estimate coefficients/ Train Model:
Paste_Image.png
- step 3: Score model: Compute score for each xi in the test set
- step 4: Evaluate model.
Logistic Regression
simple, fast, often competes with the best ML algorithms.
Paste_Image.png
Paste_Image.png
Another perspective:
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Evaluation Measures for Classifiers
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
Paste_Image.png
ROC Curves
- Started during WWII for analyzing radar signals.
- For a particular False Positive Rate(FPR), what is the True Positive Rate(TPR)?
- FPR = number of negatives that were classified by the ML algorithm as positives / total number of negatives
- TPR = number of positives that were classified by the ML algorithm as positives / total number of positives.
Paste_Image.png
TPR=7/11
FPR=3/11
Paste_Image.png
TPR=3/11
FPR=2/11
Paste_Image.png
Paste_Image.png
Paste_Image.png
