WEEK 6: CLASSIFICATION

Supervised Learning: Classification, k-NN, Decision Trees, Exercise 3 distributed

October 20, 2025

← All Weeks

Weekly Materials

🎓

Lecture slides, week 6

Professor's lecture slides (PDF)
👩‍💻

Discussion of the previous exercises; Finger exercises (with TA)

Hands-on tutorials and practice exercises

→
📝

Exercise 3

Distribution of Exercise sheet 3

→
💻

Code Examples

Introduction to matplotlib (if time permits) Multiple Demo Files
📚

References & Resources

Finger Exercises Discussion of the previous exercises; Finger exercises (with TA)

Additional Notes

Week 6: Classification

Learning Objectives

  • Understand fundamental classification concepts and algorithms
  • Master k-Nearest Neighbors (k-NN) algorithm
  • Learn how to evaluate classifier performance
  • Understand Naive Bayes classification
  • Explore decision trees and ensemble methods
  • Apply classification algorithms to real-world datasets

Topics Covered

  • Classification Fundamentals: Problem formulation and terminology
  • k-Nearest Neighbors (k-NN): Instance-based learning algorithm
  • Classifier Evaluation: Accuracy, precision, recall, F1-score, confusion matrices
  • Naive Bayes: Probabilistic classification based on Bayes’ theorem
  • Decision Trees: Tree-based classification and feature selection
  • Ensemble Methods: Boosting and Bagging for improved performance
  • Practical Applications: Introduction to matplotlib (if time permits)

Schedule

  • Lecture: Monday, October 20, 2025 (10:15 - 12:00)
  • Practice Session: Monday, October 20, 2025 (16:30 - 18:00)
  • TA Session: Discussion of exercises and finger exercises

Key Concepts

  • Supervised vs unsupervised classification
  • Training and test set evaluation
  • Cross-validation techniques
  • Overfitting in classification models
  • Feature importance and selection
  • Bias-variance tradeoff in classification

Algorithms Covered

  • k-NN: Distance-based classification
  • Naive Bayes: Probabilistic approach
  • Decision Trees: Rule-based classification
  • AdaBoost: Adaptive boosting
  • Bagging: Bootstrap aggregating

Practical Skills

  • Implementing classifiers from scratch
  • Using scikit-learn for classification tasks
  • Model evaluation and performance metrics
  • Data visualization with matplotlib
  • Handling categorical and numerical features

Assignments

  • Exercise 3: Distributed this week
  • Complete finger exercises with TA guidance
  • Practice implementing and comparing different classifiers