Machine Learning Techniques

An introduction to the main methods and models for regression, classification, and clustering, focusing on their properties and suitability for different problems.

This course was a comprehensive survey of the core algorithms that form the machine learning toolkit. Building on the mathematical foundations, we explored the three main paradigms of ML. For unsupervised learning, we covered dimensionality reduction with PCA and Kernel PCA, clustering with K-Means, and density estimation with Gaussian Mixture Models. For supervised learning, we started with regression models like Least Squares, Ridge, and LASSO. We then moved to a wide array of classification techniques, including K-NN, Decision Trees, Naive Bayes, Logistic Regression, and Support Vector Machines. The course concluded with advanced methods like ensemble learning (Bagging and Boosting) and an introduction to Artificial Neural Networks.

Instructor

Prof. Arun Rajkumar, Department of Computer Science and Engineering, IIT Madras

Course Schedule & Topics

The course is structured over 12 weeks, covering a wide range of unsupervised and supervised learning algorithms.

Week Primary Focus Key Topics Covered
1 Intro & Dimensionality Reduction Introduction to ML; Unsupervised Learning and Principal Component Analysis (PCA).
2 Non-linear Dimensionality Reduction Representation learning using Kernel PCA.
3 Clustering Algorithms Unsupervised clustering with K-Means and Kernel K-Means.
4 Gaussian Mixture Models & EM Probabilistic clustering and density estimation with GMMs via the EM algorithm.
5 Linear Regression Supervised Learning for regression: Least Squares and the Bayesian perspective.
6 Regularized Regression Improving regression models with Ridge ($L_2$) and LASSO ($L_1$) regularization.
7 Foundational Classification Models Supervised classification using K-Nearest Neighbors (K-NN) and Decision Trees.
8 Generative Models Probabilistic classification using Generative Models like Naive Bayes.
9 Discriminative Models Classification using Discriminative Models like the Perceptron and Logistic Regression.
10 Support Vector Machines (SVM) Maximal margin classifiers for linear and non-linear classification.
11 Ensemble Methods Combining models to improve performance using Bagging and Boosting (Adaboost).
12 Artificial Neural Networks (ANN) Introduction to ANNs and their application in multiclass classification problems.

Material used