Introduction to Machine Learning
Course Description
This course offers a comprehensive introduction to the foundational concepts and techniques of machine learning, equipping learners with the skills to analyze and solve real-world problems. Covering essential classical methods, the curriculum delves into classification, regression, clustering, and dimensionality reduction techniques, such as decision trees, support vector machines, k-means, and principal component analysis (PCA). With a balance of theory and practical implementation, participants will gain hands-on experience through coding exercises and case studies, ensuring they can apply machine learning concepts to a wide range of domains. Whether you’re a beginner or looking to solidify your understanding of classical approaches, this course provides a strong foundation for further exploration in the field of machine learning.
Syllabus
Foundations and Mathematics of Machine Learning:
- Linear Algebra
- Probability and Statistics
- Maximum Likelihood and Maximum A posterior
- Kernel Density Estimation
- Optimization
Probabilistic Models:
- Bayes Optimal Classifier
- Naïve Bayes Classifier
- Minimum Risk Classifier
- Gaussian Discrimination Analysis (GDA)
- KNN
- Logistic Regression
- Model Selection
- Data Splitting
- Performance Measures
Linear Models (Supervised Learning):
- Perceptron and its variants
- Least Square Methods
- Fisher linear discriminant analysis
- Support Vector Machine (SVM)
- Linear Regression
- Regularization (Ridge and LASSO)
- Bias-Variance Trade-off
Non-Linear Models (Supervised Learning)
- Basis Function Expansion
- Radian Basis Function Network (RBF-NN)
- Kernel Trick and SVM
- Multi Layer Perceptron (MLP)
- Decision Trees
- Ensemble Learning
- Bagging
- Stacking
- Boosting
- Gradient Boosting Machine (GBM)
- Gradient Tree Boosting (TreeBoost)
- Extreme Gradient Boosting (XGBoost)
Unsupervised Learning (Clustering):
- Introduction and Definitions
- Partitioning Methods (K-Means, K-Medoids, FCM, PCM)
- Model-Based Methods (GMM, EM, MeanShift)
- Density-Based Methods (DBSCAN)
- Hierarchical Methods (Agglomerative, Divisive)
- Spectral Clustering
- Competitive Learning Algorithms (SOM)
- Cluster Validity Indices
Unsupervised Learning (Dimension Reduction):
- Feature Selection
- Feature Generation
- Linear Methods (PCA, SVD, FA, LDA)
- Nonlinear Methods (Autoencoder, LEM, LLE, t-SNE, UMAP, Kernel PCA)
References:
- Probabilistic Machine Learning – An Introduction, K. Murphy, MIT Press (2022).
- Machine Learning: A Bayesian and Optimization Perspective, S. Theodoridis, Academic Press (2015).
- Pattern Recognition and Machine Learning, Ch. Bishop, Springer (2006).
- An Introduction to Statistical Learning, G. James, D. Witten, T. Hastie, and R. Tibshirani, Springer (2023).
- Mathematics for Machine Learning, M. Deisenroth, A. Faisal, and C. Ong. Cambridge University Press, (2020).
- Matrix Cookbook, K. B. Petersen and M. S. Pedersen, Technical University of Denmark (2012).