Machine Learning (ML)

Last updated Nov 14, 2024 Edit Source

The use and development of computer systems that are able to learn and adapt without following explicit instructions, by using algorithms and statistical models to analyse and draw inferences from patterns in data

• https://github.com/josephmisiti/awesome-machine-learning
• The Illustrated Machine Learning website
• Rules of ML (Google)
• Jason’s Machine Learning 101 (Google)
  • https://docs.google.com/presentation/d/1kSuQyW5DTnkVaZEjGYCkfOxvzCqGEFzWBy4e9Uedd9k/preview
• Machine Learning Glossary (Google)
• ML Resources (MIT student)
• Machine Learning & Deep Learning Tutorials
• A visual introduction to machine learning
• ML Algorithms: Strengths and Weaknesses
• A friendly introduction to linear algebra for ML (ML Tech Talks)
• Best practices for ML engineering (Google)
• Recommendation System Algorithms
• Training Machine Learning Models More Efficiently with Dataset Distillation
• Codelabs - AI & ML

• https://github.com/afshinea/stanford-cs-229-machine-learning/blob/master/en/super-cheatsheet-machine-learning.pdf
• https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-machine-learning-tips-and-tricks
• ML-AI guide

• https://github.com/trekhleb/homemade-machine-learning
• https://github.com/anhquan0412/basic_model_scratch
• https://github.com/rushter/MLAlgorithms
• https://github.com/ahmedbesbes/Neural-Network-from-scratch
• https://github.com/eriklindernoren/ML-From-Scratch

See AI/DS and DataEng/Open ML data

• #BOOK An Introduction to Statistical Learning (James 2013, SPRINGER)
  • https://github.com/JWarmenhoven/ISLR-python
• #BOOK The elements of statistical learning (Hastie 2015, SPRINGER)
• #BOOK Recommender Systems - The Textbook (Aggarwal, 2016 SPRINGER)
• #BOOK Mathematics for ML (Deisenroth, 2020 CAMBRIDGE)
• #BOOK Introduction to Machine Learning with Python - A Guide for Data Scientists (Muller, 2016 O’REILLY) - https://github.com/amueller/introduction_to_ml_with_python
• #BOOK Machine Learning for Dummies (Hurwitz, 2018 WILEY-IBM)
• #BOOK Python Machine Learning (Raschka 2019, PACKT)
• #BOOK Mastering Machine Learning with scikit-learn (Hackeling 2014, PACKT)
• #BOOK Designing Machine Learning Systems with Python (Julian 2016, PACKT)
• #BOOK Evaluating Machine Learning Models (Zheng 2015, OREILLY)
• #BOOK Introduction to Machine Learning Interviews Book

• #COURSE Machine Learning (CS229, Stanford)
  • Lecture notes
  • Cheat sheets
• #COURSE Machine Learning (Coursera-Stanford)
• #COURSE Machine Learning Crash Course with TensorFlow APIs (Google)
• #COURSE Data Mining and Machine Learning (STAT 365/665, Yale)
• #COURSE Applied machine learning (U Columbia)
• #COURSE L’apprentissage face à la malédiction de la grande dimension (College de France)
• #COURSE The Machine Learning Summer School, MLSS Tubingen 2020 (virtual)
  • https://www.youtube.com/channel/UCBOgpkDhQuYeVVjuzS5Wtxw/videos

• See AI/DS and DataEng/ML Ops and AI/DS and DataEng/Cloud platforms
• #CODE Benchmarks of ML libraries
• #CODE Ludwig - declarative machine learning framework
  • https://medium.com/predibase/ludwig-automl-for-text-classification-7c1759f3b150
• #CODE Scikit-learn
  • http://scikit-learn.org/stable/
  • Contrib packages
  • #TALK PyData tutorial by Sebastian Raschka
  • #CODE Lightning
    • Large-scale linear classification, AI/Supervised Learning/Regression and ranking (AI/Learning to rank] in Python
  • #CODE MAPIE
    • #PAPER MAPIE: an open-source library for distribution-free uncertainty quantification
    • A scikit-learn-compatible module for estimating prediction intervals for single-output regression or multi-class classification settings
  • #CODE scikit-learn-intelex
    • Intel(R) Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application
    • https://intel.github.io/scikit-learn-intelex/
• #CODE mlinsights
  • http://www.xavierdupre.fr/app/mlinsights/helpsphinx/notebooks/piecewise_linear_regression.html
• #CODE PyCaret
  • https://pycaret.org/
  • PyCaret is an open-source, low-code machine learning library in Python that automates machine learning workflows
  • PyCaret is essentially a Python wrapper around several machine learning libraries and frameworks such as scikit-learn, XGBoost, LightGBM, CatBoost, spaCy, Optuna, Hyperopt, Ray, and many more
  • PyCaret >= 2.2 provides the option to use GPU for select model training and hyperparameter tuning
• #CODE Hypertools - Python toolbox for visualizing and manipulating high-dimensional data
  • http://hypertools.readthedocs.io/en/latest/
  • https://github.com/ContextLab/hypertools-paper-notebooks
  • http://blog.kaggle.com/2017/04/10/exploring-the-structure-of-high-dimensional-data-with-hypertools-in-kaggle-kernels/
• #CODE PySAL - Python Spatial Analysis Library Meta-Package
  • http://pysal.org/pysal/
• #CODE MLxtend - Library of extension and helper modules for Python’s data analysis and machine learning libraries
• #CODE H2O
  • http://www.h2o.ai/
  • https://github.com/h2oai/h2o-3
  • https://github.com/h2oai/h2o4gpu
  • https://github.com/h2oai/h2o-tutorials
  • #TALK Getting started with H2O on Python (pydata)
  • http://www.jowanza.com/post/156015716294/why-h2o-sparkling-water
  • https://github.com/h2oai/h2o-3/tree/master/h2o-py/demos
• #CODE Dlib (C++ with python interface)
• #CODE Shogun
• #CODE Vowpal Wabbit
  • ML system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning
  • http://hunch.net/~vw/
  • https://github.com/JohnLangford/vowpal_wabbit/wiki
• #CODE DMTK (Microsoft)
  • http://www.dmtk.io/
  • Light LDA - Scalable, fast, and lightweight system for large-scale topic modeling
• #CODE [RAPIDS]( https://github.com/rapidsai, https://rapids.ai/) - GPU data science
  • #CODE cuML - RAPIDS Machine Learning Library
  • #CODE cuspatial - CUDA-accelerated GIS and spatiotemporal algorithms
  • #CODE cuSignal - RAPIDS Signal Processing Library
  • #CODE cuGraph - RAPIDS Graph Analytics Library
  • #CODE cuDF - GPU DataFrame Library
  • #CODE CuPy - NumPy-like API accelerated with CUDA
    • https://cupy.chainer.org/
    • https://docs-cupy.chainer.org/en/stable/
    • https://docs-cupy.chainer.org/en/stable/tutorial/
• #CODE ArrayFire - High performance library for parallel computing with an easy-to-use API
  • It enables users to write scientific computing code that is portable across CUDA, OpenCL and CPU devices. This project provides Python bindings for the ArrayFire library.
  • https://arrayfire.com/
• #CODE ThunderSVM - A Fast SVM Library on GPUs and CPUs
• #CODE PyGAM - Generalized Additive Models in Python
  • https://pygam.readthedocs.io
• #CODE SurPRISE - A Python scikit for building and analyzing recommender systems
• #CODE Facets
  • visualizations for understanding and analyzing machine learning datasets: Facets Overview and Facets Dive. The visualizations are implemented as Polymer web components, backed by Typescript code and can be easily embedded into Jupyter notebooks or webpages
  • https://pair-code.github.io/facets/
• #CODE PyCM
  • PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters
  • http://www.pycm.ir/
• #CODE Pycircular - Python module for circular data analysis
  • https://towardsdatascience.com/introducing-pycircular-a-python-library-for-circular-data-analysis-bfd696a6a42b

• See AI/AI-ML-DL for scientific discovery
• #PAPER Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (Raschka 2020)
• #PAPER How to avoid machine learning pitfalls: a guide for academic researchers (Lones 2021)
  • https://venturebeat.com/2021/08/23/the-dos-and-donts-of-machine-learning-research/
• #PAPER Pen and Paper Exercises in Machine Learning (Gutmann 2022)
• #PAPER How to avoid machine learning pitfalls: a guide for academic researchers (Lones 2023)
• #PAPER Questionable practices in machine learning (2024)
  • The paper categorizes the 43 questionable research practices (QRPs) in machine learning into several broad areas, such as:
    • Data Handling: Issues like cherry-picking data, inappropriate data splits, and using test data in training.
    • Model Evaluation: Inadequate baselines, selective reporting of results, and misuse of metrics.
    • Experimental Design: Running multiple experiments and only reporting successful ones.
    • Reproducibility: Lack of transparency in code and data sharing.
    • Publication Practices: Hype-driven narratives and insufficient detail in methods sections.

See AI/Supervised Learning/Feature selection

See AI/Feature learning

See AI/Anomaly and Outlier Detection

See AI/Time Series analysis and AI/Forecasting

See AI/AutoML

See AI/Deep learning/DL

See AI/Reinforcement learning

See AI/Unsupervised learning/Unsupervised learning

See AI/Supervised Learning/Supervised learning

See AI/Weakly-supervised learning. It includes these topics: AI/Semi-supervised learning, AI/Active learning and AI/Transfer learning

See AI/One, few-shot learning

See AI/Self-supervised learning

See AI/Learning to rank

See AI/Multi-task learning

See AI/Generative AI/GenAI

See AI/XAI

See AI/Federated learning

See AI/QML