Outline of machine learning
Overview of and topical guide to machine learning
- 6 min read
- 11 chapters
- 4 labs on Lably
Learn it on Lably
Machine learning (ML) is a subfield of artificial intelligence within computer science that evolved from the study of pattern recognition and computational learning theory. In 1959, Arthur Samuel defined machine learning as a "field of study that gives computers the ability to learn without being explicitly programmed". ML involves the study and construction of algorithms that can learn from and make predictions on data. These algorithms operate by building a model from a training set of example observations to make data-driven predictions or decisions expressed as outputs, rather than following strictly static program instructions.
How can machine learning be categorized?
- An academic discipline
- A branch of science
- An applied science
- A subfield of computer science
- A branch of artificial intelligence
- A subfield of soft computing
- Application of statistics
- A subfield of computer science
- An applied science
Paradigms of machine learning
- Supervised learning, where the model is trained on labeled data
- Unsupervised learning, where the model tries to identify patterns in unlabeled data
- Reinforcement learning, where the model learns to make decisions by receiving rewards or penalties.
Sign in to keep reading
10 more chapters are free with a Lably account.
Applications of machine learning
- Applications of machine learning
- Bioinformatics
- Biomedical informatics
- Computer vision
- Customer relationship management
- Data mining
- Earth sciences
- Email filtering
- Inverted pendulum (balance and equilibrium system)
- Natural language processing
- Named Entity Recognition
- Automatic summarization
- Automatic taxonomy construction
- Dialog system
- Grammar checker
- Language recognition
- Machine translation
- Question answering
- Speech synthesis
- Text mining
- Text simplification
- Pattern recognition
- Recommendation system
- Search engine
- Social engineering
Machine learning hardware
- Graphics processing unit
- Tensor processing unit
- Vision processing unit
Machine learning tools
- Comparison of machine learning software
- Comparison of deep learning software
Machine learning frameworks
Proprietary machine learning frameworks
Open source machine learning frameworks
- Apache Singa
- Apache MXNet
- Caffe
- PyTorch
- mlpack
- TensorFlow
- Torch
- CNTK
- Accord.Net
- Jax
- MLJ.jl – A machine learning framework for Julia
Machine learning libraries
- Deeplearning4j
- Theano
- scikit-learn
- Keras
Machine learning algorithms
- AdaBoost
- Almeida–Pineda recurrent backpropagation
- ALOPEX
- Backpropagation
- Bootstrap aggregating
- CN2 algorithm
- Constructing skill trees
- Decision tree learning
- Dehaene–Changeux model
- Diffusion map
- Dominance-based rough set approach
- Dynamic time warping
- Error-driven learning
- Evolutionary multimodal optimization
- Expectation–maximization algorithm
- FastICA
- Forward–backward algorithm
- GeneRec
- Genetic Algorithm for Rule Set Production
- Growing self-organizing map
- Hyper basis function network
- IDistance
- k-means clustering
- k-nearest neighbors algorithm
- Kernel methods for vector output
- Kernel principal component analysis
- Learning vector quantization
- Leabra
- Linde–Buzo–Gray algorithm
- Local outlier factor
- Logic learning machine
- LogitBoost
- Manifold alignment
- Markov chain Monte Carlo (MCMC)
- Minimum redundancy feature selection
- Mixture of experts
- Multiple kernel learning
- Naive Bayes classifier
- Non-negative matrix factorization
- Online machine learning
- Out-of-bag error
- Prefrontal cortex basal ganglia working memory
- PVLV
- Q-learning
- Quadratic unconstrained binary optimization
- Query-level feature
- Quickprop
- Radial basis function network
- Random forest
- Randomized weighted majority algorithm
- Reinforcement learning
- Repeated incremental pruning to produce error reduction (RIPPER)
- Rprop
- Rule-based machine learning
- Self-organizing map
- Skill chaining
- Sparse PCA
- State–action–reward–state–action
- Stochastic gradient descent
- Structured kNN
- Support vector machine
- T-distributed stochastic neighbor embedding
- Temporal difference learning
- Wake-sleep algorithm
- Weighted majority algorithm (machine learning)
Machine learning methods
Instance-based algorithm
Regression analysis
- Logistic regression
- Ordinary least squares regression (OLSR)
- Linear regression
- Stepwise regression
- Multivariate adaptive regression splines (MARS)
- Regularization algorithm
- Ridge regression
- Least Absolute Shrinkage and Selection Operator (LASSO)
- Elastic net
- Least-angle regression (LARS)
- Classifiers
- Probabilistic classifier
- Binary classifier
- Linear classifier
- Hierarchical classifier
Dimensionality reduction
- Canonical correlation analysis (CCA)
- Factor analysis
- Feature extraction
- Feature selection
- Independent component analysis (ICA)
- Linear discriminant analysis (LDA)
- Multidimensional scaling (MDS)
- Non-negative matrix factorization (NMF)
- Partial least squares regression (PLSR)
- Principal component analysis (PCA)
- Principal component regression (PCR)
- Projection pursuit
- Sammon mapping
- t-distributed stochastic neighbor embedding (t-SNE)
Ensemble learning
- AdaBoost
- Boosting
- Bootstrap aggregating (also "bagging" or "bootstrapping")
- Ensemble averaging
- Gradient boosted decision tree (GBDT)
- Gradient boosting
- Random Forest
- Stacked Generalization
Meta-learning
Reinforcement learning
- Q-learning
- State–action–reward–state–action (SARSA)
- Temporal difference learning (TD)
- Learning Automata
Supervised learning
- Averaged one-dependence estimators (AODE)
- Artificial neural network
- Case-based reasoning
- Gaussian process regression
- Gene expression programming
- Group method of data handling (GMDH)
- Inductive logic programming
- Instance-based learning
- Lazy learning
- Learning Automata
- Learning Vector Quantization
- Logistic Model Tree
- Minimum message length (decision trees, decision graphs, etc.)
- Probably approximately correct learning (PAC) learning
- Ripple down rules, a knowledge acquisition methodology
- Symbolic machine learning algorithms
- Support vector machines
- Random Forests
- Ensembles of classifiers
- Ordinal classification
- Conditional Random Field
- ANOVA
- Quadratic classifiers
- k-nearest neighbor
- Boosting
- SPRINT
- Bayesian networks
- Hidden Markov models
- Hierarchical hidden Markov model
Bayesian
- Bayesian knowledge base
- Naive Bayes
- Gaussian Naive Bayes
- Multinomial Naive Bayes
- Averaged One-Dependence Estimators (AODE)
- Bayesian Belief Network (BBN)
- Bayesian Network (BN)
Decision tree algorithms
Decision tree algorithm
- Decision tree
- Classification and regression tree (CART)
- Iterative Dichotomiser 3 (ID3)
- C4.5 algorithm
- C5.0 algorithm
- Chi-squared Automatic Interaction Detection (CHAID)
- Decision stump
- Conditional decision tree
- ID3 algorithm
- Random forest
- SLIQ
Linear classifier
- Fisher's linear discriminant
- Linear regression
- Logistic regression
- Multinomial logistic regression
- Naive Bayes classifier
- Perceptron
- Support vector machine
Unsupervised learning
- Expectation-maximization algorithm
- Vector Quantization
- Generative topographic map
- Information bottleneck method
- Association rule learning algorithms
Artificial neural networks
- Feedforward neural network
- Extreme learning machine
- Convolutional neural network
- Recurrent neural network
- Logic learning machine
- Self-organizing map
Association rule learning
Hierarchical clustering
- Single-linkage clustering
- Conceptual clustering
Cluster analysis
- BIRCH
- DBSCAN
- Expectation–maximization (EM)
- Fuzzy clustering
- Hierarchical clustering
- k-means clustering
- k-medians
- Mean-shift
- OPTICS algorithm
Anomaly detection
Semi-supervised learning
- Active learning
- Generative models
- Low-density separation
- Graph-based methods
- Co-training
- Transduction
Deep learning
- Deep belief networks
- Deep Boltzmann machines
- Deep Convolutional neural networks
- Deep Recurrent neural networks
- Hierarchical temporal memory
- Generative Adversarial Network
- Style transfer
- Transformer
- Stacked Auto-Encoders
Other machine learning methods and problems
- Anomaly detection
- Association rules
- Bias-variance dilemma
- Classification
- Multi-label classification
- Clustering
- Data Pre-processing
- Empirical risk minimization
- Feature engineering
- Feature learning
- Learning to rank
- Occam learning
- Online machine learning
- PAC learning
- Regression
- Reinforcement Learning
- Semi-supervised learning
- Statistical learning
- Structured prediction
- Graphical models
- Bayesian network
- Conditional random field (CRF)
- Hidden Markov model (HMM)
- Graphical models
- Unsupervised learning
- VC theory
Machine learning research
- List of artificial intelligence projects
- List of datasets for machine learning research
History of machine learning
Machine learning projects
Machine learning projects:
- DeepMind
- Google Brain
- OpenAI
- Meta AI
- Hugging Face
Machine learning organizations
Machine learning conferences and workshops
- Artificial Intelligence and Security (AISec) (co-located workshop with CCS)
- Conference on Neural Information Processing Systems (NIPS)
- ECML PKDD
- International Conference on Machine Learning (ICML)
- ML4ALL (Machine Learning For All)
Machine learning publications
Books on machine learning
- Mathematics for Machine Learning
- Hands-On Machine Learning Scikit-Learn, Keras, and TensorFlow
- The Hundred-Page Machine Learning Book
Machine learning journals
- Machine Learning
- Journal of Machine Learning Research (JMLR)
- Neural Computation
Persons influential in machine learning
- Alberto Broggi
- Andrei Knyazev
- Andrew McCallum
- Andrew Ng
- Anuraag Jain
- Armin B. Cremers
- Ayanna Howard
- Barney Pell
- Ben Goertzel
- Ben Taskar
- Bernhard Schölkopf
- Brian D. Ripley
- Christopher G. Atkeson
- Corinna Cortes
- Demis Hassabis
- Douglas Lenat
- Eric Xing
- Ernst Dickmanns
- Geoffrey Hinton
- Hans-Peter Kriegel
- Hartmut Neven
- Heikki Mannila
- Ian Goodfellow
- Jacek M. Zurada
- Jaime Carbonell
- Jeremy Slovak
- Jerome H. Friedman
- John D. Lafferty
- John Platt
- Julie Beth Lovins
- Jürgen Schmidhuber
- Karl Steinbuch
- Katia Sycara
- Leo Breiman
- Lise Getoor
- Luca Maria Gambardella
- Léon Bottou
- Marcus Hutter
- Mehryar Mohri
- Michael Collins
- Michael I. Jordan
- Michael L. Littman
- Nando de Freitas
- Ofer Dekel
- Oren Etzioni
- Pedro Domingos
- Peter Flach
- Pierre Baldi
- Pushmeet Kohli
- Ray Kurzweil
- Rayid Ghani
- Ross Quinlan
- Salvatore J. Stolfo
- Sebastian Thrun
- Selmer Bringsjord
- Sepp Hochreiter
- Shane Legg
- Stephen Muggleton
- Steve Omohundro
- Tom M. Mitchell
- Trevor Hastie
- Vasant Honavar
- Vladimir Vapnik
- Yann LeCun
- Yasuo Matsuyama
- Yoshua Bengio
- Zoubin Ghahramani
More in Artificial Intelligence
Dependent and independent variablesConcept in mathematical modeling, statistical modeling and experimental sciencesNeural networkStructure in biology and artificial intelligenceRight to privacyLegal tradition restraining actions threatening individual privacyStorytellingSocial and cultural sharing of storiesData collectionGathering information for analysisAnalyticsDiscovery, interpretation, and communication of meaningful patterns in data