Generative vs Discriminative Models
Machine learning models can be classified into two types of models – Discriminative and Generative models. In simple words, a discriminative model makes predictions on the unseen data based on conditional probability and can be used either for classification or regression problem statements. On the contrary, a generative model focuses on the distribution of a dataset to return a probability for a given example.
Discriminative Models
Discriminative models separate classes instead of modeling the conditional probability and don’t make any assumptions about the data points. But these models are not capable of generating new data points. Therefore, the ultimate objective of discriminative models is to separate one class from another.
In case of outliers present in the dataset, then discriminative models work better compared to generative models i.e, discriminative models are more robust to outliers. However, there is one major drawback of these models is the misclassification problem, i.e., wrongly classifying a data point.
Some examples are:
Scalar Vector Machine(SVMs)
Conditional Random Fields (CRFs)
Decision Trees and Random Forest
Logistic regression
Traditional Neural Networks
Nearest Neighbor
Generative Models
Generative models focus on the distribution of individual classes in a dataset and the learning algorithms tend to model the underlying patterns or distribution of the data points. These models use the concept of joint probability and create the instances where a given feature () or input and the desired output or label () exist at the same time.
These models use probability estimates and likelihood to model data points and differentiate between different class labels present in a dataset. Unlike discriminative models, these models are also capable of generating new data points.
However, they also have a major drawback – If there is a presence of outliers in the dataset, then it affects these types of models to a significant extent.
Some examples are:
Naïve Bayes
Bayesian networks
Markov random fields- Hidden Markov Models (HMMs)
Latent Dirichlet Allocation (LDA)
Generative Adversarial Networks (GANs)
Autoregressive Model
Questions
Last updated