### Naive Bayes Classification | Machine Learning

In this tutorial, we are going to learn the intuition behind the Naive Bayes classification algorithm and implement it in Python.

Naive Bayes Intuition: It is a classification technique based on Bayes Theorem. In simple terms, it is a probabilistic classifier which assumes that the presence of a particular feature in a class is not related to the presence of other features. It calculates the Posterior probability of all the events using the Bayes Theorem. Then it takes the event that has the maximum posterior probability.

To clear the idea, lets have a look at the Bayes Theorem.

Here,
P (H | D) = The conditional probability for event H to occur given that event D has occurred. This is known as the Posterior probability.

P (H) and P (D) = The probability of event H and D to occur without depending on each other.

P (D | H) = The probability of event D to occur given that even H has occurred. It is also known as the likelihood of the event.

Now lets get into work. We will apply this Bayes Theorem to make our Naive Bayes classifier. Lets take an example.

Here we have 30 data points(excluding the grey one) of people with two features Salary and Age, and their choice whether they walk or drive based on these two distinctive features. Now, for a new data point(the grey data point), we need to classify which class does it belong. That means we should find whether this new person walks or drive. For this, we will apply the Naive Bayes technique to take the decision.

First of all, we will apply Bayes Theorem to calculate the posterior probability of walking for this new data point based on the given features X. That is how likely the person walks.

In the same way, we will calculate the probability of driving

After calculating both the probabilities, the algorithm will compare them, and take the one that has the highest value.

Step 1: Now we will calculate all the prior probability, marginal likelihood, likelihood, and posterior probability of a person likely to walk.

The prior probability, P(Walks) is simply the probability of the persons who walk among all the people.                                     For marginal likelihood, P(X), we will make a circle around the new data point and calculate all the observations (including red and green). The radius of the circle depends upon you. That means you can take different radii depending on the algorithm.

The likelihood is the probability of such persons who walk to work. So, here we are concerned only with the red dots.

After calculating all these, now we can put them into the Bayes' Theorem

Step 2: Now, we will do similar calculations for P(Drives | X)

Putting all these together, we get

Step 3: Now we will compare both the probabilities. Then we will take the higher probability value as the output.

Here we can see the probability of a person likely to walk is greater than the probability for a person to drive. So we say that our new point falls into the category of people who walks.

Naive Bayes in Python: Now, we will implement the algorithm in Python. For this task, we will use the Social_Network_Ads.csv dataset. Lets have a glimpse of that dataset-

First of all, we will import all the essential libraries.

```# Importing essential libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
```

Now, we will import the dataset to our program

```# Importing the dataset

From the dataset, we take the Age and EstimatedSalary columns in the Feature matrix as they are independent features and the Purchased column in the Dependent vector.

```# Making the Feature matris and dependent vector
X = dataset.iloc[:, [2, 3]].values
y = dataset.iloc[:, 4].values
```

Now, we split our dataset into training and test sets.

```# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25, random_state = 0) ```

We need to scale the training and test sets to get a better prediction.

```#Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)```

Now, we will fit the Naive Bayes algorithm to our dataset.

```# Fitting Naive Bayes to the Training set
from sklearn.naive_bayes import GaussianNB
classifier = GaussianNB()
classifier.fit(X_train, y_train)
```

Its time to see how our model predicts the test set result.

```# Predicting the Test set results
y_pred = classifier.predict(X_test)```

We will see the performance of our model using the confusion matrix.

`# Making the Confusion Matrixfrom sklearn.metrics import confusion_matrixcm = confusion_matrix(y_test, y_pred)`

Now, we will visualize our model with the training set result.

```# Visualising the Training set results
from matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Naive Bayes (Training set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()
```

We will now see how it performs on our test set. Lets visualize this.

```# Visualising the Test set results
from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:, 0].max() + 1, step = 0.01),
np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Naive Bayes (Test set)')
plt.xlabel('Age')
plt.ylabel('Estimated Salary')
plt.legend()
plt.show()```