What is Supervised Machine Learning?

Supervised Machine Learning is the process of determining the relationship between a given set of features (or variables) and a target value, which is also known as a label or a classification. Which means building ML Models that can take in certain input data and spit out a predicted value.

Let’s understand this by taking the example of – “Whether a bank should give House Loan to an applicant or not?” based upon information submitted by applicant to bank. Let’s assume that in the House Load Application, applicant submitted Age, Sex, Education Level, Income Level, Marital Status, Demographics, Previous Loans paid.

AgeSexEducation LevelIncome Level
(per year)
Marital StatusDemographicsPrevious Load Paid
30FemaleUndergraduate98,000 USDYesNew YorkYes
27MaleUndergraduate120,000 USDYesAustinNo
29FemaleHigh School67,000 USDNoLAYes
45MaleNone54,000 USDYesBaltimoreNo
24FemaleNone43,000 USDNoGeorgiaNo
64MaleUndergraduate180,000 USDNoBay AreaYes

Based upon this data, a Supervised Machine Learning Model can be trained which can provide a yes or no answer to question “Give the loan to applicant?” for newer applicants.

Supervised Machine Learning models can be further divided into Classification Tasks and Regression Tasks.

Classification Tasks in Supervised Machine Learning

Classification Tasks are used to build models out of data with discrete categories as labels. For example – A classification task can be used to predict whether a person will pay back loan or not. You have more than two discrete categories, such as predicting ranking of a horde in a race, but they must be a finite number.

4 circles coloured as - Purple, Yellow, Blue, Brown showing clustering of data

In the above image, Machine Learning Model is classifying observation in dataset as yellow, blue, blackish or pink.
Most classification tasks output the prediction as probability of an instance to belong to each output label. The assigned label is one with highest probability.

Supervised Classification Machine Learning Algorithms

  1. Decision Trees – This algorithm follow a tree-like architecture that simulates decision process following a series of decisions, considering one variable at a time.
  2. Naive Bayes Classifier – This algorithm relies on a group of probabilistic equations based on Bayes’ theorem, which assumes independence among features. It has ability to consider several attributes.
  3. Artificial Neural Networks (ANNs) – These replicate the structure and performance of a biological neural network to perform pattern recognition tasks. An ANN consists of interconnected neurons, laid out with a set architecture. They pass information to one another until a result is achieved.

Regression Tasks in Supervised Machine Learning

Regression Tasks are used for data with continuous quantities as labels. For example – A regression task can be used for predicting house prices. This means that value is represented by a quantity and not by a set of possible outputs. Output labels can be of integer or float types.

Some commonly used Supervised Machine Learning algorithms for Regression Tasks are Linear Regression, Regression Trees, Support Vector Regression, Artificial Neural Networks (ANNs).

Evaluating Performance of a Supervised Machine Learning Model

Model evaluation is essential for the development of successful models that perform well not just on the data that was used to train the model, but also on data that it has not seen yet.  When dealing with supervised machine learning problems, the process of assessing the model is made particularly simple as ground truth is already known which can be compared to prediction of model.

When applying a model to unseen data that does not have a label class to compare it to, determining the accuracy percentage of the model is essential. For example, a model with an accuracy of 95% may lead you to believe that the chances of making an accurate forecast are great, and as a result, the model should be considered trustworthy. But definitely that assumption can be wrong as well because metric “accuracy of model” implies what? is known. Moreover a specific performance measurement metric for a Supervised Machine Learning Model should be selected on case by case basis. Because for some models it would be better to use one metric while same metric can imply something else for other model. So be careful while selecting a specific metric to measure performance of a model.

Evaluating models’ performance should be done on two types of datasets – Validation DataSet to fine-tune the model and Testing DataSet to evaluate how well model will function when applied to data which it does not know about.

Hello

Metrics used for Measuring Performance of a Supervised Classification Machine Learning Model

Confusion Metric

Confusion Matrix is a table that contains information about performance of a model. In Confusion Metric table columns represent instances which belong to a predicted class, while rows represent instances that actually belong to a class.

Let’s understand what exactly is “Confusion Metric” by taking an example of how many images in a given dataset of 500 images are of dogs.

Prediction👉
Actual👇
Dog ImageNot Dog Image
Dog Image270230
Not Dog Image150350

Each cell in a Confusion Matrix can be classified as True Positives (TP), False Positives (FP), True Negatives (TN), False Negatives (FN).

Cell of Confusion MetricDescriptionExample
True Positives (TP)Refers to instances that model correctly classified the event as positiveCorrectly classifying image of the dog as god image
False Positives (FP)Refers to the instances that model incorrectly classified the event as positiveImages of other animals being classified as Dog Images by model
True Negatives (TN)Refers to the instances that model correctly classified event as negativeImages of other animals are being classified as not images of Dog by model
False Negatives (FN)Refers to the instances that model incorrectly classified the event as negativeImages of dog being classified as not images of Dog by model

Accuracy

Accuracy of a model measures its capability to correctly classify all instances. It can be calculated by summing up number of True Positives (TP) and True Negatives (TN) then dividing by total number of instances.

Accuracy = (TP + TN)/Total Number of Instances

Precision

Precision measures the model’s ability to correctly classify positive labels by comparing it with total number of instances predicted as positive.

Precision can be calculated by taking ratio of True Positives (TP) and sum of True Positives (TP) and False Positives (FP).

Precision = TP divided by sum of TP, FP

Recall

Recall measures the number of correctly predicted positive labels against all positive labels.

Recall can be calculated by taking ratio of True Positives (TP) and sum of True Positives (TP) and False Negatives (FN).

Recall = TP divided by sum of TP, FN

Metrics used for Measuring Performance of a Supervised Regression Machine Learning Model

Considering that regression tasks are those where final output is continuous rather than being categorical, the performance of model can be measured by comparing predicted value with actual value.

For example – The performance of a Supervised Regression Machine Learning Model which makes prediction about price of a house in a locality can be measured by comparing actual price of house with that predicted by model.

Let’s say actual price of a house in a locality is 700,000 USD but our model is predicting price of that being 699,999 USD which is pretty close, so we can say that given model is efficient enough as difference between predicted and actual value is quite low.

For measuring this difference between predicted and actual values Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE) can be used.

Mean Absolute Error (MAE)

It measures average absolute difference between predicted values and actual values, without taking into account the direction of error.

    \[ M A E=\frac{1}{m} * \sum_{i=1}^{m}\left|y_{i}-\widehat{y}_{i}\right| \]

  • m = Number of total instances
  • yi = Actual Value
  • ŷi = Predicted Value
Actual House Price (yi)Predicted House Price (ŷi)Predicted – Actual (yiŷi)
500,000 USD499,012 USD988 USD
650,000 USD590,918 USD59082 USD
839,193 USD832,039 USD7154 USD
127,092 USD120,043 USD7049 USD
983,028 USD980,832 USD2196 USD

    \[\sum_{i=1}^{m}\left|y_{i}-\widehat{y_{i}}\right| \]

= 76469 USD

MAE = 76469/5 = 15293.8 USD

Root Mean Square Error (RMSE)

It measures average magnitude of error between actual value and predicted value. It can be calculated by taking square root of average of squared difference between actual, predicted values.

    \[ R M S E=\sqrt{\frac{1}{m} \cdot \sum_{i=1}^{m}\left(y_{i}-\hat{y}_{i}\right)^{2}} \]

  • m = Number of total instances
  • yi = Actual Value
  • ŷi = Predicted Value
Actual House Price (yi)Predicted House Price (ŷi)Predicted – Actual (yiŷi)Predicted – Actual
(yiŷi)2
500,000 USD499,012 USD988 USD976144 USD
650,000 USD590,918 USD59082 USD3490682724USD
839,193 USD832,039 USD7154 USD51179716USD
127,092 USD120,043 USD7049 USD49688401USD
983,028 USD980,832 USD2196 USD4822416USD

    \[ \sum_{i=1}^{m}\left(y_{i}-\hat{y}_{i}\right)^{2}\]


= 3597349401

RMSE = Square Root (3597349401/5) = Square Root (719469880.2) = 26822.93 USD

Which one metric out of MAE, RMSE to use for measuring the performance of Supervised Regression Machine Learning Model?

Both of Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) expresses the average error, in a range from 0 to infinity, where lower the value the better the performance of model. The main difference between these two metrics is that MAE assigns same weight of importance to all errors, while RMSE squares the error, assigning higher weight to larger errrors.

Consider this, RMSE metric is especially useful in cases where larger errors should be penalized, meaning that outliers are taken into account in the measurement of performance. For example – RMSE metric can be used when a value that is off by 4 is more than twice as bad as being off by 2. The MAE, on the other hand, is used when a value that is off by 4 is just twice as bad as a value off by 2.

Gagan

Hi, there I'm founder of ComputerScienceHub(Started this to bring useful Computer Science information just at one place). Personally I've been doing JavaScript, Python development since 2015(Been long) - Worked upon couple of Web Development Projects, Did some Data Science stuff using Python. Nowadays primarily I work as Freelance JavaScript Developer(Web Developer) and on side-by-side managing team of Computer Science specialists at ComputerScienceHub.io

Leave a Reply

Your email address will not be published. Required fields are marked *

Recent Posts