DL Practical 3 Loss Function

Practical 03
Title: Loss Function in ANN

Aim: To implement Python code for Loss Functions
Theory:
Loss functions in Python are an integral part of any machine learning model. Loss
functions are one of the most important aspects of neural networks, as they are
directly responsible for fitting the model to the given training data.
A loss function is a function that compares the target and predicted output values
and measures how well the neural network models the training data. These
functions tell us how much the predicted output of the model differs from the
actual output. When training, we aim to minimize this loss between the predicted
and target outputs.
Loss functions are mainly classified into two different categories:
1.Classification loss: is the case where the aim is to predict the output from
different categorical values for example, if we have a dataset of handwritten
images and the digit is to be predicted that lies between (0–9), in these kinds of
scenarios classification loss is used.
2. Regression Loss: Whereas if the problem is regression like predicting the
continuous values for example, if need to predict the weather conditions or
predicting the prices of houses based on some features. In this type of case,
Regression Loss is used.
Use of loss function:
The below equation represents how a neural network processes the input data at
each layer and eventually produces a predicted output value.
To train — the process by which the model maps the relationship between the
training data and the outputs — the neural network updates its hyperparameters,
the weights, wT, and biases, b, to satisfy the equation above. Each training input
is loaded into the neural network in a process called forward propagation. Once
the model has produced an output, this predicted output is compared against the
given target output in a process called backpropagation — the hyperparameters of
the model are then adjusted so that it now outputs a result closer to the target
output. This is where loss functions come in.
Four Important Loss Functions
1. Mean Absolute Error
Mean absolute error (MAE) also called L1 Loss is a loss function used for
regression problems. It represents the difference between the original and
predicted values extracted by averaging the absolute difference over the
data set.
Mean absolute error is used when you are doing regression and don’t want
outliers to play a big role. It can also be useful if you know that your distribution
is multimodal, and it’s desirable to have predictions at one of the modes, rather
than at the mean of them.
2. Mean Square Error
Mean Squared Error (MSE) also called L2 Loss is also a loss function used
for regression. It represents the difference between the original and
predicted values extracted by squared the average difference over the data
set.
MSE is used when doing regression, believing that your target, conditioned on
the input, is normally distributed, and want large errors to be significantly
(quadratically) more penalized than small ones.
3. Root Mean Square Error
The Root Mean Squared Error (RMSE) is a mostly used error function.
It is the differences between values predicted by a model and the actual
desired values. The RMSE can be calculated by taking the square root of
above-mentioned Mean Squared Errors (MSE) / L2 Loss.
The root mean squared error (RMSE) is a useful loss function when large
errors are more costly, and it is important to minimize them.
4. Cross-Entropy Loss
Cross-entropy loss is used when adjusting model weights during training.

The aim is to minimize the loss, i.e, the smaller the loss the better the model.
A perfect model has a cross-entropy loss of 0.
In cross entropy loss we have:
a. Binary cross entropy: which is for binary classification tasks with two
classes 0 and 1
b. Categorical cross-entropy and sparse categorical cross- entropy: Both
categorical cross entropy and sparse categorical cross-entropy have the
same loss function. The only difference between the two is on how truth
labels are defined that is categorical cross entropy is used when true
labels are one-hot encoded and sparse categorical cross-entropy is used
when truth labels are integer encoded .
Code:
Implementing Loss Functions in Python
#Python implementation for MAE is as follows :
import numpy as np
def MAE(act, pred):
print("Actual :: ", act)
print("Predict:: ",pred)
diff = act - pred
print("Difference is ::", diff)
abs_diff = np.absolute(diff)
print("Absolute Difference is ::", abs_diff)
mean_diff = abs_diff.mean()
return mean_diff
# use of python mean absolute error function
act = np.array([100,200,300])
pred = np.array([115,180,331])
MAE (act,pred)
Output:
Actual :: [100 200 300]
Predict:: [115 180 331]
Difference is :: [-15 20 -31]
Absolute Difference is :: [15 20 31]
22.0
• You can also use mean_absolute_error from sklearn to calculate MAE:
from sklearn.metrics import mean_absolute_error
mean_absolute_error(act, pred)
Output: 22.0
#Python implementation for MSE is as follows :

import numpy as np
def mean_squared_error_my(act, pred):
diff = pred - act
differences_squared = diff ** 2
mean_diff = differences_squared.mean()
return mean_diff
# use of python mean squared error function
act = np.array([100,200,300])
pred = np.array([115,180,331])
mean_squared_error_my(act,pred)
Output: 528.6666666666666
from sklearn.metrics import mean_squared_error
val = mean_squared_error(act, pred)
print("MSE :: ",val)
Output: MSE :: 528.6666666666666
# use of python mean squared error function
act = np.array([1.1,2,1.7])
pred = np.array([1,1.7,1.5])
print(mean_squared_error(act,pred))
• You can also use mean_squared_error from sklearn to calculate MSE. Here’s how the
function works:
act = np.array([1.1,2,1.7])
pred = np.array([1,1.7,1.5])
mean_squared_error(act, pred)
Output: 0.04666666666666667
#Python implementation for RMSE is as follows:

import numpy as np
def root_mean_squared_error(act, pred):
diff = pred - act
differences_squared = diff ** 2
mean_diff = differences_squared.mean()
rmse_val = np.sqrt(mean_diff)
return rmse_val
print(root_mean_squared_error(act,pred))
Output: 0.21602468994692867
act = np.array( [10,20,30])
pred = np.array([11,19,33])
print(root_mean_squared_error(act,pred))
Output: 1.9148542155126762
• You can use mean_squared_error from sklearn to calculate RMSE as well. Let’s see
how to implement the RMSE using the same function:

act = np.array([1.1,2,1.7])
pred = np.array([1,1.7,1.5])
mean_squared_error(act, pred, squared = False)
Output: 0.21602468994692867
If the parameter ‘squared’ is set to True, then the function returns MSE value. If set to False,
the function returns RMSE value.
#Python implementation for cross-entropy loss function is as follows:
from sklearn.metrics import log_loss
log_loss(["Dog", "Cat", "Cat", "Dog"],[[.1, .9], [.9, .1], [.8, .2], [.35, .65]])
Output: 0.21616187468057912
Output: 0.10536051565782628
Output: 0.5325784680133981
log_loss(["Dog", "Cat", "Cat", "Dog"],[[0,1.0], [1,0], [1,0], [0,1]])
Output: 2.2204460492503136e-16
Conclusion:
We studied different Loss Functions in ANN and also implement them in
Python.
Experiment Number Date of Performance Grade Teacher's Sign

DL Practical 3 Loss Function

Uploaded by

Copyright:

Available Formats

DL Practical 3 Loss Function

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DL Practical 3 Loss Function

Uploaded by

Copyright:

Available Formats

Practical 03

Title: Loss Function in ANN

1. Mean Absolute Error

2. Mean Square Error

3. Root Mean Square Error

Cross-entropy loss is used when adjusting model weights during training.

In cross entropy loss we have:

#Python implementation for MSE is as follows :

#Python implementation for RMSE is as follows:

from sklearn.metrics import mean_squared_error

Experiment Number Date of Performance Grade Teacher's Sign

You might also like