ML Mod 6

1. Write a short note on ISA and compare it with PCA.
Ans
2. Explain in detail Principal Component Analysis for Dimension Reduction.
Ans
Principal component analysis (PCA) is a dimensionality reduction technique
that transforms a set of correlated variables into a smaller set of uncorrelated
variables called principal components. PCA is often used for data visualization,
feature selection, and anomaly detection.
The main goal of Principal Component Analysis (PCA) is to reduce the

dimensionality of a dataset while preserving the most important patterns or
relationships between the variables without any prior knowledge of the target
variables.
PCA works by:
1. Calculating the covariance matrix of the data. The covariance matrix
measures the linear relationship between each pair of variables.
2. Finding the eigenvectors and eigenvalues of the covariance matrix. The
eigenvectors represent the principal components, and the eigenvalues
represent the amount of variance that each principal component
captures.
3. Sorting the principal components by eigenvalue, from highest to lowest.
4. Selecting the first few principal components, which capture the most
variance in the data.
5. Projecting the data onto the selected principal components. This produces
a new dataset with fewer dimensions, but which still contains most of the
information in the original dataset.
3. Why is Dimensionality reduction a very important step in Machine Learning?
Ans
Dimensionality reduction is a critical step in machine learning for several reasons:
1. Curse of Dimensionality: High-dimensional data can lead to increased

computational complexity and require more data to achieve reliable results.
2. Improved Model Performance: By reducing dimensionality, models become less

prone to overfitting, as they focus on the most important features.
3. Visualization: Reducing dimensions makes it easier to visualize and understand

data, aiding in data exploration and model interpretation.
4. Feature Engineering: Dimensionality reduction can help identify and retain the most
relevant features, improving model efficiency and effectiveness.
5. Computational Efficiency: Simplifying the dataset reduces training and inference

times, making models more practical for real-world applications.
6. Noise Reduction: It can help filter out noise and irrelevant information, enhancing
the signal-to-noise ratio in the data.
7. Data Compression: It can be used for efficient data storage and transmission,
particularly important in applications like image and signal processing.
8.Collinearity Resolution: It can mitigate issues of multicollinearity when features are

highly correlated.
Q4 . Linear Discriminant Analysis for Dimension Reduction
Ans
Linear Discriminant Analysis (LDA) is a supervised learning algorithm
used for classification tasks in machine learning. It is a technique used to
find a linear combination of features that best separates the classes in a
dataset.
LDA is a dimensionality reduction technique commonly used in the field of
machine learning and statistics. Unlike other dimensionality reduction
techniques such as Principal Component Analysis (PCA), LDA takes into
account the class labels of the data points.
LDA works by projecting the data onto a lower-dimensional space that

maximizes the separation between the classes. It does this by finding a
set of linear discriminants that maximize the ratio of between-class variance
to within-class variance.
Objective:
● LDA aims to find a linear combination of features that separates
two or more classes in the data.
Supervised Technique:
● LDA is a supervised technique, meaning it utilizes class labels
during training to maximize the separation between classes.
Eigenvalue Decomposition:
● The transformation is achieved by calculating the eigenvalues and
eigenvectors of the covariance matrix. The eigenvectors form the
new feature space.
Applications:
● LDA is often used for feature extraction and dimensionality
reduction in classification problems. It helps to improve the
performance of classifiers by reducing the number of features while
preserving class discrimination.
Before LDA 2D PLANE converted to After LDA 1D plane

ML Mod 6

Uploaded by

Copyright:

Available Formats

ML Mod 6

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ML Mod 6

Uploaded by

Copyright:

Available Formats

1. Write a short note on ISA and compare it with PCA.

The main goal of Principal Component Analysis (PCA) is to reduce the

1. Curse of Dimensionality: High-dimensional data can lead to increased

2. Improved Model Performance: By reducing dimensionality, models become less

3. Visualization: Reducing dimensions makes it easier to visualize and understand

5. Computational Efficiency: Simplifying the dataset reduces training and inference

8.Collinearity Resolution: It can mitigate issues of multicollinearity when features are

LDA works by projecting the data onto a lower-dimensional space that

You might also like