ASSIGNMENT

HARSHIT GUPTA
2K22/BMS/14
Q1. What is regression and its types? State the assumptions in a linear regression
model. How do you know that linear regression is suitable for any given data?

Ans.1-
Regression is a statistical technique used to model the relationship between a dependent
variable and one or more independent variables. The goal is to understand how changes in
the independent variables are associated with changes in the dependent variable. It is
commonly employed for prediction, forecasting, and understanding the strength and nature
of relationships between variables.

There are various types of Regression, these are:

1.Linear Regression: Assumes a linear relationship between the variables.
2. Multiple Regression: Involves more than one independent variable.
3. Polynomial Regression: Allows for non-linear relationships by including polynomial terms.
4. Ridge Regression and Lasso Regression: Used for regularization and variable selection.
5. Logistic Regression: Applied when the dependent variable is binary (two categories).
6. Time Series Regression: Applied to time-ordered data.

Assumptions in Linear Regression:

1. Linearity: Assumes a linear relationship between the independent and dependent
variables.
2. Independence: Assumes that the residuals (the differences between observed and
predicted values) are independent of each other.
3. Homoscedasticity: Assumes constant variance of the residuals across all levels of the
independent variable(s).
4. Normality of Residuals: Assumes that the residuals are normally distributed.
5. No Perfect Multicollinearity: Assumes that the independent variables are not perfectly
correlated.

Suitability of Linear Regression for Data:

To assess whether linear regression is suitable for a given dataset, consider the following:

1. Scatter Plots: Plotting the data can provide a visual indication of whether a linear
relationship exists.
2. Residual Analysis: Examining the residuals for patterns helps assess the assumptions of
linearity, independence, and homoscedasticity.
3. Correlation Coefficients: Calculating correlation coefficients between variables can
indicate the strength and direction of relationships.
4. Domain Knowledge: Understanding the nature of the variables and the underlying
problem is crucial. Linear regression might not be suitable if the relationship is inherently
non-linear.

If the assumptions are met and the relationships are linear, linear regression can be a
suitable model. However, if the assumptions are violated or the relationship is non-linear,
alternative regression techniques might be more appropriate.

Q2. How residuals help in checking the assumptions of linear regression model?

Ans.2- Residuals play a crucial role in checking the assumptions of a linear regression
model. Residuals are the differences between the observed values and the values predicted
by the regression model. Analysing residuals helps assess whether the model satisfies key
assumptions. Here's how residuals contribute to checking these assumptions:
1. Linearity:
- Assumption: Assumes a linear relationship between the independent and dependent
variables.
- Residual Check: Plot the residuals against the predicted values. A random scatter pattern
suggests linearity, while a systematic pattern may indicate non-linearity.

2. Independence:
- Assumption: Assumes that the residuals are independent of each other.
- Residual Check: Plot the residuals against the independent variables or any other
relevant variable (e.g., time). If there is no discernible pattern, independence is likely
maintained.

3. Homoscedasticity:
- Assumption: Assumes constant variance of the residuals across all levels of the
independent variable(s).
- Residual Check: Plot the residuals against the predicted values. A constant spread of
points indicates homoscedasticity, while a funnel-shaped pattern or a change in spread
suggests heteroscedasticity.

4. Normality of Residuals:
- Assumption: Assumes that the residuals are normally distributed.
- Residual Check: Construct a histogram or a Q-Q plot of the residuals. Deviations from
normality may indicate issues with this assumption.

5. No Perfect Multicollinearity:
- Assumption: Assumes that the independent variables are not perfectly correlated.
- Residual Check: Examine the variance inflation factor (VIF) for each independent
variable. High VIF values may indicate multicollinearity issues.

By examining residuals, we can identify patterns or deviations that may signal violations of
these assumptions. Addressing these issues might involve refining the model, transforming
variables, or considering alternative regression techniques.

Q.3- What is clustering and its types in machine learning.

Ans.3- Clustering in Machine Learning:

Clustering is a type of unsupervised machine learning technique that involves grouping

similar data points together. The goal is to discover inherent patterns or structures within the
data without prior knowledge of the groups. Clustering algorithms aim to maximize the
similarity within clusters and minimize the similarity between different clusters.

Types of Clustering:

1. K-Means Clustering:
- Description: Divides data into 'k' clusters based on similarity. Each cluster is represented
by its centroid.
- Use Case: Customer segmentation, image compression, document categorization.

2. Hierarchical Clustering:
- Description: Creates a tree of clusters (dendrogram) by recursively merging or splitting
existing clusters.
- Use Case: Taxonomy creation, evolutionary biology.
3. DBSCAN (Density-Based Spatial Clustering of Applications with Noise):
- Description: Identifies clusters based on density, allowing for irregularly shaped clusters.
Points not part of any cluster are considered outliers.
- Use Case: Anomaly detection, spatial data analysis.

4. Agglomerative Clustering:
- Description: Starts with individual data points as clusters and merges them based on
similarity until only one cluster remains.
- Use Case: Biological taxonomy, image segmentation.

5. Mean Shift:
- Description: Iteratively shifts cluster centroids to areas with higher point density.
- Use Case: Image segmentation, tracking objects in video.

6. Gaussian Mixture Models (GMM):

- Description: Models data as a mixture of Gaussian distributions, allowing for probabilistic
assignment of data points to clusters.
- Use Case: Density estimation, image segmentation.

7. Spectral Clustering:
- Description: Utilizes the eigenvalues of a similarity matrix to reduce the dimensionality of
the data before clustering.
- Use Case: Image segmentation, document clustering.

8. OPTICS (Ordering Points To Identify the Clustering Structure):

- Description: Identifies clusters of varying shapes and densities by analyzing the
reachability of data points.
- Use Case: Spatial data analysis, network analysis.

Choosing the appropriate clustering algorithm depends on the nature of the data, the desired
outcomes, and the characteristics of the clusters being sought. Each algorithm has its
strengths and weaknesses, making it suitable for specific scenarios.

Q.4 What are different measures of descriptive analytics.

Ans.4- Descriptive analytics involves summarizing and presenting data to gain insights into
its main features. Several measures are commonly used in descriptive analytics to
characterize different aspects of a dataset. Here are some key measures:

1. Measures of Central Tendency:

- Mean (Average): The sum of all values divided by the number of values.
- Median: The middle value when the data is sorted; it's not affected by extreme values.
- Mode: The most frequently occurring value in a dataset.

2. Measures of Dispersion (Variability):

- Range: The difference between the maximum and minimum values.
- Variance: The average of the squared differences from the mean.
- Standard Deviation: The square root of the variance, providing a measure of how spread
out values are.

3. Measures of Shape:
- Skewness: Indicates the asymmetry of a distribution. Positive skewness means a longer
tail on the right, while negative skewness means a longer tail on the left.
 - Kurtosis: Measures the "tailedness" of a distribution. High kurtosis indicates heavy tails,
and low kurtosis indicates light tails.

4. Measures of Position:
- Percentiles: Divides the data into 100 equal parts. The median is the 50th percentile.
- Quartiles: Divide the data into four equal parts. The first quartile (Q1) is the 25th
percentile, the second quartile (Q2) is the median, and the third quartile (Q3) is the 75th
percentile.

5. Frequency Distributions:
- Histograms: A graphical representation of the distribution of a dataset, showing the
frequency of values within predefined bins.

6. Categorical Data Measures:

- Mode (for categorical data): The most frequently occurring category.
- Bar Charts: Visual representation of the frequency of each category.

7. Correlation:
- Correlation Coefficient (e.g., Pearson's correlation): Measures the strength and direction
of a linear relationship between two variables.

8. Summary Tables:
- Frequency Tables: Summarize the distribution of categorical variables.
- Cross-Tabulations (Contingency Tables): Show the joint distribution of two or more
categorical variables.

These measures collectively provide a comprehensive overview of the characteristics of a

dataset.

Q.5- What is the difference between supervised and unsupervised machine learning?
When should you use classification over regression?

Ans.5- Supervised Learning vs. Unsupervised Learning:

1. Supervised Learning:
- Definition: In supervised learning, the algorithm is trained on a labeled dataset, where the
input data is paired with corresponding output labels. The goal is to learn a mapping from
inputs to outputs.
- Objective: The algorithm aims to make predictions or decisions based on new, unseen
data.
- Examples: Classification and regression are common tasks in supervised learning.

2. Unsupervised Learning:
- Definition: In unsupervised learning, the algorithm is given input data without explicit
output labels. The goal is to find patterns, relationships, or structures within the data.
- Objective: Discover the inherent structure of the data, such as grouping similar data
points (clustering) or reducing the dimensionality of the data.
- Examples: Clustering, dimensionality reduction, and association rule learning.

Choosing Between Classification and Regression:

Use Classification Over Regression When:

1. Nature of the Output:

 - Classification: When the output is categorical or belongs to a distinct class. For example,
predicting whether an email is spam or not.
- Regression: When the output is a continuous value. For instance, predicting house
prices.

2. Predictive vs. Descriptive:

- Classification: When the goal is to predict the class or category of a new instance.
- Regression: When the goal is to predict a numeric value, often for forecasting or
estimation purposes.

3. Output Interpretability:
- Classification: Provides clear, discrete labels, making it suitable for tasks with distinct
categories.
- Regression: Offers a continuous range of values, suitable for tasks where the output is on
a scale.

4. Error Interpretation:
- Classification: Evaluation typically involves metrics like accuracy, precision, recall, and F1
score.
- Regression: Evaluation metrics include mean squared error, mean absolute error, or R-
squared.

5. Example Applications:
- Classification: Spam detection, image recognition, sentiment analysis.
- Regression: Predicting sales, estimating temperature, predicting stock prices.

Therefore, the choice between classification and regression depends on the nature of the
output variable and the specific goals of the machine learning task. If the output is
categorical and the goal is to predict classes, classification is appropriate. If the goal is to
predict a continuous value, regression is the suitable choice.

Q.6- What is confusion matrix in machine learning?

Ans.6- A confusion matrix is a table used in machine learning to evaluate the performance of
a classification algorithm. It is particularly useful when the model's outputs need to be
compared to the true outcomes in a binary or multiclass classification problem. The
confusion matrix provides a summary of the predictions made by a model, highlighting
correct and incorrect classifications.

In a binary classification scenario, a confusion matrix has four entries:

1. True Positive (TP): Instances correctly predicted as positive (actual positive and predicted
positive).
2. False Positive (FP): Instances incorrectly predicted as positive (actual negative but
predicted positive).
3. True Negative (TN): Instances correctly predicted as negative (actual negative and
predicted negative).
4. False Negative (FN): Instances incorrectly predicted as negative (actual positive but
predicted negative).
The layout of a confusion matrix looks like this:

```
Actual Positive Actual Negative
Predicted Positive TP FP
Predicted Negative FN TN
```

From the confusion matrix, various performance metrics can be derived:

- Accuracy: (TP + TN) / (TP + FP + TN + FN

- Precision (Positive Predictive Value): TP / (TP + FP)
- Recall (Sensitivity or True Positive Rate): TP / (TP + FN)
- Specificity (True Negative Rate): TN / (TN + FP)
- F1 Score: 2 (Precision *Recall) / (Precision + Recall)

These metrics help assess different aspects of the model's performance, such as its ability
to correctly classify positive instances (precision), its sensitivity to positive instances (recall),
and the balance between precision and recall (F1 score).Hence, confusion matrix is a
valuable tool for understanding the strengths and weaknesses of a classification model.

Q.7- What is Logistic Regression? State its importance in machine learning.

Ans.7-Logistic Regression is a statistical method used for binary classification

problems, where the outcome variable is categorical and has two classes. Despite
its name, logistic regression is used for classification, not regression. It's called
"regression" because it uses a logistic function to model a binary dependent
variable.

The logistic function, also known as the sigmoid function, maps any real-valued
number into a range between 0 and 1. The logistic regression model calculates
the probability that a given instance belongs to a particular category.

The logistic regression model can be expressed as:

Here,

The importance of logistic regression in machine learning includes:

1. Binary Classification: Logistic regression is widely used for binary classification

problems, such as spam detection, medical diagnosis (e.g., disease
presence/absence), and credit risk analysis.

2. Interpretability: The coefficients in logistic regression provide insights into the

relationship between the features and the likelihood of the outcome. This makes it
easier to interpret and understand the impact of each feature on the prediction.
3. Efficiency: Logistic regression is computationally efficient and does not require
high computational resources. It is particularly useful when dealing with large
datasets.

4. Regularization: Logistic regression can be regularized to prevent overfitting.

Regularization techniques, such as L1 or L2 regularization, help control the
complexity of the model and improve its generalization to new data.

5. Probability Estimation: Logistic regression provides probabilities, allowing you

to assess the likelihood of an instance belonging to a particular class. This is
valuable in scenarios where knowing the certainty of predictions is crucial.

6. Feature Importance: By examining the coefficients of logistic regression, you

can identify the importance of different features in influencing the model's
predictions.

Q.8-Explain the use of ROC curve and AUC of a ROC curve.

Ans.8-The Receiver Operating Characteristic (ROC) curve is a graphical

representation of the performance of a binary classification model at various
classification thresholds. It illustrates the trade-off between the true positive rate
(sensitivity) and the false positive rate (1-specificity) across different threshold
values. The Area Under the ROC Curve (AUC-ROC) is a summary metric used to
quantify the overall performance of a classification model.

Let's break down these concepts:

ROC Curve:

- True Positive Rate (Sensitivity): This is the proportion of actual positive

instances correctly predicted by the model. It is calculated as

- False Positive Rate (1-Specificity): This is the proportion of actual negative

instances incorrectly predicted as positive by the model. It is calculated as

The ROC curve is created by plotting the true positive rate against the false
positive rate at various threshold settings. Each point on the curve represents a
different threshold. A diagonal line (the line of no-discrimination) is drawn,
representing a model that performs no better than random chance.

A good classifier's ROC curve will be positioned towards the upper-left corner of
the plot, indicating higher true positive rates and lower false positive rates across
different thresholds.

AUC-ROC (Area Under the ROC Curve):

The AUC-ROC is a single scalar value that quantifies the overall performance of a
classification model. It represents the area under the ROC curve. The AUC-ROC
value ranges from 0 to 1, where:

- AUC-ROC = 0.5 implies a model that performs no better than random chance.
- AUC-ROC > 0.5 implies a model that is better than random chance.
- AUC-ROC = 1 implies a perfect classifier.

A higher AUC-ROC indicates better discrimination between positive and negative

classes. It provides a useful summary of the model's ability to distinguish between
the two classes across various threshold settings.

Hence, the ROC curve and AUC-ROC are valuable tools for evaluating and
comparing the performance of binary classification models. They provide insights
into the trade-off between sensitivity and specificity and offer a concise summary
of the model's overall discriminative power.

Q.9 What are the methods to determine optimal cutoff probability in logistic
regression?

Ans.9-The optimal cutoff probability in logistic regression is the threshold

probability that is used to classify instances into the positive or negative class. By
default, logistic regression models often use a cutoff probability of 0.5, meaning
that if the predicted probability of an instance belonging to the positive class is
greater than or equal to 0.5, it is classified as positive; otherwise, it is classified as
negative.

However, in some cases, you might want to adjust this threshold based on the
specific needs of your application or to achieve a better balance between false
positives and false negatives. Several methods can be used to determine the
optimal cutoff probability in logistic regression:

1. ROC Curve Analysis:

- Plot the ROC curve by varying the threshold from 0 to 1.
- Identify the point on the curve that is closest to the top-left corner (ideal
sensitivity and specificity).
- The corresponding threshold is considered as the optimal cutoff.

2. Youden's J Statistic:
- Youden's J statistic is calculated as
- Identify the threshold that maximizes the Youden's J statistic.

3. Cost-Benefit Analysis:
- Assign costs to false positives and false negatives based on the specific
context of the problem.
- Choose the threshold that minimizes the total cost, taking into account the
costs associated with misclassifications.

4. Precision-Recall Tradeoff:
- Consider the precision and recall metrics at different cutoff probabilities.
- Choose the threshold that provides the desired balance between precision and
recall.
5. F1 Score Maximization:
- Calculate the F1 score at different thresholds.
- Choose the threshold that maximizes the F1 score (harmonic mean of
precision and recall).

6. Cross-Validation:
- Use cross-validation techniques to evaluate model performance at different
thresholds.
- Select the threshold that maximizes performance on the validation set.

7. Business Rules and Requirements:

- Consider any specific business rules or requirements that may guide the
choice of the optimal cutoff.
- For example, in a medical setting, you might prioritize sensitivity over
specificity.

It's important to note that the optimal cutoff may vary depending on the specific
goals and constraints of your application. The choice of the threshold involves a
trade-off between different evaluation metrics and the specific consequences of
false positives and false negatives in your domain.