Scribd Logo
Upload

Welcome to Scribd!

  • 22 views

    AI With Python-Data Preprocessing: Student Name Student Roll # Program Section

    Uploaded by

    every one
    The document discusses various techniques for preprocessing data before providing it to machine learning algorithms, including binarization, mean removal, scaling, and normalization. It provides code examples in Python of how to apply these techniques, such as using Scikit-learn's preprocessing tools to binarize data based on a threshold, center and scale data to remove the mean and standard deviation, and normalize data using L1 and L2 normalization. The code samples are meant to demonstrate how to preprocess sample input data into different formats and representations suitable for machine learning.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    AI With Python-Data Preprocessing: Student Name Student Roll # Program Section

    Uploaded by

    every one
    22 views7 pages
    The document discusses various techniques for preprocessing data before providing it to machine learning algorithms, including binarization, mean removal, scaling, and normalization. It provides code examples in Python of how to apply these techniques, such as using Scikit-learn's preprocessing tools to binarize data based on a threshold, center and scale data to remove the mean and standard deviation, and normalize data using L1 and L2 normalization. The code samples are meant to demonstrate how to preprocess sample input data into different formats and representations suitable for machine learning.

    Original Title

    Lab-06

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses various techniques for preprocessing data before providing it to machine learning algorithms, including binarization, mean removal, scaling, and normalization. It provides code examples in Python of how to apply these techniques, such as using Scikit-learn's preprocessing tools to binarize data based on a threshold, center and scale data to remove the mean and standard deviation, and normalize data using L1 and L2 normalization. The code samples are meant to demonstrate how to preprocess sample input data into different formats and representations suitable for machine learning.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    22 views7 pages

    AI With Python-Data Preprocessing: Student Name Student Roll # Program Section

    Uploaded by

    every one
    The document discusses various techniques for preprocessing data before providing it to machine learning algorithms, including binarization, mean removal, scaling, and normalization. It provides code examples in Python of how to apply these techniques, such as using Scikit-learn's preprocessing tools to binarize data based on a threshold, center and scale data to remove the mean and standard deviation, and normalize data using L1 and L2 normalization. The code samples are meant to demonstrate how to preprocess sample input data into different formats and representations suitable for machine learning.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 7

    Student Name

    Student Roll #

    Program

    Section

    AI with Python-Data
    Preprocessing
    Lab# 06

    ___________________________________________________________________________28
    Artificial Intelligence-Lab [COSC-3212]
    Preprocessing Data

    In our daily life, we deal with lots of data but this data is in raw form. To provide the data as the
    input of machine learning algorithms, we need to convert it into a meaningful data. That is where
    data preprocessing comes into picture. In other simple words, we can say that before providing the
    data to the machine learning algorithms we need to preprocess the data.

    1. DATA PREPROCESSING STEPS

    Follow these steps to preprocess the data in Python:

    Step 1: Importing the useful packages: If we are using Python then this would be the first step
    for converting the data into a certain format, i.e., preprocessing. It can be done as follows:

    import numpy as np

    from sklearn import preprocessing

    Here we have used the following two packages:

     NumPy: Basically NumPy is a general purpose array-processing package designed to efficiently


    manipulate large multi-dimensional arrays of arbitrary records without sacrificing too much speed
    for small multi-dimensional arrays.

     Sklearn.preprocessing: This package provides many common utility functions and transformer
    classes to change raw feature vectors into a representation that is more suitable for machine
    learning algorithms.

    Step 2: Defining sample data: After importing the packages, we need to define some sample data
    so that we can apply preprocessing techniques on that data. We will now define the following
    sample data:

    Input_data = np.array ([2.1, -1.9, 5.5], [-1.5, 2.4, 3.5], [0.5, -7.9, 5.6], [5.9, 2.3, -5.8])

    Step3: Applying preprocessing technique: In this step, we need to apply any of the preprocessing
    techniques. The following section describes the data preprocessing techniques.

    ___________________________________________________________________________29
    Artificial Intelligence-Lab [COSC-3212]
    2. TECHNIQUES FOR DATA PREPROCESSING

    The techniques for data preprocessing are described below:

    Binarization

    This is the preprocessing technique which is used when we need to convert our numerical values
    into Boolean values. We can use an inbuilt method to binarize the input data say by using 0.5 as
    the threshold value in the following way:

    data_binarized = preprocessing.Binarizer(threshold=0.5).transform(Input_data)
    print("\nBinarized data:\n", data_binarized)

    Output

    Now, after running the above code we will get the following output, all the values above
    0.5(threshold value) would be converted to 1 and all the values below 0.5 would be converted to
    0.

    Mean Removal

    It is another very common preprocessing technique that is used in machine learning. Basically it
    is used to eliminate the mean from feature vector so that every feature is centered on zero. We can
    also remove the bias from the features in the feature vector. For applying mean removal
    preprocessing technique on the sample data, we can write the Python code shown below. The code
    will display the Mean and Standard deviation of the input data:

    print("Mean =", input_data.mean(axis=0))

    print("Std deviation = ", Input_data.std(axis=0))

    We will get the following output after running the above lines of code:

    ___________________________________________________________________________30
    Artificial Intelligence-Lab [COSC-3212]
    Output

    Now, the code below will remove the Mean and Standard deviation of the input data:

    data_scaled = preprocessing.scale(input_data)

    print("Mean =", data_scaled.mean(axis=0))

    print("Std deviation =", data_scaled.std(axis=0))

    Provide output in following area after running the above lines of code:

    Output

    Scaling

    It is another data preprocessing technique that is used to scale the feature vectors. Scaling of feature
    vectors is needed because the values of every feature can vary between many random values. In
    other words we can say that scaling is important because we do not want any feature to be
    synthetically large or small. With the help of the following Python code, we can do the scaling of
    our input data, i.e., feature vector:

    # Min max scaling

    data_scaler_minmax=preprocessing.MinMaxScaler(feature_range=(0,1))

    ___________________________________________________________________________31
    Artificial Intelligence-Lab [COSC-3212]
    data_scaled_minmax = data_scaler_minmax.fit_transform(Input_data)

    print ("\nMin max scaled data:\n", data_scaled_minmax)

    Provide output in following area after running the above lines of code:

    Min max scaled data

    Output

    Normalization

    It is another data preprocessing technique that is used to modify the feature vectors. Such kind of
    modification is necessary to measure the feature vectors on a common scale. Followings are two
    types of normalization which can be used in machine learning:

    L1 Normalization

    It is also referred to as Least Absolute Deviations. This kind of normalization modifies the values
    so that the sum of the absolute values is always up to 1 in each row. It can be implemented on the
    input data with the help of the following Python code:

    # Normalize data

    data_normalized_l1 = preprocessing.normalize(input_data, norm='l1')

    print("\nL1 normalized data:\n", data_normalized_l1)

    ___________________________________________________________________________32
    Artificial Intelligence-Lab [COSC-3212]
    Provide output in following area after running the above lines of code:

    Output

    L2 Normalization

    It is also referred to as least squares. This kind of normalization modifies the values so that the
    sum of the squares is always up to 1 in each row. It can be implemented on the input data with the
    help of the following Python code:

    # Normalize data

    data_normalized_l2 = preprocessing.normalize(input_data, norm='l2')

    print("\nL2 normalized data:\n", data_normalized_l2)

    Provide output in following area after running the above lines of code:

    L2 normalized data:

    ___________________________________________________________________________33
    Artificial Intelligence-Lab [COSC-3212]
    ___________________________________________________________________________34
    Artificial Intelligence-Lab [COSC-3212]

    You might also like