Scribd Logo
Upload

Welcome to Scribd!

  • 147 views

    Regression in Data Mining

    Uploaded by

    poonam
    Regression analysis is a statistical technique used to explain and predict relationships between variables. Simple linear regression fits a straight line to the data to represent the relationship between a dependent variable and a single independent variable. Multiple linear regression extends this to use multiple independent variables. The output of regression includes coefficients, prediction equations, measures of fit like the R-squared value, and the standard error.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd

    Regression in Data Mining

    Uploaded by

    poonam
    147 views15 pages
    Regression analysis is a statistical technique used to explain and predict relationships between variables. Simple linear regression fits a straight line to the data to represent the relationship between a dependent variable and a single independent variable. Multiple linear regression extends this to use multiple independent variables. The output of regression includes coefficients, prediction equations, measures of fit like the R-squared value, and the standard error.

    Original Description:

    regression

    Copyright

    © © All Rights Reserved

    Available Formats

    PPT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Regression analysis is a statistical technique used to explain and predict relationships between variables. Simple linear regression fits a straight line to the data to represent the relationship between a dependent variable and a single independent variable. Multiple linear regression extends this to use multiple independent variables. The output of regression includes coefficients, prediction equations, measures of fit like the R-squared value, and the standard error.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    147 views15 pages

    Regression in Data Mining

    Uploaded by

    poonam
    Regression analysis is a statistical technique used to explain and predict relationships between variables. Simple linear regression fits a straight line to the data to represent the relationship between a dependent variable and a single independent variable. Multiple linear regression extends this to use multiple independent variables. The output of regression includes coefficients, prediction equations, measures of fit like the R-squared value, and the standard error.

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    of 15

    Regression Analysis

    Module 3
    Regression

    Dependent variable

    Independent variable (x)

    Regression is the attempt to explain the variation in a dependent


    variable using the variation in independent variables.
    Regression is thus an explanation of causation.
    If the independent variable(s) sufficiently explain the variation in the
    dependent variable, the model can be used for prediction.
    Simple Linear Regression

    Dependent variable (y)


    y = b0 + b1X

    B1 = slope
    = y/ x
    b0 (y intercept)

    Independent variable (x)

    The output of a regression is a function that predicts the dependent


    variable based upon values of the independent variables.

    Simple regression fits a straight line to the data.


    Simple Linear Regression

    Observation: y

    Dependent variable Prediction: y^

    Zero
    Independent variable (x)

    The function will make a prediction for each observed data point.
    ^
    The observation is denoted by y and the prediction is denoted by y.
    Simple Linear Regression

    Prediction error:

    Observation: y
    Prediction: y^

    Zero

    For each observation, the variation can be described as:

    y=^
    y+
    Actual = Explained + Error
    Regression

    Dependent variable

    Independent variable (x)


    A least squares regression selects the line with the lowest total sum
    of squared prediction errors.
    This value is called the Sum of Squares of Error, or SSE.
    Calculating SSR

    Dependent variable Population mean: y

    Independent variable (x)

    The Sum of Squares Regression (SSR) is the sum of the squared


    differences between the prediction for each observation and the
    population mean.
    Regression Formulas

    The Total Sum of Squares (SST) is equal to SSR + SSE.

    Mathematically,

    ^ 2
    SSR = ( y y ) (measure of explained variation)

    ^
    SSE = ( y y ) (measure of unexplained variation)

    2
    SST = SSR + SSE = ( y y ) (measure of total variation in y)
    The Coefficient of Determination

    The proportion of total variation (SST) that is explained by the


    regression (SSR) is known as the Coefficient of Determination, and is
    often referred to as R 2.

    2 SSR SSR
    R = =
    SST SSR + SSE

    The value of R 2 can range between 0 and 1, and the higher its value
    the more accurate the regression model is. It is often referred to as a
    percentage.
    Standard Error of Regression

    The Standard Error of a regression is a measure of its variability. It


    can be used in a similar manner to standard deviation, allowing for
    prediction intervals.

    y 2 standard errors will provide approximately 95% accuracy, and 3


    standard errors will provide a 99% confidence interval.

    Standard Error is calculated by taking the square root of the average


    prediction error.

    Standard Error =
    SSE
    n-k

    Where n is the number of observations in the sample and


    k is the total number of variables in the model
    The output of a simple regression is the coefficient and the
    constant A. The equation is then:

    y = A+ * x +

    where is the residual error.

    is the per unit change in the dependent variable for each unit
    change in the independent variable. Mathematically:

    y
    =
    x
    Multiple Linear Regression

    More than one independent variable can be used to explain variance in


    the dependent variable, as long as they are not linearly related.

    A multiple regression takes the form:

    1 1 2 2
    y = A + X + X + + k Xk +

    where k is the number of variables, or parameters.


    Multicollinearity

    Multicollinearity is a condition in which at least 2 independent


    variables are highly linearly correlated. It will often crash computers.

    Example table of
    Correlations
    Y X1 X2
    Y 1.000
    X1 0.802 1.000
    X2 0.848 0.578 1.000

    A correlations table can suggest which independent variables may be


    significant. Generally, an ind. variable that has more than a .3
    correlation with the dependent variable and less than .7 with any
    other ind. variable can be included as a possible predictor.
    Nonlinear Regression

    Nonlinear functions can also be fit as regressions. Common


    choices include Power, Logarithmic, Exponential, and Logistic,
    but any continuous function can be used.
    Regression Output in Excel

    You might also like