Scribd Logo
Upload

Welcome to Scribd!

  • 196 views

    KPI

    Uploaded by

    mehak rajdev
    This document discusses key performance indicators (KPIs) for evaluating forecasting models, including mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). It explains that RMSE gives more importance to larger errors compared to MAE. The document also discusses how MAE can result in biased forecasts for skewed demand distributions, while RMSE aims for the average and does not produce bias. It recommends experimenting with both MAE and RMSE as KPIs to determine the best approach for a given dataset.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd

    KPI

    Uploaded by

    mehak rajdev
    196 views10 pages
    This document discusses key performance indicators (KPIs) for evaluating forecasting models, including mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). It explains that RMSE gives more importance to larger errors compared to MAE. The document also discusses how MAE can result in biased forecasts for skewed demand distributions, while RMSE aims for the average and does not produce bias. It recommends experimenting with both MAE and RMSE as KPIs to determine the best approach for a given dataset.

    Original Description:

    Supply chain analytics

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses key performance indicators (KPIs) for evaluating forecasting models, including mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). It explains that RMSE gives more importance to larger errors compared to MAE. The document also discusses how MAE can result in biased forecasts for skewed demand distributions, while RMSE aims for the average and does not produce bias. It recommends experimenting with both MAE and RMSE as KPIs to determine the best approach for a given dataset.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    196 views10 pages

    KPI

    Uploaded by

    mehak rajdev
    This document discusses key performance indicators (KPIs) for evaluating forecasting models, including mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE). It explains that RMSE gives more importance to larger errors compared to MAE. The document also discusses how MAE can result in biased forecasts for skewed demand distributions, while RMSE aims for the average and does not produce bias. It recommends experimenting with both MAE and RMSE as KPIs to determine the best approach for a given dataset.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    of 10

    KPI

    Lab

    Data Science for Supply Chain Forecast by Nicholas Vandeput


    KPI’s to Consider
    • Bias (mean error)

    • MAE (mean absolute error)

    • RMSE (root mean square error)

    • MAPE (mean absolute percentage error)


    Data Science for Supply Chain Forecast by Nicholas Vandeput
    KPI
    • Compared to MAE, RMSE does not treat each error the same.
    • It gives more importance to the biggest errors.
    • That means that one big error is enough to get a very bad RMSE.
    • Let's imagine we want to compare two slightly different forecasts.

    • The only difference is the forecast on the latest demand observation: forecast #1
    undershot it by 7 units and forecast #2 by only 6 units.
    • Calculate the KPI: MAE and RMSE for both the forecasts.

    Data Science for Supply Chain Forecast by Nicholas Vandeput


    Which KPI to Choose?
    • Lets do another problem to understand this.
    • Take a product with a rather flat weekly demand that has from time to time
    a big order.

    • Lets look at 3 different forecasts for this product.


    • The first one predicts 2 pieces/day,
    • The second one 4
    • The last one 6.
    • Plot the demand we observed and these forecasts.
    • Calculate bias, MAPE, MAE, and RMSE on the historical period for all the 3
    forecasts.
    Data Science for Supply Chain Forecast by Nicholas Vandeput
    MAE or RMSE - Which one to choose?

    • Is it worse to aim for the median or the average of the demand?


    • The answer is not black and white.
    • Each technique has some benefits and some risks.
    • Only experimentation will reveal which technique works best for a
    dataset.
    • You can even choose to use both MAE and RMSE.
    • Let’s look at the impact of choosing either RMSE or MAE on the bias,
    the sensitivity to outliers, and the intermittent demand.

    Data Science for Supply Chain Forecast by Nicholas Vandeput


    Bias
    • For most of the products, the median is not the same as
    the average/mean demand.
    • The demand will have some peaks here and there
    resulting in a skewed distribution.
    • These skewed demand distributions are very common in
    supply chain
    • The peaks can be due to periodic promotions
    • Clients ordering in bulk.
    • This will cause the demand median to be below the
    average demand
    • This means that a forecast that is minimizing MAE will
    result in a bias.
    • On the other hand, the forecast that is minimizing RMSE
    will not result in a bias (as it aims for the average).
    • This is MAE's main weakness.
    Data Science for Supply Chain Forecast by Nicholas Vandeput
    Sensitivity to Outliers
    • Let’s look at another example
    • Calculate median and average for both the datasets
    • The only difference between the 2 datasets is that
    there is a sudden increase in the demand in period
    10 (may be an outlier)
    • Which KPI? (MAE or RMSE)
    • Would they be different for both the dataset?
    • Observations
    • The median is more robust to outliers than the average.
    • In a supply chain environment, this is rather important as
    we can have many outliers due to encoding mistakes or
    demand peaks (marketing, promotions, spot deals).
    Data Science for Supply Chain Forecast by Nicholas Vandeput
    Intermittent Demand
    • We sell a product to a single client. It
    is a highly profitable product and our
    unique client seems to make an order
    one week out of three without any
    kind of pattern. The client always
    orders the product by batches of 100.
    • Create a weekly forecast for this
    product
    • #Forecast 1 will be the average demand
    • #Forecast 2 will be the demand median
    • Calculate MAE and RMSE in both the
    situations

    Data Science for Supply Chain Forecast by Nicholas Vandeput


    Conclusion
    • MAE provides a protection against outliers whereas RMSE provides
    the assurance to get an unbiased forecast.
    • Which indicator should you use?
    • There is unfortunately no definitive answer.
    • As a supply chain data scientist, you should experiment:
    • if using MAE as a KPI results in a high bias, you might want to use RMSE.
    • If the dataset contains many outliers, resulting in a skewed forecast, you
    might want to use MAE.

    Data Science for Supply Chain Forecast by Nicholas Vandeput


    Problems
    • Demand problem
    • Assume we have demand data upto 80 periods
    • We use moving average of 10 periods to calculate our forecast for
    period 81
    • Once we have forecast for period 81, demand for 81 occurs
    • We then forecast for period 82, and demand for period 82 occurs and
    so on
    • Calculate MAE, RMSE, and MAPE
    • Do this with moving average of 20 periods
    Data Science for Supply Chain Forecast by Nicholas Vandeput

    You might also like