Committee of Donor Agencies for Small Enterprise Development

Performance Measurement for

Business Development Services to
Micro and Small Enterprises:
A Revised Framework and Guide to the
Preparation of Case Studies

by Mary McVay
September 1999

USAID's Microenterprise ILO's International

Best Practice Project Small Enterprise Programme
 i

TABLE OF CONTENTS

CHAPTER ONE
PURPOSE 1

CHAPTER TWO
OVERVIEW: FOCUS ON BDS MARKETS 3

CHAPTER THREE
BDS MARKET DEVELOPMENT INDICATORS 5
EXPANDING THE MARKET FOR BUSINESS DEVELOPMENT SERVICES .................................. 6
Market Size (demand).......................................................................................... 6
Market Size (supply)............................................................................................ 7
Market Penetration .............................................................................................. 8
DEVELOPING A HIGH QUALITY, DIVERSE, COMPETITIVE MARKET .................................... 8
Number of BDS Suppliers ................................................................................... 8
Number of BDS Products .................................................................................... 9
Average Price; Well-Distributed, Wide Price Range for BDS Services ................ 9
Number and Proportion of Multiple-User Customers ......................................... 10
Market Distortion .............................................................................................. 11
DEEPENING THE MARKET: REACHING UNDERSERVED GROUPS ....................................... 12
Extent of Underserved Market Reached............................................................. 12
Penetration of Underserved Markets .................................................................. 13

CHAPTER FOUR
BDS SUPPLIER INDICATORS 15
COST-RECOVERY OF BDS SUPPLIERS ............................................................................. 16
COST-EFFECTIVENESS OF A BDS PROGRAM .................................................................... 17
Cost-Benefit Assessment and Return on Investment .......................................... 17
Cost Per Supplier, Customer, and $1 Increase in Supplier Revenue.................... 18

CHAPTER FIVE
BDS CUSTOMER INDICATORS 21
ACQUISITION: CUSTOMER PURCHASE INFORMATION....................................................... 22
Customer Satisfaction........................................................................................ 22
Repeat Purchases ............................................................................................... 22
USE: BUSINESS PRACTICES............................................................................................. 23
BENEFITS: BUSINESS PERFORMANCE .............................................................................. 24
 1

CHAPTER ONE
PURPOSE

The purpose of this guideline is to provide instructions for assessing the performance of
business development services (BDS) programs targeted to micro, small and medium-scale
enterprises (SMEs) and for testing this recently completed framework for assessing BDS
program performance. The instructions provided in the following chapters assume that a case
study will be prepared over the course of several months, using this guideline in conjunction
with a broader guide of general research questions to address for case studies being presented
at the Donor’s Committee Conference in Hanoi, April 2000. The instructions contain a
structured format for thinking about program performance, indicators, and methodologies for
assessing performance, and research questions that the case studies should address on the
practicality and validity of the proposed indicators. In addition to contributing to learning
about best practices of BDS programs, these case studies will be part of a broader process of
developing a common performance measurement system.

It should be noted that this performance measurement guideline does not, in its current form,
contain benchmarks or performance standards. Benchmarks for the performance indicators
contained in the framework may evolve once the indicators have been sufficiently field
tested. However, these benchmarks likely will vary significantly according to each program’s
type of service, target population served, economic environment, and maturity.

Chapter One—Purpose
2
 3

CHAPTER TWO
OVERVIEW: FOCUS ON BDS MARKETS

The BDS field is undergoing a paradigmatic shift from a focus on the supply of subsidized
services to a limited population, to a focus on the development of a vibrant BDS market for
large numbers of SMEs. For many BDS suppliers that serve low-income populations or offer
services for which it is challenging to recover costs, the vision of a competitive private sector
market for services may seem far-fetched. However, more BDS suppliers are starting to
become financially sustainable, some programs are making progress in stimulating private
sector markets, and many BDS developers feel this is the direction in which the field needs to
move.

Based on this paradigmatic shift, the Committee of Donor Agencies for Small Enterprise
Development’s Virtual Conference on Performance Measurement, sponsored by the
International Labour Organization and USAID, recommended an overarching performance
measurement framework that was based on an analysis of the market for business
development services. Thus, in the framework, performance goals and indicators are
categorized according to whether they:

§ Assess the overall BDS market;

§ Assess BDS suppliers; or

§ Assess BDS customers, SMEs.

At the same time, to accommodate more traditional ways of thinking about performance,
each category is matched with an overall goal that BDS programs are typically trying to
achieve:

§ Outreach, meaning both the number of SMEs reached (scale) and the effort to provide
services to people not served by existing markets (access);

§ Sustainability of BDS provider institutions and the cost-effectiveness of program

activities; and

§ Impact on SMEs.

Within these three broad categories, the framework proposes a goal that a BDS program
might be trying to achieve, as well as the indicators that would measure success in reaching
that goal. The following chapters detail the methodologies to be used in gathering and
analyzing data, specify who typically would gather and be interested in the data, and pose
research questions on the practicality and validity of these performance indicators. Table 1
summarizes the framework.

Chapter Two—Overview: Focus on BDS Markets

 4

Table 1: BDS Performance Measurement Framework

Goal 1: Increase Outreach (scale and access)

BDS Market Development Indicators
Objective Indicators
(Report for the overall market and for the BDS program’s contribution to the market)
Market size: Number of SMEs purchasing services (or procuring through other
Expand the market for commercial transactions)*
business development Market size: Amount of sales by BDS providers*
services Market penetration: % of potential SME market reached with a business
development service
Number of BDS providers*
Number of BDS products
Develop a high- Well distributed, wide price range for business development services
quality, diverse, Average price for a unit of business development services
competitive market Number and proportion of multiple-user customers in the market (for provider, see
Goal 3)
Market distortion: Average subsidy content of a business development service*
Extent of access: Number and % of SME customers purchasing a business
Increase access
development service who represent targeted populations (women,
underserved groups to
microenterprises, exporters, etc.)*
business development
Target market penetration: % of potential SME targeted markets reached (women,
services
microenterprises, exporters, etc.)
Goal 2: Sustainability and Cost-Effectiveness
Assessing BDS Suppliers
Objective Indicators
Achieve supplier BDS supplier cost recovery of operational costs from client fees*
sustainability
Improve program cost- Simplified cost-benefit assessment comparing total program costs to aggregate
effectiveness program benefits for entrepreneurs*
Total program cost per customer served*
Total program cost per supplier assisted*
Total program cost per increase in provider revenue
Goal 3: Impact
Assessing BDS Customers, SMEs
Objective Indicators
Increase customer Customer satisfaction with a business development service*
acquisition of business Repeat customers (percent of customers who buy more than once)*
development services
Increase customer use Percent of customers who improve business practices, as defined by the provider*
of business
development services
Increase customer Change in value added (sales - raw materials)*
benefits from business
development services
* These indicators are in use in practice in a fair number of BDS programs. Note that the indicators under “market
development” with an asterisk are mainly used to assess the outreach of particular programs but are not widely used to
assess the BDS market in general.
 5

CHAPTER THREE
BDS MARKET DEVELOPMENT INDICATORS

The overarching goal of developing BDS markets is to develop vibrant, competitive, and
(primarily) private sector markets of relevant, differentiated services consumed by a broad
range and significant proportion of small businesses. This might be achieved by (1)
expanding the market for business development services; (2) developing a high quality,
diverse, and competitive BDS market; and (3) deepening the market to reach populations
with limited access to business development services. The indicators that will measure these
goals are useful for BDS suppliers to help them understand their market position and for
BDS program managers and donors to help them understand the influence of BDS program
activities on the development of BDS markets as a whole. BDS program managers or donors
will most likely collect data on wider markets, while individual providers will supply data on
their outreach.

Although developing the market for business development services is not an explicit goal of
many BDS programs, this category of objectives and indicators is still relevant for all
programs. Why? All programs affect BDS markets, whether they intend to or not, because
BDS programs add to the supply and satisfy a demand for services. They may expand the
market by offering new services to new target populations, or they may even distort the
market with subsidies and crowd out private sector providers. Thus, where possible, the
indicators listed below should be gathered for all programs. Data on the overall market and
on specific program activities should be collected so that the contribution of a specific BDS
program to changes in the market can be understood. (If the particular BDS program does not
collect market information and such collection is not feasible within the timeframe and
resources of the case study, the program should at least report information on its program.)

Table 2: Example of Goal 1 (Business Training)

Goal 1: Increase Outreach (scale and access)

BDS Market Development Indicators
Market Program
Objective Indicators
1997 1999 1997 1999
Market size: Number of SMEs purchasing N/A N/A 100 500
Expand the Market size (supply): Amount of annual sales* N/A N/A 0 $40,000
market for BDS Market penetration: % of potential SME market
N/A N/A N/A N/A
reached with business development services
Number of BDS providers* 10 15 1 2
Number of BDS products N/A 27 1 3
Develop a high- Average price per unit of business development
N/A N/A 0 $40
quality, diverse, services
competitive Well distributed, wide price range N/A N/A 0 $15-$60
market 100,
Number and proportion of multiple users N/A N/A N/A
20%
Market distortion: Average subsidy content N/A N/A 100% 50%

Chapter Three—BDS Market Development Indicators

 6

Goal 1: Increase Outreach (scale and access)

BDS Market Development Indicators
Market Program
Objective Indicators
1997 1999 1997 1999
10% 35%
Deepen the Number and % of SME customers purchasing
women women
market: reach business development services who represent N/A N/A
25% 50%
underserved targeted populations
micros micros
groups
% of potential SME target market reached N/A N/A N/A N/A

EXPANDING THE MARKET FOR BUSINESS DEVELOPMENT SERVICES

Market Size (demand)

Indicator: The number of SMEs purchasing services (or procuring through other commercial
transactions).

At the market level, one indicator is the total number of SMEs purchasing a business
development service in a market defined by the program in a given year. At the program
level, it is the number of SMEs purchasing a business development service from a provider
assisted directly by the program cumulatively and in a given year. The annual figure is used
to assess the program’s contribution to the market, and the cumulative figure to give a sense
of the scale of the program over time. In either case, a program may count the number
procuring a business development service through other commercial transactions such as an
SME supplying a product to a larger business and receiving technical assistance from the
buyer to develop the correct product. However, SMEs receiving a business development
service free or being paid to receive a business development service are not counted.

Methodology: To assess the BDS market, the BDS program needs to define the market. This
definition should contain:

§ A definition of the business development service. This should be broader than the
specific product offered by the program (i.e., CEFE training services) but narrow enough
to represent a reasonable group of similar services, such as general management training.

§ A geographic boundary, such as a town, district, or country.

§ An enterprise definition: what size enterprise, what sector, or what other characteristics
(i.e., women owned) define the market?

At the market level, researchers are free at this stage to use any methodology available to
develop this indicator, but they should document the methodology used. Because the purpose
is to assess changes in the market, the indicator requires two figures, one for last year and
one for this year. At this stage, it is recommended to report the numbers, the numeric change
over one year, and the percent change over one year.
 7

Suppliers simply report the number of SMEs procuring services last year and cumulatively
over the life of the project. The only challenges are:

§ Report only SMEs, as defined by the Committee of Donor Agencies for Small Enterprise
Development.

§ Report enterprises served, not number of purchases. Number of purchases will include
repeat customers or may include employees in addition to owners.

§ Report providers that the program directly assists.

To collect these data, the program also must clearly define:

§ The providers that are directly assisted, with what “assistance”; and

§ The definition of SME being used by the program.

Research Questions:

§ Which methodologies are in use to assess the size of the BDS market?

§ Does assessing the market over the last year capture the impact of the program, or is a
different timeframe more relevant?

§ How easy is it for suppliers to identify customers as SMEs?

§ How easy is it for suppliers to count customers, rather than transactions?

Market Size (supply)

Indicator: Amount of sales by BDS providers.

At the market level, the indicator is the total annual sales, in U.S. dollars, to SMEs of all the
BDS providers offering the particular business development service. At the program level,
the indicator is the total sales, in U.S. dollars, to SMEs, of all assisted BDS suppliers. This
should be collected for the last year and cumulatively.

Methodology: At the market level, any methodology may be used, provided the researcher
describes it in the case study. As with customers served, the indicator is assessing only
revenue from sales to SMEs. Currencies should be converted at the exchange rate for the
relevant year. Sales for 1997 should use the 1997 exchange rate, and sales for 1998 should
use the 1998 exchange rate. No adjustments for inflation should be made.

Assisted suppliers simply report last year’s sales, in U.S. dollars, using last year’s exchange
rate, this year’s sales using this year’s exchange rate, and the cumulative sales since the

Chapter Three—BDS Market Development Indicators

 8

beginning of the program. If available and relevant, annual sales before and after receiving
assistance from the program would be useful. The researcher should report the sum of these
sales for all assisted providers, and, if relevant, the change in sales before and after assistance
in U.S. dollars and as a percent growth rate.

Research Questions: Same as above.

Market Penetration

Indicator: Percent of potential SME market reached with a business development service.

At the market level, this is the percent of the potential SME customers that are receiving
business development services, as defined by the program. At the program level, this is the
portion of that market served by assisted suppliers. The framework asks for the change in
market “penetration” or market “share” over time. At the market level, collect last year’s and
this years’ market penetration. For the program, collect market penetration data before and
after assistance, if relevant and possible.

Methodology: Any methodology may be used to estimate the number of SMEs that might
procure a service. To do this, the researcher will have to estimate the number of potential
customers in the market. Any methodology may be used as long as the case study documents
how it was determined that the service was in “demand.” The methods must not be subject to
significant manipulation. For example, a program might define a market narrowly so that it
seems as if a high portion of the market is being served. Once a market is quantified, then
calculate the portion of the market that is served by any supplier and the portion served by
assisted providers, using the number of customers served as described above.

Research Question: What practical and valid methods can be used to define a target market?

DEVELOPING A HIGH-QUALITY, DIVERSE, COMPETITIVE MARKET

Number of BDS Suppliers

Indicator: The number of organizations—private sector, government, or nongovernmental—

that provide a specific type of business development service, as defined by the program, to
SMEs.

At the market level, this is the total number of BDS suppliers in the market, as defined by the
program. At the program level, this is the number of suppliers being directly assisted by the
program. A “supplier,” in this context, is an organization that provides a business
development service directly to SMEs.
 9

Methodology: Any methodology may be used at the market level, and the program should
find it easy to count these institutions. Researchers should keep in mind that the definition
includes formal and non-formal suppliers, both those assisted with development or
government funds and those operating in the private sector. The data should be compared
over time. At the market level, report the number of suppliers a year ago and the current
number. At the program level, report the number at the start of the program and the current
number. Indicate if the program was responsible for founding any of the institutions.

Research Questions:

§ What methods can be used to identify the full array of BDS suppliers?

§ Is the number of suppliers a significant factor in achieving high outreach?

§ Is a high number always a positive outcome, or are there markets in which there are too
many suppliers?

Number of BDS Products

Indicator: Within the category of business development services defined by the program and
by the assisted suppliers, the number of different specific products and services offered to
SMEs in the market.

Methodology: The program is free to define these services in the way most relevant for
capturing the level of diversity of services available to SMEs.

Research Questions:

§ How easy is it to define a broad category of business development services and then
define the number of different service types within this category?

§ Is there enough commonality in the way this is done to produce a comparable indicator?

§ Will common broad categories of BDS emerge?

§ Is a high number for an individual program a good thing?

Average Price; Well-Distributed, Wide Price Range for BDS Services

Indicator:

§ At the market level, the average unsubsidized price for the business development service
in the market. This means the price of a business development service sold to SMEs by
financially sustainable or unassisted BDS suppliers. The rational is that in a competitive

Chapter Three—BDS Market Development Indicators

 10

market, the average price for services will decline, but if the average price includes
subsidized services, then lower prices may simply indicate higher subsidies. At the
program level, assisted suppliers should simply report the average price paid for a unit of
service by SMEs, whether subsidized or not. This will allow comparison between the
price charged by the program and the average unsubsidized price in the market.

§ A well-distributed, wide price range for business development services. In a well-

differentiated market, there will be a wide range of services in a wide range of price
categories.

Methodology: Any methodology may be used to report this price information as long as data
are reported on unsubsidized services. Indicate how this was determined and what
methodology was used. Similarly, any methodology may be used to collect data on the
spread of prices in the market. If possible, report these data for this year and last year. For the
program, the prices should be for time periods before and after the BDS program assisted the
provider.

Research Questions:

§ What methodology can be used to collect and report relevant private sector pricing
information?

§ How can we be sure that the information is indicating a more competitive, varied market?

Number and Proportion of Multiple-User Customers

Indicator: The number and portion of SMEs in the market that use business development
services more than once. This is an indicator of the quality of business development services
in the market, although it is understood that not all business development services are
designed to be purchased more than once. At the market level, it is the number of SMEs that
purchase more than once from any supplier and the portion of these SMEs with respect to the
total number procuring the business development service. Provider level data are collected
under “impact.”

Methodology: Any methodology may be used for these indicators. Please explain how data
were collected and how the indicator was calculated. Note if the particular type of business
development services is not typically purchased more than once, and why. Collect data for
the indicator for last year and this year, if possible.

Research Questions:

§ What is the most practical and valid way to assess multiple use?

§ Is this a good indicator of the quality of service in the market?

 11

§ Is it relevant for most service types?

Market Distortion

Indicator: The average subsidy content of a unit of a business development service provided
to an SME.

This is equivalent to the direct and indirect operational costs of delivering a unit of service,
less the price paid by the SME, divided by the total cost: (cost - price)/cost.

For example, if the program costs of providing a training course to one entrepreneur were
$200, and the entrepreneur paid $50, the subsidy content would be 75 percent.

At the market level, it is the average subsidy content of all business development services in
the market. At the program level, it is the average subsidy content of business development
services provided to SMEs by assisted providers.

Methodology: Any method may be used to collect the information on the market in general.
If possible, collect last year’s and this year’s data. At the program level, the main challenge is
to use consistent definitions of costs. At this point, the “cost” is defined as:

§ Supplier institution costs, not costs of organizations that support suppliers;

§ Direct and indirect operational costs—that is, the incremental cost of providing a unit of
service, plus ongoing overhead costs of managing the institution that can be allocated to
the provision of that service; and

§ Not start-up, research and development, or monitoring and evaluation costs.

An extremely valuable case study would present these “subsidy content” data in a variety of
ways so that we can learn from the variations: one as noted above, one including ongoing
costs incurred by external institution supporting the program, one including start-up costs,
and so on. If relevant and possible, collect the data for the baseline year (pre-assistance) and
the latest year.

Research Questions:

§ Is it possible to assess the subsidy content of services in the wider market?

§ How easy is it to collect comparable “subsidy content” data?

§ Is it easier for some types of institution than for others?

§ Is this a good measure for “market distortion,” or is it better used as an indicator of

provider sustainability?

Chapter Three—BDS Market Development Indicators

 12

DEEPENING THE MARKET: REACHING UNDERSERVED GROUPS

All programs under consideration serve SMEs, but within this category, many programs
target particular populations that have limited access to business development services in the
private sector. In this section, BDS programs are free to identify particular populations they
may be targeting and to report on their success in reaching these populations. In addition, the
field has a general interest in tracking the performance of programs in reaching two typically
underserved groups so that the field may learn best practices in reaching them: women (or
men in rare cases where men are particularly underserved) and microenterprises, which is a
proxy indicator for reaching lower income SMEs. If all of the SMEs in a program are women
and microenterprises, and these are the main characteristics of the underserved population
being targeted, then this section is redundant, since the information provided about the target
market and SMEs served in the previous section will be the same as the information provided
here. This section is relevant for all other programs, whether the program was designed to
serve these populations or not.

Extent of Underserved Market Reached

Indicator: The number and percent of SME customers purchasing a business development
service who represent underserved populations (such as women, microenterprises, and
exporters).

Methodology: To measure this indicator, the program must first define the underserved target
population it is trying to reach, within the broad definition of SMEs. In some cases, programs
may not be making a proactive effort to reach a more specific group, other than SMEs. In this
situation, researchers should at least report on the number of women-owned enterprises
served (or explain why men are the underserved population within the context of the
program) and the number of microenterprises served.

A “women-owned business” is an enterprise of which a woman owns 50 percent or more. If a

woman is trained as an employee or family member of a business owner, she may be
counted, but her status noted. A “microenterprise” is an enterprise employing 10 or fewer
employees, with “employees” referring broadly to full- or part-time, regular or seasonal, and
paid or unpaid family workers.

Any method may be used to gather this information for the market in general, including
going outside the above definitions if necessary. Programs should clearly define the target
group they are concerned with, in addition to women and microenterprises, and report the
number and portion of SMEs purchasing services that fall into each group. This should be
reported cumulatively. Please document the strategy used to collect these data.
 13

Research Questions:

§ How easy is it to disaggregate data on the number of SMEs reached by gender, type of
enterprise, or other targeting information at the market and program levels?

§ What are the most practical and valid strategies?

Penetration of Underserved Markets

Indicator: The percent of the potential underserved SME market reached.

Earlier, the potential market for SMEs in general was assessed. In this section, we assess the
potential market within a specific target population (women, microenterprises, or other
groups). We also look at the extent to which that market has been penetrated in general and
by assisted suppliers.

Methodology: Any methodology may be used to disaggregate the number of potential

customers by gender, business type, or other category, and then calculate the portion served
by the entire market (from above) and the portion served by assisted suppliers (from above).
If relevant and possible, report on penetration before and after assistance to providers.

Research Question: Is it feasible to disaggregate the potential market by gender and business
type?

Chapter Three—BDS Market Development Indicators

14
 15

CHAPTER FOUR
BDS SUPPLIER INDICATORS

The indicators described in Chapter Three focus on the broad market for business
development services and the contribution of a specific BDS program to that market. In
contrast, the indicators described in this chapter assess the sustainability and cost-
effectiveness of organizations involved in the supply of business development services to
SMEs. When it comes to assessing the suppliers of business development services, the two
primary goals usually are sustainability and cost-effectiveness. Although these goals are easy
to measure in a microfinance institution, for business development services, the question is,
“Sustainability and cost-effectiveness of what?” The provider institution? The service? Direct
services supply? Up-front research and start-up costs?

BDS programs contain diverse institutional structures that can make performance
measurement difficult. In a few cases, they resemble microfinance programs in that
significant start-up costs are invested in a focused institution that plans on becoming
financially sustainable by selling services to SMEs. Often, however, business development
services are delivered by nongovernmental organizations (NGOs) with a wide array of
services or by international nongovernmental or U.N. organizations. These organizations do
not expect to recover costs either because cost recovery has not been part of their mission or
because their costs are so high that cost recovery is impossible. Increasingly, another BDS
model is emerging in which NGOs or government agencies hope to “facilitate” the
development of numerous BDS suppliers, whether NGO or private sector, that will be
financially sustainable. These different models have emerged in business development
services for several reasons, some appropriate and some not. One reason is the challenge of
implementing a major principle of business development services—services need to be
tailor-made to fit the needs of particular sectors and types of enterprises. This requires
significant up-front market research, service development, and monitoring costs. Another
reason is that business development services often require up-front payment, whereas with
finance, a business gets the funding first and pays later. This makes it challenging for the
poor to pay for services. Unfortunately, many programs provide expensive services that are
not in high demand by SMEs. At the same time, other organizations are supplying relevant
services to the poor, who cannot pay up-front for the service. The challenge for a
performance measurement system is to differentiate among funds well spent on research and
development to establish a sustainable supply of business development services to SMEs,
programs legitimately serving low-income SMEs, and programs subsidizing low-quality
services.

This framework, built on the performance measurement framework developed by the

Microenterprise Best Practices Project and the Committee of Donor Agencies for Small
Enterprise Development’s Virtual Conference, suggests assessing supplier performance at
two levels:

Chapter Four—BDS Supplier Indicators

 16

1) Sustainability of a BDS supplier, or of activities involved in direct service supply. The

performance measurement system would look only at how much it costs to provide
services to SMEs and what portion of those costs are recovered from client fees.

2) Cost-effectiveness of a BDS program. The performance measurement system would

calculate all cumulative program costs and compare them to benefits realized by SMEs as
a result, and/or simple cost per output ratios such as cost per SME served.

In a regular performance measurement system, BDS suppliers, program managers, and

donors will all be interested in supplier sustainability. Suppliers would likely collect these
data. Cost-effectiveness concerns mainly program managers and donors, and they would
calculate the cost-effectiveness indicators, although providers would be involved in data
collection.

Table 3: Example of Goal 2 (Business Training)

Goal 2: Sustainability and Cost-Effectiveness

Assessing BDS Suppliers
Objective Indicator 1997 1999
Achieve supplier BDS supplier cost recovery of operational
0 50%
sustainability costs from client fees
Simplified cost-benefit assessment comparing 1:4 cost:
total program costs to aggregate program benefit
N/A
benefits for entrepreneurs; return on 250%
investment (ROI) ROI
Improve program
Total program costs N/A $200,000
cost-effectiveness
Total program cost per supplier assisted N/A $100,000
Total program cost per customer served N/A $400
Total program cost per $1 increase in provider
N/A $5
revenue

COST RECOVERY OF BDS SUPPLIERS

Indicator: The percent of operational BDS supplier costs, both direct and indirect, that are
covered by the revenues from the sale of those services to SMEs.

It is recognized that it is not always feasible or appropriate for a BDS provider to be

financially sustainable, depending on the operating environment, the target population, the
service provided, and the maturity of the institution. At the same time, it is recognized that
the providers’ becoming as financially sustainable as possible will encourage them to
respond to customer demand and reduce costs. Any benchmark for cost recovery, then, needs
to be established with these criteria in mind.

For programs working with private sector suppliers, which are presumed to be profitable and
are not receiving cash grants from the BDS program, data should be reported on the
profitability of these companies, if possible. If not possible, simply note that the providers are
private sector companies.
 17

Methodology: On the revenue side, calculate the revenue earned, over the last year, from
sales of business development services to SMEs. Do not include revenue earned from other
sources. If a researcher wishes to document that income and highlight sustainability
strategies that involve cross-subsidies, this would be interesting, but at least one figure
should include only revenue from SME fees.

In cases where institutions are wholly focused on providing business development services
directly to SMEs, calculate the entire institution’s annual costs. In cases where there is a
large development program in the organization, calculate the annual costs associated with the
BDS program. However, these should include direct costs of service provision and indirect
costs, such as organizational overhead costs in-country and abroad. These costs should not
include those that would be considered research and demonstration, fundraising, monitoring
and evaluation, or other start-up costs. Only ongoing operational and management costs
should be included. (If the researcher prefers to break these costs down into direct and
indirect costs, this would be interesting, but one overall figure should include both.)

Calculate the percent of the costs that are recovered by fees: fee revenue/costs. If the BDS
supplier existed before the BDS “program,” the researcher should collect baseline
information on the cost-recovery rate prior to the program, and the previous year’s.

Research Questions:

§ How easy is it for BDS programs to allocate costs and revenues to specific services,
programs, or SME clients?

§ Is cost recovery a good proxy indicator for impact?

COST-EFFECTIVENESS OF A BDS PROGRAM

Cost-Benefit Assessment and Return on Investment

Indicator: The ratio of total program costs to total financial benefits realize by SMEs; the
percent represented by this ratio, which is referred to as the program’s return on investment
(ROI).

A cost-benefit analysis assesses total program costs—start-up and ongoing, direct, and
indirect—and compares them to the benefits realized by SMEs receiving services. It is an
indicator that, in theory, captures all aspects of a program and is comparable not only across
BDS programs but across a wide array of public and private investments. There are many
complex methodologies for calculating it. These often pose barriers for organizations wishing
to use this indicator. This framework proposes a simple, basic cost-benefit analysis that may
or may not be as powerful as hoped because of compromises in validity.

Chapter Four—BDS Supplier Indicators

 18

Methodology:

Benefits. These are described in more detail below. In sum, the methodology calls for an
assessment of the aggregate net increase in value added experienced by SMEs purchasing a
business development service, attributable to the business development service. “Value
added” refers to raw materials subtracted from sales. The “benefits” are the sum of the
increase in value added experienced by SME clients, attributable to the program. The
assessment of value added should take place within two years of assistance—preferably less.
It should not be extrapolated to estimate value added that might take place in the future.
Local currency values should be converted to U.S. dollars using the rate for the year in which
the data are relevant. No inflation adjustment should be used.

Costs. These are the cumulative, start-up, ongoing, and direct and indirect program costs,
converted to U.S. dollars at the exchange rate for the relevant year, not adjusted for inflation.

Cost-Benefit Ratio. The ratio of costs to benefits. Cost/benefits, expressed as 1:x.

Return on Investment. That ratio, expressed as a percentage—i.e., x percent of costs were

“returned” in social benefits to entrepreneurs.

Research Questions:

§ How easy is it to apply this type of cost-benefit analysis?

§ How standard are cost-allocation patterns?

§ Does this indicator, in fact, help assess total program activities?

Cost Per Supplier, Customer, and $1 Increase in Supplier Revenue

These indicators help researchers assess the cost-effectiveness of BDS programs even when
impact data for SMEs are not available.

Indicators:

§ Total program costs per SME purchasing a service.

§ Total program costs per supplier assisted.

§ Total program costs per $1 supplier increase in revenue from SMEs.

Methodology: Program costs are as described above: cumulative, start-up and ongoing, and
direct and indirect costs. The number of SMEs purchasing a service is pulled from the market
development indicators described in Chapter Three, as are the number of suppliers assisted
and total revenue from sales to SMEs. Once these numbers are in hand, calculate the total
 19

costs per indicator and express it in U.S. dollars. If the researcher would like to calculate the
ratios for different cost levels (direct, last year’s, etc.), this would be interesting.

Research Question: Do these indicators reveal enough about a program to merit their
inclusion in the framework?

Chapter Four—BDS Supplier Indicators

20
 21

CHAPTER FIVE
BDS CUSTOMER INDICATORS

The ultimate aim of any BDS program is to help SMEs become stable or grow so that they
can improve the livelihoods of families, generate employment, and contribute to economic
growth. Most BDS programs find it too challenging to assess changes in household
livelihoods and economic growth that might be attributable to BDS programs. In addition,
this information provides them with little that might help them provide better services. As a
result, this framework, based on recommendations from BDS program managers around the
world, focuses on an assessment that looks at the attribution, use, and benefits of business
development services. This approach provides significant feedback to BDS suppliers to help
them improve and sell services and gives them some information about the financial impact
of business development services on SMEs. The link between the financial impact on
enterprises and the ultimate impact on families and the economy is assumed to be strong, but
occasional evaluations could assess the extent of the connection for different types of
businesses in different contexts.

The approach of assessing acquisition, use, and benefits is a market-based approach. The
indicators for assessing customer acquisition are customer satisfaction and repeat customers.
These indicators help providers assess their performance in meeting the demand for services.
Regarding “use,” the framework assesses the percentage of customers who change their
business practices, as defined by the BDS provider. This information helps providers and
supporters understand the extent to which the business development service is having the
intended effect on the business, and it can help a BDS provider redesign services to better
meet customer wants and needs. “Benefits” are assessed in terms of “value added” (sales -
raw materials). This helps the researcher assess the program’s overall performance in adding
financial value to businesses and helps the provider assess whether the price of a service is
realistic. It may also provide advertising material for the provider to communicate to
potential buyers the magnitude of the financial benefits from its services.

Table 4: Example of Goal 3 (Business Management Training)

Goal 3: Impact
Assessing BDS Customers, SMEs
Objective Indicators 1997 1999
Increase customer Customer satisfaction
80% “expectations
acquisition of business with a business N/A
exceeded”
development services development service
(increase sales) Repeat customers N/A 25%
Percentage of
Increase customer use
customers who reduced
of business N/A 85%
costs, found new
development services
markets
Increase customer $1,600 per enterprise
benefits from business Change in value added N/A $800,000 total
development services 100% average increase

Chapter Five—BDS Customer Indicators

 22

ACQUISITION: CUSTOMER PURCHASE INFORMATION

Customer Satisfaction

Indicator: Percentage of customers reporting “exceeded expectations” in a customer survey.

Methodology: Gathering data on any of the customer indicators will require conducting a
random sample survey of SME customers. Any valid random sampling method may be used
and should be documented in the case study. SME customers should be surveyed at a time
after receiving services that will enable the survey to capture the benefits of the services, but
no more than two years after they received a service. If a program already uses a customer
satisfaction survey, then the results of this survey may be reported. The percentage of
responses in the highest satisfaction category should be reported. If a researcher is going to
conduct a new customer survey, the common question might be: “Did the services meet your
expectations?”

The responses should fall into four groups:

§ Exceeded expectations
§ Met expectations
§ Met some expectations
§ Was disappointing

Researchers should feel free to adapt the language to make it appropriate to the local culture,
but the highest level should be something indicating more than just “satisfied,” and the next
level down should be a positive response such as “satisfied.” Because customers tend to be
polite, it is a good idea to give both a “polite” option and an option that expresses high
satisfaction.

The figure reported should be the percentage of respondents who report the highest level of
satisfaction. If the researcher would like, he or she can also report the average response (i.e.,
3.2). If possible and relevant, baseline (pre-program) and most recent customer satisfaction
data should be reported.

Research Questions:

§ Is this a valid indicator, especially in different cultural contexts?

§ Will all programs report a 100 percent satisfaction rate?

Repeat Purchases

Indicator: Percentage of customers who have purchased a service more than once through
the same program.
 23

The intent of this indicator is to measure customer satisfaction by assessing whether

customers like a provider enough to use it again. This indicator is relevant only for services
that are supplied several times. For example, although an SME might not take the same
course twice, if a provider offers more than one course or a program offers vouchers for
different providers, the SME might take several different courses. If the service is not a
service that SMEs would purchase more than once, please explain why.

Methodology: Count the number of service units sold to SMEs during the program. Compare
this number to the number of SMEs served. The difference is the number of repeat
customers. Calculate this as a percentage of the number of customers served—the number of
repeat customers divided by the number of customers. Or, in the random sample of
customers, ask customers whether they purchased a service associated with the program
more than once. Calculate this number as a percentage of total survey respondents.

Research Questions:

§ Is the term “repeat customers” relevant for a sufficient number of services?

§ How easy is it to gather these data?

§ Is a repeat purchase ever a bad thing in practice?

USE: BUSINESS PRACTICES

Indicator: Percentage of SMEs purchasing a service that changed business practices as

intended and defined by the BDS program.

Every business development service is intended to help an entrepreneur change a business

practice that will ultimately increase enterprise income. For example, a technology access
program might help reduce costs or improve quality, accounting training helps staff manage
cash flow and increase profitability, and a marketing company helps increase sales. This
indicator assesses the portion of assisted SMEs that change their business practices as the
BDS program intended. Each program will define the change in business practices according
to the specific service or services offered. This is the one indicator in the framework that is
service specific.

Methodology: If a program does not already collect these data, the information should be
collected in a random sample survey along with the other customer data. The program needs
to define the specific, intended effects of the business development service on the business.
The effect may be simple—for example, an SME purchasing an oil press might start a
business processing cooking oil. Or there may be several types of effects. For example, a
product development workshop might be set up to help an entrepreneur design a new product
and then sell that product to a new market. This program may wish to track both types of
change in business practice. Once a program has clearly defined the intended change in

Chapter Five—BDS Customer Indicators

 24

business practices, some work may be needed to ask appropriately whether the business
practice is occurring and whether it is occurring as a result of the BDS service. In general,
entrepreneurs will want to tell a surveyor that the BDS helped them. Thus, it is best if a
verifiable indicator can be identified. For example, was the oil press operating at the time of
the visit? Were there samples of the new product available? Were there receipts or pictures of
a new product that had been sold? Also, if possible, it is best to conduct a baseline survey
before a BDS provider receives services to see whether the business practices were in place.
Finally, an element of the survey should ask, “Did you make this change as a result of the
business development service that you procured.” Any methodology may be used to ask this
question. These are not a requirement for the case studies because they might not be
practical. They are simply recommendations. Once the survey has been conducted, calculate
the percentage of respondents that changed business practices for each business practice.
Then create a table that lists, for each business practice, the percentage of SMEs served
which changed that practice.

Research Questions:

§ Is it possible to collect objective data on whether SME business practices changed?

§ Is it possible to get objective data on whether the change resulted from the BDS?

BENEFITS: BUSINESS PERFORMANCE

Indicator: The average net change of the value added of SMEs purchasing a business
development service in two years or less since receiving assistance; the aggregate net change
of all the SMEs purchasing a service through the program.

Although it is quite challenging to assess the financial performance of SMEs, improving

financial performance of a business is a principal aim of BDS programs. Rather than asking
researchers to collect a wide range of data, this framework focuses on one financial indicator
in the hopes that it will be more practical to collect one than many. Value added (sales - raw
materials) is a good proxy indicator for profits, and it represents an enterprise’s contribution
to gross domestic product. If this is too challenging, for the purpose of these case studies,
sales data will suffice.

Methodology: If the BDS program already collects financial data on assisted SMEs, then the
case study may use these data as long as the time lapse between when an SME received
services and the assessment is less than two years. Calculate:

§ Average net change, in U.S. dollars, in annual SME value added that is attributable to the
program;

§ Average percent change in annual SME value added that is attributable to the program;
and
 25

§ Aggregate (sum) net change in value added for all SME customers. If the above data
were collected in a sample survey, the total figure will have to be extrapolated based on
the number of SMEs served by the program.

As with other indicators, the U.S. dollar values should be calculated using the exchange rate
for the relevant year, with no inflation adjustment. The method for collecting data,
determining attribution, and calculating the above figures should be documented.

If a new survey needs to be performed, researchers are free to use any methodology as long
as they describe it, and it is statistically valid. If there is no baseline, the survey will need to
ask SMEs for the financial data both before and after they began receiving the business
development service.

A participant of the virtual conference had the following suggestion for a methodology:

Value added can be measured by estimating annual sales of the respondent and
then estimating the percentage of raw material content in the sales. Annual sales
can be determined by asking for last month’s sales and then asking for an
evaluation of whether this was a very good, good, average, or poor month, then
filling in the estimated value for the rest of the months in the year. The percentage
of raw materials can be determined by taking the three most common products of
the firm and asking for the raw material costs in each product. Most small firms
know this information. The average percentage of raw materials to sales price for
the product is a close approximation of the percentage, which can then be
multiplied by the annual sales in order to get the value added.

For assigning attribution, he suggested asking entrepreneurs to allocate what portion of their
change in value added was caused by various factors, such as the economy, policies, changes
in customer demand or market competition, services received by the BDS program, other
help received by the business, and entrepreneurial ideas and talent.

Once data have been collected, document the methodology used and calculate the figures as
described above. If a BDS program does not have or cannot avail itself of this kind of data,
the program should report its current method for assessing the impact of business
development services on SMEs.

Research Questions:

§ What methodologies are available to collect SME financial data?

§ Which are most valid and practical?

§ Is there a way to assign attribution?

§ Is value added as easy to collect as sales? Is it more valid?

Chapter Five—BDS Customer Indicators

ii
 iii

LIST OF TABLES

Table Page

1 BDS Performance Measurement Framework 4

2 Example of Goal 1 (Business Training) 5

3 Example of Goal 2 (Business Training) 16

4 Example of Goal 3 (Business Management Training) 21