The Dream Team
The Dream Team
The Dream Team
Page 1 of 51
simple yet powerful; some examples for direct response-oriented
creative insights include:
1. Web site landing pages in horizontal formats perform better than
vertical ones.
2. There should be a balance of iconography and descriptive text.
3. Flash modules perform better if they are booked ended with a call to
action.
Other examples for more brand-oriented campaigns:
1. Consistent and prominent branding improves recall.
2. Rich experiences that incorporate playful interactivity improve
engagement.
Forecasting Results
Additionally, the Analytics and Creative teams need to jointly decide
what success looks like from a metrics standpoint and jointly present
this back to their clients. Naturally, what constitutes success will vary
depending on what the objective are for the campaign. In this way both
teams have ownership in the results.
Also, it should be the job of the Analytics team to forecast the results of
the campaign before it is launched. Considerations include the
supporting media and search plans, campaign messages, creative
execution and of course the strength of the product offering. Effective
forecasting is difficult; it is equal parts art and science. And sometimes,
the Analytics team will forecast incorrectly and miss agreed upon
targets--which has nothing to do with poor creative but has everything to
do with poor forecasting. This also forces the Analytics team to have skin
in the game--and moves them from a role of just measuring to having a
meaningful deliverable that impacts the final results.
The Power of Testing
As the Creative team works through the brief, the Analytics team should
set up multivariate or A/B testing to run at the launch of the campaign.
Page 2 of 51
After a minimum number of impressions are delivered, the better
performing combination of ad and landing page elements should remain
in rotation. Instead of hindering the creative team, this can actually
empower them. It can give the Creative team more options to test and
explore. After all, this is the power of digital, so it is the job of the
Analytics team to give their Creative partners the flexibility to try
different options and test different approaches all the while providing a
safety net to ensure delivery of the campaign objectives.
Building from this idea of flexibility can be a beautiful thing. Often the
Creative team has a concept that they firmly believe is a winner, but the
client believes is too risky. Imagine a scenario where the Creative team
develops this concept, perhaps at their agency's expense, and then
presents it to the client, in combination with a more typical campaign
execution. The Analytics team can set up a simple test at the launch of
the campaign. Both executions are initially trafficked and the agreed
upon success criteria will pick a winner when a minimum number of
impressions are reached. And if the new concept wins, which I have seen
happen many times, the client agrees to keep it in rotation.
This approach is very powerful. The Analytics team has helped get
approval for the campaign that the Creative team felt would be the most
successful. This is collaboration, a true partnership, and how Analytics
and Creative should work together.
Analytics & Creative Collaboration Will Drive Better Results
Once this type of collaboration takes hold there is a powerful, free
flowing exchange of ideas. The Creative team often has great new ideas
for measurement and analysis. I have learned a great deal about better
measurement and optimization from my Creative partners. And I have
also seen the analytics team contribute to the creative development of a
successful campaign. Good ideas can come from anywhere but only if a
mutually respectful, collaborative, environment is in place.
Page 3 of 51
The primary reason to bring these two groups together is very simple--to
deliver better campaign results. To begin with, there is the improvement
from ongoing optimization while the campaign is still in flight. This
requires a quick turnaround of initial measurement against objectives
and the ability to deliver even quicker changes to creative. This only
sounds "pedestrian" to those who have not had to deliver it. It requires
seamless integration between these two teams along with a close
working relationship with the technology and account teams. Anything
but a total commitment to the success of each other teams will cause this
process to spin wildly out of control and key deadlines will certainly be
missed.
Beyond live campaign optimization, and perhaps even more valuable,
are the consumer and business insights that can be discovered when the
Analytics and Creative teams are truly integrated. Each team often
brings a different perspective and a different set of skills to bear on the
same problem. The creative team may have a hypothesis about why
engagement scores are below average and the analytics team can provide
a testing framework to confirm that hypothesis.
The partnership between Analytics and Creative should be about
exploration--an exploration of new ideas and concepts. This is
increasingly important in the ever-evolving digital landscape. These
groups have a tremendous potential to add to the success of each other. I
have seen well integrated Analytics and Creative teams deliver ROI
improvements of 3--5x what can be accomplished when the teams are
not integrated and don't care about their mutual success. But let's not
forget that the fundamental underpinning of any partnership is trust.
Trust is not something that happens by accident. It is fostered and built
up over time within an environment of shared values that include
mutual respect and a commitment to the success of others.
Page 4 of 51
In today's marketing environment, a well integrated Analytics and
Creative team is not an option it is a necessity. So take the time to make
sure that these critical groups are well aligned. Make sure they
understand and respect each other's challenges and contributions.
Create an environment were a shared responsibility in mutual success is
the norm. Better work will follow--GO TEAM!
Every CMO's monthly schedule now includes the requisite meeting with
the digital team to review last month's brand Web site dashboard.
Indeed, it would be a short-lived tenure for any CMO who doesn't know
what is going on with their own Web site. But let's be honest,
how informative are these meetings and what are we really learning? All
too often, marketers seem to be swimming in data but often come up
short on insights. Yet, it doesn't have to be that way.
Death by a thousand paper cuts
Everyone understands the importance of performance reporting for their
Web site--it is the center of many brands' marketing efforts and is
allocated an ever-increasing budget. But, the metrics tracked are often
one-dimensional and provide very little insight into the future success of
the enterprise.
For non-ecommerce Web sites, we often see the same uninformative
metrics reported on again and again:
* Unique visitors
* Page views
* Bounce rates
* Time on site
Page 5 of 51
* Source of traffic
* High value activities (lead submission, store locator, product
configuration)
* And, occasionally an engagement score (simple to complex)
And, we often see these measures compared week over week, month
over month and year over year--not very exiting or insightful. While
these measures do have some importance they don't, by themselves, tell
us much that is helpful. Do these measures tell us if we will hit our sales
objectives next quarter? Do they tell us if we are on track to successfully
deliver our next product launch? Can they tell us if our media plan is
delivering effective reach and frequency?
If your monthly scorecard can't answer these questions then something
critical is missing. Perhaps it is time to get a new scorecard populated
with more complex measures.
The value of complex measures such as a predictive ROI
At a recent trip to the doctor I was told that my HDL was 62. My first
question was "Is that good?" Metrics without context provide no real
value. In order for your scorecard to provide impactful measures, it
needs to provide two yardsticks:
1. A relationship to success
2. Specific goals or expectations.
1. Relationship to Success
You need to start by determining what activities on your site are highly
correlated with offline sales. This is another instance where the ROI
modeling discussed in previous posts continues to pay dividends
(http://www.fastcompany.com/1608700/the-time-is-now-for-brand-
building-an...). A modern ROI model should value individual Web
activities that are closely tied with sales and attribute a success value for
each activity. Even though your ROI model is a measurement model
and, therefore, based on historic data, it uses the historic relationships it
Page 6 of 51
uncovers to provide real-time reports on site and media performance.
The integration of ROI-generated values into reporting ties current
performance with your overall goals.
Now let's say you are succeeding in driving more traffic. Is more traffic
good? Are you driving the right consumers to your site? Another
function of your ROI values is to move away from such general
measures. The inclusion of advanced metrics helps ground your progress
report in sales, showcase the quality of traffic and determine spots in the
funnel where consumers are "lost."
2. Specific Goals or Expectations
Perhaps the most important yardstick is where you are compared to
expectations. After all, don't company stocks rise and fall based on
performance against expectations? Likewise, your Web site performance
should be judged against expectations. And not arbitrary Web site
centric expectations like a 5% improvement in visits against last quarter,
but concrete sales based expectations. Once you implement ROI-based
high value activity tracking, it is simple algebra to utilize the activity's
relationship with sales to determine benchmarks.
From these two additions to your scorecard (ROI-based tracking and
benchmarking), you can begin to transform your scorecard from a set of
numbers to a set of insights that include:
* What effect do your marketing activities have on key site activities?
* How far into the future does your Web site predict sales activity or
changes in upper funnel brand measures?
* How can you facilitate optimization of current and future marketing
activities utilizing your brand site scorecard?
Adding performance benchmarks also fosters a common interpretation
of your brand site metrics--either we hit our objectives or we didn't.
When relying too heavily on simple measures - such as unique visitors or
bounce rates - the same set of metrics can be viewed as a success by one
Page 7 of 51
group and failure by another. Needless to say this isn't productive or
helpful.
Getting There
It quickly becomes clear that in order to get the most out of your brand
Web site metrics, you need to have an ROI model in place that includes
Web site activity. I have outlined the predictive modeling process many
other posts (http://www.fastcompany.com/topic/Steve+Kerho) and
many of the same tools are used in measurement and ROI calculations.
Generally, predictive modeling utilizes some form of regression
modeling. Depending on what you are trying to measure, it may take a
linear or nonlinear form. We are going to touch on a couple of the tools
that will help you develop a best-in-class ROI model.
The first step in the ROI modeling process is to always determine what
data you have, what data you want, and what data you need:
Data You Need: (The is cost of entry)
*Sales - Ensure that the sales data provided is discrete enough for your
needs (often weekly DMA level or finer)
*Media - Media data at the finest level available, at least DMA. We can
always aggregate to a higher level if need be
*Web site Data - Interactive data can provide great measures of what to
expect for your brand in the future
*Pricing - Pricing information specific to the level at which you are
modeling (brand / model)
*Promotion - Data that includes in store and price cut information
specific to the level at which you are modeling (brand / model)
*Industry - Industry sales levels or macro economic factors that may
impact the baseline level of sales for your brand
Data You Want: (Nice to have but not necessary)
*External Interest Data - Measures of overall interest or brand
perception that shape the outlook for your brand
Page 8 of 51
*Social Media Data - Measures of level and tone of the conversation in
the social space
*Industry / Economic - The overall state of the economy has an
undeniable impact on most brands' sales
Once you have the appropriate data in place, you can begin building your
ROI model. It all starts with the basic form of the model:
Sales = Industry / Econ + Media + Pricing + External Interest
+ Web site Data
While this basic form seems overly simplistic, the actual incarnation will
be rather complex. Issues we have previously discussed like lagged
effects, decays, diminishing returns and saturation points all come into
play. The key aspect for our discussion is the inclusion of Web data. With
the influx of Web data we can transform static historic measurement
models into responsive ongoing measurement models and this is key!
Since we know what activities site visitors are performing, we can use
our modeled relationships to understand how this impacts sales (both
short term and long term). Rather than relying only on passive
econometric trends and company instituted factors like media spend and
pricing, we are able to include what consumers are actually doing. This
provides much tighter measures that are able to adapt to changing levels
of consumer behavior.
It is also useful to develop a predictive model that will estimate your
levels of Web site visits based upon external trends and media levels. A
full discussion of predictive modeling can be found in earlier posts. This
helps everyone to understand and quantify the causal relationships
between offline media tactics and high-value brand Web site actions.
The combination of an activity Web based ROI model and a visit
predictive model provide the most value to your business and transform
your weekly Web site metric scorecard into a discussion of media
success, campaign performance and future sales success.
Page 9 of 51
Putting insights into action
My team developed a system of models that included an ROI model, a
Web site conversion model and a Web traffic forecasting model that we
utilized for a new product launch brand site scorecard. During the pre-
launch period, our weekly scorecards included not only a discussion of
the levels of Web activity but goals and targets that should be achieved
for a successful launch, the media plans and the overall success of the
prelaunch strategy in driving interest.
By widely distributing and frequently discussing our scorecard of
predictive measures the organization was well prepared to make quick
changes to the campaign while it was still live to improve performance.
The hallmark of a good scorecard is how often it is the driver for a
meaningful changes in the campaign for items like creative messages,
SEM keyword bid strategy and dropping poor performing display media
placements.
The updated monthly scorecard
Many of the measures that were mentioned before are key ingredients to
answering these questions--but we need go further. Some examples of
the additional measures that should be added to the monthly scorecard
include:
* ROI Valued Activities
o Discussed Above
* Benchmarks
o Discussed Above
* Pathing
o There may be a user path that we never considered that could become
a starting point of site optimization
* Engagement
o Being careful to truly definine what constitutes a user being engaged
prior to site launch is key
Page 10 of 51
o Time spent on site and average number of page views is not enough.
This definition could mean that the user couldn't find what they were
looking for, not that they were engaged
* Competitive Site Performance Data
o Not only do you need to understand where your site stands on its own,
but also where your site stands relative to your competitors in the
category
Beyond these measures, every scorecard should have a summary page of
insights. A short and insightful description of what the numbers mean.
This is by far the most important aspect of the scorecard. Some examples
of key insights:
"The incremental broadcast plan did not have the desired short-term
impact. Even though 600 incremental TRPs were run in the previous
two months, there was no relevant change in qualified site traffic. Need
to study what went wrong."
"We are tracking well against our future product launch. Looking at
our lead and handraiser volume as an indication of future demand, we
are on track for our sales goal during the first quarter after launch and
we should consider building extra inventory."
"Our new geo-targeted online campaign is successful in increasing
brand opinion. Those exposed to the campaign increased "excellent
opinion" by 20%. This campaign should be rolled out to other markets
as soon as possible."
"We should pull back media spending. It looks like overall demand for
products in our segment will decline significantly in the next months."
These insights are key because this is your chance to tell the story as well
to answer the "why's" as opposed to simply pointing out the spikes and
declines in performance. These brief comments also provide the
opportunity to provide recommendations to improve performance and
outlook based on the results.
Page 11 of 51
How to make the monthly scorecard meeting more productive
The first step to make this meeting more productive is to incorporate the
complex, predictive measures we spoke of above. The next step is to
integrate this meeting into a broader review of last month's marketing
performance. The meeting should include key stakeholders from Sales,
PR, Strategy, Media, Digital, Promotions, etc.
The discussion should focus more on what your Web site can tell you
about your business and its future success and less about page views or
bounce rates. There should be disagreements, heated discussions and
the occasional Ode to Joy about what it all means and, more
importantly, what are we going to do about. Your Web site scorecard
should be a treasure trove of insights that drive critical business
decisions. If your scorecard and this meeting only serve to provide a
rearview mirror of one dimensional measures, then it is time to stop
these activities all together, simply because they are accomplishing little
of value.
Digital Media Director 2.0: Jack-of-all-Trades, Master of Many
BY FC EXPERT BLOGGER STEVE KERHOFri Jul 30, 2010
This blog is written by a member of our expert blogging community and expresses that expert's views alone.
Page 13 of 51
trading desks that sit on top of multiple ad networks to create
commodities like trading in near real time.
New to the landscape is the addition of data networks that can be used in
conjunction with the ad networks. We can combine online and offline
data and buy very specific shopping behaviors. When combined with
media inventory, we can serve ads to a pre-qualified audience based on a
staggering array of requirements. We can target "Soccer Moms, who
have two boys in pre-school, drive domestic mini-vans, reside in the
smile belt, live at odd number addresses, and have bought Lucky Charms
cereal in the last 30 days." Granted this includes a bit of hyperbole but
you get the idea. Good directors need to navigate these waters and build
complex behavioral models to reach their desired targets.
This is only a partial list of the technologies that a good director needs to
be familiar with. But the implication is clear--good digital media
directors need to be more than just tech friendly--they need to be
downright geeky.
Analytics
Digital media has always been more performance oriented compared to
broadcast and print, simply because it is more easily measured. But
performance measures have moved beyond CPMs, click-throughs and
view-throughs. ROI models that connect ad exposure to brand site
behavior and offline shopping activity are now the norm.
It is these ROI models that are key to keeping the ad exchanges honest
and ensuring that we are indeed getting the promised value for our
spending. It is not as if we can get a look "under the hood" and see that
our ad was delivered to the specified target. We need sophisticated
performance measures to ensure our results and this clearly requires a
close working relationship between the media and analytics groups.
As a result of this, effective digital media directors must be math
friendly. The analytics group will routinely employ a variety of
Page 14 of 51
econometric models, including: time series analysis, logistic regression,
attribution modeling, multiple log-on-log regressions, and Granger
causality testing.
If this sounds like a lot of heavy math, that's because it is. This is not to
say that digital media directors need to conduct these analyses
themselves. But because their plans will be evaluated using these
concepts, they need to understand the approach and choices made
within the models. Good media directors will participate in the model's
development and keep the analytics team focused on delivering model
outputs that can be actualized in the marketplace--not an easy task.
Data
The idea ought to be that digital media is really becoming a commodity
with an unconstrained supply. So, now the data is just as valuable as the
impression. We can't find an audience by context alone anymore and
expect to reach any type of scale. In the future, it is all about the data
and the ability to identify the individual to meet your campaign needs.
Questions of data integrity, storage, and hygiene are critical. And an
understanding of how data is organized from an individual record to a
meta level is also important. It is not that the media team needs to be
fluent in Business Objects or SQL code, but they need to care about the
data that increasingly drives their decisions.
Privacy Law
Five years ago few would have predicted that a law degree would be a
plus for a digital media director. But you would have to live in a cave to
not be aware of the legal battles that are currently raging within federal
and state legislators regarding digital privacy.
The very existence of ad exchanges, data networks, and ROI models are
all predicated on the ability to track online behavior. Consumers are
increasingly aware and concerned about these practices. And the recent
issues with Facebook have only heightened everyone's focus.
Page 15 of 51
The process of aligning a client's, agencies' and advertiser's privacy
"Terms & Conditions" is time consuming beyond belief and a source of
endless billing hours for legal counsel. Everyone wants to mitigate risk
while maximizing the performance of his or her media dollars. And, of
course the digital media planner is at the unwinnable center of all this,
acting as the glue that can bring about a détente of sorts among these
warring factions.
Real-Time Optimization
The most successful media planners have learned to optimize their plans
in real-time. After all, this is the promise of digital. They don't wait until
the media has run to evaluate is effectiveness and make changes for the
next campaign.
Changing a media plan while it is still in flight is no easy task. It needs to
be based on complex attribution and ROI models and executed through
ad exchanges and trading desks. Quick decisions, similar to trading
commodities in a live exchange, must be made. And the results of these
decisions are imminently transparent to everyone, either the changes
performed better or they didn't.
What is next?
As if this wasn't enough there are other changes to consider. We all
recognize that the next tectonic shift will be the merger of the PC and TV
set top box. Adding interactivity, and Web-like track ability to the TV
will be another game changer. Broadcast media will have all of the
measurability and performance pressure of digital media. And if I had to
place a bet on who would best navigate this space it would be the next
generation of digital media directors.
Where will the next generation of digital media directors come from?
Now that we have reviewed a litany of requirements, where do you find
the training to get these new skills? The simple answer is "Cross
Training."
Page 16 of 51
Focusing on digital media alone is not the answer. A rotation through
other, closely aligned areas is invaluable in providing the context and
cross discipline skills necessary to succeed. Some suggestions include:
Ad trafficking
Site side analytics
QA
Emerging technology
Offline media planning
Contract negotiations
Search specialist
In the end, an insatiable curiosity and a commitment to always be
learning will serve future digital media directors best. But the
responsibility for their development also lies with senior management.
Cross training opportunities don't happen by accident. They require
time, money and planning--all of which have been in short supply these
last few years. But, ask yourself this: can your organization afford not to
develop your next generation of digital media directors?
Page 17 of 51
What is Marketing Efficiency?
Marketing efficiency has always been important, but probably never
more so than during the recent past. Increasingly, CMOs have to
demonstrate ongoing marketing efficiency improvements with their
limited budgets. This can take many forms but the most common
approach includes year over year comparisons for "apples-to-apples"
budget items.
Management's intentions behind these measures are often well placed.
"We simply want to make sure that we are getting the best value for our
money," and "We want to improve our marketing performance" are
common refrains. This marketing efficiency process often includes
approval or vetting from groups outside of Marketing, such as
Purchasing or Finance. The thought is to add transparency and
specialized skills that the Marketing department may not have.
Frequently, this is a two-step process. In the first step, Purchasing will
often determine which elements of the marketing budget will be
measured for efficiency. All too often the simplest of measures are
chosen. Some examples include:
Macro Measures:
* Total media budget
* Web site maintenance costs
* Overall ad production costs
* Agency management fees
Channel Specific Measures:
* Media budget by channel
* Media CPM by channel
* Cost per Web site visitor
* Production cost by channel
* Agency fees by channel
Page 18 of 51
Typically, in the second part of the process, a percent improvement
target is chosen and measured against a year-over-year period. Targets
of 5 - 15% are commonly chosen. In this scenario, a 10% year-over-year
decrease in the overall media budget or media CPMs would be counted
as an improvement in Marketing efficiency.
What is the Problem with Marketing Efficiency?
It is not that these types of measures aren't important. In fact, they are
very important. But they only tell half the story. Efficiency measures
should never be looked at without also considering effectiveness. This is
because if we increase efficiency but drive down effectiveness in the
process, then we haven't accomplished anything and we could have
actually done significant harm.
There are lots of "easy" choices to gain efficiency improvements, such as
buying cheaper media (late night versus prime time), spending less to
create digital content (using static content as opposed to video), or
choosing the agency that has the lowest hourly rate. It is not difficult to
imagine how too many of these types of decisions would drive down
marketing performance.
Another challenge is that there are many "one-time" efficiency tactics
that can't be repeated. For example: switching a large percent of the
broadcast media budget from network to cable; or adding retargeting to
the digital media plan for the first time; and moving to a lower-cost
content management system for a brand Web site. Trying to repeat these
savings year-after-year can create pressure to make decisions that will
erode performance.
Yet another consideration is that a brand's marketing objectives often
vary considerably from one year to the next. A brand may be launching
four new products in one year and none the next. Or, a key competitor
may be struggling, creating an opportunity to gain market share through
additional spending. And, of course, there is the impact of market factors
Page 19 of 51
that are out of the advertisers control, such as inflation and GDP growth
to name a few. The point is that overly simplistic year-to-year measures
often don't account for changes in marketing objectives or changes in the
larger economic environment.
And finally, if Purchasing has committed to efficiency improvements
that the Marketing team doesn't support, then there will likely be in-
fighting and a culture of blame instead of one of collaboration, teamwork
and true innovation.
Taken together, these factors can lead to a decrease in marketing
performance. Ironically, this is the exact opposite of what was intended
by instituting efficiency measures in the first place.
The Solution--Marketing Efficiency + Effectiveness
I do not want imply that you should not have marketing efficiency
measures when, indeed, you absolutely should. However, a good rule of
thumb is that for every efficiency measure, you should have a
corresponding effectiveness measure. Together, these measures act as
check and balance for each other.
A challenge worth noting is that most the meaningful measures of
effectiveness are not easy to determine. And this is often the reason that
more marketers don't pursue them. A true attribute-based ROI is often
at the heart of many of these measures. Additionally, there is likely to be
some econometric modeling along with the need to combine survey and
behavioral data. However, given what is at stake, it is imperative to work
through these challenges. I have covered in detail how to calculate an
accurate ROI in several other posts that you may find helpful
(http://www.fastcompany.com/user/steve-kerho).
Here are some conceptual examples of efficiency and corresponding
effectiveness measures:
Page 20 of 51
Here is some real world data for four different upper funnel digital
media campaigns for a highly considered durable product that were
executed within the same calendar year (changes in seasonality and
macro-economic factors have been controlled for):
So what is the take away here? Simply put, relying solely on the
efficiency measures of CPM or CPM year-over-year changes would lead
us to rate Campaign C as "poor." And, in reality this was our most
effective campaign as measured by ROI and ROI year-over-year change.
In this context, is easy to understand why both measures are important.
It would be key to understand what where the factors that led to the
significant increase in CPM costs for Campaign C and if these drivers
continue will there be a negative impact in future campaign
performance.
Marketing efficiency is important and it is something that good
marketers care deeply about. But they need to care equally about
Page 21 of 51
marketing performance and they should never sacrifice one for the
other.
How Long Does Your Ad Have an Impact?
BY FC EXPERT BLOGGER STEVE KERHOTue Jun 29, 2010
This blog is written by a member of our expert blogging community and expresses that expert's views alone.
Even though media spending significantly declined during the last two
years, we experienced a continued acceleration in the evolution of how
media is consumed. Time shifting, place shifting and device shifting are
all themes we are becoming increasingly comfortable with. Control
continues to move from the content provider to the consumer, and the
consumer is increasingly comfortable creating and sharing their own
content.
All these changes speak to the need to re-examine some of the most
basic principles of good media planning – especially as marketers
commit Tarp-sized funds in the upfront. One of the most important
concepts in media planning is effective reach and frequency. This is
about understanding the cumulative number of impressions your
campaign delivers to an individual consumer. Too few impressions and
your message won’t break through. The painful result is that your entire
media budget has gone to waste. Too many impressions and you have
not only wasted part of your media budget, but you have also succeeded
in annoying the very target you were trying to swoon.
The middle ground --between not enough impressions and too many-- is
the efficient frontier. While this concept is rather pedestrian to well-
schooled media planners, its application in today’s distributed cross-
channel media world is anything but.
With any media plan, you are going to reach the most “accessible”
consumers first. Consumers who are pulled into the Internet constantly
or always have a TV on in the background are going to be easiest to reach
(tips on breaking through the clutter will be a future post). As you start
to reach more and more people, the harder it becomes and the more it
costs. Building reach beyond a certain point is never cost effective,
especially through one medium.
The second way to address the non immediate effect of media is through
‘decaying’ media. Consider that same ad that airs today. It may have it
largest impact this week, but it still retains some impact next week; some
impact in week three, and even some impact in the following weeks. In
order to capture this relationship, analysts ‘decay’ the impact of media
by using the concept of half-lives. By using a half-life of three weeks, we
say that an ad retains 50% of its original impact three weeks later.
If your brand airs 120 TRPs of television media this week, how might
that media weight react over time? Given a three week half-life the
impact of that media would be:
You may be wondering why there is such variability in these values. One
reason is the relative strength of a brand or product and consumers’
level of engagement with a particular vertical. But a much more
Page 25 of 51
important factor is the strength of the creative content or execution. The
most expensive media plan in the world will not make up for poor
creative. Conversely, powerful creative may actually require fewer TRPs
or impressions to accomplish its goal. Measuring creative performance
is very important and that is a topic for another time.
This data validates some long-held beliefs. We all realize that magazine
ads are generally a less interruptive ad format and often have very high
levels of engagement when they are in a publication that the reader has a
deep personal connection to. Our data clearly demonstrates that most
full funnel media plans would significantly benefit from having
magazine print as part of the plan.
The first group that is required for predictive analytics models are the
real-world marketers - the people who know how to create a marketing
plan and measure results. The second group will consist of hardcore
statisticians. These two groups of people are often at complete opposite
ends of the spectrum in terms of temperament and the ways in which
they work, but marketers and statisticians must have an environment
that allows them to fully express their opinions while being open to the
different perspectives of their colleagues. The path of least resistance to
team harmony is to focus everyone on the art and science of their
predictive analytics models. Root these two different camps in the real
world so they focus on changes in economics and global trends rather
than conflicting opinions. This often requires strong leadership from
senior management and creation of an “esprit de corps” that celebrates
and respects diverse opinions and skill sets. With everyone pulling
together to make sure the team’s hypothesis holds up and the numbers
dazzle management, predictive analytics models can be an effective tool
that lead to tremendous results.
For years, marketers have used predictive modeling to forecast sales and
Page 27 of 51
determine relative return among competing marketing investments.
Only in the last few years have those “competing” investments included
online and web spending. These are important marketing expenditures,
but have not readily been included in predictive models. We’re going to
demonstrate the value these new data sources, such as web data, can add
to your predictions. Through an examination of a specific scenario, I’ll
demonstrate how the inclusion of web data can help you deliver insights
beyond a simple forecast-- insights that can help shape your planning
process.
Now let’s get into our scenario, and the nuts and bolts of predictive
modeling. Generally, predictive modeling utilizes some form of
regression modeling. Depending on what you are trying to predict, it
may take a linear or nonlinear form. Let’s suppose you are working with
a client, ‘Company U,’ which sells a number of entry-level electronic
devices. Currently 80% of its sales are through brick and mortar
locations and 20% of sales occur online. A brand manager at Company
U has reached out to your team to develop an offline sales forecast for
DVD players. She provides you with historic sales, pricing and media
information and asks you to develop a monthly 12-month DVD sales
forecast.
Page 28 of 51
* Pricing - Pricing information specific to the level at which you are
modeling (brand / model)
* Industry - Industry sales levels or macro economic factors that may
impact the baseline level of sales for your brand
Once you have the appropriate data in place, you can begin building your
predictive model and it all starts with the basic form:
While this may seem overly simplistic, the devil is in the details. In
reality, the format of a predictive model is simple because it’s based on
the influences you can measure and the levers you can control. The
complexity lies in how those levers interact with sales – and with each
other.
We’ll begin by looking at media and a few tools that help us better
understand how each channel relates to sales. For instance, a TV ad’s
airing does not often immediately have an impact on sales. People see
an ad that may incite them to purchase, yet may wait a few weeks before
buying the product. That is why the relationship between media and
Page 29 of 51
sales is not always simple or straightforward.
If your brand airs 120 TRPs of television media this week, how might
that media weight react over time? Given a three-week half-life, the
impact of that media would be:
Using the concept of decayed media helps explain how media influences
Page 30 of 51
consumers and provides a smoother data set that shows the cumulative
impact of media weights. When people think of regression they often
think of the simple linear relationship, y = b*x which yields a straight
line. While this might be sufficient in some occasions, it’s not always
appropriate.
Our second tool for dealing with media impacts is the concept of
diminishing returns. By replacing the standard regression equations
with a more dynamic equation like the generalized logistic regression, y
= k / (1 + exp(-b)). In this treatment, we develop both a ceiling and a
growth rate represented by k and move away from the straight line to
any number of new possibilities. Using this format allows us the
flexibility to account for the fact that you can throw as much money as
you want into media and it won’t bring you additional value after a
certain point. This concept is often referred to as a saturation point or
the point of diminished return.
Page 31 of 51
fifth ad in ‘People’ is just as impactful as the first ad? No, we don’t. And
while I know that this is oversimplifying things – we would account for
ads per magazine and use TRPS to ensure the ads are appropriate for
our target – it is merely meant to point out the fact that you need to
move past a straight line mentality to really forecast expected sales
levels.
Now that we’ve discussed a couple of tools that we can use for our inputs
into the model, let’s shift to the data you want in your model. A lot of
people take industry/macro economic factors, pricing and media spend
as a given in a predictive model but then they stop there. While those
factors are important, and they definitely influence your predictions,
they are lumbering relationships. Now you may be asking yourself,
‘Lumbering relationships?! What the heck does that mean?’ It means
that large scale predictions are good when examining media and macro
factors. You can use these variables to forecast at the annual national
level. However, if you try to use these variables to predict at the weekly
regional level, you often run into problems. These ‘lumbering’
predictors have long term overall relationships with sales, but short-
term variations are common.
With the influx of web data, we now have more ‘reflective relationships’
that can be included in our predictive models. Rather than relying only
on passive econometric trends and company instituted factors like
media spend and pricing, we are able to include what consumers are
actually doing. This provides much tighter predictions that are able to
adapt to changing levels of consumer behavior. Let’s look at our DVD
player sales and how our marketing strategy might change by including
web data in the predictive model.
Page 32 of 51
A predictive model without web data may very well forecast sales,
particularly at a national level. However, although the forecasts of
national sales may be accurate, attributing the DVD player sales directly
to marketing investments will be misleading without including web
data. When missing from the model, the impacts of consumer web
activity are hidden by other media spending, or even other external
factors such as industry or economic trends. When included, “reflective”
data such as display ad clicks, paid search and brand website activity (i.e.
“click here for more retailers selling DVD Brand X in your area”) will
help attribute sales to both higher level marketing media (traditional
offline) and lower-level individual media online.
Combining offline media data with web data (online ad activity and
brand website data) and properly accounting for the nature of their
relationship to sales will allow marketers to attribute sales across all
marketing expenditures, and therefore, measure a truer return on
marketing investment.
The inclusion of web data into econometric models has greatly increased
their predictive power. We have seen this for a variety of products from
durable goods to consumer package goods. The inclusion of web data, as
Page 33 of 51
a measure of overall demand, has helped push the predictive power of
these models to within 5% of actual performance and in many cases to
within less than 3%.
It’s a brave new world and inclusion of this type of data combined with
the right team players and temperament can yield highly accurate
models that quantify the complex relationships between marketing
activities, sales and the economy.
Another problem with ROI in the early part of this decade was that
digital marketers gave too much credit to the last online activity a
customer performed before they purchased, a strategy known as the “last
click wins” model. While deeply flawed, there was not a readily available
toolset that allowed for simultaneous, measures of multiple touch points
in both the online and offline space. For instance, here’s a typical
consumer’s purchase path, ending with the opening of a new account – a
very tangible metric.
1. Sees TV Spot
2. Sees TV Spot Again
3. Sees Online Banner Ad
4. Clicks on Natural Search
5. Clicks on Paid Search
6. Sees Print Ad
7. Receives Brand Email
8. Visits Brand Site
9. Visits Brand Site
10. Receives Brand Email
11. Clicks on Banner Ad
12. Brand Website New Account
Page 35 of 51
1. Sees TV Spot - $0
2. Sees TV Spot Again - $0
3. Sees Online Banner Ad - $0
4. Clicks on Natural Search - $0
5. Clicks on Paid Search - $0
6. Sees Print Ad - $0
7. Receives Brand Email - $0
8. Visits Brand Site - $0
9. Visits Brand Site - $0
10. Receives Brand Email – $0
11. Clicks on Banner Ad - $500
12. Opens New Account - $500
It’s easy to ask, “What were we thinking?” in retrospect. But the fact of
the matter was that marketers had a whole new digital world thrown at
them and they grasped onto whatever metrics they could that made
sense given their limited knowledge. At least they were measuring
something and taking the first baby step in trying to be more
accountable for their digital investments. The fundamental error can be
found in the basic ROI calculation formula that was used at the time:
Page 36 of 51
Applying this information to future campaigns often lead to flawed
decision making.
Going forward, analysts need to adopt new holistic analytics models that
accurately compare investments, returns and credit across multiple
channels and touch points. Analysts also need to become data
forecasters -- not historians. Committing to an ROI before a campaign
goes live adds teeth to the optimization process and empowers
companies with the ability to change the initial strategy in mid-
campaign to alter the outcomes. This fosters better integration between
the analytics team and the rest of the marketing organization, giving the
analysts skin in the game and moving them from “observers” to “active
participants.”
Page 37 of 51
Interaction Between Media Exposures
Page 38 of 51
11. Clicks on Banner Ad - $45
12. Opens New Account - $500
As you can see, this model gives us a much different view of our
multichannel campaign’s touch points. Marketers can look for changes
throughout the ecosystem when they ramp up or down a channel to
optimize their campaigns. They can also build out sophisticated
attribution models that shed light on consumers’ use of push versus pull
media. Over time, benchmarks for an overall campaign and each discrete
element needs to be established and forecasts for overall performance
committed to before the campaign is launched.
Page 39 of 51
Media Type Last Click Shared Values ROI
Display Yes No $15.31
Paid Search Yes No $19.72
Display No Yes $23.51
Paid Search No Yes $8.78
* Over 95% of brand site leads from View Thru are occurring within 6-7
days of an impression. However, a consumer who has clicked on the ad,
and appears more engaged, may actually have the mindset to shop and
research for a longer duration of time, between 12 – 14 days.
Page 40 of 51
While it is a significant undertaking, attribute modeling is clearly the
preferred method for calculating an ROI for both digital-only and cross-
channel campaigns; otherwise, resources allocation will be flawed and
results will suffer. This is the foundation of a good ROI model. Adding
macro economic factors to the model is the next step in moving from
rear view looking to forecasting future results and war gaming various
scenarios. Shifting analytics from rear-facing to predictive is an
important part of being a digitally-savvy marketer. In my next post, I’ll
begin providing you with the framework you need to build your own ROI
crystal ball to accurately predict results, deliver successful campaigns
and and salvage a failing one.
Page 42 of 51
nor should you endorse analytics models that you aren’t intimately
familiar with.
Don’t Forget the Creative: With the CEO and CFO constantly asking
for empirical information on success, it is easy for CMOs to forget the
human element of the marketing equation. From a creative standpoint,
CMOs need to be able to give clear direction to their internal teams and
agency partners in order to develop compelling and high performing
creative content. This includes the ability to judge a campaign based
solely on a storyboard or to determine that a website redesign is off
strategy based on a 10-minute review of wireframes. Understanding how
creative ultimately impacts customers is no easy task and any CMO that
can’t make this connection is not likely to retain their title.
There is great promise and power in the future of marketing analytics,
but there is also great peril. It is ultimately the CMO who must decide
how to guide and act on the information from the analytics department.
The more comfortable the CMO is in this space, the more questions he
or she can ask about the approach and methodology behind the
numbers, the more likely that analytics will deliver their true promise to
marketing. To remain relevant, CMOs need to balance creative and
analytics more than ever before. Some may consider this an
orchestration of opposites, but it is the future of marketing – and our
future needs strong leaders who are equally comfortable in both worlds.
The reaction from marketers has been both predictable and pragmatic.
They have focused their limited, highly-scrutinized communication
resources on lower funnel, direct response activities. With inventories
growing, revenues shrinking and consumer sentiment falling off a cliff, it
is easy to understand the need for this approach. Digital marketing
efforts have been a beneficiary of this trend. The combined
accountability and real time optimization opportunities which are
inherent to digital have been the main reasons for this. There is nothing
like a serious recession to focus everyone on ROI – especially lower
funnel ROI.
If there is one thing that good analytics teaches us, it is that every brand
needs to build the top of its purchase funnel. Simply investing in, or
harvesting from, the bottom of the purchase funnel will have dire
consequences for even the healthiest of brands. Time and time again, we
have seen that solely focusing on lower funnel ROI leads to increased
acquisition costs, decreased loyalty and diminished brand attributes. It
is worth noting, however, that analytics often played an unwitting role in
this process. Namely, that it is much, much easier to create a lower
funnel ROI model than it is to create an upper funnel ROI model.
Page 44 of 51
Within the world of the good “analysts,” the ability to create an upper
funnel ROI is akin to what separates athletes in “Major” versus “Minor”
leagues. To be clear, creating a good, attribute based lower funnel ROI
isn’t easy. But it is much more difficult to create an accurate upper
funnel ROI. Imagination, creativity and a deep understanding of the
business vertical are costs of entry for this process. It often involves
more complex modeling or less direct measures than a lower funnel
ROI. Additionally, proving causality -- versus merely measuring
correlations -- is more difficult for upper funnel ROI measures. I have
outlined a detailed approach to calculating an upper funnel ROI in
previous posts (“Pouring the Predictive Analytics Foundation” posted on
February 3 and “It Takes a Village to Drive Predictive Analytics” posted
on March 24).
Here are three ways to get started with a smart upper funnel ROI
strategy:
1. Quantify the impact that upper funnel activities have on lower funnel
activities. Spending on brand focused, upper funnel campaigns,
eventually leads to an impact in lower funnel activities. Understanding
media lag or decay functions by channel will be critical to this process.
Page 45 of 51
2. Quantify the impact that changes in key brand attributes have on ROI
measures such as sales, loyalty or customer life time value. ROI
measures that are more long term, such as loyalty, more accurately
reflect the true value of upper funnel activities. The addition of survey
data and holding out control groups to better understand lift are often
necessary to quantify these causal relationships.
Many brands have trouble getting to the first stage of building predictive
analytics models. But I repeat the old adage to them, “you don’t know
where you are going until you know where you have been.” In order to
predict future success in the marketing world, you need a window into
the past combined with a large set of current behavioral data. And this
window on the past, my friends, is realized through the availability of
accurate historical data and the results of their marketing and media
campaigns.
Page 46 of 51
Unfortunately, many brands have difficulty gathering historical data and
media metrics – largely because the multiple marketing agencies they
work with don’t take the time to organize the data appropriately or in a
useful format. Therefore, many brands don’t know how they’ve spent
their marketing budgets over the years or how they have performed
against granular objectives. Brand marketing teams need to make sure
that they gather at least three years worth of their media metrics and
performance data in order to build effective predictive models. And it
will be important to break down the barriers that often exist between
various agencies and the client to obtain a holistic data set from all
parties. To be truthful, the data the team initially receives is not going
to be perfect. It’s completely okay to make assumptions or to take artistic
liberties based on your current data when developing a predictive
analytic system. Acknowledge the flaws in your data and work to
improve data collection for the future. Don’t let spotty data stop you
from ever getting started.
Page 47 of 51
digital behavioral data to interpret information and react before the
campaign is over.
Page 48 of 51
The best way to prevent this type of inaccuracy is to look at the type of
data used within the model. First, marketers would be well advised to
not rely solely on survey data; instead, marketers should look to utilize
digital behavioral data as that information is constantly available and
provides an accurate representation of how customers are acting online
without any assumptions or biases. These online behavioral data sets
are very often a measure of the total demand that the marketing
enterprise is generating. And this data greatly adds to the accuracy of
predictive forecasting models. We will discuss some of the ways to
incorporate this data in an upcoming post.
The last point I want you to consider today is that since change is the
only constant, the predictive analytics models should be treated as
living, breathing entities that need constant care and feeding. Without
this care and attention they will simply be outpaced by the current
marketplace and will lose their value. Since many brands are steeped in
the measurement mindset, they don’t want their numbers to change. If
Q3 of 2009 brought in ten million in sales then that is THE number.
The measurement doesn’t change. However, if we predict that we will
sell eleven million units in Q2 2011, but unemployment continues to rise
in early 2010, contrary to our expectations, then the forecast changes.
Eleven million is no longer the number, now 9 million is the number. In
predictive modeling we no longer have THE number. And that’s ok.
Actually it’s better than ok, because our prediction is better than it was
before.
Page 49 of 51
pointless so marketers will need to stay on their toes and ensure their
data practices do not become stale. Forecasts for 2011 and 2012 will
change based on data that is brought in throughout the year but that
does not make them any less valuable. It’s good that your forecasts
change. It means you are learning.
You will also see that these various statistical treatments, while
intimidating to the layperson, are quite manageable with the right team
in place. Remember it takes a village to deliver on the promise of
Page 50 of 51
predictive modeling so don’t get intimidated if things get ‘quant geeky’
for a while, it will all come back to the business insights in the end. So
stay tuned my friends, and before you know it you will be discussing
media inflection points AND what that means to your business.
Page 51 of 51