In advertising, you often find ‘bullshit masquerading as science’
The ‘metrics’ you get from social networks, the ‘data’ you get from consultants and the reports you get from your agency are all unreliable at best and bullshit at worst – argues Bob Hoffman
There was a time when I taught science for a living. Don’t get the wrong idea, I know very little about science.
The only reason I was doing it was that it was impossible for the New York City Board of Education to find enough qualified people.
Later on in life I served for a year as special assistant to the executive director of the California Academy of Sciences. Once again, it had nothing to do with my credentials in science, it had to do with their need to do better marketing for their scientific endeavours.
However meagre my background in science, hanging around with scientists and science teachers taught me one thing very clearly — there is a big difference between facts and bullshit.
And to put it simply, in the advertising industry the science we get is terrible.
It is mostly conducted by interested parties with a point to prove. In the real world of science, research that is conducted by interested parties is viewed with great skepticism.
In the real world of science, research must be validated and verified by disinterested third parties before anyone takes it seriously.
The ‘metrics’ you get from Facebook, the ‘data’ you get from your consultants and the reports you get from your agency are all unreliable at best and bullshit at worst.
The most unpleasant part of trying to find out just how crappy the science you are getting is is that you have to be a real prick to do it. You have to ask the people delivering the ‘science’ the following questions:
- What controls did you use in your study?
- Did you repeat the study to verify it?
- Have you had peer review to substantiate your methodology and your conclusions?
- Did you have a third party replicate your study to validate it?
In virtually all cases, the answer will be stunned silence and you will be treated with thinly disguised contempt for asking such silly questions.
We have gotten used to bullshit masquerading as science, and we accept this bullshit without appropriate scepticism.
It is remarkable that an industry that spends half a trillion dollars a year thinks it knows so much and actually knows so little. Now that our industry has caved in to the silly notion that data will be our saviour, it is more important than ever to question the science behind the data.
The closer you look, the uglier it gets.
Bob Hoffman has been the CEO of two independent agencies and is the author of the Ad Contrarian blog, where this post first appeared
Here here!
Reply“It is mostly conducted by interested parties with a point to prove.”
Sadly, this applies to just about everything we are exposed to these days.
ReplyIf only someone could do the same for TV, press, OOH, radio, cinema…
ReplySo, your argument is that all advertising data is unscientific BS and should be challenged more frequently by clients?
Yeah, I can live with that
ReplyBullshit is too strong. Sure, all research has flaws – by accident or design, unconscious or conscious bias. Bob’s criteria on reviewing the controls, consistency, peer review and replication must be a great place to start?
I guess for a typical marketer using any media measuring the inputs (e.g. reach) accurately is tricky, but measuring outputs (e.g. sales) accurately would be easier.
Even if you don’t know what the margin of error is on the input, so long as the error is consistent, and the relationship with the output is predictable, then it is not necessary to have absolute precision.
[Edited under Mumbrella’s community guidelines].
ReplyHave your say