Surveys and their potential pitfalls

Surveys and their potential pitfalls article image

Survey this, survey that, are you sure about the result? As a marketer, I'm interested in opinion surveys but for the uninitiated they pose potential traps. For instance, survey results can be misleading and may even prompt businesses to make rash strategic choices. Now, I’m not a statistician by any means, but after undertaking some investigative studies on how to lie with statistics, I now look upon stats with a different lens. One that, I must add, has made me much more wary about accepting results on face value than ever before. A global consulting firm recently published results of its global economic snapshot. While it was interesting to read, it was also with no surprise that the BRIC nations are predicted to trump over USA, eurozone and Japan as the drivers of world stimulus into 2011. It was no more surprising that the major insights are that inflation will rise and exchange rate volatility will remain. Doesn’t really speak too much of anything new does it? Sounds more like a confirmatory survey. As I perused the content, I wondered about the credibility and validity of the respondents; turns out the only information related to this is that the survey was online for four days in early December and attracted 2,076 respondents. But, there was nothing explaining who those respondents were and how they can validate the result. There was no outline of the questioning approach, it may have been biased in some way, shape or form to produce the confirmatory result the survey was covertly seeking to achieve so as to concur with popular opinion and, as such, gain enhanced perception with a particular audience. To help valid a survey result, it’s ideal to know who are the respondents, what roles do they serve in industry, and are they qualified to make opinion on the topic? At a local level, we are exposed to numerous examples of small operators and industry associations that have plucked some consensus from their database or membership. But I question how much thought is given to whether the results are biased. Is the sample representative of a valid population? What average is used, and why are there no reference to the questions and analysis of the respondents to display substance? This helps build credible survey results. While many surveys make for interesting and thought provoking reading, their benefit lies in their means to complement our individual thinking and not over-influence us one way or other. As decision makers in business, it’s imperative we question the credibility of surveys before hastily directing resources into action or initiating policy. I’m sure most of you do anyhow. Often, survey results may not be what they seem. Interpreting the relevance is all part 'n' parcel for marketing decision makers because inevitably they will either be asked to support or be influential in guiding a strategic initiative. A little devil's advocacy will assist to ensure the execution of proposed tactics remain intact and with integrity. So, next time you are presented with a survey result consider delving a bit deeper.


Leave A Comment

Spinning icon Saving your comment, please wait...
Spinning icon Saving your comment, please wait...