Presidential pollsters got it wrong, what are the implications for consumer research?

If political research were so wrong, could consumer research be so flawed?

Most of the major presidential elections were somewhat far-fetched. And while Joe Biden is on track, as expected, he has significantly lower margins than expected. The polls have really exploded, and more than in 2016.

Our inbox is filled every day with offers from search engines with offers based on consumer research: surveys on planned vacation spending, device usage, privacy, and many other topics. These studies are conducted with varying degrees of accuracy, but most claim to be comprehensive studies. Brands such as Harris, IPSOS, and Forrester are often used to increase the credibility of the results, which are used for linking and marketing purposes.

A similar methodology is used in political and consumer research

As it is now called, the “2020 disaster” made me wonder about the validity of the consumer surveys we see every day. How related are the two? And in many (or even nearly all) consumer research is it “gray” in the same way?

An ABC / Washington Post poll in late October gave Biden a 17-point lead in Wisconsin; he won by less than a point. Jamie Harrison, Democratic Senator from South Carolina, will be two points behind on the eve of the election, with a margin of error. Current Republican Lindsey Graham won by 15 points. And there are many other similar examples.

The methods used to conduct political research online are essentially the same as those used for consumer market research: “weighted” panels and samples to determine age, gender, education, marital status, and other demographic variables. to measure.

Factual research and behavior often differ

Recently, I received an offer based on consumer surveys that said, “About 1 in 3 online shoppers uses their voice assistant to make purchases at least once a month.” While I’ve researched products with virtual assistants (on smartphones and smart speakers), I’ve never bought anything through the virtual assistant, nor do I know anyone in my extensive network who owns them.

Compare the “1 in 3” statement with a 2018 report, based on internal Amazon data, that only 2% of Alexa users made a voice purchase. Although it can grow for two years, especially during the pandemic, reaching 33% of online shoppers is extremely unlikely or not impossible.

According to the report, 1,000 “regular online buyers” are being sought. On other issues, the research source revealed that respondents were predominantly millennials and that they were generally rearranged or completed themselves, that was repaired or completed, possibly by other devices. While a bit more plausible, the numbers are still very high.

As another example, research over the years has shown that most consumers prefer to buy from brands that support social and political issues with which they agree. However, behavioral data generally shows that there is little correlation between these research-based attitudes and current consumption patterns.

Look behind the scenes

Surveys are used by brands and marketers for a variety of purposes. Some use it to strengthen the market and make decisions about products and prices. It is much more used for PR and content marketing. In the latter case, methodological accuracy is somewhat less important. The fact that 60% or 70% of consumers want to spend more on holiday has limited consequences: consumers will spend more. But if product and market decisions are based on incorrect data assumptions, this can be a problem.

No research should be considered conclusive or even completely accurate. And we have to be very careful when extrapolating small samples to the whole population on the grounds that it is a “balanced census”. There are also phantom data points that repeat and take on an aura of truth as it is widely disseminated. This happens with the now-infamous prediction that ‘50% of all polls by 2020 will be polled’, which is a mistake and falsely attributed. (I still see it mentioned.)

People should look behind the scenes and not accept these things with pleasure. I’ve seen many collections of ’20 statistics on [topic] ‘filled with inaccurate information or quotes from the wrong sources. However, this doesn’t mean we should ignore market research. On the other hand, surveys should be seen as ‘directional’ at best and possibly an indication of wider consumer sentiment, but not as a 1: 1 representation of reality.

Consumer surveys are a useful tool, but they must be verified with behavioral data for a more complete and accurate picture of consumer activity. To proceed, we must receive data or studies with deep skepticism and a clear understanding of the underlying methodology.

Translate ยป