The Trouble With Analytics
Without entering into a political discussion, this week’s events present an interesting study of the challenges we have with predictive analytics. The map shows one news organization’s election map. As I watched and read coverage of the results from Tuesday’s initial results to today (Saturday, 8:00 AM central time) from several different sources, it was very interesting to see the focus shift.
Each news outlet examined all of the pre-election polls and entered the day with a perspective on how the results would play out. Some entered the night predicting a blue wave; others said the wave would be red even though many of the polls were within or very close to the margin of error. In retrospect, the polls were right in their call of a very close election. It was the various interpretations of the polls that were questionable.
The other main difference I observed was the demographics different people chose to focus on as part of their reporting. Some of the demographics that were common discussion topics were: Suburban white women, white males with less than a college degree, black voters, Latino voters, people who think unemployment is most important, and those who believe the pandemic response is most important. Each organization interpreted the data in a way that supported their earlier assumptions. As the counting progressed, they deconstructed the data to explain why the results did not follow the original premises.
What happened and why it is important to broader discussions about analytics comes down to two significant factors.
The amount of data available is immense, and the different ways to organize, reorganize, and visualize it seems almost limitless. This ability to slice data into very granular subsets enabled analysts to make assumptions about each demographic. Over time those assumptions became singular. This was a major error. One journalist I respect kept saying that suburban white women would decide the outcome. As I listened, I asked myself why they could only be one thing. Those who own small businesses may choose to focus on the economy; others who work in healthcare may care more about the pandemic.
The singular assumptions made it easier for both conscious and unconscious bias to become part of their reporting. It is reasonable to assume that everyone has a personal option on the outcome they want to see. (As of this writing, the results are not final.) People can consider both the pandemic response and economic hardship as very important, and choosing between them becomes a Sophie’s Choice.
As businesses use analytics to make business decisions, we can fall into the same trap. Our ability to analyze data can, without our realizing it, blind us to other interpretations. The more strategically important the question, the more critical it is to explore contrary points of view. This debate-style approach to analytics will improve the quality of our decision making.
