Big Data Approaches to Improve the Accuracy of Political Predictions

Published Jun-26-19

Breakthrough:
Using social media and big data a university professor correctly predicted election results that traditional pollsters got wrong.

Company:
Gold Coast University, Australia

The Story:

When it comes to local, regional and national election campaigns a lot of debates take place online with voters turning to Twitter, Facebook and other social media channels to air their views and attempt to persuade others. This provides anyone who cares to look with a lot of information about what people are thinking about an election race which in turn can be used to make predictions.

Poll Failures

The result of the 2019 Australia election was described as a 'miracle' win for the incumbent coalition of center-right political parties. Opinion poll after opinion poll consistently predicted they would lose, and many political commentators said it was a result that no one saw coming. Well, they were wrong in that assessment because at least one person correctly predicted the outcome and that was Griffith University data scientist Professor Bela Stantic.

Using Gold Coast University's Big Data and Smart Analytics lab, he crunched the data and confidently predicted the coalition would sweep to victory. That got a lot of people asking the same question: how did the opinion polls get it so wrong?

Huge Volumes of Data

One of the principal reasons according to Prof Stantic was the sample sizes they used were much smaller than his. He put millions of tweets through his programs to get a clearer understanding of what people were saying, thinking and feeling about the election. Another factor according to the data scientist was the fundamental difference in how upfront people chose to be on social media as opposed to in a phone poll. He says people are more honest when discussing matters with friends on social media channels than they are when speaking to a pollster.

Prof Stantic reckons the two million or so tweets he analyzed in a few days was a greater representation of people than polls of 1,000 individuals, estimating they equated to around half a million people.

Predictive Power

The result of the Australian election was not his only correct political prediction. He also predicted Donald Trump would win the US election, and Britain would vote to leave the European Union. These two results were not predicted by the majority of the polls.
The Smart Analytics Lab from Gold Coast University has been able to improve the accuracy of predictions with what it calls the human sensor approach that recognizes a whole person as a data collection source.

Real-time data from a variety of sources such as blogs, forum posts and tweets are brought together, and a sentiment analysis is performed. This is a method that can identify and quantify subjective information such as what people are thinking and feeling.

Greater Understanding

No matter where you stand politically, it is clear that in some countries political forecasting has been something of a failure in recent years. Big data is giving it a shot in the arm, helping us to better understand what is going on.

Despite failures to predict Brexit and the outcome of the 2016 US election, the demand for predictions is insatiable. Most people are uncomfortable with uncertainty and want to know what is come along down the track. More accurate predictions using big data techniques fulfill that need.

Next Story »