Mining Big Data for Innovative Solutions

Published Nov-13-17

Breakthrough:
Novel solutions to help policymakers better understand labor markets.

Company:
The European Commission, Belgium

The Story:

There are huge volumes of data swirling around within and between organizations. Tapping into the facts, figures, files and statistics to innovate isn't a new concept but what is a relatively recent development is using open innovation to mine the data in the most effective ways possible.

Well-managed crowdsourcing initiatives such as hackathons are enormously beneficial, drawing out value quickly and cost-effectively. Ideation to product/solution development takes time and resources but big data hackathons yield a raft of new ideas that are processed and evaluated in highly truncated timeframes. The data can be mined at rapid speeds, much faster than having an organization's workers take on the task.

Innovating Through Big Data

These were some of the motivating factors behind the European Big Data Hackathon in 2017. The open innovation event was organized by the European Commission which gathered more than 20 teams from across Europe to come up with the best data solutions to help tackle a pressing policy question. That question was how to reduce the mismatch between jobs and skills at regional level.

In several European countries, the labor markets are seeking people with specific skills, while in other countries people with those skills are looking for work. Policymakers wondered if big data could help them better understand the problem and also whether insights from it could lead to the development of a data visualization tool to help discover interesting relationships and patterns.

To help them develop their prototypes, hackers were provided with access to five datasets, including official statistical surveys and millions of web scraped job ads. Teams were evaluated by a panel of 20 judges, consisting of policymakers and industry representatives from companies such as Amazon, Oracle and IBM.

Three Winners

The first place winner was a team from Croatia for a web application that has visual, interactive dashboards to help policy analysts gain insights into four key areas: skill supply, skill demand, and skill gap and policy actions to close the gap. "The winning solution not only answered the given policy questions but also featured an interactive interface which allows further analysis," said Marko Krištof, Director General, Croatian Bureau of Statistics.

In second place was a team from France for an application that allows users to compare the skills cited in job applications against skills in job vacancy ads at macro and micro level.

In third place was a team from Estonia for a visualization tool to demonstrate the differences between supply and demand on the job market at European and regional level. Their solution also analyzes which jobs are most vulnerable to computerization.

Putting Ideas into Action

At the time of writing this article, the European Commission is looking at ways to put some of the winning ideas to use. To that end a few months after the hackathon nine of the teams took part in a seminar in Greece to continue working on their proposals and to present their improved and upgraded prototypes.

Next Story »