Excited by a Complex Challenge with a Real-Life Application
Interview with IdeaConnection Problem Solver Marco Correia
By Paul Arnold
Computer scientist Marco Correia was an academic for 15 years. He completed his master’s and PhD and worked on several projects and papers while based in the Department of Informatics at the New University of Lisbon, Portugal.
Today he runs The Solving Machine, a business selling optimization services and consulting solutions.
During the time he was transitioning careers he searched the internet for online platforms that offered remote working and came across IdeaConnection. So far, he has taken part in one challenge, Linear Programming for which he was one of the successful solvers, picking up a handsome check for his endeavors.
In this interview, Marco talks about working on the challenge and wrestling with a complex problem.
The focus of this challenge was seeing how some phenotypic traits of a plant such as its height or seed number are a function of the plant’s genetic code. So, the problem was to find a map between the genetic code and the traits the seeker was interested in knowing.
Do you have a background in genetics?
Not really. I am a computer scientist, although when I was doing my master’s we worked a little on protein structure determination. So, there was some familiarity, but this was different from what I had to do on the IdeaConnection challenge.
How difficult was this challenge?
For about half of the time while I was working on this I was thinking it’s too hard. It was an interesting problem, but it wasn’t the structure of the problem that was difficult it was the numbers. They were trying to apply this to thousands of plants which meant that when you translated it to a format that you give to the computer to solve it becomes a huge problem even for a computer. And these problems are complex. The effort that the computer takes to solve them is not linear in size. Let me explain.
Let’s say you have two plants and you have one minute to solve the problem. Then you have four plants but you don't expect to have four minutes to solve it because it's exponential. And if you have thousands of plants, well, it’s really…. I never thought it was possible. Therefore, I had to think about using other techniques that are different from what we usually we do with these problems which are typically much smaller than this one.
One thing I really liked about working on this was that it was nice to see that there was a real application directly linked to what we were doing. Sometimes you can work on things that are too abstract but, in this case, it was not.
And whatever you did worked because you were successful.
Yes, that’s right and it felt great because I thought I wouldn’t have a chance against, well the world. There are a lot of very talented people in India, China and Russia, lots of great mathematicians come from those countries and so I was not really thinking that I would solve it.
Were you learning as well as solving on this challenge?
Yes, I was. At some point, I thought if I don't win it doesn't matter because I can easily publish my work somewhere so it wouldn’t be lost. This was something good because it was a lot of work. It wasn't one of these cases where you see the problem and you say okay this is the solution. It was something really challenging where you think you might not get anything at the end it. I was employing a lot of time and effort so the fact that it started to become something I could probably publish was good. I didn't have to wait for the prize because I was already winning.
(Ed. Note: Marco would not publish any of the confidential information from the client that was included in the challenge.)
So, you devoted a lot of time to the challenge.
I was solving this during my summer vacation and at some point, I was so interested in the problem that I was just thinking about this. Even when I wanted to enjoy the vacation I couldn’t because I was kind of hooked into the problem. My kids wanted me to go to the beach with them so in the end, I thought I must win this or they won’t forgive me.
What’s your opinion of this way of solving problems - getting brilliant minds together who hail from different backgrounds?
I think it’s great. For this challenge, I worked alone but I would also like to have the opportunity of working with other people in the future. That would be interesting because I could learn even more by being a part of a team.