At Personifest earlier this week I got to hear Rick Smolan speak on the Human Face of Big Data. Among the many stories he related was one about taxis in the rain in Singapore. Like most big cities, when it rains in Singapore it’s difficult to get a cab. So what researchers did was to overlay weather data with taxi location data in hopes of determining how taxi service could be improved. In a nutshell, what the researchers learned is that when it rains in Singapore, the taxis would park their cars. That’s what the data showed.
After doing some human research (i.e., talking directly to taxi drivers), the researchers learned that taxi drivers were, in fact, parking their cars during rain storms. They did this to avoid accidents, because at the time in Singapore, if you were in an accident, even if it wasn’t your fault, you had to put up a $1,000 bond while the crash was investigated. Most cabbies didn’t want to do this, so voila, don’t drive in the rain. It was that simple. (As a result Singapore changed their laws related to this, so hopefully now it’s easier to get a cab in the rain!)
This story reminded me of something the Texas Medical Association did several years ago, which I wrote about here. They overlaid their membership data with data from their insurance provider, and found nearly a thousand non-members participating in a members-only program. TMA was able to convert almost all of those non-members to membership.
In both cases, the data analysis did not require thousands of hours of manipulation. It was simply comparing one set of data to another and making a profound discovery. Sometimes the biggest discoveries can be found in the simplest set of data.