Seven Reasons Why Big Data is Worth Shifting a Career For

At the MIT Sloan CIO symposium earlier this month, panelists laid out some of the reasons that big data is becoming such an exciting field.

Have you ever seen something that made you want to change the trajectory of your professional career?

Thomas C. Davenport has. Big data did it.

About two years ago, Davenport was considering retirement. In a long, distinguished career based at Babson College, Davenport had accomplished much of what he’d set out to do in the field of analytics. He had written or co-authored many well-regarded books including Competing on Analytics (Harvard Business Review Press, 2007) and Analytics at Work (Harvard Business Review Press, 2010), and was a pioneer in the study of business process reengineering. In the Ziff-Davis 2007 list of 100 most influential people in the IT industry, Davenport was the highest ranking business academic.

But, as Davenport explained while participating on a Big Data and Analytics panel at the MIT Sloan CIO symposium on May 22, big data came along and began to change the world he had been writing about and consulting about for decades.

What’s different about big data that would make a leading consultant and academic excited enough to hold off retirement? Here are seven highlights mentioned at the symposium:

• New tools are putting structure on so-called “unstructured” data like email and tweets.
• Big data is big business. Panelist James Noga, vice president and CIO at Partners Healthcare, said that in the US health care industry alone, the value of big data is $300 billion. Of that, $200 billion is in reduced expenses.
• Just-in-time data can prompt a flight attendant to offer a glass of champagne to the person in seat 21A, who the attendant’s airline-issued tablet says lost her luggage on a prior leg. This example came from panelist Shvetank Shah, executive director of the Corporate Executive Board.
• The field is hot and demands a crop of skilled workers. The gap between supply and demand for data scientists in the US could reach 190,000 by 2018, said panelist Michael Chui, a senior fellow at the McKinsey Global Institute. GE, for instance, is investing more than $2 billion in an analytics function for its industrial services and products, and is looking to fill hundreds of analytics positions that demand skills with business, statistics and big data tools.
• Analytics is moving out of the IT function and into research and development, into strategy, into marketing departments. Distributed analytics is on its way.
• Big data will continue to challenge our notions of privacy. As companies combine multiple datasets from inside and outside the organization, it will be increasingly difficult to maintain control over personal information. Regulations are in the works, but as Davenport says, “get used to not having privacy going forward.”
• Although big data promises significant value, it is a business trend that remains “emergent.” There are no best practices yet, if for no other reason than that the technology is so new. Sharing data also requires significant culture change in many organizations.

The panel on Big Data and Analytics was moderated by Quentin Hardy, deputy technology editor of The New York Times.

Next year’s MIT Sloan CIO Symposium is scheduled for May 22, 2013.

This blog post has been revised to reflect the following correction:
Correction: June 4, 2012
An earlier version described the global demand for data scientists incorrectly. Panelist Michael Chui noted that the gap between supply and demand for data scientists in the US could reach 190,000 by 2018. We mistakenly wrote that the demand, and not the gap, will soon reach 190,000.