How Companies Use Big Data

The challenges of mining and cataloging data are not new. In 1944, Fremont Rider, a librarian at Wesleyan University, noted that American university libraries seemed to double in size every year. Based on this observation, Rider estimated that by 2040, Yale Library's collection would number some 200,000,000 volumes, requiring a staff of 6,000 to manage all the data contained therein.¹

Of course, Rider couldn't even begin to imagine the internet, or the immense impact it would have on our ability to catalog and organize information. Besides, it turns out that organizing books isn't the issue anymore. The focus now is on organizing exponentially more information about people and their habits than even 200,000,000 books could ever contain. Another development Rider could have never foretold? How collecting data would reach far beyond the realm of university libraries to impact business decisions in nearly every industry.

The Association for Computing Machinery notes the first use of the phrase "big data" in a 1997 paper about the challenges of managing the vast amounts of unstructured data captured by businesses. In context, the phrase referred to data sets large enough to tax the capacities of main memory, local disk and remote disk. The solution? More resources, meaning they needed to acquire more computers to house an ever-increasing amount of data.¹ At that time, analysts had not yet begun to imagine what would be the eventual impact of big data on business.

Thanks to advances in online data transfer and storage protocols, including the recent emergence of cloud-based systems, many businesses are now able to store massive amounts of data about both their customers and their internal workflows, giving new meaning to the phrase "big data." With the increased storage capacity of the internet, data has become an asset, even to the smallest companies. In 2017, 48.4 percent of executives across industries reported measurable benefits due to big data investments.²

But what are these benefits? How does big data help businesses? Specifically, how does collecting data lead to improvements to a company's bottom line? Here is a glimpse at just a handful of possibilities.

Big data isn’t just useful for mining customer interactions. Data is also used by companies internally, leveraging focused analysis to cut costs and improve employee productivity and motivation.⁴ The search giant Google even has a "People Analytics" team within its human resources department. Their job? To compile and analyze employee data, then take the resulting insights to implement new ways of generating greater employee satisfaction and greater operational efficiency and productivity.⁵

In industries like retail and technology, the impact of data may seem obvious. But the reality is, data can impact businesses of any size and in almost any industry, including those you might not expect. Here are five industries that have already been revolutionized by big data applications.

Logistics: UPS runs 55,000 truck routes per day. If they're able to shave a mile off each of those routes, they stand to save $50 million annually. With a little mathematical optimization, this isn't impossible, and the benefits to UPS would be huge.⁶

Finance: Banks especially can benefit from data to improve fraud detection. Simply by having an account, customers generate massive amounts of data about their habits and usage. By comparing future usage against this existing body of information, banks can better predict and avoid fraud.⁷

Entertainment: Sometimes, it's nice to just let Netflix pick a movie for you. But how does Netflix manage to predict that just because you liked “Star Trek,” you might also like “Mission: Impossible”? Netflix has over 800 data engineers behind the scenes, crunching the numbers to figure out how you might like to spend your Friday night.⁸

Sports: Team managers in virtually every sport rely on stats to understand how players are performing and make crucial management decisions. With advances in wearable tech like the Fitbit, players are now able to share health and performance data with coaching staff, which allows more informed decision making on the field. But that’s not all. Off-field stats are also coming into play for team owners who want an edge in fan engagement. When the New England Patriots analyzed how fans interact with their website, they discovered the cheerleader page was the second most visited on the site, so they decided to add a “cheerleader cam” on game day.⁹ It’s a small part of the overall game experience, but demonstrates that a combination of on-field and off-field analytics can build a high-performing team that knows how to retain its fans.

Insurance: For an insurance company, the bottom line is the minimization of losses. Data analysts can help traditional insurance adjusters by spotting patterns that may indicate fraud, flagging claims that have the greatest likelihood of leading to lengthy and costly litigation and precisely predicting required loss reserves.¹⁰

75 years ago, data analysts worried about how many miles of shelves might be required to store all of the books they'd need to do their jobs. Now, the problem of storage has been so well handled that most businesses have more data than they know what to do with. That's where you come in.

As storage capacities increase and computer scientists invent ever more refined ways to capture data, analysts will have to be savvy not just in the science of data mining, but also in the art of business decision making. By understanding both, analysts will continue to be at the forefront of helping businesses reduce costs, improve employee performance and give their customers what they truly want.

Think you might be the business-minded data analyst the world needs? Explore the many reasons why it’s well worth your time to get an Online Master’s in Business Analytics, and do more with data in your career.