Predictions 2016: the Impact of Real-Time Data


Eric Frenkiel

SingleStore Co-Founder

Predictions 2016: the Impact of Real-Time Data

prediction-1-the-industrial-internet-moves-to-real-time-data-pipelinesPrediction 1. The industrial internet moves to real-time data pipelines

The industrial internet knits together big data, machine learning, and machine-to-machine communications to detect patterns and adjust operations in near real time. Soon the industrial internet will expand by definition to include the Internet of Things.

The detection of patterns and insights often comes with a price: time. While the goal of machine learning is to develop models that will prove useful, dealing with large data sets means it can take days, weeks or months to reach meaningful discoveries.

We predict that in the very near future, real-time data streams will transform what is possible across the industrial internet, so users can ask critical questions, adjust a process, or see a pattern in the moment. Entire industries such as energy, pharmaceutical and even agriculture will be dramatically impacted by the ability to analyze real-time and historical data together to make business decisions faster.

prediction-2-consumer-visibility-into-business-gets-granularPrediction 2. Consumer visibility into business gets granular

The world today moves at a different pace than a generation ago. Applications on handheld devices that move us through our day tell us where to eat, how to get from point A to point B, what is the fastest route, everything that is happening in the world, and even what our friends are buying. Data is driving the course of business – and dramatically impacting the consumer experience.

We predict that in a few short years, consumer visibility into business operations will get more granular. For example, look at the transparency that already exists with companies such as Blue Apron and FedEx. Not only do we know exactly what is on the menu week to week at Blue Apron, we can opt out if it is something we do not like, or adjust the delivery times. And FedEx allows consumers to track the entire journey of a package and sometimes even reroute a package to a new delivery destination. More and more companies will adopt transparency for consumers, and in doing so, will build brand loyalty and satiate growing consumer appetite for on-demand services.

prediction-3-the-cost-of-doing-business-declinesPrediction 3. The cost of doing business declines

Just a few years ago the cost of storage was a board room conversation, where CIOs had to justify the rising cost of IT associated with growing data volumes. For many CIOs, storage was an IT line item that was on track to outpace profitability.

Today, the conversation around data storage has changed. Storage is cheap and highly accessible—any business unit within an organization can tap into the cloud. Access to commodity hardware makes rapidly scaling a business possible.

The cost of doing business will further decline as in-memory technologies set computing on a new course. While companies like Amazon provide access to more than a terabyte of memory for just a few dollars an hour via public or private clouds, other companies have created technology that provides relatively low-cost access to terabytes of non-volatile memory, which developers can use instead of storage. In-memory databases use vast stores of memory close to the compute to rapidly process data. Access to more memory means that programmers will be able to write different types of software—propelling the industry toward what is perhaps a new era of applications built on commodity hardware.

We are already seeing verticalized trends for data analytics and applications that can serve up real-time value across healthcare, manufacturing, and retail. What’s next?

prediction-4-the-crowdsourcing-of-analyticsPrediction 4. The crowdsourcing of analytics

The world of artificial intelligence (AI) used to lie solidly in the hands of physicists, scientists and researchers and well beyond mass population. Today, AI has shifted and is empowering people all over the world to participate in the analytics process. Crowdflower, for example – now Appen – blends a human-in-the-loop process alongside data science and machine learning to generate insights. Kaggle, another company crowdsourcing for analytics, has built one of the world’s largest communities of data scientists to solve complex challenges through a competitive approach to data science.

Data analysis will be more pervasive, and new applications that empower the data collection process will be broadly embraced. Consider the power of Waze and INRIX, both in use today to crowdsource traffic congestion. While the requirement is the participation of a social community at large, the upside potential is felt much more broadly. The same data collection process could be applied to many more applications to affect and improve society.