The Real-Time Track at Gartner BI
SingleStore is headed to Gartner Business Intelligence and Analytics Summit next week. We’ll be focusing on real-time analytics with our own VP of Engineering kicking off a live demo on day one of the show.
For the following days, we’ll be tracking the hot topics in real-time analytics, the new Magic Quadrant for Data Warehouses, IoT, Spark, Relational Databases, In-Memory Computing, and Machine Learning
5 Big Data Themes – Live from the Show Floor
We spent last week at the Big Data Innovation Summit in Boston. Big data trade shows, particularly those mixed with sophisticated practitioners and people seeking new solutions, are always a perfect opportunity to take a market pulse.
Here are the big 5 big data themes we encountered over the course of two days.
Real-Time Over Resuscitated Data
The action is in real time, and trade show discussions often gravitate to deriving immediate value from real-time data. All of the megatrends apply… social, mobile, IoT, cloud, pushing startups and global companies to operate instantly in a digital,connected world.
While there has been some interest in resuscitating data from Hadoop with MapReduce or SQL on Hadoop, those directions are changing. For example, Cloudera recently announced the One Data Platform Initiative, indicating a shift from MapReduce
this initiative will enable [Spark] to become the successor to Hadoop’s original MapReduce framework for general Hadoop data processing
With Spark’s capabilities for streaming and in-memory processing, we are likely to see a focus on those real-time workflows. This is not to say that Spark won’t be used to explore expansive historical data throughout Hadoop clusters.
But judge your own predilection for real-time and historical data. Yes, both are important, but human beings tend to have an insatiable desire for the now.
Data Warehousing is Poised for Refresh
When the last wave of data warehousing innovation hit mainstream, there was a data M&A spree that started with SAP’s acquisition of Sybase in May 2010. Within 10 months, Greenplum was acquired by EMC, Netezza by IBM, Vertica by HP, and Aster by Teradata.
Today, customers are suffering economically with these systems which have become expensive to maintain and do not deliver the instant results companies now expect.
Applications like real-time dashboards push conventional data warehousing systems beyond their comfort zone, and companies are seeking alternatives.
Getting to ETL Zero
If there is a common enemy in the data market, it is ETL, or the Extract, Transform, and Load process. We were reminded of this when Riley Newman from Airbnb mentioned that
ETL was like extracting teeth…no one wanted to do it.
Ultimately, Riley did find a way to get it done by shifting ETL from a data science to a data engineering function (see final theme below), but I have yet to meet a person who is happy with ETL in their data pipeline.
ETL pain is driving new solution categories like Hybrid Transactional and Analytical Processing, or HTAP for short. In HTAP solutions, transactions and analytics converge on a single data set, often enabled by in-memory computing. HTAP capabilities are the forefront of new digital applications with situational awareness and real-time interaction.
The Matrix Dashboard is Coming
Of course, all of these real-time solutions need dashboards, and dashboards need to be seen. Hiperwall makes a helpful solution to tie multiple monitors together in a single, highly-configurable screen. The dashboards of the future are here!
Five Data Persistence Dilemmas CIOs Will Face
At SingleStore, we see an in-memory, distributed approach to big data as the path forward to cost-effective deployments. Recently, Gartner released a report titled “Five Data Persistence Dilemmas That Will Keep CIOs Up at Night”, which reinforces this approach to data management.The report outlines three key impacts of utilizing new technologies across HTAP, or Hybrid Transaction/Analytical Processing, for in-memory processing, the compromises of NoSQL DBMSs, and the growing importance of agile cloud computing approaches.Download a complimentary copy of the Gartner Report: Five Data Persistence Dilemmas That Will Keep CIOs Up at NightKey Impacts from the Gartner ReportThe convergence of transaction and analytic database systems in hybrid transaction/analytical processing (HTAP) systems that use in-memory processing reduces the need for separate dedicated environments and shortens the time to value for new data, but it requires IT Leaders to make process compromises and changes to applications to maximize ROI.NoSQL DBMSs compromise a priori data models and strong levels of consistency to offer IT leaders high-throughput operations and scale-out architectures.Agile deployment approaches like cloud computing will present new opportunities that IT leaders and line-of-business heads must seize.The CIO DilemmasThe five dilemmas covered in the report generate a number of questions that CIOs must ask with any new database technology. We receive these questions daily from customers seeking to maximize opportunities with HTAP, scalable SQL databases, and flexible cloud deployments.The Single-Database DilemmaFor decades, data processing has been split into databases for Online Transaction Processing and data warehouses for Online Analytical Processing. HTAP, largely enabled by in-memory computing, collapses the single database model and allows for the definition of new classes of applications, like those that fuse real-time and historical analysis.The HTAP Adoption DilemmaMoving from split OLTP and OLAP to converged HTAP requires thorough cost and capacity planning that does not happen overnight. Fully realizing the benefits of HTAP means transactions and analytics are easily integrated with existing or net new applications. HTAP results in less precomputation and more real-time queries.The Consistency DilemmaNoSQL databases gave up traditional consistency, and abandoned SQL, to achieve scalability. Fortunately, you can have scalability, performance, simplicity and SQL with an in-memory database like SingleStore.The Schema DilemmaYou can define your structure up front, or define it later. However, multi-model databases like SingleStore support structured SQL and semi-structured data types like JSON, so you get the best of both worlds.The Cloud DilemmaWhile some database offerings restrict deployment choice, SingleStore can be deployed on-premises or in the cloud.Try HTAP, Scalable SQL, and Cloud Databases TodayIf you would like a hands-on look at HTAP, scalable SQL, and cloud deployments with in-memory databases, try SingleStore Community Edition, available for free with unlimited capacity and scale. If you would like support or high availability features, try SingleStore Enterprise Edition free for 30 days.
