Author
Dale Deloy
Sales Engineer for SingleStore Central US and Canada. Architecting and delivering data solutions for customers.
Data Intensity
SingleStore Meets the Microservices Architecture
Microservices use distributed computing to isolate and scale application service components, and this is the hot trend for application development. SingleStore sees this trend continuing because of the prolific increase of real-time applications. These applications require deep analytics and a strategy for manageable components delivering speed and scale.
Each application service component has its own independent service deployment and flexible data model. The goal is to allow developers to focus on building business logic rather than coordinating with corporate models and infrastructure constraints.
Microservices has promise for improving developer productivity but also has known drawbacks that need to be addressed. Decisions and struggles with Microservices include distributed transactions, eventual consistency and high performance analytics. One approach to addressing these challenges is to define the most appropriate data repository for your services, which also provides the flexibility to modify the data model at a service level while also delivering high performance joins to other objects. In addition the repository will also need to provide high performance analytics across all the data within a domain.
By making a few base decisions and leveraging SingleStore as the data repository a highly flexible, scalable and performant solution can be delivered. The first choice is to leverage a single database table per service. The second choice is to leverage JSON as the model for your service data.
SingleStore and Microservices Architecture
Read Post
Case Studies
SingleStore Manages Smart Meter Data with Leading Gas and Electric Utility Enterprise
Smart gas and electric meters produce huge volumes of data. A small SingleStore cluster of 5 nodes easily handles massive quantities of data like the workloads from leading gas and electric utility enterprises.
Read Post
Engineering
Massive Data Ingest and Concurrent Analytics with SingleStore
The amount of data created in the past two years surpasses all of the data previously produced in human history. Even more shocking is that for all of that data produced, only 0.5% is being analyzed and used. In order to capitalize on data that exists today, businesses need the right tools to ingest and analyze data.
At SingleStore, our mission is to do exactly that. We help enterprises operate in today’s real-time world by unlocking value from data instantaneously. The first step in achieving this is ingesting large volumes of data at incredible speed. The distributed nature of the SingleStore environment makes it easy to scale up to petabytes of data! Some customers use SingleStore to process 72TB of data a day, or over 6 million transactions per second, while others use it as a replacement for legacy data warehouse environments.
SingleStore offers several key features for optimizing data ingest, as well as supporting concurrent analytics:
High Throughput
SingleStore enables high throughput on concurrent workloads. A distributed query optimizer evenly divides the processing workload to maximize the efficiency of CPU usage. Queries are compiled to machine code and cached to expedite subsequent executions. Rather than cache the results of the query, SingleStore caches a compiled query plan to provide the most efficient execution path. The compiled query plan does not pre-specify values for the parameters, which allows SingleStore to substitute the values upon request, enabling subsequent queries of the same structure to run quickly, even with different parameter values. Moreover, due to the use of Multi-Version Concurrency Control (MVCC) and lock-free data structures, data in SingleStore remains highly accessible, even amidst a high volume of concurrent reads and writes.
Query Execution Architecture
SingleStore has a two-tiered architecture consisting of aggregators and leaves. Aggregators act as load balancers or network proxies, through which SQL clients interact with the cluster. Aggregators store metadata about the machines in the cluster and the partitioning of the data. In contrast, leaves function as storage and compute nodes.
Read Post
Case Studies
Rethinking Lambda Architecture for Real-Time Analytics
Big data, as a concept and practice, has been around for quite some time now. Most companies have responded to the influx of data by adapting their data management strategy. However, managing data in real time still poses a challenge for many enterprises. Some have successfully incorporated streaming or processing tools that provide instant access to real-time data, but most traditional enterprises are still exploring options. Complicating the matter further, most enterprises need access to both historical and real-time data, which require distinct considerations and solutions.
Of the many approaches to managing real-time and historical data concurrently, the Lambda Architecture is by far the most talked about today. Like the physical aspect of the Greek letter it is named for, the Lambda architecture forks into two paths: one is a streaming (real-time) path, the other a batch path. Thus, it accommodates real-time high-speed data service along with an immutable data lake. Oftentimes a serving layer sits on top of the streaming path to power applications or dashboards.
A Fork in the Road
Many Internet-scale companies, like Pinterest, Zynga, Akamai, and Comcast have chosen SingleStore to deliver the high-speed data component of the Lambda architecture. Some customers have chosen to fork the input stream in order to push data into SingleStore and a data lake, like HDFS, in parallel.
Here is an example of the Comcast Lambda Architecture:
Read Post
