Build Real-Time Data Pipelines with SingleStore Streamliner
Data Intensity

Build Real-Time Data Pipelines with SingleStore Streamliner

SingleStore Streamliner is now generally available! Streamliner is an integrated SingleStore and Apache Spark solution for streaming data from real-time data sources, such as sensors, IoT devices, transactions, application data and logs. The SingleStore database pairs perfectly with Apache Spark out-of-the-box. Apache Spark is a distributed, in-memory data processing framework that provides programmatic libraries for users to work with data across a broad set of use cases, including streaming, machine learning, and graph data processing. SingleStore and Spark share many design principles: they are in-memory, distributed, and data-centric. Spark provides an amazing interface to the unique functionality in SingleStore: fast and durable transactions, a real-time hybrid row/column-oriented analytics engine, and a highly concurrent environment for serving complex SQL queries. The SingleStore Spark Connector, released earlier this year, allows Spark and SingleStore integration, facilitating bi-directional data movement between Spark and SingleStore. The connector generated a lot of interest from users who saw the benefits of using Spark for data transformation and SingleStore for data persistence. A consistent theme in the use cases we saw was the desire to use Spark to stream data into SingleStore with Spark Streaming. SingleStore Streamliner is the result of our work to productize this workflow into an easy, UI-driven tool that makes this process dead simple. Let’s review the thinking behind some of the decisions we made as we were developing Streamliner. Early work with Pinterest Pinterest showcased a Kafka+Spark+SingleStore solution in Strata+Hadoop World last February, which was a collaborative effort with SingleStore. See the Pinterest blog post and the Pinterest demo to learn more. The Pinterest solution leveraged Spark Streaming to quickly ingest and enrich data from Kafka, and then store it in SingleStore for analysis.
Read Post
How We Hire Remarkable Engineers
Engineering

How We Hire Remarkable Engineers

Read Post
Forrester
SingleStore Recognized In

The Forrester WaveTM

Translytical Data
Platforms Q4 2022

Seven Databases in Seven Weeks: SingleStore at CMU
Engineering

Seven Databases in Seven Weeks: SingleStore at CMU

Last week I gave a presentation as part of the “Seven Databases in Seven Weeks” seminar series designed to showcase new technologies available for front-end application developers. Hosted by Carnegie Mellon University, the series features leading developers of NoSQL and NewSQL database management systems. Watch the presentation below to learn about Hybrid Transactional and Analytical Processing (HTAP) and SingleStore’s key innovations that enable us to deliver on being a powerful, in-memory, HTAP-driven database.
Read Post