Product
SingleStoreDB Self-Managed 5 Ships with LLVM-based Code Generation for SQL Queries
We are proud to announce general availability for SingleStoreDB Self-Managed 5 today. A key milestone in this release is a full fledged SQL compiler resulting in faster query processing across the board. Making this happen was a result of several months of hard work, which featured a large uplift of our existing database execution engine.
This new SQL compiler is using LLVM for code generation. This modern compilation strategy is capable of supporting dynamic compilation of programming languages.
In addition to performance, SingleStoreDB Self-Managed 5 has many new features and smarter query optimization.
Building the Fastest SQL Compiler
Query processing in database engines such as Oracle, SQL Server, or Postgres can benefit significantly from code generation, especially for complex analytical workloads. However, too often, code generation does not get enough attention, teams have lacked enough compiler experts, or the need was not acute. Today, with the rise of big data, proper code generation is critical, especially in memory optimized systems where I/O is no longer a bottleneck.
Building code generation is a compilers project. We knew it would be an extremely ambitious engineering undertaking, but that the competitive advantages gained would greatly outweigh the cost.
Building a new SQL compiler within months requires a world class team. For us it started with Drew Paroski, who came to SingleStore as an architect specifically to design and lead our code generation efforts. He spent his initial weeks on the project understanding the current code and its shortcomings and prototyping new designs, working closely with SingleStore engineers Michael Andrews and David Stolp (aka Pieguy).
One big design decision was how far we should push code generation away from the classic volcano model for query processing. The trade-off was between using the volcano model and generating code for just parts of the SQL query, or abandoning that model to generate code for the whole query, including error handling and corner cases. We chose the latter approach for its composability and precise control over performance. This required more work, but ultimately it provides a significantly better experience for our customers.
Download SingleStoreDB Self-Managed 5 Today
SingleStoreDB Self-Managed 5 is generally available now. Download it now and enjoy its real-time capabilities. You can build real-time data pipelines and take them to unprecedented levels of scale and sophistication. Our customers solve their hardest data problems with SingleStore and we are thrilled to continue driving database innovation for the industry.
Read Post
Product
Introducing SingleStoreDB Self-Managed 5 Beta
A post from our co-founders Eric Frenkiel, CEO and Nikita Shamgunov, CTO
Today SingleStoreDB Self-Managed 5 Beta is publicly available! SingleStore customers have been able to achieve remarkable results with our database, and we look forward to feedback on this upcoming release from our user community.
Read Post
Product
The Lambda Architecture Simplified (Guide)
Modern businesses need to support an increasing variety of data workloads and uses cases that require both fault-tolerance and scalability. This has led to widespread adoption of the Lambda Architecture.
Lambda is designed to model everything that happens in a complex computing system as an ordered, immutable log of events. Processing the data (for example, totaling the number of web visitors or transactions) is completed as a series of transformations that output to new tables or streams.
SingleStore combines database and data warehouse workloads, enabling both transactional processing and analytics. Gartner refers to this as HTAP or hybrid transaction/analytical processing. SingleStore often fulfills the speed layer of the Lambda architecture, providing in-memory performance to ingest and process streaming data. In our experience with Lambda implementations, we find that most organizations get hung up on details of the Lambda Architecture, introducing unnecessary technologies and workarounds to fit within the model. It doesn’t have to be this way.
Free Guide: The Lambda Architecture Simplified
To make sense of the Lambda Architecture and get you on track to a successful implementation, we are launching a new guide: The Lambda Architecture Simplified. This guide demystifies complexity surrounding the Lambda Architecture, and will reframe the way you think about the model by providing simplified data frameworks and real world use cases.
You’ll learn
What defines the Lambda Architecture, broken down by each layerHow to simplify the Lambda Architecture by consolidating the speed layer and batch layer into one systemHow to implement a scalable Lambda Architecture that accommodates streaming and immutable dataHow companies like Comcast and Tapjoy use Lambda Architectures in production
Read Post
Product
Why SingleStore Placed a Bet on SQL
In the technology industry, when products or innovations last for a long period of time, they are often here to stay. SQL is a great example of this – it has been around for over 30 years and is not going away anytime soon. When Eric Frenkiel and Nikita Shamgunov founded SingleStore in 2011, they were confident in choosing the SQL relational model as the foundation for their database. But the database industry during that era was clamoring around NoSQL, lauding it as the next great innovation, mostly on the themes of scalability and flexibility. When SingleStore graduated from Y Combinator, a prominent tech incubator, that same year it was the only distributed SQL database in a sea of non-SQL offerings.
SQL has since proven its ability to scale and meet today’s needs. Business analysts seek easy interfaces and analytics for the problems they are trying to solve. Customers want SQL, and like Dan McCaffrey, VP of Analytics at Teespring, happily cite that as a reason for choosing SingleStore. Dan states: “What I really liked about SingleStore was the ANSI SQL support for dynamic querying needs at scale, in a reliable, robust, easy-to-use database.”
Now, with the reconquista of SQL, we are seeing two funny things happening in the market.
One, companies that monetize the Hadoop Distributed File System are adding layers of SQL on top of the Hadoop platform. Two, NoSQL databases are incorporating SQL. NoSQL databases are essentially key value stores, and adding SQL gives them the ability to do some analytics. However, adding a SQL layer is no substitute for the richness of advanced SQL that was built into the SingleStore database. SQL as a layer is just a band-aid solution.
The Gartner Magic Quadrant for Operational Database Management Systems
The latest Gartner Magic Quadrant for Operational Database Management Systems confirms something we have been championing for a while:
“By 2017, all leading operational DBMSs will offer multiple data models, relational and NoSQL, in a single DBMS platform… by 2017, the “NoSQL label will cease to distinguish DBMSs, which will result in it falling out of use.”
For years, SingleStore has supported both a fully-relational SQL model, and a “NoSQL” model, together in the same cluster of machines. This was a bet made by our original engineering team – they understood the powerful appeal of SQL to business users, but also knew the value of the “NoSQL” model of vast scale. For that reason, SingleStore is multi-model, and databases of the future will need to support multiple operations to survive.
Our co-founders were confident back in 2011, and we remain confident with validation from the market, research firms like Gartner, and most importantly from our customers, that SQL is the path forward. We will continue to hone the SQL aspects of our database and champion the lingua franca of the database world.
Read Post
Product
Digital Ocean Tutorial Gets You Up and Running in Minutes
As fun as it is to squirrel around inside the guts of some new technology, it’s sometimes nice to follow a recipe and end up with something that Just Works. For years, Digital Ocean, an up and coming cloud provider, has been producing quality tutorials on how to set up cool software on their virtual machines. Today Ian Hansen published an in-depth tutorial on setting up a three-node SingleStore cluster. Check it out here.
Go to the Digital Ocean tutorial and learn how to install SingleStore in minutes
Once the cluster is running, Ian walks through our DB Speed Test. He then dives into interacting with SingleStore using the stock MySQL client and handling structured and instructed data with our JSON datatype. The next tutorials in the series will deal with sharding strategies, replication, and security.
We’re also lucky to have Ian here at Strata / Hadoop World in NYC to give a talk called “Big Data for Small Teams”, about how Digital Ocean uses SingleStore to unify and analyze their clickstream data with a minimum of fuss.
Read Post
Product4 min Read
Understanding SingleStore in 5 Easy Questions
The funny thing about SingleStore is that it is simultaneously familiar and leading edge. On one hand, it is a relational database…
Read Post
Product
SingleStore Community Edition Available on AWS and Azure Marketplaces
Good news, everyone! Today we’re releasing SingleStore Community Edition on the Amazon AWS and Microsoft Azure Marketplaces. Many of our customers run SingleStore in the cloud, and often their entire infrastructure. An even larger number try SingleStore first on the cloud before pulling it into their production systems.
A great example is VCare, currently using SingleStore deployed on AWS cloud instances to power its online, real-time charging platform. VCare supports Mobile Virtual Network Operators (MVNOs), whose customers pay for a certain number of minutes. With its end-to-end software solution, VCare can verify that there is sufficient balance available for users to make phone calls or send text messages. The entire process takes place in milliseconds to ensure there is no lapse in service, supported by the cloud.
Now it is easier than ever to get started. With just a few clicks you can launch a “cluster in a box” on a single virtual machine, and run our DB Speed Test within minutes. The DB Speed Test is a 30-second performance benchmark that comes with SingleStore Ops, and current frequently push over 1 million inserts per second on a single virtual machine.
Read Post
Product
Deploy SingleStore with Mesosphere
SingleStore is now certified for deployment on the Mesosphere DCOS (Datacenter Operating System). With a few simple commands, users can launch a SingleStore cluster on DCOS.
The Mesosphere DCOS (Datacenter Operating System) is a commercially supported product built on top of Apache Mesos that serves as a datacenter abstraction layer. DCOS deploys distributed applications with a few simple command-line interface (CLI) steps, and handles application provisioning, resource management, and fault tolerance seamlessly. It also centralizes management of a distributed system in a single interface for easy deployment.
SingleStore is a relational, in-memory database optimized for multi-machine deployment. Its distributed nature makes it a good match for DCOS.
The package needed to deploy SingleStore on DCOS can be found in the Mesosphere Universe repository, which allows all DCOS users to install SingleStore anytime simply by running `dcos package install memsql` from a DCOS console. SingleStore leverages the Marathon scheduler to pull a SingleStore Docker image to deploy the SingleStore software, which is tightly integrated with DCOS.
SingleStore supports a variety of deployment topologies, and the Mesosphere DCOS adds another option to this growing list. SingleStore can be deployed on:
bare metal machines in your datacentervirtual machines in your datacenterDocker containersAmazon EC2, Microsoft Azure and other public cloud platform providersMesosphere DCOS
Additional Links
Read the full press release here: Mesosphere Launches Developer Program, VIP Partner Program and SDK for Building Distributed Datacenter-Scale Services on Mesosphere DCOS
Try SingleStore today! Run ``dcos package install memsql`` on your DCOS system, or download the free SingleStore Community Edition with unlimited scale and capacity.
Read Post
Product
New SingleStore Ops Comes with Database Speed Test
SingleStore Ops 4.0.31 is now available for download! For this release, we have maintained a focus on providing the smoothest possible on-ramp for developers to quickly get productive with SingleStore. We believe enterprise-class software should be easy to install or use.
Here are the new features of the latest SingleStore Ops release, available today. Note that aside from features, this release includes bug fixes, and is a free upgrade for existing SingleStore Ops users.
Read Post
Product
SingleStoreDB Self-Managed 4: Market and Strategy CEO Q&A
SingleStoreDB Self-Managed 4 represents a leap forward in relational database innovation with new capabilities such as real-time geospatial intelligence and high-throughput connectivity to Apache Spark. It also includes a free forever, unlimited capacity Community Edition, enhancements to the Optimizer for distributed joins, and a new version of the SingleStore Ops management framework.
Start Using SingleStore Community Edition Now
Unlimited scale and capacity. Free forever.
Download Community Edition →
As Big Data abounds, understanding every company and product can be tricky. To make easier, here are a few questions and answers about SingleStore.
The database landscape is big. Where does SingleStore fit?
SingleStore is the leading database for real-time transactions and analytics. That means we’re operational by nature, much more akin to SQL Server, Oracle, or SAP HANA than Hadoop. SingleStore is also:
In-Memory
Providing the utmost performance for today’s demanding workloadsDistributed
Enabling cost-effective, horizontal scale-out on-premises or in the cloudRelational and multi-model
Allowing companies to use in-house SQL tools, applications, and knowledgeWith JSON and Geospatial data formats supportedSoftware
Designed to run on commodity hardware for costs savings
You talk about transactions and analytics. Can you explain in more detail?
To meet real-time demands, companies must be able to capture information across millions to hundreds of millions of sensors or mobile applications. They also want to analyze that data up to the last transaction. So with real-time operations you don’t have the luxury or the pain of ETL. You need to bypass ETL by transacting and analyzing in a single database designed to support these concurrent workloads. It boils down to analytics on changing datasets, today’s critical capability.
SingleStoreDB Self-Managed 4 includes the SingleStore Spark Connector and the SingleStore Loader for HDFS and S3. How should we think about SingleStore with Spark and Hadoop?
SingleStore and Spark work well together as they both have memory-optimized, distributed frameworks. Spark is a processing framework that enables real-time transformation and advanced analytics, but Spark itself does not have a storage or persistence ability. By storing data permanently in SingleStore, customers get an easy way to build operational applications, and the ability to take operational data and ‘round-trip’ it to Spark for advanced analytics.
With Hadoop, customers frequently build simplified Lambda architectures using SingleStore. All data can go directly to HDFS for long term archiving. Simultaneously, data can go directly into SingleStore, bypassing Hadoop for the real-time path. Should historical data be needed for analysis, SingleStore can import that data from HDFS using the SingleStore Loader.
Many folks say not everything needs to be in-memory. How do you respond?
We agree! While in-memory computing remains critical for many applications, the pace of data growth still eclipses memory-only solutions. This is exactly why SingleStore ships with an integrated column-store optimized for disk and flash. Now customers can create tables entirely in-memory, or across a combination of memory and disk and flash. And starting with SingleStoreDB Self-Managed 4, we license the software based on DRAM capacity so the use of disk or flash storage in the column store is unlimited at no additional cost.
We are big believers that customers will see the benefit of placing data in a structured format from the beginning, and the SingleStore column store will let them do that affordably.
You have always focused on SQL. All the while NoSQL has received a lot of discussion. Where is SingleStore in this?
SQL is the lingua franca of the data processing world, having been well adopted over its decades-long history. Companies can build SQL applications quickly, and then immediately use in house analytics tools and practices to derive insights.
Prevailing wisdom used to be that SQL could not scale. That is untrue and companies are now discovering the powerful combination SingleStore delivers of a relational, in-memory, distributed database with speed, scale, and simplicity.
The future is multi-model and SingleStore also includes JSON and Geospatial datatypes to fulfill this promise.
The Community Edition is freely available but not open-source. What strategy is SingleStore pursuing?
Our strategy is to build a scalable software business. This means running on all the platforms customers want to use, including public and private clouds, in the way they want to use them. Community Edition is a way to allow more people to use SingleStore without limitations on time or capacity.
There is a wide spectrum of commercial database offerings. On one extreme there are proprietary hardware / software combinations. Our belief is that proprietary hardware is a down elevator. On the other end of the spectrum there are open-source projects with businesses built around them. But that is essentially selling consulting hours, not software. There are many successful businesses in the middle of the spectrum, such as MySQL’s dual commercial / open-source licensing, and we maintain open-source projects, including SingleStore Loader for Hadoop and S3, a large number of general-purpose Python packages, and the SingleStore Spark Connector.
At SingleStore, we want to offer performance that trumps legacy vendors, deliver it a fraction of the cost, and provide complete deployment choice across hardware, data centers, and clouds. In doing so we hope to help more companies achieve their goals to become real-time enterprises.
Read Post
Product
Launching Our Community Edition
We started SingleStore with a vision to enable every company to be a real-time enterprise, and to do that by delivering the leading real-time database for transactions and analytics. Since then, the forces shaping our digital economy have only added wind to our sails. The world is more mobile, interconnected, interactive, and on the cusp of several industry transformations such as the Internet of Things.
Real-time processing is the secret to keeping up, and in-memory solutions are the foundation. Yet existing options have been too expensive or too complex for companies to adopt.
That changes today with the release of SingleStoreDB Self-Managed 4 and our new Community Edition, a free unlimited capacity, unlimited scale offering of SingleStore that includes all transactional and analytical features.
By sharing SingleStore capabilities with the world, for free, we expect many developers and companies will have a chance to explore what is possible with in-memory computing. As the pace of business advances, SingleStore will be there.
Start Using SingleStore Community Edition Now
Unlimited scale and capacity. Free forever.
Download Community Edition →
We hope you enjoy working with our Community Edition. Please feel free to share feedback at our Community Edition Help page.
Eric Frenkiel, CEO and co-founder, SingleStore
Nikita Shamgunov, CTO and co-founder, SingleStore
Community FAQ
Read Post
Product
Celebrating SingleStore Availability Two Years In
Today, I couldn’t be more excited to mark the two year anniversary of SingleStore general availability! SingleStore began with a simple idea to build a new database that would give any company the ability to operate in real-time and make their business more data-driven, responsive, and scalable. Since releasing SingleStore, it’s been an amazingly fun journey as the company has grown by leaps and bounds every quarter.
To celebrate our second birthday, I wanted to take a brief moment to reflect on what we’ve been able to accomplish in the two years since releasing SingleStore.
People
SingleStore started in the Y-Combinator winter class of 2011 with just two people – co-founder and CTO Nikita Shamgunov, and myself. Since then, we’ve grown the company to more than 50 people who bring great experience, energy, and passion to the company. We’ve also added database visionaries like Jerry Held and Chris Fry to our executive team to help us see our vision come to fruition.
Customers
We’ve added 40+ enterprise customers over the past 2 years, including top brands like Comcast, Samsung and Shutterstock. It’s been incredibly rewarding to see our customers use SingleStore in ways we never imagined, truly pushing the boundaries of what is possible in Big Data.
Product
Since launching with general availability in 2013, we’ve expanded the SingleStore platform to scale with growing market demand. Major additions to the platform include:
Going beyond memory by including a flash-optimized column store that is closely integrated with the in-memory row store to provide a single database for real-time and historical analyticsWorking with Apache Spark by shipping a bi-directional connector to operationalize Spark models and resultsIncorporating real-time geospatial intelligence to help customers build location-aware applications and analytics
What’s Next?
The most exciting times are still ahead!
Big data has been traditionally thought of as a mechanism for extracting insights from yesterday’s data. We seek to change that way of thinking, empowering businesses to be more responsive by operating with real-time data in the here and now. As demand for real-time and in-memory databases increases, we plan to be there helping customers achieve phenomenal results.
Read Post
Product
Harnessing the Enterprise Capabilities of Spark
As more developers and data scientists try Apache Spark, they ask questions about persistence, transactions and mutable data, and how to deploy statistical models in production. To address some of these questions, our CEO Eric Frenkiel recently wrote an article for Data Informed explaining key use cases integrating SingleStore and Spark together to drive concrete business value.
The article explains how you can combine SingleStore and Spark for applications like stream processing, advanced analytics, and feeding the results of analytics back into operational systems to increase efficiency and revenue. As distributed systems with speedy in-memory processing, SingleStore and Spark naturally complement one another and form the backbone of a flexible, versatile real-time data pipeline.
Read the full article here.
Get The SingleStore Spark Connector Guide
The 79 page guide covers how to design, build, and deploy Spark applications using the SingleStore Spark Connector. Inside, you will find code samples to help you get started and performance recommendations for your production-ready Apache Spark and SingleStore implementations.
Download Here
Read Post
Product
Operationalizing Spark with SingleStore
Combining the data processing prowess of Spark with a real-time database for transactions and analytics, where both are memory-optimized and distributed, leads to powerful new business use cases. SingleStore Spark Connector links at end of this post.
Data Appetite and Evolution
Our generation of, and appetite for, data continues unabated. This drives a critical need for tools to quickly process and transform data. Apache Spark, the new memory-optimized data processing framework, fills this gap by combining performance, a concise programming interface, and easy Hadoop integration, all leading to its rapid popularity.
However, Spark itself does not store data outside of processing operations. That explains that while a recent survey of over 2000 developers chose Spark to replace MapReduce, 62% still load data to Spark with the Hadoop Distributed File System and there is a forthcoming Tachyon memory-centric distributed file system that can be used as storage for Spark.
But what if we could tie Spark’s intuitive, concise, expressive programming capabilities closer to the databases that power our businesses? That opportunity lies in operationalizing Spark deployments, combining the rich advanced analytics of Spark with transactional systems-of-record.
Introducing the SingleStore Spark Connector
Meeting enterprise needs to deploy and make use of Spark, SingleStore introduced the SingleStore Spark Connector for high-throughput, bi-directional data transfer between a Spark cluster and a SingleStore cluster. Since Spark and SingleStore are both memory-optimized, distributed systems, the SingleStore Spark Connector benefits from cluster-wide parallelization for maximum performance and minimal transfer time. The SingleStore Spark Connector is available as open source on Github.
SingleStore Spark Connector Architecture
There are two main components of the SingleStore Spark Connector that allow Spark to query from and write to SingleStore.
A `SingleStoreRDD` class for loading data from a SingleStore queryA `saveToSingleStore` function for persisting results to a SingleStore table
Read Post
