SingleStore Introduces Direct Ingest from Amazon S3 and HDFS with SingleStore Loader

SingleStore Introduces Direct Ingest from Amazon S3 and HDFS with SingleStore Loader

SAN FRANCISCO, December 30, 2014 - SingleStore, the leader in real-time and historical Big Data analytics, today introduced a new productivity tool that dramatically improves data ingest from popular data stores like Amazon Web Services S3 and the Hadoop Distributed File System (HDFS). Whereas typical data loading often requires multiple steps, the SingleStore Loader enables direct streaming from the originating datastore in a single transfer. SingleStore allows for multiple parallel input streams, further increasing performance and reducing time-consuming repetitive operations. The SingleStore Loader is also available as open source, providing developers the ability to adapt it to their favorite data source.

“The SingleStore Loader is another innovation of simplifying SingleStore implementations with production data workflows,” said Nikita Shamgunov, Chief Technology Officer and co-founder, SingleStore. “After working with customers during their SingleStore deployments, we found a simple way to eliminate steps in data pipelines, saving them time and reducing complexity,” he continued. “By streaming directly from popular data stores like Amazon S3 and HDFS, we offer customers an easy way to get started, and an efficient way to integrate the real-time transactions and analytics of SingleStore into existing environments.”

Data ingest often involves multiple files or objects, particularly with scalable storage services like S3. In certain cases, customers may have hundreds or thousands of files to import. Other import methods operate at the job level, meaning that if just one file fails, the entire job must be restarted. SingleStore Loader supports loading batches of thousands of files or objects automatically without having to specify files individually. This enables a synchronization path, only loading new or changed files as they are updated, or restarting at a specific file in case of any import issues.

SingleStore Loader automates loading processes and enables queuing of jobs further simplifying ingest. Now SingleStore administrators have a thorough loading solution that eliminates steps from the process, scales performance across a distributed database, monitors file level granularity, and offers connectivity to common data stores like S3 and HDFS.

To download SingleStore Loader, visit the SingleStore Github site.

Read the technical blog post at:

About SingleStore

SingleStore enables companies to merge real-time and historical Big Data analytics with its distributed in-memory database. SingleStore instantly accesses data through a familiar SQL interface on a horizontally scalable architecture using commodity hardware across on-premises or cloud deployments.

Innovative enterprises with data intensive environments use SingleStore to accelerate insights, extract previously untapped value in their data, and drive new revenue opportunities. Based in San Francisco, SingleStore is a Y Combinator company funded by Accel Partners, Khosla Ventures, First Round Capital, and Data Collective.

Follow SingleStore @SingleStoreDB or visit at

Media Contact

Gina von Esmarch
VP Marketing & Communications