Author
Gary Orenstein
Former Chief Marketing Officer, SingleStore
Trending
Real-Time Geospatial Intelligence with Supercar
Today, SingleStore is showcasing a brand new demonstration of real-time geospatial location intelligence at the Gartner Business Intelligence and Analytics Summit in Las Vegas.
The demonstration, titled Supercar, makes use of a dataset containing the details of 170 million real world taxi rides. By sampling this dataset and creating real-time records while simultaneously querying the data, Supercar simulates the ability to monitor and derive insights across hundreds of thousands of objects on the go.
Read Post
Trending
In The End We Seek Structure
In Short:
A range of assumptions led to a boom in NoSQL solutions, but in the end, SQL and relational models find their way back as a critical part of data management.
In the End We Seek Structure. Why SQL and relational models are back as a critical part of data management – Click to Tweet
Background
By the mid 2000s, 10 years into the Netscape-inspired mainstream Internet, webscale workloads were pushing the limits of conventional databases. Traditional solutions could not keep up with a myriad of Internet users simultaneously accessing the same application and database.
At the time, many websites used relational databases like MySQL, SQL Server from Microsoft, or Oracle. Each of these databases relied on a relational model using SQL, the Structured Query Language, which emerged nearly 40 years ago and remains the lingua franca of data management.
Genesis of NoSQL
Scaling solutions is hard, and in particular scaling a relational, SQL database proved particularly challenging, in part leading to the emergence of the NoSQL movement.
FIGURE 1: Interest in NoSQL 2009 – 2015 Source: Google Trends
Read Post
Trending
SingleStore at Spark Summit East
We are happy to be in New York City this week for Spark Summit East. We will be sharing more about our new geospatial capabilities, as well as the work with Esri to showcase the power of SingleStore geospatial features in conjunction with Apache Spark.
Last week we shared the preliminary release of SingleStore geospatial features introduced at the Esri Developer Summit in Palm Springs. You can read more about the live demonstration showcased at the summit here.
The demonstration uses the “Taxistats” dataset: a compilation of 170 million real-world NYC taxi rides. It includes GPS coordinates of the pickup and dropoff, distance, and travel time. SingleStore is coupled with the new version of Esri’s ArcGIS Server, which has a new feature to translate ArcGIS queries into external database queries. From there we generate heatmaps from the raw data in sub-second time.
This week we launched the official news release of SingleStore geospatial capabilities.
By integrating geospatial functions, SingleStore enables enterprises to achieve greater database efficiency with a single database that is in-memory, linearly scalable and supports the full rage of relational SQL and geospatial functions. With SingleStore, geospatial data no longer remains separate and becomes just another data type with lock-free capabilities and powerful manipulation functions.
Read Post
Trending
SingleStore at the AMP Lab
Please join us next week as two members of the SingleStore engineering team present at the AMPLab at Berkeley on Wednesday March 11th from 12:00pm to 1:00pm.
AMP SEMINAR
Ankur Goyal and Anders Papitto, SingleStore,
A Distributed In-Memory SQL Database
Wednesday 3/11, Noon, 405 Soda Hall, Berkeley
Talk Abstract
This talk will cover the major architectural design decisions with discussion on specific technical details as well as the motivation behind the big decisions. We will cover lockfree, code generation, durability/replication, distributed query execution, and clustering in SingleStore. We will then discuss some of the new directions for the product, including some ideas on leveraging Spark.
Speakers
Ankur Goyal is the Director of Engineering at SingleStore. At SingleStore he has focused on distributed query execution and clustering, but has touched most of the engine. His areas of interest are distributed systems, compilers, and operating systems. Ankur studied computer science at Carnegie Mellon University and worked on distributed data processing at Microsoft before SingleStore.
Anders Papitto is an engineer at SingleStore, where he has worked on distributed query execution, column store storage and query execution, and various other components. He joined SingleStore shortly before completing his undergraduate studies at UC Berkeley.
About the AMPLab
AMP: ALGORITHMS MACHINES PEOPLE
TURNING UP THE VOLUME ON BIG DATA
Working at the intersection of three massive trends: powerful machine learning, cloud computing, and crowdsourcing, the AMPLab is integrating Algorithms, Machines, and People to make sense of Big Data. We are creating a new generation of analytics tools to answer deep questions over dirty and heterogeneous data by extending and fusing machine learning, warehouse-scale computing and human computation. We validate these ideas on real-world problems including participatory sensing, urban planning, and personalized medicine with our application and industrial partners.
Read Post
Trending
Data Stores for the Internet of Things
Like the world wide web, the Internet of Things is personal. It represents a near complete connectedness, including the industrial world, and never ending possibilities of new applications and services.The Internet of Things also represents a need to examine conventional assumptions on databases and data stores to support real-time data pipelines.In an article on Silicon Angle, Designing data stores for the Internet of Things, SingleStore CEO and co-founder Erik Frenkiel shares his insight on the critical requirements to support new interconnected devices, interactive applications, and the analytics to understand their use.Principles of data store design for the Internet of ThingsCapture EverythingSave Data While Serving DataFit the EcosystemOnline All the TimeBe sure to read the entire a article at Designing data stores for the Internet of Things.
Read Post
Trending
Big Data, Big Fun! Visit SingleStore at Strata Booth 1015
The fun begins at Strata + Hadoop World this week in San Jose. Be sure to check out SingleStore at Booth 1015 for the latest product details, demonstrations, games, and prizes. Here is a quick rundown of our activity at the show.
SingleStore Introduces Seamless Spark Connectivity for Enterprise Deployments
Last week, SingleStore announced a high-performance parallel connector for SingleStore and Spark. You can read all the details here and see it live at the show.
SingleStore and Pinterest Showcase Operationalizing Spark at Strata + Hadoop World 2015
We partnered with our friends at Pinterest to share the latest and greatest with Spark and SingleStore. Read all of the details here.
Keynote: Close Encounters with the Third Kind of Database
9:10am-9:15am Thursday, February 19th, Grand Ballroom 220
Join us for this engaging presentation by our CEO and co-founder Eric Frenkiel.
Tutorial Session: Bringing OLAP Fully Online: Analyze Changing Datasets in SingleStore and Spark with Pinterest Demo
10:40am-11:20am Thursday, February 19th, Room LL20D
This session includes appearances from Robert Stepeck, CTO, Novus and Yu Yang, Software Engineering, Pinterest.
Test Your Skills with Query Kong
Win an Estes Proto X Drone after proving your low-latency skills with Query Kong, the breakout game sensation of Strata + Hadoop World!
Visit the SingleStore Booth 1015
We have cool t-shirts for all visitors during the show expo hours:
Wednesday, February 18, 5:00pm – 6:30pm
Thursday, February 19, 10:00am – 4:30pm and 5:30pm – 7:00pm
Friday, February 20, 10:00am- 4:00pm
See you there!
We look forward to sharing great technical insights and fun times at booth 1015!
http://www.singlestore.com/events
Read Post
Product
Operationalizing Spark with SingleStore
Combining the data processing prowess of Spark with a real-time database for transactions and analytics, where both are memory-optimized and distributed, leads to powerful new business use cases. SingleStore Spark Connector links at end of this post.
Data Appetite and Evolution
Our generation of, and appetite for, data continues unabated. This drives a critical need for tools to quickly process and transform data. Apache Spark, the new memory-optimized data processing framework, fills this gap by combining performance, a concise programming interface, and easy Hadoop integration, all leading to its rapid popularity.
However, Spark itself does not store data outside of processing operations. That explains that while a recent survey of over 2000 developers chose Spark to replace MapReduce, 62% still load data to Spark with the Hadoop Distributed File System and there is a forthcoming Tachyon memory-centric distributed file system that can be used as storage for Spark.
But what if we could tie Spark’s intuitive, concise, expressive programming capabilities closer to the databases that power our businesses? That opportunity lies in operationalizing Spark deployments, combining the rich advanced analytics of Spark with transactional systems-of-record.
Introducing the SingleStore Spark Connector
Meeting enterprise needs to deploy and make use of Spark, SingleStore introduced the SingleStore Spark Connector for high-throughput, bi-directional data transfer between a Spark cluster and a SingleStore cluster. Since Spark and SingleStore are both memory-optimized, distributed systems, the SingleStore Spark Connector benefits from cluster-wide parallelization for maximum performance and minimal transfer time. The SingleStore Spark Connector is available as open source on Github.
SingleStore Spark Connector Architecture
There are two main components of the SingleStore Spark Connector that allow Spark to query from and write to SingleStore.
A `SingleStoreRDD` class for loading data from a SingleStore queryA `saveToSingleStore` function for persisting results to a SingleStore table
Read Post
Trending
Closing the Batch Gap
Read Post
Trending
The Rise of the Cloud Memory Price Drop
Last week, Data Center Knowledge published a piece on Microsoft’s ‘Monster’ Azure instances, with RAM capacities approaching half a terabyte at 448 GB.
Microsoft Azure has launched its most powerful cloud instances to date. The new G-series instances go up to 32 cores, 448 GiB of RAM, and 6,596 GB of local SSD storage.
Microsoft Azure Launches Monster Cloud Instances, Data Center Knowledge, 8 January 2015
The article continues to detail
The highest-memory instance available on Google Compute Engine is 104 GB
and
The Azure announcement comes before the expected roll-out of new high-octane cloud instances by AWS. …the upcoming C4 instances, which will go up to…60 GB of RAM.
However, the R3 instances from Amazon, optimized for memory, reach capacities of up to 244 GB.
This week, Business Insider published a post outlining “The Vicious Price War Going On In Cloud Computing,” that details in finer granularity the precipitous drop of average monthly cost per GB of RAM. The chart comes from RBC Capital’s Mark Mahaney.
Read Post
Data Intensity
A Sensor In It, A Database Behind It
On opening day, Molly Wood at The New York Times recounted the official CES theme emerging as “Put a sensor in it.”
The unofficial theme seemed to be: Put a sensor in it.
Later that day Samsung declared its entire product portfolio will be connected within 5 years.
“In five years,” every single one of Samsung’s products will be a connected “Internet of Things” device, Samsung chief executive Boo-Keun Yoon said today during the opening keynote at the 2015 Consumer Electronics Show in Las Vegas.
–Venturebeat
The Consumer Electronics Show reigns as the tech industry’s annual device bonanza, and a large part of this year’s euphoria relates to connected devices that form the Internet of Things.
The theme and message remain clear. Sensors, and interconnected devices will stampede ahead.
Less frequently discussed is what happens behind those devices, and in particular, the expectations users have about their daily device interactions and demand for data.
Fulfilling Data for The Internet of Things
Both companies and users have a lot at stake.
Device and application providers aim to serve users with a rich engaging set of functionality. They also seek to instrument service delivery to monitor and react to different situations.
Users invest time and money in connected devices and applications to fill needs and desires. But along with that comes a set of expectations.
Examples include:
Fitness device users want their information up to the last stepVideo watchers expect streams to be delivered efficientlyDrivers expect hassle free connected services
Ultimately, all of the edge data drives demand and application requirements at the other end of the network, in the data center. Specifically, that infrastructure must include a database that can:
Support massive data ingest across millions of devices and connections
Database systems must keep up with the incoming flood of data to ensure no loss, and that every user or device has a complete picture of its history.
Serve as the system of record while simultaneously providing real-time analytics
In a real-time world, there is no luxury or pain of transferring data between systems, commonly referred to as Extract, Transform and Load (ETL). Systems of record for the Internet of Things need to mix transactions and analytics seamlessly and simultaneously.
Respond to and integrate well with familiar ecosystems
With sensor data touching everything from business intelligence to ad networks, connecting to multiple systems must be painless and simple.
Allow for online scaling and online operations
The world stops for no one, and successful services will be judged by their ability to grow and provide enterprise level service quality.
It will be fun to see the possibilities of devices, drones, automated equipment and the emerging services they will power as part of the Internet of Things. As that happens, mountains of data will need to be captured and applied quickly to provide the richest user experience.
We built SingleStore to give organizations behind the Internet of Things a head start. Every day, we work with development and operations teams to make sure they can ingest large amounts of sensor data with ease, make sound decisions in real-time, and manage complex, real-world data models with the familiarity of SQL. If that sounds like something that might help, give us a ring, or download SingleStore and try it for 30 days.
Read Post
Product
Full Speed Ahead
Nearly ten years ago I received a phone call about a startup in Silicon Valley solving application performance problems with memory.
From my data center infrastructure experience, I knew the days of mechanical disk drives were limited. I had to get on the memory train, so I went.
That experience led me to meet the co-founders of Fusion-io and ultimately join them in 2010. When Fusion-io went public in 2011 revenue was on its way from $36 million to $197 million annually. The time was right for flash memory and Fusion-io had the products to deliver.
Companies like Facebook and others jumped at the opportunity to supercharge their databases and infrastructure, going so far as to deploy all solid-state data centers to meet the needs of a globally connected population interacting with data and images around the clock.
During the next several years I watched customers deploy solutions for Oracle, SAP HANA, Microsoft SQL Server, and MySQL to achieve great results. Ultimately however, these solutions remained available to only a portion of the population. They were driven by adding remarkable hardware to good software.
What if we could change the equation so customers could work with remarkable software and hardware that didn’t break the bank?
My excitement for SingleStore comes from this very premise. Far beyond making the databases of yesteryear look good, SingleStore has rethought the database itself.
With a ground-up focus on memory and DRAM, distributed systems, and the ability to deploy anywhere from bare metal to a cloud container or VM, SingleStore has designed a product for today’s interconnected and interactive world. It scales out, handles the most torturous workloads without breaking a sweat, delivers analytics in the midst of massive data capture, and preserves the SQL goodness that has served as the enterprise analytics lingua franca for decades.
The SingleStore team has made phenomenal progress in the last few years, delivering a solid product with incredible market opportunity. A real-time world awaits as we experience the growth of data, applications, and touch points in our daily lives.
The SingleStore path is flanked by customers generating new revenue, driving down solution costs, and innovating with data-driven solutions in ways that had not been possible before.
I’m thrilled to be a part of it!
Read Post
