Author
Neil Dahlke
Senior Solutions Engineer
Trending
SingleStore, Tableau, and the Democratization of Data
“We love fast databases. It makes the experience of interacting with your database that much more enjoyable.” – Tableau
Today’s business decisions are about seconds, not minutes. To accommodate this trend, businesses have moved to evidence-backed decision making and widespread data access. Modern business intelligence tools abound, making it easier for the average analyst to create compelling visualizations. In this post, I’ll address how this new mode of thinking about data, the Democratization of Data, comes with two challenges – making data easily available and making it actionable in real time.
Making Data Available
Companies are migrating to a new model of data distribution – shared access to a centralized database with both historical data and real-time data. This is a far cry from the traditional approach of using many small database instances with stale data, isolated silos, and limited user access. Now, raw data is available to everyone. Employees are empowered to dive into the data, discover new opportunities, or close efficiency gaps in a way that has never been done before. The need for data now coupled with scalability has attracted many developers to in-memory, clustered databases.
Making Data Actionable in Real Time
Innovations in data visualization have produced powerful, usable tools that afford companies the opportunity to be data-driven. One tool we see embedded across different industries is Tableau. With its mature ecosystem and rich featureset, the business intelligence platform makes it easy for individuals to create compelling, interactive data visualizations. It is a very attractive package for different business levels because it does not require expertise or a degree in visual design or information systems. Any user can create meaningful, actionable dashboards providing views of the business from thirty thousand feet as well as at ground level.
But even with a Tableau license in hand, users still face issues – the dashboards are slow or the data is stale. The problem often lies in the database layer. It is imperative that data is up-to-date to be relevant to today’s fast moving business operations. Common issues include:
Read Post
Data Intensity
Election 2016: Analyzing Real-Time Twitter Sentiment with SingleStore Pipelines
November is nearly upon us, with the spotlight on Election 2016. This election has been amplified by millions of digital touchpoints. In particular, Twitter has risen in popularity as a forum for voicing individual opinions as well as tracking statements directly from the candidates. Pew Research Center states that “In January 2016, 44% of U.S. adults reported having learned about the 2016 presidential election in the past week from social media, outpacing both local and national print newspapers.” The first 2016 Presidential debate “between Donald Trump and Hillary Clinton was the most-tweeted debate ever. All told, there were 17.1 million interactions on Twitter about the event.”
By now, most people have probably seen both encouraging and deprecating tweets about two candidates: Hillary Clinton and Donald Trump. Twitter has become a real-time voice for the public watching along with debates and campaign announcements. We wanted to hone in on the sentiments expressed in real time. Using Apache Kafka, SingleStore, Machine Learning and our Pipelines Twitter Demo as a base, we are bringing real-time analytics to Election 2016.
Read Post
Case Studies
How Pinterest Measures Real-Time User Engagement with Spark
Setting the Stage for Spark
With Spark on track to replace MapReduce, enterprises are flocking to the open source framework in effort to take advantage of its superior distributed data processing power.
IT leads that manage infrastructure and data pipelines of high-traffic websites are running Spark–in particular, Spark Streaming which is ideal for structuring real-time data on the fly–to reliably capture and process event data, and write it in a format that can immediately be queried by analysts.
As the world’s premiere visual bookmarking tool, Pinterest is one of the innovative organizations taking advantage of Spark. Pinterest found a natural fit in SingleStore’s in-memory database and Spark Streaming, and is using these tools to find patterns in high-value user engagement data.
Pinterest’s Spark Streaming Setup
Here’s how it works:
Pinterest pushes event data, such as pins and repins, to Apache Kafka.Spark Streaming ingests event data from Apache Kafka, then filters by event type and enriches each event with full pin and geo-location data.Using the SingleStore Spark Connector, data is then written to SingleStore with each event type flowing into a separate table. SingleStore handles record deduplication (Kafka’s “at least once” semantics guarantee fault tolerance but not uniqueness).As data is streaming in, Pinterest is able to run queries in SingleStore to generate engagement metrics and report on various event data like pins, repins, comments and logins.
Visualizing the Data
We built a demo with Pinterest to showcase the locations of repins as they happen. When an image is repinned, circles on the globe expand, providing a visual representation of the concentration of repins by location.
Read Post
Engineering
The Internet of Furbies
At SingleStore engineering there are few things we love as much as building great products. One, however, is a good laugh. This past weekend a team from SingleStore set out to make our mothers proud at Cultivated Wit’s Comedy Hack Day, an event for comedians and hackers to get together in attempt to make the funniest hack. And drink whiskey.
I’m very excited to share with you our product today, the grand prize winner, and the future of wearable technology. Engineering at SingleStore goes to great lengths to ensure that our technology enhances your life, rather than detracting from your special moments. Our engineering and design teams have determined that the best place for strapping technology to your body is not to your face or to your wrist, but rather, to your shoulder.
The wait is over, the next furry thing is here. We present to you: AwwCog – the future of wearable technology.
Read Post
