Tapjoy is an early and enthusiastic customer of SingleStore. They described their move to SingleStore on the Tapjoy engineering blog. This blog post gives some of the high points of Tapjoy’s experience, spelling out why Tapjoy moved – and what they now get from SingleStore. (This blog post is updated from the original version, which appeared several years ago. It includes more details, from several sources, including recent updates to SingleStore.)
Tapjoy is a leader in advertising and app monetization. They use video ads, offers, and rewards to help app publishers grow their user base and monetize mobile apps. They have been very successful, with their SDK embedded in more than 30,000 mobile apps. Tapjoy reaches more than 800 million active users a month and has more than a dozen offices worldwide.
Tapjoy’s engineering team is also global. They have built up a robust data processing backend, which processes billions of data points a day. The backend includes Kafka for data streaming, Amazon’s RDS, MySQL, PostgreSQL, and Google BigQuery. They’ve established solid procedures for problem-solving, vendor assessment, and change management.
The company faced performance issues with a workload running in MySQL. They carried out a careful assessment of alternatives, tested them, then made their move – the move to SingleStore.
What Tapjoy Needed
Tapjoy’s growth often leads them to improve their stack with new components that scale better. Several years ago, the company had a key workload, including financial data, in MySQL. This workload was an exception to a key TapJoy rule: no applications of large size and scale on MySQL. Widely known problems with MySQL include:
- Not scalable for performance. (See the link, and the MySQL vs. SingleStore comparisons below.)
- The database is owned by Oracle, which is not transparent about the MySQL roadmap.
- MySQL is suffering defections from many users.
Any key workload that stops performing well causes Tapjoy business risks similar to those of any digitally-driven company: inability to grow the business efficiently, greater need for technical personnel, difficulty meeting service level agreements (SLAs), and lower quality of service.
The team at Tapjoy assessed alternatives. They wanted to move fast, so they used a strict set of criteria:
- ACID-compliant. The new product needed to be a relational database – either a traditional RDBMS or a NewSQL contender. NoSQL was not an option.
- Scalable. The system needs to be designed for scalability. While Tapjoy has a strong DevOps team, they weren’t going to be asked to support a system that wasn’t ready to scale.
- 10x-capable. In particular, any new system needed to scale to meet demand at least an order of magnitude greater than was already occurring at the time.
- Drop-in replacement. The new solution needed to “swap in” smoothly for MySQL; the change needed to be quick, easy, and as code-compatible with MySQL as possible.
The need for scalability was where MySQL fell down, and that’s not only a MySQL issue; RDBMS alternatives tend to lack the ability to scale to the extent Tapjoy needs.
What Tapjoy Considered
With all these constraints, Tapjoy only found a few alternatives worthy of serious consideration:
- Sharded MySQL. Many organizations meet scalability demands by sharding MySQL. Neither engineering nor Ops wanted to take on this challenging task, nor to stay with a system that was already failing to do the job.
- AuroraDB. The AuroraDB option in AWS Relational Data Service (RDS) is wire-compatible with MySQL, a big advantage. However, when tested, it didn’t clear the bar for performance. AuroraDB is scalable by design, but it did not meet Tapjoy’s needs for scalability, in practice.
- SingleStore. SingleStore was already in use by the Tapjoy data science team. This team described SingleStore as ACID-compliant and fully scalable, offering stellar performance. And, like AuroraDB, it’s MySQL wire-compatible.
SingleStore is designed from scratch to meet demanding requirements like Tapjoy’s.
SingleStore Offers Huge Performance Gains
As part of their consideration of alternatives, the team compared SingleStore to alternatives, and here’s where SingleStore really stood out: the legacy system was far slower than SingleStore.
Sean Kelly, who wrote the Tapjoy engineering blog post about the move to SingleStore, sets the stage: “SingleStore had everything we wanted, and after an initial pilot, we confirmed it would meet our stated requirements. But the proof is in the pudding. We set out to build a cluster, shadow-write data to it, and put it through a battery of tests to make sure it would hold up to our use cases.”
The tests included putting SingleStore under 20x stress test loads and running it with twice the amount of historical data available as the previous database solution, while that database was run under existing cluster load.
Summing a financial data point for a single partner – an operation that needs to be repeated over and over, given Tapjoy’s huge reach in gaming – took more than a minute (68 seconds) on the old solution. It took 1.23 seconds – 55 times faster – on SingleStore.
Monthly reporting can be crucial, especially when customers are pulling their financial reports together – and paying their bills. Under the old system, summing a single financial data point for the entire platform over a month took nearly half an hour (28 minutes); with SingleStore, it took 1.1 seconds, an increase of more than 1500x.
Along with Finance, Marketing is a big constituent for any in-house data system, and there’s a regular need to pull lists of recently active customers, for a variety of purposes. On the old system, this took 24 seconds; with SingleStore, running under 20 times the load, it took three seconds, an eight-fold improvement.
As indicated by load test results, SingleStore met all of Tapjoy requirements and turned out to be a clear winner in the head-to-head load tests.
SingleStore also met Tapjoy’s other requirements: ACID compliance, SQL compatibility, scalability, and the ability to drop in as a replacement for MySQL.
Conclusion
After initially adopting SingleStore for this single use case, Tapjoy went on to use SingleStore for several use cases. SingleStore and Tapjoy announced the results of the two companies’ work together.
You can see a more recent presentation of Tapjoy’s architecture in this presentation, Building a Real-Time Data Science Service for Mobile Advertising. And, if you’re interested in trying SingleStore for your own challenging data issues, you can try SingleStore for free or contact SingleStore.
