Ant Money Migrates from PostgreSQL to SingleStore on AWS, Boosting Performance 20-100X and Reducing TCO by 90%

Investing has come a long way from the days when you would get on a phone call, pay someone a few dollars, and ask them to buy a stock for you. Apps are democratizing access to investments, but those who are not tech-savvy are left behind, and even those who are comfortable accessing apps are left largely on their own once in the app. As a result, about half of Americans are still not participating in the stock market.

Ant Money, founded in March 2020, wanted to change this investment landscape by bringing investments to the websites and applications many people already use. It created an embedded finance solution, joining an emerging market that’s expected to reach $7.2 trillion in revenue by 2030. Embedded finance is when non-financial companies offer their customers access to credit through their technology platform. Customers can be individuals or businesses, and the credit can be offered by the company or by a third party.

Inderpreet Singh, Chief Technology Officer, Ant Money, explained the company’s main focus. “We strongly believe in bringing more people over to the other side of investment. Our goal is a very simplified interface with the least amount of friction to get individual users set up for an investment account.” 

“We work with different third-party apps to see if they can put in some micro income or seed the user’s investment account. This goes a long way in actually getting people to realize the benefit of investing, and brings a very valuable user to the customer app who is engaged with the investment platform.”

Challenges/Goals 

The success of Ant Money’s embedded finance solution has resulted in aggressive growth including 3X user growth, 5X increase in revenue, 1,200+ sponsors and partners, and $20+ million in series A funding. However, this growth has led to technical challenges.

Ant Money launched its platform with PostgreSQL on Amazon RDS for first-party data and Amazon Quicksight for analytics, as it was easy to prototype, ingest data, and perform basic reports with this configuration. However, this was a costly, fragile system not well-suited for the long term. 

Amazon Quicksight was not only costly, but required giant PostgreSQL replica sets. Ant Money felt it was not the best choice to transform and process data, and ingesting data from different sources was tricky. It was more of a reporting platform rather than the real-time ingestion system it needed.  

The system was slow, with queries taking seconds to minutes to process, and lacked coverage for emergent data sources such as ATM.com or financial services. The replicated data needed to be re-ingested, and the dashboard only refreshed once every 24 hours, leading to a serious and unacceptable lag in data freshness.

Most apps using Ant Money didn’t store a lot of information due to performance constraints on writes to PostgreSQL, and data from other partners was nearly impossible to obtain so Ant Money could enrich its first-party data. Ant Money had to do a bulk load of the data and it was so time-consuming that certain data was skipped. Ant Money was dropping approximately 2.4GB per day of data due to these issues.

“Everyone should be able to slice and dice data in the company. If someone needed data before, they had to talk to someone in engineering or the data team. We have a variety of internal users who are fairly savvy with looking up data, but they were blocked and unable to get additional insight or feedback in terms of how to shape some of our data,” said Singh. 

While Ant Money could solve this problem with PostgreSQL and Amazon QuickSight, it was cost-prohibitive to do so. It would need to supersize its Postgres instances to scale the system, and the use of multiple AWS availability zones meant each growth stage would result in a 4X cost increase.

Ant Money needed a way to improve performance, lower TCO, achieve cost-effective scaling, and empower its teams with real-time analytics.

Technology Requirements 

Ant Money needed to connect all of its embedded installs and microservices, each with its own data, into a single, unified data store. Its technology requirements included: 

• A performance analytics engine with columnstore to democratize data access across the organization.
• Near real-time analytics with very fast reads and quick ingestion. Ant Money wanted the data to be available as soon as possible to make data-driven decisions faster. 
• Support for unstructured data types, including clickstream data within apps that have been embedded inthe Ant Money SDK. 
• Scalability to support Ant Money’s accelerated growth trajectory. 
• A managed service, so it could focus on its platform rather than its data store. 
• A data warehouse that could scale for reads and other workloads including UPSERTs. 
• Ingesting data from external partners such as mobile marketing partners. 
• The ability to manage non-read workloads such as data and column compression. 

Why SingleStore 

“Being a small startup with a lean engineering team, we did not want to spend time working on the ins and outs of scaling databases, doing sharding, and figuring out what partitioning would look like, then embedding some of that logic in our backend systems,” said Singh. “For this reason, we went with a managed service, as it allows us to focus on what we are really here to do.”

The company never got to the breaking point with its previous solution, but saw plenty of warning signs. “Queries were running, but getting slow; the Amazon RDS instance was getting bigger and bigger; and cost was escalating. We realized it was a good time to have a data warehouse and started looking for a partner. We did a cost and performance analysis and chose SingleStore,” said Singh.

This powerful modern database offers ultimate data ingestion performance at scale, supporting built-in batch loading and real-time data pipelines. It integrates with many streaming ingest feeds, including Apache Kafka, Amazon S3, Azure Blob, and Hadoop Distributed File System (HDFS), with no additional middleware required.

Solution 

“We have a lot of different data stores, and didn’t want folks to need to connect to each of them. We wanted to pull this data together for a unified story,” said Singh.

Once Ant Money selected SingleStore Helios, it revisited its platform’s architecture to determine what to change in its data pipelines. 

Emmanuel Kala, Director of Engineering, Ant Money, detailed the data ingestion changes. “With SingleStore, we were able to standardize our ingestion path to data streaming services to support external sources we couldn’t ingest before. Clickstream data, financial services data, data from affiliate networks, and our own applications are now coming in.”

The applications log into Amazon Kinesis, which streams into Amazon S3 using a delivery stream. SingleStore offers a primitive called pipelines, making it easy to tap into different data sources. SingleStore ingest data in less than a minute, whereas the PostgreSQL data got pushed into Amazon S3 for an UPSERT workload.

“It can be an S3 bucket, Kafka, Azure, or Google storage buckets. SingleStore handles this for us internally. All we do is write the commands, point it to where the data needs to go, and specify how the data needs to be shaped as it's ingested into SingleStore. As for the internals and the magic of how that works, that's what SingleStore is doing for us,” explained Kala.

Ant Money has an occasional need to re-ingest data. By using Amazon S3, which is low-cost and offers long-term archival storage, it can always replay the data without overly burdensome expenses. It can get a better picture of the business by replaying the event stream from one point in time to another.

Ant Money also moved from Amazon QuickSight to Metabase and Tableau for its long-term analytics solution after adopting SingleStore. These tools plug in directly into SingleStore, making it easy for teams to slice and dice data and thus making the democratization of analytics simpler and less costly than what QuickSight could offer.

Outcomes 

Ant Money has generated significant business and technology gains by investing in SingleStore:

60X Improvement in Data Freshness and Performance

Ant Money now has a system that is truly data-driven and offers the freshest available data for all, with insights in real time instead of an hour. Product, marketing, and other teams all consume this data daily. 

“We're a lot more responsive with regards to freshness of data. We can now get data on the minute as opposed to on the hour or every 24 hours,” said Kala.

50X Increase in Actionable Data 

Ant Money is now ingesting and processing 50x more actionable data.

“We are able to have self-service because users outside the engineering team now have access to the data warehouse. They can run ad hoc queries as needed. We can shape the data in the way we want to slice and dice it, making it a lot more usable. Our reporting is much better as well,” explained Kala. “We're also a lot more future-proof. We are properly set up to ingest additional types of workloads and unlock other capabilities such as AB testing and data science work.”

90% Reduction in TCO 

Ant Money has reduced TCO 90% by moving to SingleStore Helios. It now has a future-proof system with workload and tooling scalability. In addition to lowering Ant Money’s direct data warehouse costs, SingleStore also delivered several indirect benefits.

“Because of SingleStore’s separation of storage and compute, we can score some tremendous savings on the scaling front as well. We sleep a lot better now, as we only worry about the application itself rather than the application and the underlying data store,“ said Kala.

Freedom to Focus on Core Competencies 

“As part of our savings overall, we are now able to focus on what we know best and let SingleStore do what it does best for us,” said Kala. “At the same time, we're also able to scale with them. We are having a lot more satisfaction internally with what we can do. We are in a much, much better place than we were a year ago.”

“It is really good to have a data platform that can model relational Postgres data with big data, right off the bat. It’s hard to put a price on ‘It just works.’ With SingleStore, we can set it and forget it for all of our common workloads,” added Kala.

Up to 11X Higher Engagement and up to 4X Increase in CLV for App Partners

The apps embedding Ant Money’s investment products are also reaping value from: some apps have seen an 11X increase in engagement and a 4X increase in customer lifetime value (CLV).

---

To hear Ant Money discuss its innovative implementation of SingleStore, watch this on-demand webinar: Migrating from PostgreSQL to Drive 20-100x Faster Performance

SingleStore is helping companies compete and win across all verticals. Learn more →