You're Using Too Many Databases. You Only Need One

Data infrastructure complexity is rampant in our industry, with modern applications continuing to be built on top of multiple, special-purpose databases, with even more being added throughout product’s and organization’s lifespans.

Peruse GitHub and you’ll typically find the following database mix for a web application:

• A traditional relational database, like MySQL, MariaDB or PostgreSQL for storing and retrieving content. Sometimes, that can be a NoSQL database, such as MongoDB or Cassandra (although, you really shouldn’t do that).
• A faster, often in-memory, key-value database, such as Redis or Memcached, for content caching or tracking state of high-speed background job queue.
• A specialist database for a particular use-case or feature, like Elastic for full-text search, or a time series database.
• A Data Warehouse for pulling data back together and running analytics.

This is a common practice beyond self-hostable web applications, and Cloud Service Providers seem to be incentivizing this behavior. We’ve already called to stop this insanity, which we dubbed the “data infrastructure sprawl”.

Well, I’m here to tell you that you don’t need to use multiple databases of your web apps. You only need one database: SingleStore.

The advantages of using multiple databases

Let’s assume you start your project with a single database. There are many ways in which you can reach its limits:

• You start spinning up background jobs, and all of a sudden you run out of connections, causing your team to scramble to control concurrency limits.
• You need to run some complex joins to provide some analytical data in your app, which your database takes too long to run, slowing down your response times and disappointing your customers. Again, your team scrambles to work around the issue.
• You need to format data in a specific way, and you find yourself tinkering with app logic to circumvent the feature-set your database was meant to offer. And again, your team is working around your database limitations.

Adding new databases seems like a sensible approach:

1. Right tool for the job: Your initial database can’t do all the things, so you extract part of your app to rely on another database that is good for a specific task. This solves your performance problems.
2. It’s not that hard to set up: Frameworks and libraries are widely available for a number of different databases. Your dev team can just run docker-compose up to spin up multiple services in development, and you can use DBaaS (Database as a Service) offerings to easily run different databases in production. Job done.
3. It’s a well-accepted common practice: Like a self-fulfilling prophecy, the more apps do this, the more apps keep doing this. So you find lots of tutorials, forum entries, documentation, libraries, etc., making it easier for you to follow this pattern.

Separation of concerns can indeed apply to the database layer. Multiple database technologies can be used with monolithic applications, and can even seem more natural in a microservices environment, where each service would have its own database.

This approach, however, is not bulletproof. Far from it, actually. In the next section, we’ll explore some of the common problems that arise when you choose to use multiple database technologies to work around the limitations of your main, or initial, database.

The downsides of using multiple databases

When designing and building software, you often find yourself managing trade-offs. In the previous section, we listed some of the advantages that come with using multiple databases to back your application. It’s not all fun and games, though.

You’ll find that the number of issues and potential problems does not grow linearly with the number of database services you add to your infrastructure; in fact, you get an exponential number of headaches:

1. Not just a steeper learning curve: There are now multiple learning curves. Developers aren’t well versed in the intricacies of each database technology, by default. When you’re onboarding new team members, or when your team members need to touch a new part of the codebase, the multiplicity of database technologies can very well slow down their learning or result in technical debt, if the new code they write doesn’t leverage the database properly. If you added a new database service to speed up your development velocity initially, you may find later on that you’ve actually only punted the problem.
2. More points of failure! You now need to implement disaster recovery strategies that cover multiple database technologies, each having their own way of allowing you to manage data backup and restoration. You’ll also need to monitor more services, each of them having different relevant metrics. Not only that, you also need to consider how your application’s features that rely on different databases coordinate with each other, so that you have a good grasp of how your users, and your team, will deal with failures in a given database, or several of them. Imagine that your app’s search bar stops working because the specific database that backs the search feature is down -- how does your application behave then? Or if your background jobs are suddenly lost because the database that manages the queues ran in-memory and the server crashed -- will your app degrade gracefully, or will problems start piling up for your users and your team and result in an entangled mess? Orchestrating additional points of failure is hard and will take time and resources away from building and polishing your application.
3. Increased business costs. We’ve seen how it can be more expensive for your team to continue to grow your application, but you also need to consider the multiplication of direct business costs. Each database vendor will price their Cloud offerings, self-hosted licenses and support services differently, each of course charging healthy margins on their services. As your application grows, you’ll find yourself hitting different tier limits with the different vendors. You’ll often find yourself over-provisioning not for one, but for several services, paying more than you need several times over.

Using multiple database technologies is a costly affair. It will take time away from your team, slowing down onboarding of new members and development of new features over time. It will add complexity to your architecture and to operations management, making it harder to maintain your quality of service. And it will cost your business more dollars as your application grows.

This tradeoff between capability, performance and cost is accepted by most organizations, as there have been no real alternatives to achieving the right performance and capabilities for modern, data-intensive applications given the limitations of available database technologies.

But we believe that there’s a far better way to achieve your goals without taking on the problems described above.

Just use SingleStore

Over the years, the SingleStore database has been used to replace a variety of best-in-class database technologies by hundreds of companies. It turns out that for an overwhelming majority of cases, you don’t need an entirely different database technology to overcome specific limitations of your first, or main, database; you’d only need that database to have a few more features, and for it to be faster across the board.

A customer in the Cybersecurity space has recently shared with us that they replaced hundreds of instances of two different database technologies, Elastic and Postgres, with a single cluster running a couple dozen units of SingleStore. This represents not only massive dollar savings, but tremendous simplification in maintainability, feature development and team growth.

Another customer, Fathom Analytics, a privacy-first web analytics service, started out with SingleStore when their cloud-based MySQL deployment started to produce slow dashboards for their fast growing user base. A few months in, they’re already looking to replace other database technologies with SingleStore to simplify their architecture. Theirs is a voluntarily small, bootstrapped company. If you read through their technical blog posts, you’ll see that their goal is to provide the best and fastest analytics service in the world, while staying small. SingleStore enables this at the database level.

We have a number of case studies available on our website describing how customers simplified their architecture by replacing at least two databases with just SingleStore:

• Case Study: How Fanatics powered their way to a better future
• Case Study: How GoGuardian stores and queries high throughput data with SingleStore
• Case Study: Insite360 Uses SingleStore Pipelines to Deliver IoT in the Cloud

There’s no end to the benefits. 

Fast and familiar. SingleStore is an incredibly fast relational database that can support both transactional and analytical queries at any scale, leveraging in-memory and on-disk storage capabilities. In addition, it also supports a variety of data formats, ranging from JSON to geospatial to time series data. It is MySQL-wire compatible, making it a drop-in replacement for MySQL and MariaDB: you don’t need to rewrite code, retool or learn an entirely new technology!

Scalable and versatile. When you start your project with SingleStore, you’ll be able to scale your usage forever: not only will the database remain fast for your main use case at all sizes, you will also be able to leverage its versatility to build new use cases without needing to deploy additional, specialized, database technologies.

• You’ll rationalize your architecture.
• Your team will always be in familiar ground.
• You’ll save money.
• You can rest assured that your application won’t find a bottleneck at the database layer.

That’s why companies of all sizes, from startups to unicorns and Fortune 10 companies have already adopted SingleStore.

You can try SingleStore for free in the Cloud or deploy it yourself. Give it a shot today. I promise you won’t regret it. Create your account now!