Intel’s new Optane DC persistent memory adds a new performance option for SingleStore users. After careful analysis, we’ve identified one area in which SingleStore customers and others can solve a potentially urgent problem using Optane today, and we describe that opportunity in this blog post.
We also point out other areas where SingleStore customers and others should keep an eye on Optane-related developments for the future. If you need broader information than what’s offered here, there are many sources for more comprehensive information about Optane and what it can do for you, beginning with Intel itself.
What Optane Offers
Intel’s Optane is a new kind of memory option that’s less expensive than RAM, and also nonvolatile. Optane is only available on systems running some of the latest hardware and software from Intel.
Optane offers a new way to improve the performance of systems, especially servers. However, it will take the industry – from hardware vendors, to operating system developers, to application developers – a good while to take full advantage of it.
Optane runs in two modes, each with different advantages and trade-offs: memory mode, which we will concentrate on here, and application mode. In memory mode, the computer’s RAM is used as a fast, volatile cache above larger, lower-cost Optane memory modules, managed entirely by Intel’s built-in memory management. In this mode, taking advantage of Optane does not require any changes to application code.
In application mode, Optane can be explicitly written to and read from, separate from traditional RAM or hard disk storage. We will only discuss application mode briefly here.
Optane has two features that make it interesting: price and capacity. First, Optane is much cheaper, on a like-for-like basis, than traditional RAM. For traditional DRAM, a 128GB ECC DDR4 module is the largest available today, and costs roughly $4,500. For Optane, a 128GB DIMM is the smallest available, and costs about $850 at this writing – more than 80% less than traditional RAM.
Of just as much interest, however, is capacity. Optane offers DIMMs up to 512GB in capacity – four times the capacity of traditional DRAM. A 512GB Optane DIMM module costs about $7,000, or nearly ten times the price of the 128GB DIMM. However, the Optane 512GB DIMM is still less than half the price of four 128GB ECC DDR4 modules of traditional RAM, at about $18,000.
A Brief Mention of Application Mode
Use cases for Optane in application mode are interesting, but we won’t go into much depth on that here. When running Optane in application mode, since you can address Optane memory directly, you can treat it almost as a replacement for disk. That’s because writes to the Optane memory become permanent, or “persistent.”
The main benefit of application mode is persistence. This allows databases that keep information in memory to restart faster, and experiment with new architectures for making update transactions permanent. SingleStore’s scale-out architecture already allows us to restart quite fast, since each processor core reads data locally to re-populate RAM, in a fully parallel fashion. Restart time has not been a significant problem for our customers to date. So while we are intrigued by the possibilities of using application mode for SingleStore, we have not so far made changes to our code to use it.
Under the Covers of Memory Mode
What’s more widely interesting for now is the use of Optane in memory mode. When you add Optane persistent memory to your system, the total Optane capacity acts as your addressable RAM capacity. Traditional RAM then serves as a cache over the Optane persistent memory.
In memory mode, you need to consider three factors in gauging performance:
- Traditional RAM performance. Adding Optane does not change performance when the data you need resides in traditional RAM. The Intel memory controller and drivers work to maximize the number of times needed data is fetched from traditional RAM rather than Optane memory.
- Optane memory performance. When you add Optane, and application code and data in your address space exceed the capacity of traditional RAM, some RAM accesses will miss in traditional RAM. These accesses will then be served by Optane memory instead. The total time to access will be the traditional RAM access time plus the Optane memory access time, which can be a multiple of the traditional RAM access time.
- Cache hit percentage. When you access data in RAM, some of it will be served by a cache hit – the data is in traditional RAM – and some by a cache miss – the access to traditional RAM fails, and an additional access to Optane memory is required. The percentage of cache hits determines whether you get something close to traditional RAM performance or something closer to Optane memory performance.
For most applications, with Optane memory at perhaps four times the size of traditional RAM, the overall performance reduction may average around 25%, compared to the same total RAM size made up of traditional RAM only. This applies across the board, including for queries as well as size-of-data operations like backup, restore, and snapshots. But the traditional RAM + Optane memory combination will be much less expensive – and, as we describe below, you can get far more Optane memory into a single server than traditional RAM.
Because no one has to rewrite their software to take advantage of memory mode, and because operationally savvy users can take full advantage of Optane against suitable workloads, it’s expected that Optane will be very popular with cloud providers and in large data centers. These deployments can offer large cost savings for the server owners, with an impact on end users that may not even be noticeable.
The “Hidden” Feature of Optane – and the Impact on SingleStore Customers
What’s most exciting about Optane, when used in memory mode, is not necessarily the direct cost savings on like-for-like servers. The exciting part is that the total RAM capacity of a single machine can be multiplied by a large amount. For instance, where an organization currently supports servers with 256GB of RAM, it can cost-effectively add support for servers with 256GB of traditional RAM plus, perhaps, 1TB of Optane memory. Where logical RAM capacity is the gating factor for the number of servers used, one such Optane-enabled server may be able to replace up to four traditional servers.
This applies directly to SingleStore customers who have very large rowstore tables. If the total rowstore table size is 2TB, it would take eight or nine servers with 256GB of RAM to support the leaf nodes for the tables, plus additional servers as aggregators (this assumes no redundancy — using high availability with “redundancy 2” would double the number of needed servers).
It would only require two leaf node servers, though, if each is provisioned with a mix of 256GB of traditional RAM and 1TB of Optane memory. The total hardware cost would be far less than for eight servers sporting traditional RAM, and management and maintenance of the server pool would be far easier.
SingleStore is so efficient that it’s common for a server that’s maxed out on RAM to be only using 20-30% of its CPU capacity. However, past RAM limitations prevented some customers from getting more useful performance from each server. With Optane, effective RAM capacity can be upgraded much further, and quite cost-effectively.
Some SingleStore customers have been held back from deploying applications with truly large rowstore tables, totaling 10TB and more, due to the cost and operational complexity of managing the number of servers that would be required. With Optane, these projects become quite feasible.
These same dynamics apply to other workloads that use rowstore tables in SingleStore, and these are the workloads that will be moved to Optane-powered servers within cloud providers and in large data centers. It’s no accident that launch partners for Optane persistent memory include Google Cloud, as well as server makers like Cisco, Dell EMC, and Lenovo, plus consultancy Accenture.
It’s also possible for Optane to be useful with columnstore tables, or in applications that use a mix of rowstore and columnstore. However, columnstore needs for large databases are already served effectively by SSDs, and it will take careful testing to establish where Optane will or won’t offer a clear price/performance benefit.
What You Should Do Today
If you are a current SingleStore customer, running very large tables (1TB-plus total data size) in rowstore today – or if you have a near-term need to do so – reach out to your SingleStore contacts soon. You may be able to save money on your current deployment and cost-effectively grow your application’s memory footprint and effectiveness.
If you are not running workloads of this type, Optane may not have immediate benefit for you in the short term, at least for the database part of your applications. As applications – everything from operating systems, to business applications, to end user applications – are rewritten to take advantage of Optane, you will have the opportunity to leverage those improvements by including Optane in your server mix.
If you are not a SingleStore customer, but are running (or contemplating) applications that will require high rates of ingest, high transaction performance, fast query response, and a high degree of concurrent usage, Optane helps make SingleStore an even more attractive option. We encourage you to try SingleStore for free today, or contact us to learn how SingleStore can help with even very ambitious data management goals.
