Our performance engineering team is committed to delivering high quality tools. Since we released
7 months ago, it has been widely adopted across our engineering and sales teams as the definitive tool for testing database workloads. Today we are announcing availability of a new version of
, as well as a package of high level tools to enhance it.
In this latest release, we enhanced both the flexibility and ease of use of the tool. We augmented capabilities of
and added a tutorial to help new sales engineers get started. In addition to these enhancements, we released a package of internal tools specifically designed to support high level workflows that use
. These changes improve our technical proof-of-concept (POC) process for our customers and open up
for new workloads and use cases. This version of
not only increases the performance and power of this benchmark testing tool, but also makes it easily accessible to anyone interested in using it.

dbbench in Action

One of the new features of
is the ability to display latency histograms during the final analysis to easily detect and understand outliers. In the example below, we can see that most of the queries executed between 0.2 and 0.5 ms, but there were outliers as high as 30ms:
dbbench query latency historgram
To support more complicated and varying data,
can read arguments from a
file and parameterize queries via the
parameter. The snippet below shows this parameter specified in a configuration.ini file for
query=insert into table colors (?)
In addition, the
parameter allows
to chain multiple jobs together, and makes it possible to have external tools in between each of these jobs.


is a package of sales engineering tools, used by the SingleStore team to easily describe and efficiently run a customer workload during a technical POC. The most important question to answer during a POC is “how will SingleStore scale to more concurrent users?”
is one of several tools provided by the
package to help answer this question.
runs a
-defined workload at different levels of concurrency to find the limits of the database installation.
For example, I was doing a POC where I wanted to determine how many concurrent inserts a small SingleStore cluster could handle. I set up a simple
query=create table test(a varchar(20), b float, c float, d float, \
    shard key a (a), key b (b), key c (c), key d (d))

query=drop table test

[batch sized 128 multi-inserts]
query=insert into test values (?, ?, ?, ?), (?, ?, ?, ?), …, (?, ?, ?, ?)
I ran
with this configuration to determine that this small test cluster could sustain 100,000 inserts per second in batches of 128, as depicted in the top half of the chart below. In the bottom half of the chart, we see that using more connections allows us to insert more records per second (RPS) until we reach 32 connections, after which there is no benefit to using more connections and we see a hugely adverse effect on query latency:
$ dbbench-scaler --database=db --concurrency=1,2,4,8,16,32,64,128,256,512 --output out.png /tmp/test.ini
dbbench scaler results
In this example, not only did
answer the question “will SingleStore handle the scale of many users”, it also found the ideal number of users for this application.
Access the new version of
here and get
here. Learn even more about the new features in our tutorial.