Summary

The article presents a performance comparison between Redis and KeyDB, focusing on their capabilities as cache servers using GET/SET commands, with KeyDB's multithreading being a key differentiator.

Abstract

The author conducted a series of benchmark tests to compare the performance of Redis and KeyDB, a multithreaded fork of Redis. The tests were executed using Docker with 4 vCPUs, and the performance was measured under different configurations of server threads (1, 2, and 4) with 500 concurrent clients issuing a total of 50,000 queries. The results indicated that KeyDB's performance did not scale linearly with the number of threads, with the best performance often achieved with fewer threads, sometimes even matching Redis' single-threaded performance. Despite KeyDB's claim of superior performance through multithreading, the tests suggested that Redis could be more efficient in certain scenarios. However, the author also notes that KeyDB offers an "On Flash" feature, which allows using SSDs as an extension of RAM, potentially reducing memory costs while maintaining performance, although this feature was not successfully tested due to an error.

Opinions

The author suggests that KeyDB's multithreading capability does not necessarily translate to better performance over Redis, especially in a cache server scenario using GET/SET commands.
It is implied that Redis may be more resource-efficient, as it achieved comparable or better performance with a single thread compared to KeyDB with multiple threads.
The author points out that KeyDB's "On Flash" feature could be advantageous for memory-intensive applications, despite not being able to test it.
The article conveys a nuanced view that while KeyDB has potential benefits, such as multithreading and "On Flash," these do not universally lead to better performance than Redis in all use cases.
The author's conclusion indicates a preference for Redis under the tested conditions, while acknowledging that KeyDB's features might be beneficial in different scenarios not covered by the benchmarks.

Redis vs. KeyDB — My Performance Test and Conclusions

Is KeyDB faster than Redis?

KeyDB is a fork of Redis. KeyDB project claims that the Redis implementation approach is not correct and can be better, in terms of performance.

KeyDB can run more than one thread, compared to Redis approach which only runs one.

The KeyDB website claims that it’s faster than Redis. I did some quick tests to check the performance as cache server (it means that I tested commands like GET / SET) and in this post I’ll share my results.

How Did I Test?

For my test, I used Docker for running Redis and KeyDb. I configured my Docker Engine with 4 vCPUs to take the advantage of KeyDB multithreading:

I also increased the OS limits to allow a high number of concurrent connections for the tests:

> ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-v: address space (kbytes)          unlimited
-l: locked-in-memory size (kbytes)  unlimited
-u: processes                       11136
-n: file descriptors                256

Tests Conditions

I did a test with these parameters:

Total queries: 50.000
Command tested: GET / SET
Concurrent clients: 500
I executed the same test 5 times.

KeyDB results

KeyDB Version: 6.3.1

Server with 4 threads

> docker run -p 6777:6379 eqalpha/keydb keydb-server --server-threads 4

Executing tests:

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 23775.56 requests per second, p50=14.559 msec
GET: 22758.31 requests per second, p50=13.439 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 18903.59 requests per second, p50=16.247 msec
GET: 22872.83 requests per second, p50=13.359 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 23430.18 requests per second, p50=14.583 msec
GET: 20729.69 requests per second, p50=14.687 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 17325.02 requests per second, p50=19.263 msec
GET: 19723.87 requests per second, p50=16.591 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 25706.94 requests per second, p50=13.191 msec
GET: 19817.68 requests per second, p50=14.839 msec

As a summary:

Server with 2 threads

> docker run -p 6777:6379 eqalpha/keydb keydb-server --server-threads 2

Executing tests:

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 31605.56 requests per second, p50=11.095 msec
GET: 33692.72 requests per second, p50=10.815 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 30395.14 requests per second, p50=12.271 msec
GET: 32959.79 requests per second, p50=13.639 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 29691.21 requests per second, p50=13.087 msec
GET: 32467.53 requests per second, p50=13.159 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 28216.71 requests per second, p50=14.295 msec
GET: 30693.68 requests per second, p50=14.103 msec

> redis-benchmark -p 6777 -t get,set -q -n 50000 -c 500
SET: 31486.14 requests per second, p50=12.543 msec
GET: 35087.72 requests per second, p50=12.543 msec

As a summary:

Server with 1 thread

I did tests with only one thread, but the results were the same as with 2 threads.

Redis Results

Redis Version: 7.0.2

Executing server:

> docker run -p 6379:6379 redis

Executing tests:

> redis-benchmark -p 6379 -t get,set -q -n 50000 -c 500
SET: 37622.27 requests per second, p50=12.527 msec
GET: 28392.96 requests per second, p50=12.919 msec

> redis-benchmark -p 6379 -t get,set -q -n 50000 -c 500
SET: 27442.37 requests per second, p50=13.439 msec
GET: 31446.54 requests per second, p50=15.527 msec

> redis-benchmark -p 6379 -t get,set -q -n 50000 -c 500
SET: 27808.68 requests per second, p50=13.751 msec
GET: 35335.69 requests per second, p50=13.391 msec

> redis-benchmark -p 6379 -t get,set -q -n 50000 -c 500
SET: 30156.82 requests per second, p50=14.631 msec
GET: 37009.62 requests per second, p50=12.711 msec

> redis-benchmark -p 6379 -t get,set -q -n 50000 -c 500
SET: 33068.79 requests per second, p50=13.359 msec
GET: 40225.26 requests per second, p50=10.999 msec

As a summary:

Conclusion

Ok. Results are curious:

KeyDB says that they are faster than Redis because they can run in a multithreading way, but in tests, results appear that it runs faster when the number of threads tens to 1… like Redis.
This means that KeyDB needs more resources to reach the same performance as Redis.
KeyDB threads need to count your hardware resources; otherwise, Redis.

We also have to keep in mind that KeyDB has a great feature that Redis doesn’t have (at least in their Open-Source version) like:

The “On Flash” feature. This feature uses SSD disks as an extension of RAM, spending less RAM with very similar performance.

Redis on flash: https://redis.com/redis-enterprise/technology/redis-on-flash/
KeyDb on flash: https://docs.keydb.dev/docs/flash/

I tried to run KeyDB server with On Flash feature, but I couldn’t. It raised me the error: “FLASH Is Not yet Supported”.