Creating a Redis Cluster

6 min readJul 3, 2019

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. — Redis Website

In this story i will share how to setup a redis cluster.

Redis is a valuable multifaceted tool that helps a developer in many ways. You can use it as a cache or session management store for your application as it is blazing fast. Click here to read benchmarks. You can also read this that says redis is roughly 2x faster for reads and 3x faster for writes when compared to mongoDB. It also has a built in pub/sub message broker. This means that you can use redis for various reactive implementations. For example — A notification system that subscribe to a channel on redis message broker. Notifications can be published on this channel through various publishers. Apart from this redis supports range of data structures including raw JSON with redis v4.0 or greater with support of a modules. check out RedisJSON module that enables raw JSON support for redis. All in all it’s a neat tool.

Enough of selling redis. Now let’s get into creating a distributed environment.

For creating a distributed environment of redis you have 2 options as follows-

Redis Sentinel — Use sentinel when speed isn’t your primary concern, which makes it an excellent option for smaller implementation with high availability concerns.
Redis Cluster — It provides high availability plus clustering solution. Its an excellent choice to ensure high availability while keeping fast access speed in consideration to access your data

Bottom Line — If you need an automatic failover solution without going to a full cluster solution, then use sentinel. For getting a complete clustering clustering solution using sharding go for clustering.

For this story i will create a full cluster solution therefore i will be going the clustering way.

Understanding redis

Every Redis Cluster node requires two TCP connections open. The normal Redis TCP port used to serve clients, for example 6379, plus the port obtained by adding 10000 to the data port, so 16379. Make sure you open both ports in your firewall, otherwise Redis cluster nodes will be not able to communicate.

Redis Cluster does not use consistent hashing, but a different form of sharding where every key is conceptually part of what we call an hash slot. There are 16384 hash slots in Redis Cluster.

Every node in a Redis Cluster is responsible for a subset of the hash slots, so for example you may have a cluster with 3 nodes, where:

Node A contains hash slots from 0 to 5500.
Node B contains hash slots from 5501 to 11000.
Node C contains hash slots from 11001 to 16383.

Creating and using a Redis Cluster

A redis cluster uses master-slave configuration to support distributed environment. In this example we will create 3 master nodes and 3 slave nodes. Each master node having at least 1 slave.

if you want to get a cluster up and running in no time with minimal configuration then use the `create-cluster` script shipped by default in redis package. There are many tutorials on how to create cluster using `create-cluster` script so i am not going to cover that. Rather i will showcase how to do it in a manually which gives you great degree of freedom to tweak with cluster configuration parameters.

Installation

Since we are simulating a 6 node cluster(3 masters and 3 slaves) we will create 6 folders namely 6001, 6002, 6003, 6004, 6005, 6006. Here name of the folder represents the port number on which each instance will run. On each node i.e. inside each folder, download and make the redis package by executing following commands-

$ wget http://download.redis.io/releases/redis-5.0.5.tar.gz
$ tar xzf redis-5.0.5.tar.gz
$ cd redis-5.0.5
$ make

You can also simply download the package in a folder and copy it to other folders. This will save you some download time and bandwidth.

The binaries that are now compiled are available in the src directory.

Running Redis in cluster mode

Now that you have redis, you can see a `redis.conf` file insidesrc directory. This is the configuration file in which you should define all cluster configuration parameters. Following is an example of minimal configuration you should have in `redis.conf` to start with-

port 6001
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

As you can see what enables the cluster mode is simply the cluster-enabled directive. Every instance also contains the path of a file where the configuration for this node is stored, which by default is nodes.conf. This file is never touched by humans; it is simply generated at startup by the Redis Cluster instances, and updated every time it is needed. Note that the minimal cluster that works as expected requires to contain at least three master nodes.

Create a redis.conf file inside each of the directories, from 6001 to 6006. As a template for your configuration file just use the small example above, but make sure to replace the port number 6001 with the right port number according to the directory name.

Finally open 6 terminal tabs in your favorite terminal application. Start every instance like that, one every tab:

./src/redis-server ./redis.conf

Following is a screenshot from one of the nodes after startup. You can see As you can see from the logs of every instance, since no nodes.conf file existed, every node assigns itself a new ID.

This ID will be used forever by this specific instance in order for the instance to have a unique name in the context of the cluster. Every node remembers every other node using this IDs, and not by IP or port. IP addresses and ports may change, but the unique node identifier will never change for all the life of the node. We call this identifier simply Node ID.

Creating the cluster

Now open a new terminal window and execute following command being inside src directory.

./redis-cli --cluster create 127.0.0.1:6001 127.0.0.1:6002 127.0.0.1:6003 127.0.0.1:6004 127.0.0.1:6005 127.0.0.1:6006 --cluster-replicas 1

The option --cluster-replicas 1 means that we want a slave for every master created. The other arguments are the list of addresses of the instances you want to use to create the new cluster.

Obviously the only setup with our requirements is to create a cluster with 3 masters and 3 slaves.

You can see in following screenshot it says [OK] All 16384 slots covered

In above screenshot, following lines means exactly what they say.

Adding replica 127.0.0.1:6005 to 127.0.0.1:6001
Adding replica 127.0.0.1:6006 to 127.0.0.1:6002
Adding replica 127.0.0.1:6004 to 127.0.0.1:6003

Nodes running on ports 6001, 6002 and 6003 are masters and nodes running on ports 6005, 6006 and 6004 are slaves of those masters respectively.

The cluster is now up and running. You can test it by executing commands as shown in following screenshots

Following command indicates that i am using redis-cli in cluster mode and connecting to node running on port 6001.

./src/redis-cli -c -p 6001

After that you can see key value pairs are getting stored in nodes depending on which hash slots they are getting saved into. It also shows i can access any key from any node. In above screenshot key foo was saved in node 6003 but i was able to retrieve it even when i was on node 6002.

Your redis cluster is now up and fully functioning.

What’s next?

Once you are done with creating a redis cluster. Read following relevant stories in continuation-