Distributed Redis

We love #Redis on #Rails here at Nowcado. We also love #scalability. Redis has a distributed flavor that automagically shards your keys across multiple nodes (though of course keep an eye on Redis Cluster which looks exciting and hopefully will be stable soon!)

This is a rough layout of how our Redis initializer for Rails looks so that in our app we can reference all of our distributed Redis nodes with $redis. It is creating however many processes we define evenly distributed across however many hosts we define in our Redis config.

This is a snippet of our redis initializer to handle the distributed redis nodes defined in our config YAML which is set up to support development, test, and production environments.

# REDIS_NUMPROCS defines how many redis processes we will use.
# REDIS_HOST is the array of available redis server hosts.
# REDIS_FIRST_PORT is the first sequential TCP port for the redis server array.

require 'redis/distributed'
redis_procs_per_host = REDIS_NUMPROCS / REDIS_HOST.length
# Set up the initial master processes
$master_arr = Array.new(REDIS_NUMPROCS) {|i|
    :host => REDIS_HOST[i/redis_procs_per_host],
    :db => 0,
    :port => REDIS_FIRST_PORT + i%redis_procs_per_host,
    :driver => :hiredis
if $redis.nil?
  # set up the global $redis variable
  $redis = Redis::Distributed.new $master_arr

Make sure that all your redis nodes are set up in the manner this is expecting, which means you should have redis_procs_per_host number of process running on each host with ports ranging from REDIS_FIRST_PORT to REDIS_NUMPROCS / REDIS_HOST.length. Note that we use the hiredis driver to improve performance.

From an application perspective this becomes dead simple to work with (just have to reference $redis as mentioned), though having all of these distributed nodes means high availability becomes very important. Stay tuned for when we cover how we set up site-resilient slave nodes tomorrow!


One thought on “Distributed Redis

  1. […] more complete HA (High Availability) solution, we wanted a simple solution that would work for our distributed Redis nodes we recently described, which unfortunately will not work with Sentinel. We recommend you have slave nodes (ideally in a […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s