So you dressed up your terminal with your favorite bash profile, tuned up your monitor lighting to an ambient level and your favorite chillstep music is playing in the background, in other words, you're ready for Redis and Chill. Welcome to our tutorial series on how to master the art of Redis!
On my (new) mac I can use spork and redis seems to be already running. However on my Ubuntu system (11.10) I get. Connection refused - Unable to connect to Redis on localhost:6379 (Errno::ECONNREFUSED) I can get around this with redis-server which start redis as a foreground process in that window.
- Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. Get Started with Redis and understand the basic knowledges of Redis.
- When I connect locally on the Redis instance, everything works, but when I try to connect from the other instance on the subnet I get: Connection refused - Unable to connect to Redis I setup two separate VPC security groups, one for the redis server, allowing inbound connections on port 6123 using the other instance's security group as the source.
What is Redis?
Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.
Created by Salvatore Sanfilippo aka antirez, Redis is one of the most popular piece of software on the web! I'm very excited to give you a tutorial today and I hope to demonstrate you the power behind this amazing database.
Installation
You don't need to have Redis installed in order to start try out commands, but we're all about being practical, so I'm going to show you how to install Redis in three different ways:
- Standard installation
- Digital Ocean pre-build droplet
- Docker Container
Standard Installation
Assuming you're on a *nix based OS (sorry Windows users):
Head over to http://redis.io/download, download the current stable version (3.2.3 at the time of writing):
verify redis is working by turning on Redis Server in the background
then open another terminal tab and start a Redis Client by:
When you see the terminal prompt changed into
type in INFO
and you should see something like this:
That's it! Fast and simple. Now you don't want to run Redis server in a new tab everytime you need to develop your app, so we should make Redis Server to run automatically in the background.
Follow this tutorial if you're running on Mac.
Digital Ocean Pre-built Droplet
The fastest, easiest way for us to set up a fresh server with Redis pre-installed, is via a Digital Ocean droplet. Digital Ocean is an amazing cloud hosting company similar to AWS, Google App Engine or Microsoft Azure where you can start your own private servers. If you would like an intro tutorial on Digital Ocean, I recommend you to read my tutorial on How to create a blog in 5 minutes with Digital Ocean
Ok, head over to Digital Ocean, assuming you're signed in, we'll need to create a droplet (a droplet is Digital Ocean's term for a server), select Redis 3.2.1 on 14.04 under One Click Apps:
Select your droplet size and location and name it something you can easily reference it by, then click Create to launch your server. Within 60 seconds, you will receive an email with password to your server! That's it, that server now has Redis pre-installed.
To test it out, SSH into the server
enter the password that was sent to you to your email and you should be able to log in:
follow the instruction above from the Standard Installation section and you should see a similar screen that displays your Redis server information.
Your remote Redis server is now set up! If you wish to connect to it, you would do this on your local server:
Hmm, why is this not working? That is because by default, Digital Ocean turned off connections to the server from outside. This is a good security practice because only IPs from a whitelist can connect to the server.
For the sake of this example, let's let anyone from anywhere connect to the server.
On your droplet, type:
This will open up your Redis Config file using Vim, we need to look for the line:
and comment it out
You also need to turn off Protective Mode by changing this line from:
to
Save (:wq) and restart the Redis Server by:
Now on your local computer, connect to the Redis droplet by
where 104.131.18.33 is your server IP, and you should be able to connect and run the INFO command!
Of course the above was just to demonstrate how you would connect from a remote location, in practice, you should always limit connection with an IP whitelist and set a password to your Redis server. Digital Ocean's droplet comes with a lot of best practices built in, so it's definitely my preferred method of installation.
Docker
Docker is the one of the hottest Containerization technology around the block. With Docker, you can isolate Redis on a 'container' on your computer to reach true isolation while maximizing your server utilization, This part of the tutorial is useful if you are thinking about adding Redis to your existing stack. If you need an intro to Docker, you can read about my tutorial here: Using Docker to deploy Apache, Nginx, WordPress and Nodejs containers with Digital Ocean and Creating a LEMP stack with Docker Compose
To start, we need to create a directory on either your local machine or your server:
open up your favorite editor and edit the docker-compose.yml file, in my case I use Visual Studio Code:
We're going to install Redis and a very helpful companion app call Redis Commander.
Start a docker terminal session and type:
and you should see your containers being created:
believe it or not, we're done! Now we should be able to connect to the containerized Redis server from inside Docker. Let's find the IP address of the container instance:
this will return the IP of our docker virtual machine. In my case it returned 192.168.99.101 , we also know the port of our Redis server because we defined it in our docker-compose file, the port to the redis server is 9000. Now that we have the information we need to connect to our Redis instance, let's start a connection.
With our local Redis client installed, we can connect to our container via the following command:
Amazing isn't it? Let's set a key in Redis so we can demonstrate the power of Redis Commander:
Redis Commander was set up on port 9001, so we need to head over to http://192.168.99.101:9001/ in order to access the web UI:
The UI is really straight forward, now you can VISUALLY see all your keys and perform operations on them!
Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.
Created by Salvatore Sanfilippo aka antirez, Redis is one of the most popular piece of software on the web! I'm very excited to give you a tutorial today and I hope to demonstrate you the power behind this amazing database.
Installation
You don't need to have Redis installed in order to start try out commands, but we're all about being practical, so I'm going to show you how to install Redis in three different ways:
- Standard installation
- Digital Ocean pre-build droplet
- Docker Container
Standard Installation
Assuming you're on a *nix based OS (sorry Windows users):
Head over to http://redis.io/download, download the current stable version (3.2.3 at the time of writing):
verify redis is working by turning on Redis Server in the background
then open another terminal tab and start a Redis Client by:
When you see the terminal prompt changed into
type in INFO
and you should see something like this:
That's it! Fast and simple. Now you don't want to run Redis server in a new tab everytime you need to develop your app, so we should make Redis Server to run automatically in the background.
Follow this tutorial if you're running on Mac.
Digital Ocean Pre-built Droplet
The fastest, easiest way for us to set up a fresh server with Redis pre-installed, is via a Digital Ocean droplet. Digital Ocean is an amazing cloud hosting company similar to AWS, Google App Engine or Microsoft Azure where you can start your own private servers. If you would like an intro tutorial on Digital Ocean, I recommend you to read my tutorial on How to create a blog in 5 minutes with Digital Ocean
Ok, head over to Digital Ocean, assuming you're signed in, we'll need to create a droplet (a droplet is Digital Ocean's term for a server), select Redis 3.2.1 on 14.04 under One Click Apps:
Select your droplet size and location and name it something you can easily reference it by, then click Create to launch your server. Within 60 seconds, you will receive an email with password to your server! That's it, that server now has Redis pre-installed.
To test it out, SSH into the server
enter the password that was sent to you to your email and you should be able to log in:
follow the instruction above from the Standard Installation section and you should see a similar screen that displays your Redis server information.
Your remote Redis server is now set up! If you wish to connect to it, you would do this on your local server:
Hmm, why is this not working? That is because by default, Digital Ocean turned off connections to the server from outside. This is a good security practice because only IPs from a whitelist can connect to the server.
For the sake of this example, let's let anyone from anywhere connect to the server.
On your droplet, type:
This will open up your Redis Config file using Vim, we need to look for the line:
and comment it out
You also need to turn off Protective Mode by changing this line from:
to
Save (:wq) and restart the Redis Server by:
Now on your local computer, connect to the Redis droplet by
where 104.131.18.33 is your server IP, and you should be able to connect and run the INFO command!
Of course the above was just to demonstrate how you would connect from a remote location, in practice, you should always limit connection with an IP whitelist and set a password to your Redis server. Digital Ocean's droplet comes with a lot of best practices built in, so it's definitely my preferred method of installation.
Docker
Docker is the one of the hottest Containerization technology around the block. With Docker, you can isolate Redis on a 'container' on your computer to reach true isolation while maximizing your server utilization, This part of the tutorial is useful if you are thinking about adding Redis to your existing stack. If you need an intro to Docker, you can read about my tutorial here: Using Docker to deploy Apache, Nginx, WordPress and Nodejs containers with Digital Ocean and Creating a LEMP stack with Docker Compose
To start, we need to create a directory on either your local machine or your server:
open up your favorite editor and edit the docker-compose.yml file, in my case I use Visual Studio Code:
We're going to install Redis and a very helpful companion app call Redis Commander.
Start a docker terminal session and type:
and you should see your containers being created:
believe it or not, we're done! Now we should be able to connect to the containerized Redis server from inside Docker. Let's find the IP address of the container instance:
this will return the IP of our docker virtual machine. In my case it returned 192.168.99.101 , we also know the port of our Redis server because we defined it in our docker-compose file, the port to the redis server is 9000. Now that we have the information we need to connect to our Redis instance, let's start a connection.
With our local Redis client installed, we can connect to our container via the following command:
Amazing isn't it? Let's set a key in Redis so we can demonstrate the power of Redis Commander:
Redis Commander was set up on port 9001, so we need to head over to http://192.168.99.101:9001/ in order to access the web UI:
The UI is really straight forward, now you can VISUALLY see all your keys and perform operations on them!
Head over to Redis Commander website for more info on what it's capable of.
Of course the above Docker workflow can be performed within your Digital Ocean droplet as well (Digital Ocean also provide droplets pre-built with Docker).
That is it for our intro lesson! We went over three ways to set up Redis today, and we set up Redis Commander to help us see our data inside Redis better. Next time we'll talk about how we can use what we setup today to power our applications. I'll see you there!
If you enjoyed this tutorial, make sure to subscribe to our Youtube Channel and follow us on Twitter @pentacodevids for latest updates!
You've heard that Redis has an embedded scripting language, but haven't given ita try yet? Here's a tour of what you need to understand to use thepower of Lua with your Redis server.
Hello, Lua!
Our first Redis Lua script just returns a value without actually interacting withRedis in any meaningful way:
This is as simple as it gets. The first line sets up a local variable with ourmessage, and the second line returns that value from the Redis server to theclient. Save this file locally as hello.lua and run it like so:
Connectivity Problems?
This redis-cli example assumes that you're running a Redisserver locally. If you're working with a remote server like RedisGreen, you'llneed to specify host and port information. Find the Connect button onyour RedisGreen dashboard to quickly copy the login info for your server.
See also: Could not connectto Redis at 127.0.0.1:6379: Connection refused.
Running this will print 'Hello, world!'. The first argument ofEVAL is the complete lua script — here we're using the catcommand to read the script from a file. The second argument is the number of Rediskeys that the script will access. Our simple 'Hello World' script doesn'taccess any keys, so we use 0.
Accessing Keys and Arguments
Suppose we're building a URL-shortener. Each time a URL comes in we want to storeit and return a unique number that can be usedto access the URL later.
We'll use a Lua script to get a unique ID from Redis using INCR andimmediately store the URL in a hash that is keyed by the unique ID:
We're accessing Redis for the first time here, using the call() function.call()'s arguments are the commands to send to Redis: first we INCR ,then we HSET . These two commands will run sequentially— Redis won't do anything else while this script executes, and it will runextremely quickly.
We're accessing two Lua tables, KEYS and ARGV. Tables are associativearrays, and Lua's only mechanism for structuring data. For ourpurposes you can think of them as the equivalent of an array in whateverlanguage you're most comfortable with, but note these two Lua-isms that trip upfolks new to the language:
Redis Connection Refused
Tables are one-based, that is, indexing starts at 1. So the first element in
mytableismytable[1], the second ismytable[2], etc.Tables cannot hold nil values. If an operation would yield a table of
[ 1,nil, 3, 4 ], the result will instead be[ 1 ]— the table istruncated at the first nil value.
When we invoke this script, we need to also pass along the values for the KEYSand ARGV tables. In the raw Redis protocol, the command looks like this:
When calling EVAL, after the script we provide 2 as the number ofKEYS that will be accessed, then we list our KEYS, and finally we providevalues for ARGV.
Normally when we build apps with Redis Lua scripts, the Redisclient library will take care of specifying the number of keys. The above codeblock is shown for completeness, but here's the easier way to do this on at thecommand line:
Redis Connection Refused Mac Download
When using --eval as above, the comma separates KEYS[] from ARGV[] items.
Just to make things clear, here's our original script again, this time withKEYS and ARGV expanded:
When writing Lua scripts for Redis, every key that isaccessed should be accessed only by the KEYS table. The ARGV table is used for parameter-passing — here it's the value ofthe URL we want to store.
Conditional Logic: increx and hincrex
Our example above saves the link for our URL-shortener, but we also need totrack the number of times a URL has been accessed. To do that we'll keep acounter in a hash in Redis. When a user comes along with a link identifier,we'll check to see if it exists, and increment our counter for it if it does:
Each time someone clicks on a shortlink, we run this script to track that thelink was shared again. We invoke the script using EVAL and pass inlinks:visits for our single key and the link identifier returned from ourprevious script as the single argument.
The script would look almost the same without hashes. Here's ascript which increments a standard Redis key only if it exists:
SCRIPT LOAD and EVALSHA
Remember that when Redis is running a Lua script, it will not run anything else.The best scripts simply extend the existing Redis vocabulary of small atomic dataoperations with the smallest bit of logic necessary. Bugs in Lua scripts can lockup a Redis server altogether — best to keep things short and easy to debug.
Even though they're usually quite short, we need not specify the full Lua scripteach time we want to run one. In a real application you'll instead register eachof your Lua scripts with Redis when your application boots (or when you deploy),then call the scripts later by their unique SHA-1 identifier.
An explicit call to SCRIPT LOAD is usually unnecessary in a live applicationsince EVAL implicitly loads the script that is passed to it. An applicationcan attempt to EVALSHA optimistically and fall back to EVAL only if the scriptis not found.
If you're a Ruby programmer, take a look at Shopify's Wolverine,which simplifies the loading and storing of Lua scripts for Ruby apps. For PHPprogrammers, Predis supports adding Luascripts to be called just as though they were normal Redis commands. If youuse these or other tools to standardize your interaction with Lua, let me know— I'd be interested to find out what else is out there.
When to use Lua?
Redis support for Lua overlaps somewhat with WATCH/MULTI/EXECblocks, which group operations so they are executed together. So how do youchoose to use one over the other? Each operation in a MULTI block needs to beindependent, but with Lua, later operations candepend on the results of earlier operations. Using Lua scripts can also avoidrace conditions that can starve slow clients when WATCH is used.
From what we've seen at RedisGreen, most apps that use Lua will also useMULTI/EXEC, but not vice versa. Most successful Lua scripts are tiny, and justimplement a single feature that your app needs but isn't a part of the Redisvocabulary.
Visiting the Library
The Redis Lua interpreter loads seven libraries: base, table,string,math,debug,cjson,and cmsgpack. The firstseveral are standard libraries that allow you to do the basic operations you'dexpect from any language. The last two let Redis understand JSON and MessagePack— this is an extremely useful feature, and I keep wondering why I don'tsee it used more often.
Web apps with public APIs tend to have JSON lying around all over. So maybe youhave a bunch of JSON blobs stored in normal Redis keys and you want to accesssome particular values inside of them, as though you had stored them as a hash.With Redis JSON support, that's easy:
Here we check to see if the key exists and quickly return nil if not. Then weget the JSON value out of Redis, parse it with cjson.decode(), and return therequested value.
Loading this script into your Redis server lets you treat JSON values stored inRedis as though they were hashes. If your objects are reasonably small, this isactually quite fast, even though we have to parse the value on each access.
If you're working on an internal API for a system that demands performance,you're likely to choose MessagePack over JSON, as it's smaller and faster.Luckily with Redis (as in most places), MessagePack is pretty much a drop-inreplacement for JSON:
Crunching Numbers
Lua and Redis have different type systems, so it's important to understandhow values may change when crossing the Redis-Lua border. When a number comes fromLua back to a Redis client, it becomes aninteger — any digits past the decimal point are dropped:
When you run this script, Redis will return an integer of 3 — you lose theinteresting pieces of pi. Seems simple enough, but things get a bit more trickywhen you start interacting with Redis in the middle of the script. An example:
The resulting value here is astring: '3.2' Why? Redis doesn't have a dedicated numeric type. When we first SET the value, Redis saves it as a string, losing all record of the fact that Lua initially thought of the value as a float. When we pull the value out later, it's still a string.
Values in Redis that are accessed with GET/SET should be thought of as strings exceptwhen numeric operations like INCR and DECR are run against them. Thesespecial numeric operations will actually return integer replies (andmanipulate the stored value according to mathematical rules), but the'type' of the value stored in Redis is still a string value.
Gotchas: A Summary
These are the most common errors that we see when working with Lua in Redis:
Tables are one-based in Lua, unlike most popular languages. Thefirst element in the KEYS table is
KEYS[1], the second isKEYS[2],etc.A nil value terminates a table in Lua. So
[ 1, 2, nil, 3 ]willautomatically become[1, 2]. Don't use nil values in tables.redis.callwill raise exception-style Lua errors, whileredis.pcallwill automatically trap any errors and return them astables that can be inspected.Lua numbers are converted to integers when being sent to Redis — everythingpast the decimal point is lost. Convert any floating point numbers to stringsbefore returning them.
Be sure to specify all the keys you use in your Lua scripts in the
KEYStable, otherwise your scripts will probably break in future versions ofRedis.Lua scripts are just like any other operation in Redis: nothing else runswhile they're being executed. Think ofscripts as a way to expand the vocabulary of the Redis server — keep themshort and to-the-point.
Further Reading
There are lots of great resources for Lua and Redis online — here are a few Iuse:
