Beginner's Guide to Redis and Caching with NodeJS

Contents

Today’s APIs rely heavily on data, often sourced from multiple layers of abstractions and sources, from databases to AI models, other APIs, and third-party services. But gathering the information you need for your request is just a step. You still need to process all that information and return it to the client to be useful for your applications.

All these gathering and processing takes time, but what if information that is frequently accessed is already available in fast and efficient storage for us to either serve directly to clients or reduce the processing efforts on each request?

In a nutshell, that’s what we can do with caching. Caching is the process of storing frequently accessed information in a fast and efficient layer. Hence, it is more quickly available than retrieving it from slower sources or having to compute it each time.

Of course, managing this data structure and its updates is another challenge. That’s where Redis comes into play.

What Is Redis?

Redis is an open-source, in-memory data structure store used as a database, cache, and message broker. It has multiple uses, like caching NodeJS applications and API responses for faster performance.

You can think of it as a No-SQL database, which stores data as a key-value pair in the system memory. Redis supports disk-persistent data storage, too, if needed.

Redis supports storing multiple data structures and types , including strings, lists, hashes, sets, and sorted sets. Supported data structures give Redis versatility for many use cases.

Redis Use Cases

Caching

One of the most popular use cases of Redis is caching.

Since Redis is an in-memory database, its data access operations are faster than any disk-bound database could deliver.

Redis promises sub-millisecond-long data processing operations. It makes Redis a perfect candidate for applications that rely on real-time data analysis.

For example, an e-commerce application used for product comparison could benefit from using Redis to cache the data of all its products. This way, the data won’t need to be retrieved from a remote database whenever required and will be available in near real-time without any latency issues.

Redis for session management

If your application uses sessions to track authenticated users and manage user-specific data, Redis is a perfect fit to use as session storage. Using Redis could significantly improve the system’s performance while making it easier to process users’ data, including credentials, recent activities, and even a shopping cart-like system.

Redis as a Queue

You can use Redis to queue application tasks that take a long time to complete. You can implement FIDO (first-in, first-out) queues or create delayed queues to delay task implementation until a pre-scheduled time.

For example, you can use Redis as a queue for sending scheduled emails or queuing system tasks.

Run a Redis Instance

To follow this tutorial, you’ll need a running instance of Redis. It could be running locally or somewhere on the internet as long as it’s accessible from your local machine.

If you don’t already have a Redis instance, you have a few options to get one.

Run a free Redis instance from redis.io
Install Redis locally, here’s how .

Run Redis with Docker (my default for development)

docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest

Run Redis on Linodo (cheap but not free)

Implement Caching in NodeJS Using Redis

In this tutorial, you’ll build an Express API that retrieves data from an external API using Axios. Still, because our third-party dependency could be relatively slow, we’ll implement a caching mechanism to increase the performance of the API and provide a better user experience.

I recommend that you follow the step-by-step tutorial to set up the API and fully understand the code, but if you prefer, you can download the full source code from GitHub .

The first step is to initialize our project. We’ll create a new folder to host our files and initiate npm.

mkdir nodejs_loves_redis && cd nodejs_loves_redis

Then initialize npm

npm init -y

To run the app, we’ll need three dependencies, express, axios, and redis.

npm i express axios redis

Finally, create a new app.js file and add the following code to the file

const express = require('express');
const app = express();
const port = 3000;

app.get('/', (req, res) => {
  res.send('Hello World!');
});

app.listen(port, () => {
  console.log(`NodeJS loves Redis listening on port ${port}`);
});

So far we created a simple express app that will return “Hello World!” when we access the server. To try it out, simply run:

node app.js

If you followed along, you’d read the phrase “NodeJS loves Redis listening on port 3000” on your terminal, and if you access the application in your browser or terminal, you’ll read “Hello World!”

curl http://localhost:3000

Before we move on, there’s nome more thing you can do. So far we are running the NodeJS API with the node command, and it works, but there’s a catch. Each time you change your code, you’ll have to manually restart the node server. To enable hot reload so that the API restarts automatically after your code changes, use nodemon:

npx nodemon app.js

Alternatively, you can install nodemon as a dev dependency. You can read more about this NPM package on the official Nodemon docs .

Retrieving Data from an External API

In this step, you’ll connect your NodeJS API to an external API. You can use any RESTful API . For this tutorial, you’ll use the ToDo API from JSONPlaceHoler , in particular, the endpoint: https://jsonplaceholder.typicode.com/todos?completed=false which returns a list of pending ToDos like the following:

[
  {
    "userId": 1,
    "id": 1,
    "title": "delectus aut autem",
    "completed": false
  },
  {
    "userId": 1,
    "id": 2,
    "title": "quis ut nam facilis et officia qui",
    "completed": false
  }
]

Let’s set it up in the code, back to the app.js file.

const express = require('express');
const axios = require('axios'); // 👈 new code
const app = express();
const port = 3000;

// 👇 new code
async function fetchToDos(completed) {
  const apiResponse = await axios(`https://jsonplaceholder.typicode.com/todos?completed=${completed}`);
  console.log('Data requested from the ToDo API.');
  return apiResponse.data;
}
// ☝️ new code

app.get('/', async (req, res) => {
  res.send(await fetchToDos(req.query.completed)); // 👈 updated code
});

app.listen(port, () => {
  console.log(`NodeJS loves Redis listening on port ${port}`);
});

In summary, your changes added a function to fetch data from the API, passing one parameter for when you need to fetch completed or uncompleted tasks. The variable is a boolean.

Then, on the primary endpoint, you call the new function reading the completed state from the query string and sending the result back to the client.

If you now visit your endpoint, e.g.: http://localhost:3000/?completed=false, you’ll see a long list of ToDos.

And on the terminal, you’ll read:

[nodemon] starting `node app.js`
NodeJS loves Redis listening on port 3000
Data requested from the ToDo API.
Data requested from the ToDo API.
Data requested from the ToDo API.
Data requested from the ToDo API.

Each time you reload the endpoint, a new message, “Data requested from the ToDo API.” appears. This is not very efficient if the external source is slow or unavailable.

Caching the API Results Using Redis

In this section, you’ll optimize your API endpoint by caching the results from the external API so that your endpoint will fetch the external API only the first time a client visits it. All subsequent requests will return the data from the Redis cache.

Adding Redis to your NodeJS API is a two-step process:

Initialize the Redis Cache

First, you need to initialize and connect to the Redis cache. This is normally done once during the API setup.

Connecting to Redis is as simple as registering a client and calling the function connect, for example:

const redisClient = redis.createClient({ url: 'redis://localhost:6379' });
(async () => {
  redisClient.on("error", (error) => console.error(`Ups : ${error}`));
  await redisClient.connect();
})();

If you connect to a local Redis with default setup parameters, you can skip the url parameter from the createClient function.

Reading and Writing the Redis Database

Second and final, you’ll have to update your fetching code to ensure it reads from the cache first. When the cache isn’t available, then you can default back to query the source of the data, e.g., the external API.

To read data from a Redis Database you can use the method get from the Redis Client. For example:

redisClient.get('MY_CACHE_KEY');

Since the key is simply a string, when you need to cache and access data from Redis that depends on other variables, you can create a dynamic key, for example:

redisClient.get('TODOS_COMPLETED');
redisClient.get('TODOS_UNCOMPLETED');

Or to make it easier to work with:

redisClient.get('TODOS_TRUE');
redisClient.get('TODOS_FALSE');

Which is very easy to put together

redisClient.get(`TODOS_${completed}`);

Totally up to you how you use keys. However, as with variable names, be descriptive and document the keys and what type of information they store in comments and the project’s documentation. It can get quite messy for larger projects or shared Redis instances across projects.

If you want to add data to the cache, use the set method.

redisClient.set('MY_CACHE_KEY', 'VALUE');

Redis supports multiple data types out of the box. However, to store certain information, like JSON objects, you’ll have to convert them into strings or other supported data types.

Adding Caching to the API

Now that you know how to connect, read, and write from a Redis server, you can update your API accordingly.

When reading and setting data from/to a Redis instance, it is a good practice to wrap the code on a try…catch block, as you never know if the server will be available at the time, and you may want to default to the source in case Redis is not available.

Here is how the updated code looks like:

const express = require('express');
const axios = require('axios');
const redis = require('redis'); // 👈 new code
const app = express();
const port = 3000;

// 👇 new code
// Initiate and connect to the Redis client
const redisClient = redis.createClient();
(async () => {
  redisClient.on("error", (error) => console.error(`Ups : ${error}`));
  await redisClient.connect();
})();
// ☝️ new code

// 👇 Updated code
async function fetchToDos(completed) {
  const cacheKey = `TODOS_${completed}`;

  // First attempt to retrieve data from the cache
  try {
    const cachedResult = await redisClient.get(cacheKey);
    if (cachedResult) {
      console.log('Data from cache.');
      return cachedResult;
    }
  } catch (error) {
    console.error('Something happened to Redis', error);
  }

  // If the cache is empty or we fail reading it, default back to the API
  const apiResponse = await axios(`https://jsonplaceholder.typicode.com/todos?completed=${completed}`);
  console.log('Data requested from the ToDo API.');

  // Finally, if you got any results, save the data back to the cache
  if (apiResponse.data.length > 0) {
    try {
      await redisClient.set(cacheKey, JSON.stringify(apiResponse.data));
    } catch (error) {
      console.error('Something happened to Redis', error);
    }
  }

  return apiResponse.data;
}
// ☝️ Updated code

app.get('/', async (req, res) => {
  res.send(await fetchToDos(req.query.completed));
});

app.listen(port, () => {
  console.log(`NodeJS loves Redis listening on port ${port}`);
});

In the code above, you followed the implementation for connecting to Redis, and you implemented the read-and-write functionality following the best practice of defaulting to the source when the Redis cache isn’t available.

If you try our new code using the same endpoint as before: http://localhost:3000/?completed=false, you’ll get the same results, even when we refresh the page a few times.

Let’s now look at the console and see what outputs we got this time:

[nodemon] starting `node app.js`
NodeJS loves Redis listening on port 3000
Data requested from the ToDo API.
Data from cache.
Data from cache.
Data from cache.
Data from cache.

That’s great. The first time the client accesses the API, it reads the data from the external API, and all subsequent requests fetch it from the cache.

But what happens if you restart the server and try it again? Let’s find out:

[nodemon] starting `node app.js`
NodeJS loves Redis listening on port 3000
Data from cache.
Data from cache.

This time, you never hit the external API, and each time you fetch the data from the cache, but why? The simple answer is that the data on Redis remains there even if the application or API is closed, and we never removed it.

So the next question you may have is, when will the API fetch new information from the external source? As it is, it is only when the Redis instance is shut down or cleaned, which could be a problem for your application as you may want to have the cache cleaned often.

There are a lot of strategies for handling data caching, and some are pretty complex, but here are a few:

Set cache expiration when saving data into Redis, e.g., Invalidate the ToDo cache after 1 hour.
Trigger cache deletions when data updates from APIs, e.g., invalid the cache when a ToDo is created, updated or deleted.
Run periodical background jobs to generate the data and set the cache on Redis. E.g., with heavy operations where it’s not acceptable to, for example, let the client wait even for the first request.

Those are only some options, and I won’t get into the details. If you want to learn more about caching strategies, I recommend the free ebook by Lee Atchison that covers the topic in detail.

In this tutorial, you’ll implement the first strategy and set an expiration time for the cache. For this tutorial, it’s not critical that the information the API sends to clients is the latest and greatest. It’s ok if it refreshes from time to time.

Implementing Time-Based Cache Invalidation

As discussed, your API has a problem that needs to get fixed. It never updates the cache. To fix that, you’ll update your code to tell Redis that the cache should only be valid for a given period, say ten seconds, after which the cache won’t be valid anymore, and Redis will discard it.

Fortunately for us, this is not something we need to code ourselves, but rather options of the set method. Check out the official Redis Set command documentation to learn more about cache invalidation and other options.

Considering you’d like to test this quickly, you’ll set up a cache validity of ten seconds.

And this is all the code we would need:

redisClient.set('MY_CACHE_KEY', 'VALUE', {
  EX: 10, // Set the specified expire time, in seconds.
});

So, how would this look like in our code?

...

async function fetchToDos(completed) {
  const cacheKey = `TODOS_C_${completed}`; // 👈 updated code

  // First attempt to retrieve data from the cache
  try {
    const cachedResult = await redisClient.get(cacheKey);
    if (cachedResult) {
      console.log('Data from cache.');
      return cachedResult;
    }
  } catch (error) {
    console.error('Something happened to Redis', error);
  }

  // If the cache is empty or we fail reading it, default back to the API
  const apiResponse = await axios(`https://jsonplaceholder.typicode.com/todos?completed=${completed}`);
  console.log('Data requested from the ToDo API.');

  // Finally, if you got any results, save the data back to the cache
  if (apiResponse.data.length > 0) {
    try {
      await redisClient.set(
        cacheKey,
        JSON.stringify(apiResponse.data),
        { EX: 10 }
      ); // 👈 updated code
    } catch (error) {
      console.error('Something happened to Redis', error);
    }
  }

  return apiResponse.data;
}

...

Note that I also changed the cache key, this is because we used the previous key without any expiration. You could also remove the key using the Redis CLI .

Rerunning the API queries you can get an output like the following:

[nodemon] starting `node app.js`
NodeJS loves Redis listening on port 3000
Data requested from the ToDo API.
Data from cache.
Data from cache.
Data from cache.
Data from cache.
Data requested from the ToDo API.
Data from cache.
Data from cache.
Data from cache.

The initial request to your API fetches the external API, and all subsequent requests serve from the cache, but after ten seconds, the cache will invalidate, and your API will query the source again.

And for the requirements that we have, that would be it. Congrats 🎉!

Conclusion

In this tutorial, you’ve seen how NodeJS and Redis can be used together to improve the performance of your API by caching responses. You’ve learned about different strategies for cache invalidation, like tge time-based invalidation, that we have used in our example.

The source code for this project can be found in GitHub .

Thanks for reading!

If you liked what you saw, please support my work!

Donate on GitHub Donate on Patreon

Juan Cruz Martinez

Juan has made it his mission to help aspiring developers unlock their full potential. With over two decades of hands-on programming experience, he understands the challenges and rewards of learning to code. By providing accessible and engaging educational content, Juan has cultivated a community of learners who share their passion for coding. Leveraging his expertise and empathetic teaching approach, Juan has successfully guided countless students on their journey to becoming skilled developers, transforming lives through the power of technology.