Distributed cache using Redis and ASP.NET Core

What is Redis ?

Redis is a super fast non-relational database that uses keys to map to different data types. It is a key value data store (NoSQL) allowing to solve efficiently many different problem sets. Redis was created by Salvatore Sanfilippo in 2009, and Sanfilippo still remains the lead developer of the project today. It is a mature and hugely popular open source project, being used by many companies and in countless mission-critical production environments.

Here is an interview with the inventor of Redis, Salvatore Sanfilippo.

Why is Redis popular?

Not only is it extremely effective, but it is also relatively simple. Getting started with Redis is quite fast, and it usually takes only a few minutes to set up and get them working within an application. Thus, a small investment of time and effort can have an immediate, dramatic impact on performance of the application.

Just to name two cases when Redis is helpful:

• Redis is used at Pinterest – see use case
• or at Twitter, where Raffi Kirkorian, VP of Engineering at Twitter, explains how Redis helps to support over 30 billion timeline updates per day based on 5000 tweets per second or 400,000,000 tweets per day – see here the presentation.

Easy access the code

The full client web application that display the Twitter messages could be found on Github – https://github.com/fpetru/redis-aspnetcore-caching.

To install

Here are all the things needed to be installed locally:

• Visual Studio Community 2015 with Update 3 (or just Update 3 if you have already installed) and .NET Core 1.0.1 – VS 2015 Tooling Preview 2
• .NET Core 1.1 SDK

Installing Redis Server on Windows

Create a new ASP.NET Core project, and from Nuget select for installing Redis-64. The server is installed in the default Nuget path. To start the server run next in command prompt:

C:\Users\[logged in user]\.nuget\packages\Redis-64\[version installed]\tools>redis-server.exe
# in my case
C:\Users\petru\.nuget\packages\Redis-64\3.0.503\tools>redis-server.exe

In the same folder, there is a document describing how to install Redis as a Windows Service (check the file Windows Service Documentation.docx). For running the simple scenarios from this article, running just from command line should be enough.

Caching in ASP.NET Core using Redis

To use Redis in ASP.NET Core, we need to reference Microsoft.Extensions.Caching.Redis.Core package. Additionally, in our sample we would need also the Session package. Installing these could be done either using Nuget or extending the project.json file:

"Microsoft.Extensions.Caching.Redis.Core": "1.0.3",
"Microsoft.AspNetCore.Session": "1.1.0"

To enable Redis in the application we need to add AddDistributedCache method in ConfigureServices method.

public void ConfigureServices(IServiceCollection services)
{
    services.AddDistributedRedisCache(options =>
    {
        options.InstanceName = "Sample";
        options.Configuration = "localhost";
    });
    services.AddMvc();
}

To enable the session, we need to do changes in both ConfigureServices and Configure:

public void ConfigureServices(IServiceCollection services)
{
    services.AddDistributedRedisCache(options =>
    {
        options.InstanceName = "Sample";
        options.Configuration = "localhost";
    });

    services.AddSession();
    services.AddMvc();
}

public void Configure(IApplicationBuilder app, 
    IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    app.UseSession();
    app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });
}

How do we access Redis?

To cache a value in Redis we use:

var valueToStoreInRedis = Encoding.UTF8.GetBytes("This is a cached value from Redis");
HttpContext.Session.Set("TestProperty", valueToStoreInRedis);

To retrieve the value from cache we use:

var valueFromRedis = default(byte[]);
if (HttpContext.Session.TryGetValue("TestProperty", out valueFromRedis))
    valueToDisplay = Encoding.UTF8.GetString(valueFromRedis);

Twitter client example

External interfaces to connect to social platforms are usually relatively slow to access. If we would do a web application showing messages from Twitter, it would take a couple of seconds to load the page (to make the search and retrieve). If the same user would connect again to our application, Redis could retrieve from memory the same messages, without accessing the server again. Caching the results, and updating them only when new messages appear, bring a huge improvement to the overall performance.

To keep the things simple, I have earlier written a small client in Python, that connects to Twitter and save the details in a JSON file. You can read more details in the article Visual Studio Code – Connect to Twitter with Python. It includes the full source code, and very fast you can access and save data in a JSON file.

In our sample here, we start from an already available JSON file.

[
 {
        "Id": 0,
        "ProfileImage": "https://pbs.twimg.com/profile_images/1772973596/inogiclogo_normal.png",
        "ProfileDescription": "Microsoft Partner with Gold Competency in #Dynamics365 #MSDynCRM providing Solns/Services. Innovators of @Maplytics & Inolink #QuickBooks Int #MSDyn365 #MVPBuzz",
        "Username": "Inogic",
        "Text": "Execute the Global Action Using Web API in Dynamics CRM https://t.co/DAuzP6L7FE #MSDyn365 #webapi https://t.co/v0XgyotaFn",
        "ScreenName": "@inogic"
    },
    ....
]

Retrieval of data from Twitter takes a couple of seconds. To simulate this, we add a delay of 20 seconds in our code. Here is the code from the controller.

public IActionResult Index()
{
	var watch = Stopwatch.StartNew();
	string jSONText = RetrieveOrUpdateRedis();
	watch.Stop();

	TempData["DataLoadTime"] = watch.ElapsedMilliseconds;
	var itemsFromjSON = JsonConvert.DeserializeObject>(jSONText);
	return View(itemsFromjSON);
}

private string RetrieveOrUpdateRedis()
{
	var valueFromRedis = default(byte[]);
	string valueToReturn = string.Empty;
	if (HttpContext.Session.TryGetValue("TwitterDataset", out valueFromRedis))
	{
		// Retrieve from Redis
		valueToReturn = Encoding.UTF8.GetString(valueFromRedis);
		TempData["DataLoadType"] = "From Redis";
	}
	else
	{
		// read the file and update the URLs
		var jSONText = System.IO.File.ReadAllText("twitter.json");
		valueToReturn = GetUpdatedFileContent(jSONText);

		// add an artificial delay of 20 seconds (simulating the search is done directly against a Twitter server)
		Thread.Sleep(20000);

		// store values in Redis
		var valueToStoreInRedis = Encoding.UTF8.GetBytes(valueToReturn);
		HttpContext.Session.Set("TwitterDataset", valueToStoreInRedis);
		TempData["DataLoadType"] = "From file";
	}

	return valueToReturn;
}

// a minimum data processing, updating the URLs
private string GetUpdatedFileContent(string jSONText)
{   
	var itemsFromjSON = JsonConvert.DeserializeObject>(jSONText);
	foreach (var item in itemsFromjSON)
	{
		Regex r = new Regex(@"(https?://[^\s]+)");
		item.Text = r.Replace(item.Text, "$1");
	}

	return JsonConvert.SerializeObject(itemsFromjSON);
}

How fast does the application run with Redis?

The initial time to load is displayed below (this includes also the artificial timeout of 20 seconds):

Just taking the results from Redis runs much faster. This exclude any “connection” to Twitter or any other internal update. Here is the print screen with the data retrieved from memory:

To replicate this multiple times, and not just at the beginning, use command flushall to clean entire Redis cache:

C:\Users\[logged in user]\.nuget\packages\Redis-64\[version installed]\tools>redis-cli.exe

# in my case
C:\Users\petru\.nuget\packages\Redis-64\3.0.503\tools>redis-cli.exe

# and then
127.0.0.1:6379> flushall

Is Redis just a cache? When to use it?

Redis is much more than just a cache. Like any cache, Redis stores indeed [key, value] pairs. More than this, Redis allows us to use values in a very efficient ways. Using different data structures (not just strings), we gain a lot of power (such as the ability to fine-tune cache contents and durability) and greater efficiency overall. Once we start using the data structures, the efficiency boost becomes tremendous for specific application scenarios.

Here is a summary of the data structures, along with few concrete use cases associated with each type: