How to use ByteBuffer to get and set data on Apache Redis

In this article we will see, how one can reduce memory usage on redis while storing/getting data in form of ByteBuffer. The Java application will get and set key-values on redis using byte arrays, while converting byte array to/from ByteBuffer.

Let's assume we have following data of per minute site visits, to be stored on redis:

HourMinuteCount
120110
120220
120315


1) Storing data as String key value pair

First let's try to store above data in simple String key-vale pair, above table data can be represented in following string format:

1201:10,1202:20,1203:15

Where 1201, represents 12 th Hour 01 th Minute and so on and value after ":" i.e. represents visit count for that minute.

		String key = "k";
		String value = "1201:10,1202:20,1203:15";

		Jedis jedis = new Jedis("127.0.0.1");
		jedis.set(key.getBytes(),value.getBytes());
After running above code, lets evaluate total memory taken by key-valye in redis using "MEMORY USAGE" command, The MEMORY USAGE command reports the number of bytes that a key and its value require to be stored in RAM. Read More: Here


127.0.0.1:6379> MEMORY USAGE k
(integer) 68


Here one thing is to be noted that, these 68 bytes are not all occupied by key-value data but every key requires a constant extra bytes depending upon the os, even empty key-value requires some bytes for administering purpose, as shown below:
jedis.set("","");

127.0.0.1:6379> MEMORY USAGE ""
(integer) 46


This indicates that for my os the above key "k" took around (68-46) ~ 22 bytes where 1 byte is taken by the key "k" and 20-21 bytes by the data i.e. "1201:10,1202:20,1203:15";

2) Storing data as ByteBuffer key value pair

Now lets try to store above key value using ByteBuffer and evaluate the memory usage, the data represented in table can be stored in a ByteBuffer as follows:

We can store (12 01 10) and (12 02 20) and (12 03 15) in 9 bytes, because 1 byte can store number upto 127 and 12,01 ... will take 1 byte each.

		// Allocating 9 bytes
		ByteBuffer buffer = ByteBuffer.allocate(9);

		// Storing first row: Hour > Minute > Count
		buffer.put((byte) 12);
		buffer.put((byte) 01);
		buffer.put((byte) 10);

		// Storing first row: Hour > Minute > Count
		buffer.put((byte) 12);
		buffer.put((byte) 02);
		buffer.put((byte) 20);

		// Storing first row: Hour > Minute > Count
		buffer.put((byte) 12);
		buffer.put((byte) 03);
		buffer.put((byte) 15);
Now lets store this value to redis:

		String key = "k";

		Jedis jedis = new Jedis("127.0.0.1");
		jedis.set(key.getBytes(), buffer.array());
Lets evaluate total memory taken by key-valye in redis using "MEMORY USAGE" command:


127.0.0.1:6379> MEMORY USAGE "k"
(integer) 54


Here the memory usage is approx 54-46 ~ 9 bytes, as was assigned to byte buffer, this is around 1/2 of the memory used by same data represented as string.

If you try to look for the value of key in redis-cli you will see something like this:


127.0.0.1:6379> GET k
"\x0c\x01\n\x0c\x02\x14\x0c\x03\x0f"


Now lets try to get value of stored ByteBuffer in application and construct actual value thet was stored:

byte [] value= jedis.get(key.getBytes());
		ByteBuffer valueBuffer = ByteBuffer.wrap(value);
		
		System.out.println(valueBuffer.get()+","+valueBuffer.get()+","+valueBuffer.get());
		System.out.println(valueBuffer.get()+","+valueBuffer.get()+","+valueBuffer.get());
		System.out.println(valueBuffer.get()+","+valueBuffer.get()+","+valueBuffer.get());
Output: Output of following piece of code will be something like this:


12,1,10
12,2,20
12,3,15


We have seen how to use ByteBuffer to store data on redis, read more about Redis Cluster Setup on Ubuntu and Spring Data with Standalone Redis.