CODEWITHSUNDEEP
javaredisspring-data-redislettuce
maslan
maslan

Reputation: 2178

Spring Data REDIS - hash keys with weird prefixes, and HSCAN not returning results correctly

I am using spring boot (irrelevant) with spring-data-redis:jar:2.0.9, which uses lettuce to connect to my REDIS. I am using a hash structure that contains around 100 keys. Under those keys I put some objects which type is also irrelevant:

private static final String HASH_KEY_NAME = "myspecialhashes:somekey";

@Autowired
private RedisTemplate<String, MyDto> myDtoRedisTemplate;

Now I am simply placing a list of my objects into the hash using their id as the key:

myDtoRedisTemplate.opsForHash().put(HASH_KEY_NAME, dto.getId(), dto);

This works fine, and retrieval of all elements from the hash is fine, as well as retrieving keys only

List allDtosRaw = myDtoRedisTemplate.opsForHash().values(HASH_KEY_NAME);

Also when listing keys:

myDtoRedisTemplate.boundHashOps(HASH_KEY_NAME).keys()

it seems fine and the set of keys returned starts with:

(java.util.LinkedHashSet<E>) [fakeservicetest:dummy3:write, fakesingle:dummy:sub1:write, ....

Since there are alot of keys I wanted to be able to filter list of objects STARTING with a token using HSCAN instead of getting all keys and filtering them in my Java app. So, this is how I did the HSCAN to get all hash entries starting with "fake"

List filteredDtosRaw = new LinkedList<>();
ScanOptions scanOptions = ScanOptions.scanOptions().match("fake*").count(10000).build();
Cursor cursor = myDtoRedisTemplate.boundHashOps(HASH_KEY_NAME).scan(scanOptions);
        cursor.forEachRemaining(filteredDtosRaw ::add);

Unfortunately this returns zero results. I was trying various ways to fix that and get some results. Eventually I turned to redis command line to see what is REDIS thinking of all of this

redis-cli HSCAN "myspecialhashes:somekey" 0 MATCH "fake*" COUNT 1000

zero results indeed. The next thing was to view all keys in that has and see what is actually in the hash

redis-cli HGETALL "myspecialhashes:somekey"

The result is like : 

1) "0"
2)  1) "\xac\xed\x00\x05t\x00\x1cfakeservicetest:dummy3:write"
    2) "{\"@class\":\"
.....

So it appears, that the keys contain a prefix that are some Unicode characters. This is probably due to String serialization (I have checked the strings before being put to REDIS with a debugger, and they do not contain any invisible characters at the beginning). So I now have a workaround that works : I can search for "*fake*" and it works both in REDIS CLI and in Spring Data Redis. And since I want to only have only those that begin with "fake" I can filter this in my Java app using String.startsWith. But since I do not like workarounds I would like to know If I am using spring data redis incorrectly, or there is some inconsistency when serializing string for putting into REDIS and for those that are used in SCANs?

Upvotes: 5

Views: 5898

Answers (1)

maslan
maslan

Reputation: 2178

Ok I know now. I have been configured redis serializer for string serializer, but it appears that hash keys need to have serializer for hash keys set separately. Those "weird Unicode chars" are a result of JdkSerializer

@Bean
public RedisTemplate<String, MyDto> redisTemplateMyDto() {
    final RedisTemplate<String, MyDto> template = new RedisTemplate<String, MyDto>();
    template.setConnectionFactory(redisConnectionFactory);
    template.setKeySerializer(new StringRedisSerializer());
    template.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
    template.setValueSerializer(new GenericJackson2JsonRedisSerializer());
    return template;
}

Changed to 

@Bean
public RedisTemplate<String, MyDto> redisTemplateMyDto() {
    final RedisTemplate<String, MyDto> template = new RedisTemplate<String, MyDto>();
    template.setConnectionFactory(redisConnectionFactory);
    template.setHashKeySerializer(new StringRedisSerializer());
    template.setKeySerializer(new StringRedisSerializer());
    template.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
    template.setValueSerializer(new GenericJackson2JsonRedisSerializer());
    return template;
}

Upvotes: 9

Related Questions