0

I hope somebody can help since I really have no more idea on how to solve the following problem: I want so serialize some objects of Entity and ThumbnailUrlEntity (see code below) with Spring Data Redisin a respective CrudRepository. Since the classes have members I don't want to serialize, I tried to ignore them in various ways by either annotating the Classes themselves with @JsonIgnoreProperties or by annotating the respective members / properties with @JsonIgnore or with Filters in the RedisConfig.java by a CustomConversion.

Now the Problem is that no matter which way of ignoring I try I always get the same Error org.springframework.data.keyvalue.core.UncategorizedKeyValueException: Path to property must not be null or empty.; nested exception is java.lang.IllegalArgumentException: Path to property must not be null or empty. when I try to save the respective object. (see Stacktrace below for further details)

I even tried it with MixIn's for those classes but it doesn't work as well...

So the question: How to correctly ignore properties when serializing an object of a certain class with Spring-Data-Redis?

Entity.java

@EqualsAndHashCode(exclude = "domainObject")
@ToString(exclude = "domainObject")
@NoArgsConstructor
//@JsonIgnoreProperties("domainObject")
@JsonFilter("myEntityFilter")
public abstract class Entity<T extends DomainObject> {
    @TimeToLive
    protected final static long TIME_TO_LIVE = 60 * 60 * 24;

    @Id
    @Indexed
    private String id;

//    @Transient
//    @JsonIgnore
    private T domainObject;

    public Entity(@NotNull T domainObject) {
        this.domainObject = domainObject;
    }

    public abstract Entity<T> createFromDomainObject(@NotNull T domainObject);

    public abstract void updateFromDomainObject(T domainObject);

    public void updateFromDomainObject() {
        this.updateFromDomainObject(this.getDomainObject());
    }

    public String getId() {
        return id;
    }

//    @Transient
//    @JsonIgnore
    public T getDomainObject() {
        return domainObject;
    }

    public void setDomainObject(T domainObject) {
        this.domainObject = domainObject;
    }
}

ThumbnailUrlListEntity.java

@AllArgsConstructor
@NoArgsConstructor
@EqualsAndHashCode(callSuper = false, exclude = "parentList")
@ToString(exclude = "parentList")
@RedisHash("thumbnail_url_entity")
//@JsonIgnoreProperties("parentList")
@JsonFilter("myThumbnailUrlEntityFilter")
public class ThumbnailUrlEntity extends Entity<ThumbnailUrl> {

    @Indexed
    private String url;

    private Integer priority;

//    @Transient
//    @JsonIgnore
    private ThumbnailUrlListEntity parentList;

    public ThumbnailUrlEntity(@NotNull ThumbnailUrl domainObject) {
        super(domainObject);
        this.updateFromDomainObject(domainObject);
    }

    @Override
    public Entity<ThumbnailUrl> createFromDomainObject(@NotNull ThumbnailUrl domainObject) {
        ThumbnailUrlEntity entity = new ThumbnailUrlEntity();
        entity.url = domainObject.getUrl();
        entity.priority = domainObject.getPriority();
        return entity;
    }

    @Override
    public void updateFromDomainObject(@NotNull ThumbnailUrl domainObject) {
        this.url = domainObject.getUrl();
        this.priority = domainObject.getPriority();
    }

    public String getUrl() {
        return url;
    }

    public Integer getPriority() {
        return priority;
    }

    public void setPriority(Integer priority) {
        this.priority = priority;
    }

//    @Transient
//    @JsonIgnore
    public ThumbnailUrlListEntity getParentList() {
        return parentList;
    }

    public void setParentList(ThumbnailUrlListEntity parentList) {
        this.parentList = parentList;
    }
}

RedisConfig.java

@Configuration
@EnableRedisRepositories(basePackages = "nlp.floschne.thumbnailAnnotator.db")
public class RedisConfig {

    @Bean
    RedisConnectionFactory connectionFactory() {
        return new LettuceConnectionFactory();
    }

    @Bean
    RedisTemplate<?, ?> redisTemplate(RedisConnectionFactory connectionFactory) {

        RedisTemplate<String, byte[]> template = new RedisTemplate<>();
        template.setConnectionFactory(connectionFactory);
        template.setKeySerializer(new StringRedisSerializer());

        ObjectMapper mapper = new ObjectMapper();
        mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
        mapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
        mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_NULL);
        mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_EMPTY);


        SimpleBeanPropertyFilter theFilter = SimpleBeanPropertyFilter.serializeAllExcept("domainObject");
        FilterProvider filters = new SimpleFilterProvider()
                .addFilter("myEntityFilter", theFilter)
                .addFilter("myThumbnailUrlEntityFilter", theFilter);

        mapper.setFilters(filters);


        template.setValueSerializer(new GenericJackson2JsonRedisSerializer(mapper));

        return template;
    }

    @Bean
    public RedisCustomConversions redisCustomConversions() {
        return new RedisCustomConversions(Arrays.asList(
                new EntityToBytesConverter(),
                new BytesToEntityConverter(),
                new ThumbnailUrlEntityToBytesConverter(),
                new BytesToThumbnailUrlEntityConverter()));
    }

    @WritingConverter
    public class EntityToBytesConverter implements Converter<Entity<?>, byte[]> {

        private final GenericJackson2JsonRedisSerializer serializer;

        public EntityToBytesConverter() {
            ObjectMapper mapper = new ObjectMapper();
            mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
            mapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_NULL);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_EMPTY);

            SimpleBeanPropertyFilter theFilter = SimpleBeanPropertyFilter.serializeAllExcept("domainObject");
            FilterProvider filters = new SimpleFilterProvider().addFilter("myEntityFilter", theFilter);

            mapper.setFilters(filters);

            serializer = new GenericJackson2JsonRedisSerializer(mapper);
        }

        @Override
        public byte[] convert(Entity value) {
            return serializer.serialize(value);
        }
    }

    @ReadingConverter
    public class BytesToEntityConverter implements Converter<byte[], Entity<?>> {

        private final GenericJackson2JsonRedisSerializer serializer;

        public BytesToEntityConverter() {
            ObjectMapper mapper = new ObjectMapper();
            mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
            mapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_NULL);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_EMPTY);


            SimpleBeanPropertyFilter theFilter = SimpleBeanPropertyFilter.serializeAllExcept("domainObject");
            FilterProvider filters = new SimpleFilterProvider().addFilter("myEntityFilter", theFilter);

            mapper.setFilters(filters);

            serializer = new GenericJackson2JsonRedisSerializer(mapper);
        }

        @Override
        public Entity convert(byte[] value) {
            return (Entity) serializer.deserialize(value);
        }
    }

    @WritingConverter
    public class ThumbnailUrlEntityToBytesConverter implements Converter<ThumbnailUrlEntity, byte[]> {

        private final GenericJackson2JsonRedisSerializer serializer;

        public ThumbnailUrlEntityToBytesConverter() {
            ObjectMapper mapper = new ObjectMapper();
            mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
            mapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_NULL);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_EMPTY);

            SimpleBeanPropertyFilter theFilter = SimpleBeanPropertyFilter.serializeAllExcept("parentList");
            FilterProvider filters = new SimpleFilterProvider().addFilter("myThumbnailUrlEntityFilter", theFilter);

            mapper.setFilters(filters);

            serializer = new GenericJackson2JsonRedisSerializer(mapper);
        }

        @Override
        public byte[] convert(ThumbnailUrlEntity value) {
            return serializer.serialize(value);
        }
    }

    @ReadingConverter
    public class BytesToThumbnailUrlEntityConverter implements Converter<byte[], ThumbnailUrlEntity> {

        private final GenericJackson2JsonRedisSerializer serializer;

        public BytesToThumbnailUrlEntityConverter() {
            ObjectMapper mapper = new ObjectMapper();
            mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
            mapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_NULL);
            mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_EMPTY);


            SimpleBeanPropertyFilter theFilter = SimpleBeanPropertyFilter.serializeAllExcept("parentList");
            FilterProvider filters = new SimpleFilterProvider().addFilter("myThumbnailUrlEntityFilter", theFilter);

            mapper.setFilters(filters);

            serializer = new GenericJackson2JsonRedisSerializer(mapper);
        }

        @Override
        public ThumbnailUrlEntity convert(byte[] value) {
            return (ThumbnailUrlEntity) serializer.deserialize(value);
        }
    }
}

ThumbnailUrlRepository

public interface ThumbnailUrlEntityRepository extends CrudRepository<ThumbnailUrlEntity, String> {
}

StackTrace w/ Exception

org.springframework.data.keyvalue.core.UncategorizedKeyValueException: Path to property must not be null or empty.; nested exception is java.lang.IllegalArgumentException: Path to property must not be null or empty.

    at org.springframework.data.keyvalue.core.KeyValuePersistenceExceptionTranslator.translateExceptionIfPossible(KeyValuePersistenceExceptionTranslator.java:55)
    at org.springframework.data.keyvalue.core.KeyValueTemplate.resolveExceptionIfPossible(KeyValueTemplate.java:459)
    at org.springframework.data.keyvalue.core.KeyValueTemplate.execute(KeyValueTemplate.java:345)
    at org.springframework.data.keyvalue.core.KeyValueTemplate.insert(KeyValueTemplate.java:158)
    at org.springframework.data.redis.core.RedisKeyValueTemplate.insert(RedisKeyValueTemplate.java:125)
    at org.springframework.data.keyvalue.core.KeyValueTemplate.insert(KeyValueTemplate.java:140)
    at org.springframework.data.keyvalue.repository.support.SimpleKeyValueRepository.save(SimpleKeyValueRepository.java:101)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:377)
    at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:200)
    at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:629)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
    at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:593)
    at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:578)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
    at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
    at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy43.save(Unknown Source)
    at nlp.floschne.thumbnailAnnotator.db.repository.ThumbnailUrlEntityRepositoryTests.whenSaving_thenAvailableOnRetrieval(ThumbnailUrlEntityRepositoryTests.java:34)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:73)
    at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:83)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
    at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
    at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.IllegalArgumentException: Path to property must not be null or empty.
    at org.springframework.util.Assert.hasText(Assert.java:276)
    at org.springframework.data.redis.core.convert.Bucket.put(Bucket.java:72)
    at org.springframework.data.redis.core.convert.MappingRedisConverter.writeToBucket(MappingRedisConverter.java:750)
    at org.springframework.data.redis.core.convert.MappingRedisConverter.writeInternal(MappingRedisConverter.java:571)
    at org.springframework.data.redis.core.convert.MappingRedisConverter.write(MappingRedisConverter.java:396)
    at org.springframework.data.redis.core.convert.MappingRedisConverter.write(MappingRedisConverter.java:122)
    at org.springframework.data.redis.core.RedisKeyValueAdapter.put(RedisKeyValueAdapter.java:208)
    at org.springframework.data.keyvalue.core.KeyValueTemplate.lambda$insert$0(KeyValueTemplate.java:165)
    at org.springframework.data.keyvalue.core.KeyValueTemplate.execute(KeyValueTemplate.java:343)
    ... 58 more

Update I tried using a custom serializer (see code below) and configured it in my RedisConfig (again see code below) but that still doesn't help and I get the exact same exception..

By the way I already took the answer of "Christoph Strobl" here: spring-data-redis Jackson serialization into account :-)

EntityJsonSerializer.java

 public class EntityJsonSerializer implements RedisSerializer<Entity> {

        private final ObjectMapper mapper;

        public EntityJsonSerializer() {
            this.mapper = new ObjectMapper();
            this.mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL, JsonTypeInfo.As.PROPERTY);
            this.mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
            this.mapper.setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
            this.mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_NULL);
            this.mapper.setDefaultPropertyInclusion(JsonInclude.Include.NON_EMPTY);


            SimpleBeanPropertyFilter theFilter = SimpleBeanPropertyFilter.serializeAllExcept("domainObject", "parentList");

            FilterProvider filters = new SimpleFilterProvider()
                    .addFilter("myEntityFilter", theFilter)
                    .addFilter("myThumbnailUrlEntityFilter", theFilter);

            this.mapper.setFilters(filters);
        }

        public EntityJsonSerializer(ObjectMapper mapper) {
            this.mapper = mapper;
        }

        @Override
        public byte[] serialize(Entity t) throws SerializationException {
            try {
                return mapper.writeValueAsBytes(t);
            } catch (JsonProcessingException e) {
                throw new SerializationException(e.getMessage(), e);
            }
        }

        @Override
        public Entity deserialize(byte[] bytes) throws SerializationException {

            if (bytes == null) {
                return null;
            }

            try {
                return mapper.readValue(bytes, Entity.class);
            } catch (Exception e) {
                throw new SerializationException(e.getMessage(), e);
            }
        }
    }

Updated RedisConfig.java

@Configuration
@EnableRedisRepositories(basePackages = "nlp.floschne.thumbnailAnnotator.db")
public class RedisConfig {

    @Bean
    RedisConnectionFactory connectionFactory() {
        return new LettuceConnectionFactory();
    }

    @Bean
    RedisTemplate<?, ?> redisTemplate(RedisConnectionFactory connectionFactory) {

        RedisTemplate<String, byte[]> template = new RedisTemplate<>();
        template.setConnectionFactory(connectionFactory);
        template.setKeySerializer(new StringRedisSerializer());

        template.setValueSerializer(new EntityJsonSerializer());

        return template;
    }

    @Bean
    public RedisCustomConversions redisCustomConversions() {
        return new RedisCustomConversions(Arrays.asList(
                new EntityToBytesConverter(),
                new BytesToEntityConverter(),
                new ThumbnailUrlEntityToBytesConverter(),
                new BytesToThumbnailUrlEntityConverter()));
    }

    @WritingConverter
    public class EntityToBytesConverter implements Converter<Entity<?>, byte[]> {

        private final EntityJsonSerializer serializer;

        public EntityToBytesConverter() {
            serializer = new EntityJsonSerializer();
        }

        @Override
        public byte[] convert(Entity value) {
            return serializer.serialize(value);
        }
    }

    @ReadingConverter
    public class BytesToEntityConverter implements Converter<byte[], Entity<?>> {

        private final EntityJsonSerializer serializer;

        public BytesToEntityConverter() {
            serializer = new EntityJsonSerializer();
        }

        @Override
        public Entity convert(byte[] value) {
            return (Entity) serializer.deserialize(value);
        }
    }

    @WritingConverter
    public class ThumbnailUrlEntityToBytesConverter implements Converter<ThumbnailUrlEntity, byte[]> {

        private final EntityJsonSerializer serializer;

        public ThumbnailUrlEntityToBytesConverter() {
            serializer = new EntityJsonSerializer();
        }

        @Override
        public byte[] convert(ThumbnailUrlEntity value) {
            return serializer.serialize(value);
        }
    }

    @ReadingConverter
    public class BytesToThumbnailUrlEntityConverter implements Converter<byte[], ThumbnailUrlEntity> {

        private final EntityJsonSerializer serializer;

        public BytesToThumbnailUrlEntityConverter() {
            serializer = new EntityJsonSerializer();
        }

        @Override
        public ThumbnailUrlEntity convert(byte[] value) {
            return (ThumbnailUrlEntity) serializer.deserialize(value);
        }
    }
}
Sumesh TG
  • 2,367
  • 1
  • 12
  • 29
p0w3r
  • 113
  • 2
  • 12
  • Use @jsonignore. Your error not related to serialization. – Sumesh TG Sep 06 '18 at 10:22
  • Well as already stated, I tried this before but it doesn't help. And I'm pretty sure the problem is somewhere in the Redis serialization since I tried to serialize without Redis (simply to a String with Jackson) and the annotations to ignore properties or a custom ObjectMapper work perfectly.. only when I try to do it with Spring Data Redis I get into those problems. – p0w3r Sep 06 '18 at 18:47
  • try `@transient` annotation too – Sumesh TG Sep 06 '18 at 19:51
  • Yeah I tried @Transient too :) the problem with that is, that it does not ignore the property but set it to null when serializing. I want to get totally rid of the property – p0w3r Sep 06 '18 at 19:54
  • Their docs say that basic interface serialization and deserialization of Objects to byte arrays (binary data). It is recommended that implementations are designed to handle null objects/empty arrays on serialization and deserialization side. Note that Redis does not accept null keys or values but can return null replies (for non existing keys). – Sumesh TG Sep 07 '18 at 05:54

1 Answers1

2

I think you can do one of the following:

First, you can try to get the current version work, it is not easy it seems.

Have you configured the correct serializer for Redis? On the link there is this answer:

Use StringRedisTemplate to replace RedisTemplate.

By default, RedisTemplate uses Java serialization, StringRedisTemplate uses StringRedisSerializer.

This could solve your problem too...

Second option to use dedicated POJOs for Redis. You can do the value exchange manually or by using Java bean mappers, like MapStruct.

Third option could be to use a custom serializer

m4gic
  • 1,339
  • 8
  • 16
  • Thanks for your answer! I just read through the MapStruct docs and I think I'll stick with that – p0w3r Sep 06 '18 at 19:53