I'm writing real time application that needs fast access to some resources. I'm using a relational database and redis. I use the relational database for safe storing of the resources and redis for fast access of those same resources.
The problem I'm facing is that the code for maintaining consistency between these two stores becomes very complicated. I need to check if writes are being done to both and if one of those fails, undo the one that worked.
I thought of using something like kafka where the write would be sent to a specific topic and having different consumers (SQLConsumer and RedisConsumer) that would write to the respective databases. This way, the consumers would retry indefinitely and achieve eventual consistency. Ideally, the message wouldn't be committed until the write was successful.
Is this a common/correct approach? Is there other way in which I could improve my architecture?
