Is redis data kept indefinitely?

[c3s4r]

Hi. I was trying your module for an orchestrator microservice.

From the tests, it looks like even after the saga is complete, the data remains in redis. Is that the intended design?

My use case doesn't require the steps to be persisted after completed. Should I clean the redis DB manually? (I'm worried that I will be storing unnecessary data I don't need anymore).

[dmerrill6]

Hi @c3s4r!

That's right, currently Brokkr doesn't delete sagas after they are finished in case you want to set up some kind of tracking (for example a dashboard where you can see the status of all pending and finished sagas).

I understand your need to keep data usage low though, specially in the case your orchestrator processes thousands of sagas. I'd love to include a flag so that this could be configured when initializing Brokkr, but don't have the bandwidth right now to do it (would be happy to receive PRs though 😄). Another option is, as you say, cleaning the data manually. You could do this by using the saga id and directly executing Redis DEL on that key.

[c3s4r]

Thank you for your prompt response. Data is not critical for our use case, so I'm thinking of just using the built-in memory client (or Redis without persistence), but I wanted to confirm if it's a good idea... I wouldn't like to find out that it just keep increasing the memory usage.

If I use the built-in memory client, would that be something to worry about?

[dmerrill6]

In-memory client was built mainly to run tests without Redis dependency, it will definitely work in production, but the downside is that if your node server crashes, you won't be able to pick up the jobs from where it was left off. Also if you run the in-memory client you won't be able to have multiple orchestrator instances.

I think I can make up some time next week to add a Redis clean-up mechanism. I'd recommend you to use the Redis client and wait for the next release if that's possible.

[c3s4r]

No worries. I will use the Redis client. Thanks for sharing the module!