Add reconciliation of sandbox kv store key counts. #6
Conversation
|
Not at all ready for review, although the plan is fairly well documented and a skeleton of solution is in place. |
| * If there is a reconciliation in progress, continue scanning | ||
| keys from where the work stopped, updating counts as needed. | ||
| * After doing some work, write the results back to the reconciliation | ||
| status in Redis. |
There was a problem hiding this comment.
I know this isn't yet ready for review, but its a bit unclear to me what part the locks play in these steps. Could you explain?
There was a problem hiding this comment.
The lock is to prevent multiple workers from doing reconciliation work at the same time.
There was a problem hiding this comment.
Ah, I see. Missed this bit, sorry:
Only one worker performs reconciliation work at a time. This reduces load on Redis and simplifies the implementation.
This step confused me:
If there is a reconciliation in progress, continue scanning keys from where the work stopped, updating counts as needed.
I thought that meant that another recon can start while another is in progress, and it'd continue work from where the other recon is going to be ending.
Looking at the implementation, it seems like each looping call does a bit of work at a time, and then leaves the rest for the next call. Is that what happens?
No description provided.