Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow checkpointing from the Ready state. #2

Open
nataren opened this issue Oct 2, 2015 · 0 comments
Open

Allow checkpointing from the Ready state. #2

nataren opened this issue Oct 2, 2015 · 0 comments

Comments

@nataren
Copy link

nataren commented Oct 2, 2015

Hello there:

First, _thanks a lot_ for writing this library.

Second, I am trying to use it, and I am performing accumulation of records in my workers (IRecordProcess implementation), and after some period of time, out of band, I fire a function that finally processes the accumulated records and stores them, and it is at that point that I know the _SequenceId_ of the last record that I am 100% has been successfully processed and _checkpoint_ right there. But, since that function fires out of band (meaning, it is not happening on the ProcessRecords method invocation), the state in which the state machine can be anything, in particular I noticed that it would be in the _Ready_ state, therefore when I try to checkpoint at that moment I am getting failures, since it is an invalid transition.

So, I was wondering if it would reasonable to be able to _Checkpoint_ when the state machine is on the _Ready_ state, that would allow me to persist my data after I have accumulated enough records or recurrently after certain period of time, without having to wait to be tickled by the _ProcessRecords_ call.

One such stacktrace looked like this:

System.Collections.TimeRecurrentAccumulator`2[[Amazon.Kinesis.ClientLibrary.Record, AmazonKinesisClientLibrary,ion=1.0.0.0, Culture=neutral, PublicKeyToken=null],[Amazon.Kinesis.ClientLibrary.Record, AmazonKinesisClientLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]((null)) - RecurringAccumulator handler failed on.Kinesis.ClientLibrary.MalformedActionException: trigger BeginCheckpoint is invalid for state Ready

Amazon.Kinesis.ClientLibrary.DefaultKclProcess.<ConfigureStateMachine>b__0 (State state, Trigger trigger) [0x00000] in <filename unknown>:0 Stateless.StateMachine`2[Amazon.Kinesis.ClientLibrary.DefaultKclProcess+State,Amazon.Kinesis.ClientLibrary.DefaultKclProcess+Trigger].InternalFire (Trigger trigger, System.Object[] args) [0x00000] in <filename unknown>:0

Stateless.StateMachine`2[Amazon.Kinesis.ClientLibrary.DefaultKclProcess+State,Amazon.Kinesis.ClientLibrary.DefaultKclProcess+Trigger].Fire (Trigger trigger) [0x00000] in <filename unknown>:0

Amazon.Kinesis.ClientLibrary.DefaultKclProcess+InternalCheckpointer.Checkpoint (System.String sequenceNumber, Amazon.Kinesis.ClientLibrary.CheckpointErrorHandler errorHandler) [0x00000] in <filename unknown>:0

Amazon.Kinesis.ClientLibrary.Checkpointer.Checkpoint (Amazon.Kinesis.ClientLibrary.Record record, Amazon.Kinesis.ClientLibrary.CheckpointErrorHandler errorHandler) [0x00000] in <filename unknown>:0

What do you think?

Regards
césar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant