From e05c860f00824c837f6a9e4f74bb0043baa293a0 Mon Sep 17 00:00:00 2001 From: Piotr Konopka Date: Thu, 30 May 2024 16:55:42 +0200 Subject: [PATCH 1/5] OCTRL-901 Document the order of actions performed during SOR and EOR --- docs/handbook/operation_order.md | 110 +++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 docs/handbook/operation_order.md diff --git a/docs/handbook/operation_order.md b/docs/handbook/operation_order.md new file mode 100644 index 00000000..008109dc --- /dev/null +++ b/docs/handbook/operation_order.md @@ -0,0 +1,110 @@ +# Environment operation order + +This chapter attempts to document the order of important operations done during environment transitions. +Since AliECS is an evolving system, the information presented here might be out-of-date, thus please refer to event handling in [core/environment/environment.go](core/environment/environment.go) and plugin calls in [ControlWorkflows/workflows/readout-dataflow.yaml](https://github.com/AliceO2Group/ControlWorkflows/blob/master/workflows/readout-dataflow.yaml) for the ultimate source of truth. +Also, please report to the ECS developers any inaccuracies. + +[State Machine Callbacks](docs/handbook/configuration.md#State-machine-callbacks) documents the order of callbacks that can be associated with state machine transitions. + +## START_ACTIVITY (Start Of Run) + +This is the order of actions happening at a healthy start of run. + +### before_START_ACTIVITY + +- `"run_number"` is set. +- `"run_start_time_ms"` is set using the current time. It is considered as the SOR and SOSOR timestamps. +- `before_START_ACTIVITY` hooks are executed: + - `trg.PrepareForRun()` at `-200` + - `trg.RunLoad()`, `bookkeeping.StartOfRun()` at `-100` + - `bookkeeping.RetrieveFillInfo()` at `-99` + - `kafka.PublishStartActivityUpdate()` at `-50` + - `dcs.StartOfRun()`, `odc.Start()` (does not need to return now), `ccdb.RunStart()` at `0` + +### before_event + +- `before_event` hooks are executed + +### leave_CONFIGURED + +- `leave_CONFIGURED` hooks are executed + - `kafka.PublishLeaveStateUpdate()` at `0` + +### leave_state + +- `leave_state` hooks are executed + +### Transition START_ACTIVITY + +- Fill Info previously retrieved by the BKP plugin is read from the variable stack and put into transition message to tasks +- Tasks are transitioned to `RUNNING` +- If everything succeeds up to this point, we report that the run has started + +### enter_RUNNING + +- `enter_RUNNING` hooks are executed + - `o2-roc-ctp-emulator` for all ROC CTP emulator endpoints, `kafka.PublishEnterStateUpdate()` at `0` + +### enter_state + +- `enter_state` hooks are executed + +### after_START_ACTIVITY +- `"run_start_completion_time_ms"` is set using current time. It is considered as the EOSOR timestamp. +- `after_START_ACTIVITY` hooks are executed: + - `trg.RunStart()` at `0` + - waiting until `odc.Start()` executed at `before_START_ACTIVITY` completes at `0` + - `bookkeeping.UpdateRunStart()`, `bookkeeping.UpdateEnv()` at `+100` + +### after_event + +- `after_event` hooks are executed + +## STOP_ACTIVITY (End Of Run) + +This is the order of actions happening at a healthy end of run. + +### before_STOP_ACTIVITY + +- `"run_end_time_ms"` is set using the current time. It is considered as the EOR and SOEOR timestamps. +- `before_STOP_ACTIVITY` hooks are executed: + - `trg.RunStop()`, `odc.Stop()` (does not need to return now) at `0` + +### before_event + +- `before_event` hooks are executed + +### leave_RUNNING + +- `leave_RUNNING` hooks are executed + - `kafka.PublishLeaveStateUpdate()` at `0` + +### leave_state + +- `leave_state` hooks are executed + +### Transition STOP_ACTIVITY + +- Tasks are transitioned to `STOP` +- If everything succeeds up to this point, we consider that the run has stopped + +### enter_CONFIGURED + +- `enter_CONFIGURED` hooks are executed + - `kafka.PublishEnterStateUpdate()` at `0` + +### enter_state + +- `enter_state` hooks are executed + +### after_STOP_ACTIVITY +- `"run_end_completion_time_ms"` is set using current time. It is considered as the EOEOR timestamp. +- `after_STOP_ACTIVITY` hooks are executed: + - `trg.RunUnload()` at `-100` + - `ccdb.RunStop()`, `dcs.EndOfRun()` at `0` + - waiting until `odc.Stop()()` executed at `before_STOP_ACTIVITY` completes at `0` + - `bookkeeping.UpdateRunStop()`, `bookkeeping.UpdateEnv()` at `+100` + +### after_event + +- `after_event` hooks are executed \ No newline at end of file From 67b6333e9f180d97173d7cc3b232277af118ca2a Mon Sep 17 00:00:00 2001 From: Piotr Konopka Date: Fri, 31 May 2024 11:16:16 +0200 Subject: [PATCH 2/5] fix links --- docs/handbook/operation_order.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/handbook/operation_order.md b/docs/handbook/operation_order.md index 008109dc..f7ab93d2 100644 --- a/docs/handbook/operation_order.md +++ b/docs/handbook/operation_order.md @@ -1,10 +1,10 @@ # Environment operation order This chapter attempts to document the order of important operations done during environment transitions. -Since AliECS is an evolving system, the information presented here might be out-of-date, thus please refer to event handling in [core/environment/environment.go](core/environment/environment.go) and plugin calls in [ControlWorkflows/workflows/readout-dataflow.yaml](https://github.com/AliceO2Group/ControlWorkflows/blob/master/workflows/readout-dataflow.yaml) for the ultimate source of truth. +Since AliECS is an evolving system, the information presented here might be out-of-date, thus please refer to event handling in [core/environment/environment.go](https://github.com/AliceO2Group/Control/blob/master/core/environment/environment.go) and plugin calls in [ControlWorkflows/workflows/readout-dataflow.yaml](https://github.com/AliceO2Group/ControlWorkflows/blob/master/workflows/readout-dataflow.yaml) for the ultimate source of truth. Also, please report to the ECS developers any inaccuracies. -[State Machine Callbacks](docs/handbook/configuration.md#State-machine-callbacks) documents the order of callbacks that can be associated with state machine transitions. +[State Machine Callbacks](configuration.md#State-machine-callbacks) documents the order of callbacks that can be associated with state machine transitions. ## START_ACTIVITY (Start Of Run) From 1959ffdf59a2102ef226391493be117c430d9b2f Mon Sep 17 00:00:00 2001 From: Piotr Konopka Date: Fri, 31 May 2024 11:38:48 +0200 Subject: [PATCH 3/5] remove generic targets in the operation order doc, since they are not supported --- docs/handbook/operation_order.md | 34 +------------------------------- 1 file changed, 1 insertion(+), 33 deletions(-) diff --git a/docs/handbook/operation_order.md b/docs/handbook/operation_order.md index f7ab93d2..95ce3f85 100644 --- a/docs/handbook/operation_order.md +++ b/docs/handbook/operation_order.md @@ -21,19 +21,11 @@ This is the order of actions happening at a healthy start of run. - `kafka.PublishStartActivityUpdate()` at `-50` - `dcs.StartOfRun()`, `odc.Start()` (does not need to return now), `ccdb.RunStart()` at `0` -### before_event - -- `before_event` hooks are executed - ### leave_CONFIGURED - `leave_CONFIGURED` hooks are executed - `kafka.PublishLeaveStateUpdate()` at `0` -### leave_state - -- `leave_state` hooks are executed - ### Transition START_ACTIVITY - Fill Info previously retrieved by the BKP plugin is read from the variable stack and put into transition message to tasks @@ -45,10 +37,6 @@ This is the order of actions happening at a healthy start of run. - `enter_RUNNING` hooks are executed - `o2-roc-ctp-emulator` for all ROC CTP emulator endpoints, `kafka.PublishEnterStateUpdate()` at `0` -### enter_state - -- `enter_state` hooks are executed - ### after_START_ACTIVITY - `"run_start_completion_time_ms"` is set using current time. It is considered as the EOSOR timestamp. - `after_START_ACTIVITY` hooks are executed: @@ -56,10 +44,6 @@ This is the order of actions happening at a healthy start of run. - waiting until `odc.Start()` executed at `before_START_ACTIVITY` completes at `0` - `bookkeeping.UpdateRunStart()`, `bookkeeping.UpdateEnv()` at `+100` -### after_event - -- `after_event` hooks are executed - ## STOP_ACTIVITY (End Of Run) This is the order of actions happening at a healthy end of run. @@ -70,19 +54,11 @@ This is the order of actions happening at a healthy end of run. - `before_STOP_ACTIVITY` hooks are executed: - `trg.RunStop()`, `odc.Stop()` (does not need to return now) at `0` -### before_event - -- `before_event` hooks are executed - ### leave_RUNNING - `leave_RUNNING` hooks are executed - `kafka.PublishLeaveStateUpdate()` at `0` -### leave_state - -- `leave_state` hooks are executed - ### Transition STOP_ACTIVITY - Tasks are transitioned to `STOP` @@ -93,18 +69,10 @@ This is the order of actions happening at a healthy end of run. - `enter_CONFIGURED` hooks are executed - `kafka.PublishEnterStateUpdate()` at `0` -### enter_state - -- `enter_state` hooks are executed - ### after_STOP_ACTIVITY - `"run_end_completion_time_ms"` is set using current time. It is considered as the EOEOR timestamp. - `after_STOP_ACTIVITY` hooks are executed: - `trg.RunUnload()` at `-100` - `ccdb.RunStop()`, `dcs.EndOfRun()` at `0` - waiting until `odc.Stop()()` executed at `before_STOP_ACTIVITY` completes at `0` - - `bookkeeping.UpdateRunStop()`, `bookkeeping.UpdateEnv()` at `+100` - -### after_event - -- `after_event` hooks are executed \ No newline at end of file + - `bookkeeping.UpdateRunStop()`, `bookkeeping.UpdateEnv()` at `+100` \ No newline at end of file From d7582bf26d0c965bc1c84f8a470400faa379c827 Mon Sep 17 00:00:00 2001 From: Piotr Konopka Date: Fri, 31 May 2024 11:41:19 +0200 Subject: [PATCH 4/5] fix detail --- docs/handbook/operation_order.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/handbook/operation_order.md b/docs/handbook/operation_order.md index 95ce3f85..23a8f79e 100644 --- a/docs/handbook/operation_order.md +++ b/docs/handbook/operation_order.md @@ -61,7 +61,7 @@ This is the order of actions happening at a healthy end of run. ### Transition STOP_ACTIVITY -- Tasks are transitioned to `STOP` +- Tasks are transitioned to `CONFIGURED` - If everything succeeds up to this point, we consider that the run has stopped ### enter_CONFIGURED From dc433859befd8f5f50de3817a92c9fd93f818422 Mon Sep 17 00:00:00 2001 From: Piotr Konopka Date: Fri, 31 May 2024 11:42:06 +0200 Subject: [PATCH 5/5] spurrious brackets --- docs/handbook/operation_order.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/handbook/operation_order.md b/docs/handbook/operation_order.md index 23a8f79e..77b2bc5d 100644 --- a/docs/handbook/operation_order.md +++ b/docs/handbook/operation_order.md @@ -74,5 +74,5 @@ This is the order of actions happening at a healthy end of run. - `after_STOP_ACTIVITY` hooks are executed: - `trg.RunUnload()` at `-100` - `ccdb.RunStop()`, `dcs.EndOfRun()` at `0` - - waiting until `odc.Stop()()` executed at `before_STOP_ACTIVITY` completes at `0` + - waiting until `odc.Stop()` executed at `before_STOP_ACTIVITY` completes at `0` - `bookkeeping.UpdateRunStop()`, `bookkeeping.UpdateEnv()` at `+100` \ No newline at end of file