✨ Enable hourly runs in create-a-derived-table

[moj-supratikc]

Describe the feature request.

See Slack thread here for full details: https://mojdt.slack.com/archives/C06TFT94JTC/p1728488760638649

We want to have the ability to run hourly workflows for a subset of models, similar to the bold-daily workflows on create-a-derived-table.

Describe the context.

Context:
We have been building out the Avature recruitment pipelines and derived models. We are just entering a UAT period with analysts on the current user-facing facts/ dimensions.
Currently all the models are views. However the data is quite large (and growing quickly every day), and there is some fairly complex logic under the hood to generate the user-facing models. As such, the final facts/ dimensions have unusable query times (upwards of 30 seconds).
We have found that materializing the end models is much faster and results in far less data scanned (as expected!), however our operational users would not be able to use data that is a day out of date (as it would be with daily) - they need the data as close to live as possible.
Because of this, we'd want to materialize those tables hourly at minimum to meet their need.

Value / Purpose

Will enable operational analysts to have recruitment data on prison officers that is correct to within the hour, for allocation and other operational purposes, while also having fast query times.

User Types

Analysts, Operational Users

[simon-pope]

@moj-supratikc @pricemg I have invited you to the Request Forum on 14th October to present this to the team. Any questions, please let me know