From 24e74fa919a515fe6695c8b1dc54baa7cb62c30f Mon Sep 17 00:00:00 2001 From: Yaraslau Kurmyza Date: Tue, 28 May 2024 15:16:20 +0200 Subject: [PATCH] Update the proposal from task-run level to the task and task group levels --- README.md | 2 +- rfcs/0190-queue-change-task-run-priority.md | 56 ++++++++++++++------- rfcs/README.md | 2 +- 3 files changed, 41 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index e663e44..954f386 100644 --- a/README.md +++ b/README.md @@ -68,4 +68,4 @@ See [mechanics](mechanics.md) for more detail. | RFC#180 | [Github cancel previous tasks](rfcs/0180-Github-cancel-previous-tasks.md) | | RFC#182 | [Allow remote references to .taskcluster.yml files processed by Taskcluster-GitHub](rfcs/0182-taskcluster-yml-remote-references.md) | | RFC#189 | [Batch APIs for task definition, status and index path](rfcs/0189-batch-task-apis.md) | -| RFC#190 | [Queue change task run priority](rfcs/0190-queue-change-task-run-priority.md) | +| RFC#190 | [Queue change task / task group priority](rfcs/0190-queue-change-task-run-priority.md) | diff --git a/rfcs/0190-queue-change-task-run-priority.md b/rfcs/0190-queue-change-task-run-priority.md index 4a7a18d..29a9982 100644 --- a/rfcs/0190-queue-change-task-run-priority.md +++ b/rfcs/0190-queue-change-task-run-priority.md @@ -1,4 +1,4 @@ -# RFC 0190 - Queue change task run priority +# RFC 0190 - Queue change task / task group priority * Comments: [#0190](https://github.com/taskcluster/taskcluster-rfcs/pull/190) * Proposed by: @lotas @@ -7,45 +7,67 @@ Since we refactored queue internals it became possible to see what tasks are pending for a particular worker pool / task queue. -This RFC proposes new API method that would allow to change the priority of the existing task run. +This RFC proposes new API method that would allow to change the priority of the existing task run / task group. ## Motivation There might be several use-cases when some worker pool has a lot of pending tasks and there is a need to move some more important tasks to the top of the waiting list, or also move down less important ones. -Instead of cancelling existing tasks we can instead choose to change task run priorities. +Instead of cancelling some existing tasks we can change task run priorities. + +Use-case examples: + +* We have a high-priority patch running tasks on Try. We want to bump the priority of all tasks in this task group +* We have a chemspill release or high priority nightly to get out. We want to bump the priority of all tasks in one or more groups +* We are waiting for a specific task in a low resource pool (eg: mac hardware). We want to bump the priority of that specific task +* There are several tasks with the same highest priority and one of them is more important. We want to lower the priority of the less important task ## Details -I propose we implement this on a task run level and not the task itself: +Queue service will expose new methods: + +* `queue.changeTaskPriority(taskId, newPriority)` +* `queue.changeTaskGroupPriority(taskGroupId, newPriority)` + +New priority would be stored along the task definition for a given task or all tasks within the task group. -* tasks are supposed to be immutable -* changing the priority makes more sense in the context of a specific run, not the whole task +The process to change single task will be as follows: -Queue service will expose new method `queue.changeTaskRunPriority(taskId, runId, newPriority)`. +* new task priority will be stored in the database as `task.priority_override` column +* if task was scheduled already, its priority in the `queue_pending_tasks` table will be updated +* if the tasks run fails and new one is created, it will check if `task.priority_override` is set and use it instead of the original priority -This will try to do the following: +The process to change the whole task group will be as follows: -* update `task.runs[runId]` in the database to reflect priority change (possibly extra field to keep the original) -* update `priority` field in the `queue_pending_tasks` table +* all tasks in the task group will be updated to store the new priority in the `task.priority_override` column +* all currently scheduled tasks would be attempted to be updated in the `queue_pending_tasks` table +* all unscheduled tasks and tasks that will be restarted will use the new priority -If any of this fails (for example task is being changed by some other process), this method will also fail. +### New scopes This will also require introduction of the new scopes (anyOf): -* `queue:change-task-run-priority-in-queue:` -* `queue:change-task-run-priority:` -* `queue:lower-task-run-priority:` to only allow lowering the priority -* `queue:raise-task-run-priority:` to only allow raising the priority +* `queue:change-task-priority-in-queue:` +* `queue:change-task-priority:` +* `queue:lower-task-priority:` to only allow lowering the priority +* `queue:raise-task-priority:` to only allow raising the priority + +And for task group: + +* `queue:change-task-group-priority:/` (for consistency with existing task group scopes) +* `queue:lower-task-group-priority:` to only allow lowering the priority +* `queue:raise-task-group-priority:` to only allow raising the priority + +### Priority calculation Current order for picking up tasks is based on the task priority and insertion time (FIFO). This RFC proposes to change the priority only, and leave the insertion time as is. ### Other considerations -Chain of Trust validation might be broken if we change too much of the task definition or run. -We need to make sure we don't break it with this change. +To insure that Chain of Trust validation is not affected, we aim to keep the original task definition +and store the new priority in a separate column. ## Implementation diff --git a/rfcs/README.md b/rfcs/README.md index 50d283f..2b6b41c 100644 --- a/rfcs/README.md +++ b/rfcs/README.md @@ -56,4 +56,4 @@ | RFC#180 | [Github cancel previous tasks](0180-Github-cancel-previous-tasks.md) | | RFC#182 | [Allow remote references to .taskcluster.yml files processed by Taskcluster-GitHub](0182-taskcluster-yml-remote-references.md) | | RFC#189 | [Batch APIs for task definition, status and index path](0189-batch-task-apis.md) | -| RFC#190 | [Queue change task run priority](0190-queue-change-task-run-priority.md) | +| RFC#190 | [Queue change task / task group priority](0190-queue-change-task-run-priority.md) |