Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[15.0][ADD] queue_job_cron_jobrunner #415

Merged
merged 1 commit into from
Apr 11, 2022

Conversation

ivantodorovich
Copy link
Contributor

This module implements a simple queue.job runner using ir.cron triggers.

It's meant to be used on environments where the regular job runner can't be run, like on Odoo.sh.

Unlike the regular job runner, where jobs are dispatched to the HttpWorkers, jobs are processed on the CronWorker threads by the job runner crons. This is a design decision because:

  • Odoo.sh puts HttpWorkers to sleep when there's no network activity
  • HttpWorkers are meant for traffic. Users shouldn't pay the price of background tasks.

For now, it only implements the most basic features of the queue_job runner, notably no channel capacity nor priorities.
Please check the ROADMAP for further details.


For the easiest case, no configuration is required besides installing the module.

To avoid CronWorker CPU timeout from abruptly stopping the job processing cron, it's recommended to launch Odoo with --limit-time-real-cron=0, to disable the CronWorker timeout altogether. (In Odoo.sh, this is done by default)

Parallel execution of jobs can be achieved by leveraging multiple ir.cron records:

  • Make sure you have enough CronWorkers available (Odoo CLI --max-cron-threads)
  • Duplicate the queue_job_cron cron record as many times as needed, until you have as much records as cron workers.

Related to:

ping @simahawk @guewen

queue_job_cron_jobrunner/models/queue_job.py Outdated Show resolved Hide resolved
queue_job_cron_jobrunner/__manifest__.py Show resolved Hide resolved
queue_job_cron_jobrunner/models/queue_job.py Outdated Show resolved Hide resolved
queue_job_cron_jobrunner/models/queue_job.py Outdated Show resolved Hide resolved
queue_job_cron_jobrunner/models/queue_job.py Outdated Show resolved Hide resolved
queue_job_cron_jobrunner/models/queue_job.py Outdated Show resolved Hide resolved
Copy link
Contributor

@simahawk simahawk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG and FTR we are testing it on odoo.sh these days ;)

Copy link
Member

@guewen guewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@OCA-git-bot
Copy link
Contributor

This PR has the approved label and has been created more than 5 days ago. It should therefore be ready to merge by a maintainer (or a PSC member if the concerned addon has no declared maintainer). 🤖

@simahawk
Copy link
Contributor

@ivantodorovich wanna squash or we merge like this?

@ivantodorovich
Copy link
Contributor Author

@simahawk done ;)

@simahawk
Copy link
Contributor

/ocabot merge nobump

@OCA-git-bot
Copy link
Contributor

This PR looks fantastic, let's merge it!
Prepared branch 15.0-ocabot-merge-pr-415-by-simahawk-bump-nobump, awaiting test results.

@OCA-git-bot
Copy link
Contributor

Congratulations, your PR was merged at d6a5c13. Thanks a lot for contributing to OCA. ❤️

@OCA-git-bot OCA-git-bot merged commit b0cd0da into OCA:15.0 Apr 11, 2022
@ivantodorovich ivantodorovich deleted the 15.0-add-queue-job-cron-runner branch April 11, 2022 12:36
@StefanRijnhart
Copy link
Member

For the record, this allows you to run jobs on the 'hybrid model of multiple threaded workers' of Odoo.sh as mentioned in #169, right?

@ivantodorovich
Copy link
Contributor Author

That's right, by getting rid of the job runner. Instead it uses async ir.crons to process jobs

@chrisb-c01
Copy link

chrisb-c01 commented Oct 31, 2022

I've (tried to) create a backport from 15.0 to 14.0. As known, 14.0 does not feature the ir.cron.trigger model that 15.0 introduced. Instead I'm just setting the cron.nextcall attribute to the current datetime (link to code).

@ivantodorovich would be great if you could have a look at the PR. Thanks for the initial commit.

@guewen
Copy link
Member

guewen commented Nov 1, 2022

Hi @amigrave, this is the module we spoke about in Brussels :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants