[Question] Is it possible to add support for Recurring Tasks?

[weizheheng]

First of all, thank you very much for the gem. I am using it for my personal project and hosting it on Fly.io with LiteFS just works.

I am not very familiar with how all these works together, so please bear with me if the question sounds stupid.

One thing I am missing now with the setup is the ability to be able to run cron jobs. I see SolidQueue recently merged in Recurring tasks which was inspired by GoodJob's cron manager, and I am wondering if it's possible to have something like that in Litejob?

What I have looked into:

1. Try out solid queue with Fly.io + LiteFS, but I just couldn't set it up. There weren't many resources out there so I gave up.
2. Setting up cron on Fly.io, there are a few suggestions on scheduled machine and Crontab with Supercronic, but those require extra machine.

[misterhtmlcss]

This is probably not helpful, but I thought why not consider it depending on your openness to self resolution on this point.

1. Create a recursive job? Look at how fugit is used and the litejob gem and guide and see what can be done to tweak it to work for you?

2. A free cron job service: https://cron-job.org/ or another one as this is not a recommendation only an illustration.

[weizheheng]

This is probably not helpful, but I thought why not consider it depending on your openness to self resolution on this point.

1. Create a recursive job? Look at how fugit is used and the litejob gem and guide and see what can be done to tweak it to work for you?
2. A free cron job service: https://cron-job.org/ or another one as this is not a recommendation only an illustration.

Thanks for the suggestion. I did spend some time looking around for alternative, and this is more of a general question on whether it's feasible or not since I am not really well-versed in sqlite3.

Found a reddit comment from the author and it seems like it's possible. With that in mind, I will explore more on adding support to litestack.

I am asking this question in litestack because the goal of litestack is to be a all-in-one solution for web application data infrastructure and having support for cron jobs is definitely a good addition.

[oldmoe]

Support is definitely possible, it will require a few changes to the schema and I want to do that after I implement more durable execution (where jobs are not removed from the queue until after execution completes). Which will interfere with this particular feature

[timuckun]

I also need this feature. I am wondering if something like this could work

require 'litestack'
class AddJob
  include Litejob
  # select which specific named queue your jobs should go to
  self.queue = 'my_job_queue'
  # perform must be implemented and can have any number of parameters
  def perform(a, b)
    a + b
   Addjob.perform_in(delay, 1, 1)
  end
end

I don't know if this going to work because it might cause stack issues or worse.  I used to do things like this in erlang but erlang has tail call optimizations so it doesn't cause problems there.

[oldmoe]

This is going to work, since the execution of the new job is out of band (it is just slotted into the queue and ran later) Note that if the job fails, scheduling the next one will not happen until a retry of the failing job succeeds, if all the retries fail then the next job will not be scheduled (unless you catch the errors yourself of course) I am thinking the Litestack options for a recurring job support are: 1. Come up with a simple solution for a config file for recurring jobs 2. Match the SolidQueue approach, even use the same config syntax/file location etc. 3. Add the recurring option to the job itself, basically hiding your implementation above behind a sleeker API

…

On Mon, Sep 30, 2024, 13:20 Tim Uckun ***@***.***> wrote: I also need this feature. I am wondering if something like this could work require 'litestack'class AddJob include Litejob # select which specific named queue your jobs should go to self.queue = 'my_job_queue' # perform must be implemented and can have any number of parameters def perform(a, b) a + b Addjob.perform_in(delay, 1, 1) endend I don't know if this going to work because it might cause stack issues or worse. I used to do things like this in erlang but erlang has tail call optimizations so it doesn't cause problems there. — Reply to this email directly, view it on GitHub <#101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEQZVGKHTCNTYONMWCEJLZZEXXVAVCNFSM6AAAAABPDBW2MWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBSHEYDKNRTGQ> . You are receiving this because you commented.Message ID: ***@***.***>

[timuckun]

I think the easiest thing for the users would be to implement the solid queue approach. If anything else it would save you writing a lot of documentation.

[oldmoe]

Easier said than done, the main issue with that approach is parsing the different time periods, it can be done, but I would rather roll something of my own to parse it than add ActiveSupport as a dependency.

…

On Tue, 1 Oct 2024 at 05:14, Tim Uckun ***@***.***> wrote: I think the easiest thing for the users would be to implement the solid queue approach. If anything else it would save you writing a lot of documentation. — Reply to this email directly, view it on GitHub <#101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAEQZRUI7ITPPR7QVJKJRLZZIHSRAVCNFSM6AAAAABPDBW2MWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBUGY4TKOBSGI> . You are receiving this because you commented.Message ID: ***@***.***>

[timuckun]

Makes sense.

A very long time ago (more than a decade I think) I wrote something that would run background jobs and scheduled tasks. This was way before rails had anything like this. I did it the dumb way by creating a table of jobs and a postgres trigger to send a NOTIFY when a job was entered. I then had listeners that would perform the job. They could also do the polling method of checking for jobs.

For cron I used rufus scheduler. It was a pretty simple thing. On the scheduled time it just inserted the job and the let the handlers do the rest. Miraculously it all worked but I wasn't using it for heavy workloads or anything .

Rufus had a wonderful DSL for scheduling jobs so maybe you can grab that

[timuckun]

Hey everybody I tried this with rufus scheduler and it seems to work just fine. I used both falcon and puma following the instructions on the falcon website and the rufus web site regarding the puma plugin. It was super easy to set up. I created a simple job

class RufusJob < ApplicationJob
  queue_as :default

  def perform(*args)
    puts "ping"
  end
end

Then I created a scheduler.rb

scheduler = Rufus::Scheduler.new

scheduler.every "1s" do
  RufusJob.perform_now
end
scheduler.join

That's it. It "just worked". In the case of falcon scheduler.rb was in the initialisers and in the case of puma in the config directory. Note: if you are using falcon leave out the scheduler.join line in the initialiser.

@oldmoe I think this is a fine workaround to the issue and you may want to toss this in the documentation someplace.