Simple job queue in bash using atd for Linux.
Because cron is not a job queue.
bash-work-queue will run a set of scripts defined on a jobs file:
- one by line
- in batch mode
- on a predefinied interval
- when the cpu usage lowers a predefinied threeshold
Optional. You can place workqueue.bash in /etd/init.d/ and get it started at boot time.
Download and install as a service:
wget https://raw.githubusercontent.com/varas/bash-work-queue/master/workqueue.bash -O /etc/init.d/workqueue.bash
chmod +x /etc/init.d/workqueue.bash
Configure and launch:
service workqueue.bash start
Set up your jobs on a plain text file, one task by line. Example:
/path/to/script1.sh
sudo -u otheruser nice -n 15 /path/to/script2.sh
Tune up the config variables on the file:
JOBS_FILE='/path/to/your/workqueue.jobs'
# load_avg maybe be something like: number of cores - 1
LOAD_AVG=3
# minimum interval in seconds between the start of two batch jobs
TIME_WINDOW=30
You can run it manually:
/path/to/workqueue.bash [start|stop|status|dump TASK_ID]
Or in case you have placed it at /etd/init.d/ you can run it as a service:
service workqueue.bash [start|stop|status|dump TASK_ID]
bash-work-queue gets queued itself in order to respawn the workqueue and is treated as a last job item.