Dynamic scheduled tasks

[cupatea]

Fixes #186

Add resque-scheduler style dynamic schedules feature, allowing you to add or remove recurring tasks at runtime without touching your static config file.

What’s new:

• Dynamic scope (static: false) on SolidQueue::RecurringTask model.
• Renamed methods/vars in SolidQueue::Scheduler::RecurringSchedule to distinguish static vs. dynamic schedules.
• On boot, @configured_tasks now includes static and dynamic tasks.
• Added SolidQueue::Scheduler::RecurringSchedule.update_scheduled_tasks:
  • Schedules any new dynamic tasks added since the last run.
  • Unschedule any dynamic tasks removed from the database.
  • Refreshes @configured_tasks to match the current DB state.
• SolidQueue::Configuration no longer requires a non-blank static config file - pure dynamic scheduling is now supported.
• SolidQueue::Scheduler watches for changes after launch and updates its metadata so the running process always reflects the true set of recurring tasks.

Tests verify that adding or dropping dynamic tasks at runtime correctly updates what’s scheduled.

[cupatea]

Hi @rosa 👋, could you please take a look at this PR when you have a moment? Thanks so much!

[rosa]

Hey @cupatea, thanks for this! It's a good start, but it needs a few changes. The main ones are:

• We need to make the sleep interval for the scheduler configurable, as a sort of polling interval, to reload dynamic tasks.
• The way to use this needs to be a bit friendlier, without users having to go and figure out the details to create a SolidQueue::RecurringTask record. Something like SolidQueue.schedule_recurring_task could work, and it needs to be documented.
• We'd need the ability to delete dynamically scheduled tasks as well, by task ID. If the task is static, this could simply raise an error.

And then some other more specific changes that I'll note in the code.

[rosa]

This doesn't need to change names. It's clear the tasks here are static since this comes from the recurring.yml configuration. We don't need to rename anything here.

[rosa]

This code is mixing actions at very different levels, making it aware of details it shouldn't need to know, like how to update the metadata for its registered process record, or whether the recurring schedule changed. It should change, perhaps to something like

recurring_schedule.reload!
if recurring_schedule.changed?
  refresh_registered_process
end

And refresh_registered_process would go in SolidQueue::Processes::Registrable.

[cupatea]

Hi @rosa, thank you for the feedback! I think it's ready for the second round of review

[bruno-berchielli]

@cupatea , in the docs files you mention SolidQueue.delete_recurring_task but I think it should be SolidQueue.destroy_recurring_task, also, you might consider using the key instead of id, because that's is what probably gonna be easily available on the app level, no?
SolidQueue::RecurringTask.find_by(key: solid_queue_key)&.destroy!

[cupatea]

@cupatea , in the docs files you mention SolidQueue.delete_recurring_task but I think it should be SolidQueue.destroy_recurring_task, also, you might consider using the key instead of id, because that's is what probably gonna be easily available on the app level, no? SolidQueue::RecurringTask.find_by(key: solid_queue_key)&.destroy!

Thanks for noticing, on a way to fix that!
And yes, using a key instead of ID makes more sense to me as well. To prevent deleting a static task, this method will raise a record not found error in case of specifying a static task key (or if you try deleting a task that does not exist)