Background work: queues and cron

Which one to use

• A queue when a request needs to trigger work that can complete afterwards. Sending an email is the standard case: the request records the email and returns; the queue delivers it.
• Cron when work recurs on a clock rather than in response to a request, such as expiring old sessions or refreshing a cache every minute.

Registering a lock-based queue

A module registers a queue in its index file with QueueConfig::AddQueue. You give the module name, a queue name, a rate (max processes per second) and a ceiling (max in flight), and two callbacks: one that locks and returns the next item ids, and one that processes a single item:

QueueConfig::AddQueue(
    'Messaging',
    'email',
    10,   // max processes per second
    100,  // max in flight
    'MessagingEmailQueueDatabase::LockNext',
    'MessagingEmailQueueDatabase::Process'
);

The queue is lock based. LockNext claims rows with SELECT ... FOR UPDATE SKIP LOCKED so two workers never take the same item, and Process runs inside a transaction and must update the row so it no longer matches the lock query. If Process throws, the transaction rolls back and the item returns to the queue. The same callback that locks the next ids is used both to count what is waiting (with a limit of 100) and to claim one item to run (with a limit of 1). Enqueuing is a plain insert: the code above is exactly how Messaging wires up its email queue, and MessagingEmail::Send enqueues by inserting an unsent row.

Registration only records the queue; the work of draining it is exposed as HTTP endpoints by a separate QueueApi module, and the framework polls the registered set on its own cadence. Because the whole loop is PHP running inside the web process, this queue suits ordinary application work: sending a message, resizing an image, updating a counter.

The lease-based task queue

For heavier workloads that a PHP request should not run, the module also offers a lease-based task queue, registered with QueueTaskConfig::AddQueue. Here an external worker (for example a GPU fleet) leases an item, does the work, and reports back. Instead of a lock-and-process pair, you supply callbacks for the stages of a lease: one that lists pending candidates, one that builds the payload a worker needs, and separate callbacks that apply a result, a failure, or progress:

QueueTaskConfig::AddQueue(
    'Clip',
    'thumbnail',
    120,   // reassign after this many seconds if a worker goes quiet
    3600,  // overall timeout for one item
    'ClipThumbnailQueue::Pending',
    'ClipThumbnailQueue::Payload',
    'ClipThumbnailQueue::ApplyResult',
    'ClipThumbnailQueue::ApplyFailure'
);

An item is leased for a fixed window. If the worker does not report back within the reassign window it is handed to another worker, and the overall timeout caps how long one item may take before it is treated as failed. The registration shape mirrors the lock-based queue; the difference is that the worker lives outside the web process and talks to the queue over the QueueApi task endpoints. Choose this form when the work is too slow or too specialised to run inside a request.

Registering a cron task

A module registers a recurring task with CronConfig::Minute, giving the module name, a task name, and a callback to run:

CronConfig::Minute(
    'MyModule',
    'refresh',
    'MyModuleCrons::Refresh'
);

The callback is an ordinary static method:

final class MyModuleCrons {
    static function Refresh(): void {
        // recurring work goes here
    }
}

Registered tasks are driven by Cron::Run, called from the cron entry point once a minute. It walks every task registered with CronConfig::Minute and runs each one through Cron::RunCallback, which records a log row when a task starts and closes it when the task finishes, capturing any output and any error. That log is how you see when a task last fired and whether it threw, so a task that fails quietly is still visible.

Next: Caching and locks.