Integrate Symfony Scheduler

This document describes how to integrate and configure Symfony Scheduler module into a Spryker project.

Install

Install the required modules using Composer:

composer require spryker/symfony-scheduler spryker/symfony-messenger

Configure

To configure the Symfony Scheduler module, you need to define your scheduled tasks and their execution intervals. With the current implementation you can add them in the module config or provide them via a plugin.

Configure via Module Config

If you want to execute some console commands on a cron schedule, you can define them in the getCronJobs method of the SymfonySchedulerConfig class. This configuration will be then processed by the CompiledCronTransportsHandlerProviderPlugin and for each job a transport will be created in the Symfony Messenger module with the name of the job. The command will be executed by a handler that is also provided by the same plugin and it will execute the command in a subprocess.

src/Pyz/Client/SymfonyScheduler/SymfonySchedulerConfig.php

<?php

namespace Pyz\Client\SymfonyScheduler;

use Spryker\Client\SymfonyScheduler\SymfonySchedulerConfig as SprykerSymfonySchedulerConfigAlias;
use Spryker\Shared\MessageBroker\MessageBrokerConstants;

class SymfonySchedulerConfig extends SprykerSymfonySchedulerConfigAlias
{
    public function getCronJobs(): array
    {
        $jobs = [
            'queue-worker-start' => [
                'command' => '$PHP_BIN vendor/bin/console queue:worker:start',
                'schedule' => '* * * * *',
            ],
            'check-oms-conditions' => [
                'command' => '$PHP_BIN vendor/bin/console oms:check-condition',
                'schedule' => '* * * * *',
            ],
            'check-oms-timeouts' => [
                'command' => '$PHP_BIN vendor/bin/console oms:check-timeout',
                'schedule' => '* * * * *',
            ],
            'clear-oms-locks' => [
                'command' => '$PHP_BIN vendor/bin/console oms:clear-locks',
                'schedule' => '0 6 * * *',
                'priority' => 100,
            ],
        ];

        return $jobs;
    }
}

The job name is an unique key of job definition and it will be used as a transport name in the Symfony Messenger module. The command is the console command that you want to execute. The schedule is the cron expression that defines when the job should be executed. You can also use aliases like @hourly, @daily, etc. The no_lock option is optional and defines whether the job runs without acquiring a lock. By default (no_lock omitted or false), the cron jobs builder guards each job’s schedule with a lock created through the Lock client, so the same job is never executed by more than one worker at the same time — even when its transport is consumed by several parallel workers. When you set no_lock to true, no lock is acquired, so the same job can be executed by multiple parallel workers simultaneously. Only set it for jobs that are idempotent or otherwise safe to run concurrently. The priority option is optional and defines the consumption order of the job’s transport by the worker. The higher the number, the earlier the job is polled. When omitted, priority defaults to 0. In addition you can also provide a store or a region, which works in the same way as originally in jenkins.php

Configure via new plugin

If your use case is more complex than just executing a console command, you can create a new plugin that implements \Spryker\Shared\SymfonySchedulerExtension\Dependency\Plugin\SchedulerHandlerProviderPluginInterface. In example below you can see that we define messages, handlers and schedules for the job. Messages and handlers are concepts from the Symfony Messenger module. They will be used to trigger and process the job. Schedules are the expressions that define when the job should be executed. It can be simple cron expression or a more complex one like callback trigger, combination of multiple triggers or even a custom trigger that you can implement by yourself. You don’t need to map messages and handlers separately in the SymfonyMessenger module, because the SymfonyScheduler module will take care of things like defining the transport and mapping the message to the handler for the transport.

<?php

namespace Pyz\Client\FooBar\Plugin\SymfonyScheduler;

use Symfony\Component\Scheduler\RecurringMessage;
use Symfony\Component\Scheduler\Schedule;

class FooBarSchedulerHandlerProviderPlugin implements SchedulerHandlerProviderPluginInterface
{
    public function getHandlers(): array
    {
        return [
            RecurringReportGenerationMessage::class => [ //You can have multiple handlers for the same message
                new ReportGenerationHandler(),
            ],
        ];
    }
    public function getSchedules(): array
    {
        $schedule = new Schedule();
        $schedule->add(RecurringMessage::cron('* * * * *'), (new RecurringReportGenerationMessage('report for today'))) // every day at midnight

        return [
            'report-generation' => $schedule // transport name will be "report-generation"
        ];
    }
}

Wiring plugins

If you define your jobs with the first option (via config) or with separate plugin, you need to wire plugins in the Symfony Scheduler Dependency Provider by adding the following code:

src/Pyz/Client/SymfonyScheduler/SymfonySchedulerDependencyProvider.php

<?php

namespace Pyz\Client\SymfonyScheduler;

use Spryker\Client\SymfonyScheduler\Plugin\SymfonyScheduler\CompiledCronTransportsHandlerProviderPlugin;
use Spryker\Client\SymfonyScheduler\SymfonySchedulerDependencyProvider as SprykerSymfonySchedulerDependencyProvider;

class SymfonySchedulerDependencyProvider extends SprykerSymfonySchedulerDependencyProvider
{
    /**
     * @return array<\Spryker\Shared\SymfonySchedulerExtension\Dependency\Plugin\SchedulerHandlerProviderPluginInterface>
     */
    protected function getSchedulerHandlerProviderPlugins(): array
    {
        return [
            new CompiledCronTransportsHandlerProviderPlugin(), //Plugin that provides handlers for jobs defined in the config
            new FooBarSchedulerHandlerProviderPlugin(), //Plugin that provides scheduled jobs separately
        ];
    }
}

Register the scheduler transports with Symfony Messenger

Scheduled jobs are executed through the Symfony Messenger worker, so the scheduler ships a set of plugins that register its transports, message-to-handler mapping, transport factory, and the Back Office control plugins with the Symfony Messenger module. Wire them in the Symfony Messenger dependency provider.

src/Pyz/Client/SymfonyMessenger/SymfonyMessengerDependencyProvider.php

<?php

namespace Pyz\Client\SymfonyMessenger;

use Spryker\Client\SymfonyMessenger\SymfonyMessengerDependencyProvider as SprykerSymfonyMessengerDependencyProvider;
use Spryker\Client\SymfonyScheduler\Plugin\SymfonyMessenger\CompiledCronTransportGroupAwarePlugin;
use Spryker\Client\SymfonyScheduler\Plugin\SymfonyMessenger\DisabledSchedulerJobTransportGuardPlugin;
use Spryker\Client\SymfonyScheduler\Plugin\SymfonyMessenger\SchedulerAvailableTransportConfigProviderPlugin;
use Spryker\Client\SymfonyScheduler\Plugin\SymfonyMessenger\SchedulerMessageMappingProviderPlugin;
use Spryker\Client\SymfonyScheduler\Plugin\SymfonyMessenger\SchedulerTransportFactoryProviderPlugin;

class SymfonyMessengerDependencyProvider extends SprykerSymfonyMessengerDependencyProvider
{
    /**
     * @return array<\Spryker\Shared\SymfonyMessengerExtension\Dependency\Plugin\TransportFactoryProviderPluginInterface>
     */
    protected function getTransportFactoryProviderPlugins(): array
    {
        return [
            new SchedulerTransportFactoryProviderPlugin(),
        ];
    }

    /**
     * @return array<\Spryker\Shared\SymfonyMessengerExtension\Dependency\Plugin\AvailableTransportConfigProviderPluginInterface>
     */
    protected function getAvailableTransportConfigProviderPlugins(): array
    {
        return [
            new SchedulerAvailableTransportConfigProviderPlugin(),
        ];
    }

    /**
     * @return array<\Spryker\Shared\SymfonyMessengerExtension\Dependency\Plugin\MessageMappingProviderPluginInterface>
     */
    protected function getMessageMappingProviderPlugins(): array
    {
        return [
            new SchedulerMessageMappingProviderPlugin(),
        ];
    }

    protected function getGroupAwareTransportsPlugins(): array
    {
        return [
            new CompiledCronTransportGroupAwarePlugin(),
        ];
    }

    /**
     * @return array<\Spryker\Shared\SymfonyMessengerExtension\Dependency\Plugin\TransportConsumeGuardPluginInterface>
     */
    protected function getTransportConsumeGuardPlugins(): array
    {
        return [
            new DisabledSchedulerJobTransportGuardPlugin(),
        ];
    }
}

• SchedulerAvailableTransportConfigProviderPlugin registers one Messenger transport per scheduled job, together with the job’s priority. It supersedes the deprecated SchedulerAvailableTransportProviderPlugin.
• DisabledSchedulerJobTransportGuardPlugin lets the worker skip the transport of a job that has been disabled from the Back Office. See Pause a transport at runtime.
• SchedulerTransportFactoryProviderPlugin, SchedulerMessageMappingProviderPlugin, and CompiledCronTransportGroupAwarePlugin provide the scheduler transport factory, the message-to-handler mapping, and the transport grouping respectively.

Configure the job status storage

The Back Office scheduler page and the enable/disable and run-now controls persist their state (job statuses, disabled markers, and run requests) in Redis, through a connection the module configures under its own SymfonySchedulerConstants keys. This is a separate connection configuration, not a separate Redis instance: by default it resolves to the same key-value store Redis (same host, port, and database) as the storage, and the scheduler’s own key prefixes (scheduler:job:*) keep its entries distinct from storage keys. Configure the connection in config/Shared/config_default.php:

config/Shared/config_default.php

<?php

use Spryker\Shared\SymfonyScheduler\SymfonySchedulerConstants;

// >>> SYMFONY SCHEDULER

$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_PERSISTENT_CONNECTION] = true;
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_SCHEME] = getenv('SPRYKER_KEY_VALUE_STORE_PROTOCOL') ?: 'tcp';
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_HOST] = getenv('SPRYKER_KEY_VALUE_STORE_HOST');
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_PORT] = getenv('SPRYKER_KEY_VALUE_STORE_PORT');
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_USER] = getenv('SPRYKER_KEY_VALUE_USERNAME');
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_PASSWORD] = getenv('SPRYKER_KEY_VALUE_PASSWORD');
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_DATABASE] = $keyValueRegionNamespaces[$namespaceKey]['namespace'] ?? getenv('SPRYKER_KEY_VALUE_STORE_NAMESPACE') ?: 1;
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_DATA_SOURCE_NAMES] = json_decode(getenv('SPRYKER_KEY_VALUE_STORE_SOURCE_NAMES') ?: '[]', true) ?: [];
$config[SymfonySchedulerConstants::SYMFONY_SCHEDULER_REDIS_CONNECTION_OPTIONS] = json_decode(getenv('SPRYKER_KEY_VALUE_STORE_CONNECTION_OPTIONS') ?: '[]', true) ?: [];

The following SymfonySchedulerConstants keys are available:

The storage key prefixes and their time-to-live are defined in \Spryker\Client\SymfonyScheduler\SymfonySchedulerConfig and can be overridden at the project level:

Run the Scheduler

To run a scheduler you need to run the SymfonyMessenger consumer with the transport name that is configured for the job.

vendor/bin/console symfonymessenger:consume queue-worker-start report-generation

The scheduler transport is safe to consume in parallel — each scheduled job is guarded by a lock that the cron jobs builder creates through the Lock client, so the same schedule is never executed by more than one worker at the same time. You can therefore add the --parallel (-p) option to spread the load across several worker processes:

vendor/bin/console symfonymessenger:consume compiled-cron-scheduler --parallel=4

Jobs configured with no_lock set to true are the exception: they are not guarded by a lock, so parallel workers can execute such a job simultaneously. Use no_lock only for jobs that are safe to run concurrently (see Configure via Module Config).

Monitor and control scheduled jobs in the Back Office

The module adds a Scheduler page in the Back Office under Maintenance > Scheduler. It lists every scheduled job in a live table (refreshed every 5 seconds) with the following columns: Name, Command, Schedule, Priority, Status, Started, Finished, Duration, and Actions.

The Status reflects the latest recorded execution state of each job:

Statuses are recorded by the CommandHandler while a job runs and are stored through the scheduler’s Redis connection configuration (see Configure the job status storage). Disabled is derived from the disabled marker and is never persisted as a status record.

Each row provides the following actions:

• View — opens a detail page with the job’s name, command, status, start/finish timestamps, and error message (if any).
• Enable / Disable — toggles the job. Disabling writes a marker to Redis; the DisabledSchedulerJobTransportGuardPlugin then makes the worker skip that job’s transport, so the job is no longer executed until it is enabled again. Enabling removes the marker.
• Run now — requests an immediate, one-off execution ahead of the cron schedule. A run-request marker is written to Redis; the scheduler’s run-request-aware trigger consumes it on the next worker poll and fires the job exactly once. The action is unavailable while a job is disabled or already running.

How it works

The Back Office controls add two decision points to this flow:

• Before iterating a transport, the worker asks every transport consume-guard plugin whether the transport may be consumed. DisabledSchedulerJobTransportGuardPlugin returns false for a disabled job, so its transport is skipped for that iteration.
• When a Run now request exists for a job, the run-request-aware trigger returns the current time as the next run date, pre-empting the cron schedule so the message is yielded immediately. The request marker is consumed atomically, guaranteeing exactly one extra run.

Running consumer as a background process

In order to run the consumer you can use a Jenkins in order to run it or any other manager like Stable Workers.

Jenkins example:

config/Zed/cronjobs/jenkins.php

<?php

$jobs[] = [
        'name' => 'consume-queue',
        'command' => $logger . '$PHP_BIN vendor/bin/console symfonymessenger:consume queue-worker-start --time-limit=3600',
        'schedule' => '* * * * *',
        'enable' => true,
    ];
    $jobs[] = [
        'name' => 'consume-other-jobs',
        'command' => $logger . '$PHP_BIN vendor/bin/console symfonymessenger:consume compiled-cron-scheduler --time-limit=3600',
        'schedule' => '* * * * *',
        'enable' => true,
    ];

if (getenv('SPRYKER_CURRENT_REGION')) {
    foreach ($jobs as $job) {
        $job['region'] = getenv('SPRYKER_CURRENT_REGION');
    }
}

As you see we defined 2 jobs, one for consuming the queue worker messages and another one for consuming the rest of the jobs. Queue worker process runs at least a minute so consumer will try to re-schedule the job every minute and this will leave rest of the jobs waiting. Because of that queue worker have its own consumer process. The rest of the jobs can be consumed by another consumer process that will be running in parallel. To scale a single consumer across multiple worker processes instead of defining additional Jenkins jobs, add the --parallel=N option to the command (see Run the Scheduler).