Overlapping Scheduled Jobs - Choose a Lock That Matches the Work

Written by Adam Muiz 22 Sep 2026 Updated: 22 Sep 2026 3 min read

A job is scheduled every five minutes, but one run occasionally needs six. When the next trigger arrives, should it start a second copy, wait behind the first, or leave this run for later?

The schedule alone cannot answer that question. Concurrent runs may be harmless for a read-only report and dangerous for a job that rotates files, publishes content, or updates the same records. Preventing overlap begins with a policy decision, followed by a lock whose scope matches every process that can run the job.

This article examines a scheduled PHP command on a Debian server. It uses systemd timer behavior, local file locks, and MariaDB named locks as three different coordination boundaries. None of them creates a universal “exactly once” guarantee, and that limitation is as important as the configuration.

Decide what a competing run should do

“Do not overlap” still leaves three possible policies:

Skip: if another run is active, record that fact and exit successfully. This can fit a refresh job whose next scheduled run will be sufficient.
Wait: remain blocked until the current run releases the lock. This preserves every trigger, but waiting processes can accumulate if work stays slower than the schedule.
Queue: persist one or more pending jobs and let workers claim them deliberately. This is more machinery, but it is usually the honest choice when every requested run represents distinct work that must not be discarded.

A lock implements exclusion; it does not select the right policy. A non-blocking attempt naturally supports skipping. A blocking attempt implements waiting, but still needs a timeout or other bound if indefinite waiting is unacceptable. A queue models pending work rather than hiding it behind sleeping processes.

The choice also affects monitoring. A skipped refresh may be normal, while a skipped invoice export may mean missing work. Give a lock conflict its own log event or metric instead of making it indistinguishable from success or failure.

Let the scheduler serialize when its contract is enough

A systemd timer activates another unit, normally a service with the same base name. The Debian systemd.timer(5) manual states that if the target unit is already active when the timer elapses, systemd leaves it running. It does not restart the unit or spawn another service instance.

That gives a simple Type=oneshot service a useful single-unit boundary:

# /etc/systemd/system/catalog-refresh.service
[Unit]
Description=Refresh the local catalog

[Service]
Type=oneshot
User=www-data
ExecStart=/usr/bin/php /srv/example/bin/refresh-catalog.php

If catalog-refresh.service is still active, its matching timer will not start a second instance of that unit. This is narrower than “the job can never overlap.” An administrator could run the PHP command directly, another service unit could invoke it, or a second host could run the same application. Those paths are outside this timer-to-unit boundary.

Version details matter when changing timer cadence. For example, DeferReactivation= was added in systemd 257 and changes how a calendar timer schedules its next elapse after the service becomes inactive. It should not be copied into older releases, and it does not replace application-level coordination when multiple launch paths exist.

Use a dedicated file lock for one host

When every possible runner sees the same local filesystem, a file lock can put the boundary inside the PHP command. PHP's official flock() documentation describes an advisory reader/writer lock. “Advisory” means the protection works only when competing programs cooperate by locking the same object.

Here is a compact skip policy:

<?php

declare(strict_types=1);

$lockPath = '/run/lock/example/catalog-refresh.lock';
$lock = fopen($lockPath, 'c');

if ($lock === false) {
    fwrite(STDERR, "Cannot open lock file.\n");
    exit(1);
}

if (!flock($lock, LOCK_EX | LOCK_NB)) {
    fwrite(STDOUT, "Another run is still active; skipping.\n");
    fclose($lock);
    exit(0);
}

try {
    refreshCatalog();
} finally {
    flock($lock, LOCK_UN);
    fclose($lock);
}

LOCK_EX requests an exclusive lock, while LOCK_NB makes the request non-blocking. The dedicated lock directory and file must be writable by the service user and should not be writable by unrelated users. Create that directory during deployment or service startup with deliberate ownership; silently falling back to an unlocked run would defeat the boundary.

The stream handle is not incidental. The PHP manual says that closing the stream, or allowing it to be garbage-collected, releases the lock. It therefore has to remain reachable for the whole critical section. The example also releases it in finally, although process termination and stream closure normally release it as well.

The example was syntax-checked and exercised locally with two concurrent CLI processes: the first held the lock, the second took the skip path, and a later process acquired the lock after release. That verifies this small example on the checked host. It does not establish behavior for a different filesystem, PHP runtime, or deployment topology.

Do not treat the lock file's existence as the lock

The file can remain on disk between runs. Its existence is not evidence that a process is still working; the held kernel lock is the relevant state. Deleting and recreating a lock file can be actively misleading because different processes may then hold file descriptors referring to different underlying files.

For the same reason, a PID written into a file is diagnostic information, not a complete locking protocol. PIDs can be reused, and a stale number does not prove that the old critical section is active. Let the locking primitive decide ownership and use metadata only to make diagnosis easier.

Put the lock outside PHP when the command is the boundary

The util-linux flock(1) manual documents a command form that holds a lock while a child command runs. A cron entry can therefore express the same non-blocking policy without changing the PHP program:

*/5 * * * * flock --nonblock /run/lock/example/catalog-refresh.lock /usr/bin/php /srv/example/bin/refresh-catalog.php

This is attractive when every invocation is controlled by the scheduler. The lock surrounds the complete child process, and it is dropped when the relevant file descriptor closes. The tradeoff is visibility: by default, lock conflict and an ordinary child failure can both lead to a non-zero command result. The manual provides --conflict-exit-code when automation needs to distinguish them.

A shell wrapper and an in-program lock should not casually be stacked with different files or names. Two boundaries that do not agree can give a false sense of safety. Choose one authoritative local lock identity, document it, and make every competing launch path use it.

Move coordination to MariaDB when one filesystem is not shared

A local file lock cannot coordinate two application hosts that do not share the lock's filesystem. If all runners use the same MariaDB server, a named advisory lock can provide a different scope. MariaDB's official GET_LOCK() documentation describes names as server-wide and recommends application- or database-specific naming to reduce collisions.

SELECT GET_LOCK('example.catalog-refresh', 0);
-- Run the protected work on this connection.
SELECT RELEASE_LOCK('example.catalog-refresh');

With a zero-second timeout, GET_LOCK() returns immediately: 1 means acquired, 0 means the attempt timed out because the lock was unavailable, and NULL indicates an error. Application code must distinguish all three outcomes.

The connection is part of this lock's lifetime. MariaDB releases the named lock when that connection ends, including abnormal termination, and COMMIT does not release it because named locks do not interact with transactions. A connection pool or abstraction that swaps connections beneath the job can therefore break an otherwise tidy-looking implementation. Hold one known connection until release.

This remains cooperative advisory locking. Another client can ignore the convention, or even acquire the same poorly chosen name. It is also scoped to one MariaDB server, not magically to independent servers. MariaDB warns that statements using GET_LOCK() are unsafe for statement-based replication, which is another reason to review the actual database topology rather than treating it as a universal distributed lock.

A lock does not make the work repeat-safe

Mutual exclusion answers, “Can two cooperating runs be in this critical section now?” It does not answer, “Did the previous run finish every side effect?”

Imagine that a job calls a remote publishing API, the API accepts the request, and the PHP process crashes before recording completion. The lock is released when the process ends. A later run can acquire it cleanly and repeat the API call. There was no concurrent overlap, yet the external effect may still happen twice.

That failure requires a separate design: an idempotency key accepted by the downstream API, a durable local state transition, a uniqueness constraint, or reconciliation against the authoritative system. The appropriate mechanism depends on the effect. A lock can protect a state transition, but it cannot extend a local transaction across an unrelated remote service.

Long lock duration deserves scrutiny too. Holding a lock around slow network work may be necessary to prevent overlap, but it also increases skipped runs or waiting time. Sometimes the better boundary is brief: claim a durable job under a transaction, release the claim lock, then perform work whose retries are designed explicitly.

Review the complete boundary

List every path that can launch the job: timer, cron, command line, web request, queue worker, and other hosts.
Choose whether a conflict should skip, wait with a bound, or become durable queued work.
Use one stable, namespaced lock identity shared by all cooperating runners.
Keep the file handle or database connection alive for the complete critical section.
Treat failure to create or acquire the lock as a deliberate outcome, never permission to run unlocked.
Log lock conflicts separately from job failures and successful work.
Check filesystem and mount behavior before relying on file locks across shared storage.
Design external side effects to be repeat-safe even after crashes and ambiguous timeouts.
Test contention, normal release, abrupt termination, and the next scheduled run.

Conclusion

The smallest correct overlap control is the one whose visibility matches every runner. A systemd timer can serialize one active unit. A PHP or util-linux file lock can coordinate cooperating processes that see one suitable filesystem. A MariaDB named lock can coordinate clients of one database server when a local file is not a shared boundary.

None should be described as exactly-once execution. First decide whether a competing run should skip, wait, or queue. Then hold the chosen lock for the intended critical section, make conflicts visible, and separately design the work for safe recovery. The useful question is not merely “Do we have a lock?” but “Which executions can see it, and what remains uncertain after it is released?”