Redis Distributed Locks in Python: SET NX, Redlock, Fencing Tokens

In January 2020, Jepsen tested etcd’s lock exactly the way etcd’s docs suggested. About 18% of acknowledged updates were lost, and nothing was broken except a few paused processes.

A Redis lock fails the same way. Worker A takes a Redis distributed lock, reads a counter and stalls for 3 seconds. The lock’s timeout was 2. So worker B gets the lock, reads the same value and writes it plus one. Then A wakes up and writes its stale value over B’s. Two increments, final value 1, and nothing warned A until it tried to let go.

Leave the timeout out, which is redis-py’s default, and you get the opposite bug. A worker that dies while holding the lock blocks that job forever.

Updated 4 November 2026 · Tested with Python 3.12, redis-py 8.1.0, psycopg 3.3.6, Redis 8.10.2, PostgreSQL 18.6

Part 5 of the Reliable Python services series · ← Idempotent consumers

A Redis distributed lock is a key that one worker creates with SET key token NX PX ttl. Other workers wait or skip until the holder deletes it or it expires. redis-py’s Lock wraps exactly that. So it’s a lease with a timer, and the timer doesn’t care whether you’re done.

My position: use a Redis distributed lock to save work, not to protect data. If a double run would corrupt something, make the database refuse the stale write with a fencing token.

By the end, you’ll have a lock you can copy, the two redis-py defaults to change, and a fencing token Postgres enforces. A, B and one counter row stay with us the whole way. Guess before each result.

The short answer

• Take the lock with SET key token NX PX ttl. Release it with a compare-and-delete: DELEX key IFEQ token on Redis 8.4+, a four-line Lua script before that.
• In redis-py, always pass timeout and blocking_timeout. Both default to None: “never expires” and “wait forever”.
• A stall longer than the TTL lets two workers in. At 5% stalls, 8 workers lost 1,650 of 10,000 updates. With a fencing token claimed before the read, they lost none.
• Redlock survives a crashed Redis server. It doesn’t fix a paused worker.

Don’t take my word for either bug. GitLab’s Sidekiq docs describe both. If a job dies, its deduplication key blocks new copies until the TTL runs out, 10 minutes by default. Once it does, “duplicate jobs can happen”. So GitLab lowers the TTL only for jobs that can tolerate duplicates. This post is about the jobs that can’t.

How does a Redis distributed lock work? One SET, one careful delete

One command takes the lock and one gives it back, and both are atomic. SET lock:report <token> NX PX 30000 creates the key only if it’s missing. It also sets a 30-second expiry in the same step. Release deletes the key only if it still holds your token.

Why one command? Because SETNX then EXPIRE has a gap. Die between the two and the key never expires. It’s the same trap as INCR then EXPIRE in the rate limiting algorithms post. Redis 8.10’s command docs even mark SETNX deprecated since 2.6.12.

Why the token? Because leases expire. Say A’s lease runs out and B takes the lock. A plain DEL from A would now delete B’s lock and let a third worker in. With a token, A can only ever delete its own. Here’s the whole pattern, from simple_lock.py.

"""The whole Redis lock pattern: SET NX PX to take it, compare-and-delete to give it back."""
import random
import secrets
import time

import redis

# For Redis older than 8.4 (and Valkey/managed services without DELEX): the same
# compare-and-delete as one atomic script. redis-py's own Lock uses this approach.
RELEASE_LUA = """
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
end
return 0
"""


def acquire(r: redis.Redis, name: str, ttl_ms: int, wait_s: float) -> str | None:
    """Return our token, or None if the lock was still busy after wait_s seconds."""
    token = secrets.token_hex(16)  # unique per holder, so we can only ever delete our own lock
    deadline = time.monotonic() + wait_s
    while True:
        if r.set(name, token, nx=True, px=ttl_ms):  # one atomic command: value + expiry
            return token
        if time.monotonic() >= deadline:
            return None
        time.sleep(random.uniform(0.05, 0.15))  # random, so waiters don't retry in lockstep


def release(r: redis.Redis, name: str, token: str, use_delex: bool = True) -> bool:
    """Delete the lock only if it still holds our token. False: it had already expired."""
    if use_delex:  # Redis 8.4+
        return r.execute_command("DELEX", name, "IFEQ", token) == 1
    return r.eval(RELEASE_LUA, 1, name, token) == 1

Which release command does your Redis have?

Use DELEX if you can. DELEX arrived in Redis 8.4. Valkey 9.0 and ElastiCache for Valkey 9.0 call it DELIFEQ. Everywhere else, including ElastiCache for Redis OSS, use the Lua script. That’s also what redis-py’s own Lock runs.

redis-py 8.1.0’s r.delex() prints an “experimental” warning, hence the raw command. The random retry delay follows the Redis docs. The tests cover both release paths, against Redis 8.10.2 only.

Try it: python -m pytest tests/test_simple_lock.py -q. Every “Try it” runs in the repo for this post, after the setup in its README. Every number here comes from my own runs, so yours will differ.

One warning if you search for this. Redis’s own “Redis Lock” glossary page still calls setex inside an if. But SETEX always sets the key, so every caller “acquires” the lock. The curly quotes in its samples won’t even run.

How do you use redis-py’s Lock? Change these defaults first

Call r.lock(name, timeout=30, blocking_timeout=5) and use it as a context manager. Pass both numbers, every time. In redis-py 8.1.0, timeout defaults to None, so the key never expires. And blocking_timeout defaults to None, so acquire() waits forever.

Here are the defaults from the installed redis/lock.py. A test pins them, so an upgrade that changes them fails loudly.

With those set, here’s a “one at a time” wrapper. It’s the sync half of redispy_lock.py.

from collections.abc import Awaitable, Callable

import redis
import redis.asyncio as aredis
from redis.exceptions import LockError, LockNotOwnedError


def run_exclusive(r: redis.Redis, name: str, job: Callable[[], None],
                  ttl: float = 30, wait: float = 5) -> bool:
    """Run job() under the lock. False if another worker held it for `wait` seconds."""
    try:
        with r.lock(name, timeout=ttl, blocking_timeout=wait):
            job()
    except LockNotOwnedError:
        raise  # the lease ran out mid-job, so another worker may have run it too
    except LockError:
        return False  # never got the lock (LockNotOwnedError is a subclass: catch it first)
    return True

Order matters in those except lines. LockNotOwnedError is a subclass of LockError. So a bare except LockError would treat “my lease expired mid-job” like “someone else had it”. The first one means the job may have run twice.

Why thread_local=True? Say thread 1’s lease expires and thread 2 acquires through the same Lock object. Without it, thread 1’s release() would use thread 2’s token and delete its lock.

asyncio is where that protection runs out. redis.asyncio has its own Lock with the same defaults, and the other half of redispy_lock.py uses it with async with. But tasks share one thread. In the test, two tasks sharing one Lock deleted each other’s lock. Create one Lock per task.

How long should the lock’s TTL be?

Start from the critical section’s p99 and add a margin. A p99 of 8 seconds suits a TTL of 30. If the job sometimes runs for ten minutes, don’t stretch the TTL to match. Renew a shorter one instead, and still fence the write.

How do systems that hand out leases for a living pick the number? Here are their defaults, from each project’s docs, source or paper.

Two camps show up. Chubby, Kubernetes and DynamoDB keep leases short, 12 to 20 seconds, and renew well inside them. DynamoDB’s heartbeat is a quarter of its lease. Celery sets one long timeout instead, and doesn’t renew at all.

The TTL is rarely the whole story, though. client-go says outright that it doesn’t fence. Chubby’s paper even describes this post’s fix: send “the lock acquisition count” with each write, and reject a lower one. So size the TTL from your p99, and let a fencing token handle the stall you didn’t plan for. Leader election needs the same fence.

What happens if the lock expires while you’re still working?

Nothing tells you. Redis deletes the key when the TTL runs out, and the next worker takes the lock. Your code carries on as if it still held it. You only find out at release, when redis-py raises LockNotOwnedError. By then your write has landed on top of someone else’s.

Think of a parking meter. When the time runs out, the spot goes to the next car whether or not you’re back. And the meter doesn’t come looking for you.

What makes a worker overrun? Usually something dull: CPU throttling, a network call with no timeout, a blocked event loop. Garbage collection is the villain in Kleppmann’s Java example, but CPython frees most objects by reference counting. And Redis expires keys by the wall clock, as its docs warn, so a clock jump can end a lease early.

Let’s break it on purpose. A takes a 2-second lock, reads the counter, stalls for 3 seconds, then writes value + 1. B starts half a second later. Here’s break_it.py, minus the run() that resets the row and starts both threads.

import os
import pathlib
import threading
import time

import psycopg
import redis
from redis.exceptions import LockNotOwnedError

REDIS_URL = os.environ.get("REDIS_URL", "redis://localhost:56379/5")
PG_DSN = os.environ.get("PG_DSN", "postgresql://patterns:patterns@localhost:55432/locks")
r = redis.Redis.from_url(REDIS_URL)


def worker(name: str, stall: float) -> None:
    with psycopg.connect(PG_DSN, autocommit=True) as db:
        lock = r.lock("lock:counter", timeout=2, blocking_timeout=10)
        if not lock.acquire():
            return
        (value,) = db.execute("SELECT value FROM counters WHERE id = 1").fetchone()
        time.sleep(stall)  # A: 3 s. Nothing tells A that its lease ended at 2 s.
        db.execute("UPDATE counters SET value = %s WHERE id = 1", (value + 1,))
        try:
            lock.release()
        except LockNotOwnedError:
            print(f"{name}: the lock expired before release")

$ python break_it.py
A: the lock expired before release
final value: 1

Two increments, final value 1. The test runs it 20 times, and it ended at 1 all 20 times. B got the lock when A’s lease ran out at 2 s, read 0 and wrote 1. A woke at 3 s and wrote its stale 0 + 1 over it.

Try it: python break_it.py. It prints final value: 1.

One stall, one lost update. Now scale it up. Guess first: 8 worker processes, 10,000 increments, and 5% of critical sections stall past the TTL. How many updates go missing? About 5%?

In my runs, the counter ended a median 1,650 short. That’s 16.5%, more than three times the stall rate. A stalled worker doesn’t just lose its own update. Its stale write erases every increment the others made while it slept. If that counter tracked seats sold, 1,650 sold seats would still look free.

How a 90-second network freeze outlasted GitHub’s heartbeat timeouts

GitHub hit the same shape of bug with servers instead of workers. On 22 December 2012, during a scheduled switch upgrade, traffic between its access switches was blocked “for roughly a minute and a half” (GitHub’s write-up). Its fileservers ran in active/passive pairs. When the network froze, many exceeded their heartbeat timeouts and decided to take over.

Each sent a STONITH command (shoot the other node in the head) to its partner, and some never got through. Some pairs ended up with both nodes expecting to be active, then with both stopped. Restarting much of the file storage took over five hours.

My lesson: a fence that has to reach the stale node fails in the outage that makes you need it. STONITH had to cross the network that had just frozen. A fencing token never has to reach anyone. The stale worker brings its old token to the database, and the write matches no row.

How much does a pause lose? Jepsen measured 18%

Back to the 18% from the top. For its etcd 3.4.3 analysis in January 2020, Kyle Kingsbury protected an in-memory set with etcd’s lock, as etcd’s documentation suggested. Each client read the set, then about a second later wrote it back plus one element.

With 2-second lease TTLs, five processes and process pauses every five seconds, that’s where the 18% came from. It’s the same ballpark as my 16.5%. Kingsbury puts it bluntly: “Multiple processes can hold an etcd lock concurrently, even in healthy clusters with perfectly synchronized clocks.” His fix is a fencing token too: the lock key’s revision.

Can you renew a Redis lock? Yes, and it still won’t save you

Yes. Call lock.extend(ttl, replace_ttl=True) from a background thread at about a third of the TTL. Stop the work as soon as an extend fails. That keeps a slow worker’s lease alive. It can’t help a frozen one, because a paused process pauses its renewer too.

Here’s a renewer that tells the work when the lease is gone (renewing.py, sync half).

import asyncio
import contextlib
import sys
import threading
import time
from collections.abc import AsyncIterator, Iterator

import redis
import redis.asyncio as aredis
from redis.exceptions import LockError


@contextlib.contextmanager
def renewing_lock(r: redis.Redis, name: str, ttl: float, wait: float) -> Iterator[threading.Event]:
    # thread_local=False: the renewer thread must see the token the main thread got
    lock = r.lock(name, timeout=ttl, blocking_timeout=wait, thread_local=False)
    if not lock.acquire():
        raise LockError(f"{name} is busy", lock_name=name)
    lost, stop = threading.Event(), threading.Event()

    def renew() -> None:
        while not stop.wait(ttl / 3):  # renew at a third of the TTL
            try:
                lock.extend(ttl, replace_ttl=True)
            except LockError:  # expired or taken: someone else may be working now
                lost.set()
                return

    renewer = threading.Thread(target=renew, daemon=True)
    renewer.start()
    try:
        yield lost  # the work checks lost.is_set() between steps and stops if it's set
    finally:
        stop.set()
        renewer.join()
        if not lost.is_set():
            lock.release()  # LockNotOwnedError here means it expired after the last renewal

Note the thread_local=False. With the default, the renewer thread can’t see the main thread’s token. Then extend() raises “Cannot extend an unlocked lock”. On Redis 8.4+, SET key token IFEQ token PX ttl also extends a lease in one command.

What happens when the whole process freezes?

You need a real freeze to test this. time.sleep() pauses one thread while the renewer keeps going, so renewal looks perfect. So the repo’s test freezes the whole process instead: SIGSTOP on Linux, NtSuspendProcess on Windows. (docker pause does the same to a container.)

Guess first: a lock with a 1-second TTL, a renewer, and a 2-second freeze. Does A keep the lock?

No. The key was gone within 2 seconds, and another worker took the lock. The frozen holder printed “lost” only after it was thawed.

Try it: python -m pytest tests/test_renewing.py -q. The freeze helper is tests/procfreeze.py, and it works on Linux, macOS and Windows.

The opposite failure is real too. A renewer that runs on its own keeps a hung worker’s lock alive. In the test, the main thread was stuck for five TTLs, and nobody else could get the lock. python-redis-lock’s auto_renewal renews from a daemon thread “for as long as the lock is held”. So cap the total hold time.

asyncio fails the first way. A 0.9-second time.sleep() inside renewing_lock_async(…, ttl=0.3) starved the renewal task. The key expired while the coroutine still “held” it.

So renewal makes an overrun less likely, and that’s all it does. To stop the stale write, the resource itself has to check.

What is a fencing token? Make Postgres say no to the stale worker

A fencing token is a number that grows every time the lock is granted. The worker sends it with each write. The database keeps the highest token it has seen and rejects any write with a lower one. A stale worker always holds an older, lower token. So its write fails, even though it still thinks it has the lock.

Think of the ticket at a deli counter. Once 34 has been called, 33 doesn’t get served, however hard it waves. Kleppmann’s diagram used the same two numbers, so A and B get them here too.

Issue the token inside the acquire. A token fetched afterwards can come out higher for the stale holder. So one Lua script sets the lock and bumps a counter. A subclass keeps everything else redis-py’s Lock does. This is fencing.py.

from collections.abc import Callable

import psycopg
import redis
from redis.lock import Lock

# Take the lock and issue the next token in one atomic step. A token fetched
# after acquiring could be handed out in the wrong order to a stale holder.
ACQUIRE_LUA = """
if redis.call('SET', KEYS[1], ARGV[1], 'NX', 'PX', ARGV[2]) then
  return redis.call('INCR', KEYS[2])
end
return 0
"""


class FencedLock(Lock):
    """redis-py's Lock (same blocking, release and extend), plus a fencing token.

    Name it with a hash tag, e.g. "{lock:job}", so the lock key and
    "{lock:job}:fence" land in the same Redis Cluster slot.
    """

    def __init__(self, r: redis.Redis, name: str, timeout: float, **kwargs):
        super().__init__(r, name, timeout=timeout, **kwargs)
        self.fence_key = f"{name}:fence"
        self._acquire = r.register_script(ACQUIRE_LUA)

    def do_acquire(self, token: bytes) -> bool:
        fence = self._acquire(keys=[self.name, self.fence_key],
                              args=[token, int(self.timeout * 1000)])
        self.local.fence = int(fence)
        return self.local.fence > 0

    @property
    def fence(self) -> int:
        return self.local.fence


Update = Callable[[int], int]  # old value -> new value; the slow part of the job

The shared hash tag {lock:counter} keeps both keys in one Redis Cluster slot. One honest limit: the counter lives in Redis, so a failover that loses the lock can roll it back too.

Next, the row gets a fence column, and the write checks the token in the same statement.

-- The protected resource: one counter row per id.
-- fence = the highest fencing token that has claimed or written this row.
CREATE TABLE IF NOT EXISTS counters (
    id    int    PRIMARY KEY,
    value bigint NOT NULL DEFAULT 0,
    fence bigint NOT NULL DEFAULT 0
);

def fenced_write(db: psycopg.Connection, row_id: int, fence: int, update: Update) -> bool:
    """Read, work, then write only if no newer token has written. False: we're stale."""
    (value,) = db.execute("SELECT value FROM counters WHERE id = %s", (row_id,)).fetchone()
    cur = db.execute(
        "UPDATE counters SET value = %s, fence = %s WHERE id = %s AND fence < %s",
        (update(value), fence, row_id, fence))
    return cur.rowcount == 1  # 0 rows: a newer holder wrote first. Abort and redo.

Can two workers both pass that WHERE? No. Under Postgres’s default READ COMMITTED, if another transaction updated the row first, this UPDATE waits. Then it re-checks its WHERE on the new row version (the docs spell it out). A rowcount of 0 means you’re stale: abort and redo with a fresh lock.

The strict < lets each token write once. If a lease writes the row several times, use <=. Just know that a duplicated token could then write twice.

Now rerun A and B. fence_it.py is break_it.py with those changes.

$ python fence_it.py pause write-only
B: token 34 accepted
A: token 33 REJECTED
A: token 33 had expired before release
A: token 35 accepted
final value: 2

A’s token 33 was rejected. So A took the lock again with 35 and redid its increment. Final value 2.

Try it: python fence_it.py pause write-only.

How many lost updates does a fencing token stop?

Nearly all of them, and all of them if you claim the row before you read it. To check, 8 worker processes share one row and one lock. Each critical section reads, works 5 ms and writes +1. Some stall for 1.5× the TTL first.

That’s A and B’s story scaled down 10×: a 200 ms TTL and a 300 ms stall. Rejected writes are redone, so lost updates are 10,000 minus the final value. Each setting ran 5 times, with no fencing, fenced_write() (write-only) and claim_then_write() (claim-first, explained next).

Without fencing, each stall erased about 3.2 increments at 5%, by my arithmetic (1,650 ÷ 511). Write-only fencing lost 3 updates over the 5 runs at 5%, never more than 1 in a run, and none at the lower rates. Claim-first fencing lost nothing in any run. Postgres rejected the stale writes, and the workers redid them.

Try it: python measure_fencing.py. It takes about 35 minutes.

Is fencing the write enough if you read first?

Not always. Checking the token at write time protects the write, not the read before it. Lorin Hochstein showed this with a formal model in 2025. Fencing tokens can still fail when two holders’ requests overlap at the resource.

Here it is in slow motion. The row’s fence is 32. A (token 33) reads, then stalls past its lease. B (34) gets the lock, reads the same value and starts 1.5 s of work. A wakes and writes. 33 beats 32, so Postgres accepts it. Then B writes with 34 and overwrites A’s increment.

$ python fence_it.py overlap write-only
A: token 33 accepted
A: token 33 had expired before release
B: token 34 accepted
final value: 1

The fix is to claim before you read. Stamp your token on the row and get the value back in one statement. Then write only if the stamp is still yours.

def claim_then_write(db: psycopg.Connection, row_id: int, fence: int, update: Update) -> bool:
    """Stamp our token on the row before reading it, then write only if it's still ours."""
    row = db.execute(
        "UPDATE counters SET fence = %s WHERE id = %s AND fence < %s RETURNING value",
        (fence, row_id, fence)).fetchone()
    if row is None:
        return False  # a newer holder already claimed the row
    cur = db.execute(
        "UPDATE counters SET value = %s WHERE id = %s AND fence = %s",
        (update(row[0]), row_id, fence))
    return cur.rowcount == 1  # 0 rows: a newer holder claimed it while we worked

$ python fence_it.py overlap claim-first
A: token 33 REJECTED
A: token 33 had expired before release
B: token 34 accepted
A: token 35 accepted
final value: 2

Try it: python fence_it.py overlap claim-first.

So why did write-only fencing do so well in the big run? A stalled worker woke 100 ms after its lease ended, when the next holder had nearly always written. The event logs of its 3 losses show one rare pattern: three holders in a row stalled, and nobody newer wrote in between.

To make the overlap common, a second run used 100 ms of work (half the TTL) and random stalls of 100 to 300 ms. It ran 5% stalls over 2,000 sections, 5 runs. Guess first: how much of the damage does write-only fencing stop now?

About half. Write-only fencing lost a median 50 updates per run (252 over 511 stalls). With no fencing at all, it was 92. Claim-first lost 0 in every run.

My take: if critical sections are short next to the TTL, write-only fencing catches almost every stale writer. If they’re long, or you can’t bound them, claim first. It’s one extra statement, and the same version check an idempotent consumer does.

Is Redlock safe? What Kleppmann and antirez actually disagreed about

Redlock is safe against a Redis server dying, not against a stalled client or a jumping clock. It’s a Redis distributed lock taken on a majority of five independent Redis primaries. Losing one server doesn’t lose the lock. But Redlock issues no fencing token, so the stale-writer problem is exactly as real.

Why one Redis with a replica can hand the lock out twice

Start with the problem Redlock solves. Redis replication is asynchronous. The primary answers OK to your SET, then ships it to the replica a moment later. If it crashes in between, the promoted replica has never heard of your lock. So the next client gets it too. The Redis docs mark this “SAFETY VIOLATION!”

WAIT 1 100 after acquiring narrows the window. But the WAIT docs say it “does not make Redis a strongly consistent store”. Part 1’s idempotency keys in distributed systems carry the same caveat for their SET NX keys.

How does Redlock work, and what did the debate settle?

Redlock avoids replicas instead. You try SET NX PX with one random value on all five nodes. The lock is yours only if three agreed within the TTL. Your validity time is the TTL minus the elapsed time minus clock drift. For restarts, the docs suggest appendfsync always or delayed restarts: keep a crashed node down longer than the longest TTL.

For Python, the Redis docs list redlock-ng, pottery and aioredlock. redlock-ng replaced the unmaintained redlock-py there on 17 January 2026. I didn’t measure Redlock across several nodes, because the test setup runs one Redis.

So is it safe? That question started a famous argument. Here it is in dates.

• 8 February 2016. Martin Kleppmann: Redlock assumes bounded delays, pauses and clock error, and has no fencing token. Correctness needs consensus plus fencing at the storage.
• February 2016. antirez, Redis’s creator: the random value allows a check-and-set at the resource. And storage that checks tokens is already linearizable.
• 3 March 2025. Hochstein’s models: fencing can fail when requests overlap.
• 24 March 2026. Kleppmann and Chris Riccomini announce the second edition of Designing Data-Intensive Applications. The first made 33 and 34 famous.
• Today. The Redis lock page says: implement fencing tokens, and TTLs use the wall clock.

Both sides agree a lock alone doesn’t protect data. They disagree on whether Redlock’s timing assumptions are acceptable. My verdict: Redlock buys availability, not safety. If a double run would hurt, fence the write, and one Redis will do.

Do you even need a distributed lock?

Often not. Ask what a double run costs. If it only wastes time, like a report built twice or a cache warmed twice, a plain Redis lock is fine. If it corrupts data or moves money, the database has to enforce the rule anyway. Then the lock is only an optimization.

My runs say the same. At 5% stalls, the lock let a median 541 stale workers through per 10,000 sections. With claim-first fencing, not one of them cost an update. The lock saved work, and the UPDATE kept the counter right. Before you add a lock, check these.

• A unique constraint or an idempotency key, when the danger is doing something twice: charging, emailing, inserting. Use the key instead of a lock.
• SELECT … FOR UPDATE, or SKIP LOCKED when workers pull jobs from a table. The transactional outbox relay in Python claims its rows this way.
• An atomic update. UPDATE counters SET value = value + 1 needs no lock. break_it.py only breaks because it reads first.
• A version column. WHERE id = %s AND version = %s is optimistic locking: fencing without a lock server.

Three recipes for common stacks. Celery beat singleton: a Redis lock with timeout above the task’s time_limit, so it can’t expire while the task may still run. Add blocking=False, so an overlapping run skips. Django: select_for_update() inside transaction.atomic(). FastAPI double submit: an idempotency key, not a lock.

How Modern Treasury guards a balance with a version number, not a lock

Modern Treasury’s Ledgers API is the “moves money” case. Several requests debit one account at once, and the balance mustn’t go negative. Andy Qin wrote up their design in August 2021. Checking the balance and then debiting isn’t safe, because another debit can land in between.

They’re a Rails shop on PostgreSQL, and they picked optimistic locking with Rails’ lock_version over a row lock. They expected most ledger operations to be reads, and didn’t want reads to block writes.

The account row turned out to be the wrong home for lock_version, because editing an account’s details would conflict with a debit. So each ledger account got a separate “ledger account version” row that moves only when entries are posted. If the lock_version a client sends doesn’t match, the transaction rolls back and the request fails.

That’s break_it.py‘s race, with this post’s answer: a number the database checks in the write itself. The separate version row makes the claim-first point too. Check what you read (the entries behind the balance), not just the row you write.

Kubernetes does this for every resource. An update with a stale resourceVersion fails with a 409 Conflict, and the client re-reads and retries. Neither needs a lock server to stay correct. I’d put a Redis lock in front only if the retries got expensive.

Do AI agents need distributed locks?

Agents make long critical sections common. A tool that rewrites a shared document can hold a lock across a model call, and a slow call is exactly the stall from the lab. So set the model client’s timeout below the lock’s TTL, and claim the row before the call. If your agents go through an LLM gateway, its timeout is the number to check.

Redis lock vs Postgres advisory lock vs etcd or ZooKeeper: which should you pick?

Pick by what a double run costs and what you already run. Redis SET NX suits cheap efficiency locks. A Postgres row or advisory lock fits when the data lives in Postgres, because the lock ends with the transaction. etcd or ZooKeeper fit when the lock must survive a node failure and you want a fencing number built in.

Postgres advisory locks have no TTL. A hung client keeps a session-level lock until its connection closes. Session locks also stack, and they don’t work through PgBouncer’s transaction pooling, which PgBouncer lists as unsupported. The transaction-level lock avoids both. Hash its name with blake2b, not Python’s per-process salted hash().

"""The no-Redis option: a Postgres advisory lock that ends with the transaction.

    with psycopg.connect(PG_DSN) as db, db.transaction():
        if try_lock(db, "nightly-report"):
            build_report(db)   # released at COMMIT or ROLLBACK, or if the connection dies
"""
import hashlib

import psycopg


def lock_id(name: str) -> int:
    # Not hash(name): str hashes are salted per process, so each worker would get a different id.
    return int.from_bytes(hashlib.blake2b(name.encode(), digest_size=8).digest(), "big", signed=True)


def try_lock(db: psycopg.Connection, name: str) -> bool:
    """Call inside a transaction. True: the lock is ours until the transaction ends."""
    return db.execute("SELECT pg_try_advisory_xact_lock(%s)", (lock_id(name),)).fetchone()[0]

In the test, a second connection gets False until the first transaction ends. And lock_id() matched across processes with different PYTHONHASHSEED values, while hash() didn’t. For etcd, the lock call returns a key that lives as long as your lease.

Try it: python -m pytest tests/test_advisory.py -q.

Speed rarely decides it. Uncontended, an acquire plus release took a median 0.68 ms with redis-py’s Lock and 0.65 ms with SET NX PX and DELEX. The FencedLock took 0.69 ms, so by my arithmetic the fencing token adds about 0.01 ms. pg_try_advisory_xact_lock in its own transaction took 0.88 ms. All of it ran over loopback on the same machine.

What does redis-py’s polling cost you?

A waiter doesn’t get told when a lock frees up. It polls with SET NX every sleep seconds, 0.1 by default. Guess first: how late does one waiter notice a released lock?

In the repo’s test, a median 51.5 ms late (p95 96.3 ms). With sleep=0.01, it was 5.4 ms. Eight waiters starting on the same cron tick took 49.5 ms, against 8.4 ms for eight arriving at random. redis-py sleeps a fixed interval with no jitter, so waiters that start together poll in lockstep.

Try it: python measure_polling.py. It takes about 10 minutes.

Which Python lock library in 2026?

redis-py’s own Lock covers most jobs. Here’s what else is on PyPI, by last release.

None of them hands out a fencing token. That’s the 20 lines of fencing.py above.

Redis distributed lock mistakes to check before you ship

Each of these passes the happy-path test: one worker, no stalls, nothing crashes. They show up the first time a container gets throttled.

• No timeout. redis-py’s default never expires, so a dead worker blocks the job for good. Set it from the p99.
• No blocking_timeout. acquire() waits forever. Set a few seconds, or blocking=False to skip.
• A plain DEL to release. After an expiry, it deletes the next holder’s lock. Compare and delete instead.
• Trusting renewal. A frozen process freezes its renewer too.
• A lock as the only guard. At 5% stalls it lost 1,650 of 10,000 updates. Fence the write, and claim first if sections are long.

If you do one thing this week, search your services for .lock(. Any call without both timeout and blocking_timeout can block a job forever.

And A and B? This time B’s increment survives. A’s stale write bounces off Postgres, A takes the lock again with token 35, and the counter ends at 2. The lock saved work. The database kept the count right.

Over to you: would a double run of your cron job cost money, or just time? And if you think I’m too hard on Redlock, or you’ve fenced something that isn’t Postgres, tell me in the comments. I’m happy to be proven wrong by a test.

The lock, the fencing code, the 45 tests and the measurement scripts are on GitHub. Found a stall they miss? Open an issue or a pull request.

That’s part 5 of Reliable Python services: idempotency keys, rate limiting, the outbox, idempotent consumers and now locks. All five live in the system design hub, and the newsletter brings the next one.