Redis Distributed Locks: The Lock Expired, But the Process Is Still Running

We often use Redis to coordinate work across multiple application instances.

A common solution is a distributed lock.

At first glance, implementing one looks simple:

SET lock:report:123 unique-token NX EX 30

If the key doesn't exist, Redis creates it. If another process already holds the lock, the acquisition fails.

But here's a question that makes distributed locking more interesting:

What happens when the lock expires while the process holding it is still running?

Does Redis stop the process?

Does the process automatically know that it lost ownership?

Can another server start performing the same operation?

Let's explore.

1. Why Do We Need Distributed Locks?

Imagine we have a backend service deployed across three application instances.

All three servers receive requests to generate a report for the same customer.

Generating the report involves:

• Fetching a large amount of data
• Aggregating the data
• Generating a PDF
• Uploading the report to storage

Without coordination, all three servers might generate the same report simultaneously.

This is unnecessary work and could lead to conflicting writes.

We need a mechanism that allows only one participating server to acquire ownership of the resource at a time.

That's where a Redis distributed lock comes in.

2. Acquiring a Lock Atomically

Redis provides a simple way to acquire a lock:

SET lock:report:123 token-A NX EX 30

Let's break this down.

The important part is that Redis executes this as a single atomic command.

Suppose Server A and Server B attempt to acquire the same lock.

Server B tries next:

Server A wins.

Redis stores:

lock:report:123 → token-A

Server B cannot acquire the same lock while the key exists.

So far, everything looks good.

But now let's introduce a failure.

3. The Lock Expires, But the Process Is Still Running

Suppose Server A acquires the lock with a TTL of 30 seconds.

However, generating the report takes 60 seconds.

T0: Server A acquires the lock
    TTL = 30 seconds

T0: Server A starts generating the report

T30: Redis expires the lock

T31: Server B acquires the same lock

T35: Server A is still generating the report

At T31, both servers are executing the same operation.

Server A: Generate report -------------------- Finish
                       |
                       | Lock expires
                       |
Server B:              Generate report -------- Finish

This is the first important lesson:

Redis expiration changes the state of the lock. It does not control the execution of the process.

Redis doesn't:

• Kill Server A's process
• Cancel its JavaScript execution
• Throw an error inside Server A
• Automatically stop its report generation

Server A is still alive.

It is simply no longer the owner of the lock.

This distinction is fundamental to understanding distributed locks.

4. Redis Locks Are Leases

A Redis lock with a TTL is better understood as a lease.

A lease grants temporary ownership of a resource.

For example:

Server A acquires lock
        |
        v
Ownership valid for 30 seconds
        |
        v
Lock expires
        |
        v
Ownership is no longer valid

The expiration time tells Redis when the lock can become available again.

It does not guarantee that the original process will finish within that time.

This is why choosing a TTL is important.

A TTL that's too short can allow overlapping work.

A TTL that's too long can delay recovery when a process crashes.

We need to think about both cases.

5. A Second Race Condition: The Old Process Deletes a New Lock

Let's consider another failure scenario.

Server A acquires a lock:

SET lock:report:123 token-A NX EX 30

After 30 seconds, the lock expires.

Server B acquires the same lock:

SET lock:report:123 token-B NX EX 30

Now Server B owns the lock.

But Server A eventually finishes its operation and executes:

DEL lock:report:123

What happens?

Redis deletes the key.

Unfortunately, the key now belongs to Server B.

Time     Server A                 Redis                 Server B
-----------------------------------------------------------------
T0       Acquires lock             token-A
T30                                Lock expires
T31                                token-B              Acquires lock
T60       Executes DEL             Key deleted

Server A has accidentally deleted Server B's lock.

This is a different problem from the original overlapping execution.

The first problem was:

Server A continues working after losing ownership.

The second problem is:

Server A deletes a lock that belongs to another process.

Both can happen because the original lock expired.

6. The Solution: Unique Tokens and Atomic Release

When acquiring a lock, we generate a unique token.

const token = crypto.randomUUID();

Redis stores the token as the lock's value.

lock:report:123 → token-A

When Server A releases the lock, it must verify that the stored token still matches its own.

We cannot safely do this:

const value = await redis.get(key);

if (value === token) {
  await redis.del(key);
}

Why?

Because Redis can process another command between GET and DEL.

This is a classic TOCTOU race condition:

Time Of Check To Time Of Use.

Instead, we perform the comparison and deletion atomically using Lua.

if redis.call("GET", KEYS[1]) == ARGV[1] then
    return redis.call("DEL", KEYS[1])
else
    return 0
end

The script checks whether the current value matches the token.

If it matches, Redis deletes the key.

Otherwise, nothing happens.

This prevents an old process from deleting a newer process's lock.

But notice something important:

This solves the lock-release problem. It does not stop the old process from continuing its work.

7. What If We Renew the Lock?

One solution is to renew the lock periodically while the operation is running.

For example:

Lock TTL = 30 seconds

Server A:
  Acquire lock
       |
       |-- 10 seconds --> Renew lock
       |
       |-- 10 seconds --> Renew lock
       |
       |-- 10 seconds --> Renew lock
       |
       |-- Operation completes
       |
       Release lock

The renewal must also verify ownership.

We cannot blindly execute:

EXPIRE lock:report:123 30

The lock might have expired and been acquired by another server.

A safe renewal uses Lua:

if redis.call("GET", KEYS[1]) == ARGV[1] then
    return redis.call("PEXPIRE", KEYS[1], ARGV[2])
else
    return 0
end

This checks the token and extends the TTL atomically.

If the renewal fails, the process knows it no longer owns the lock.

However, there's a subtle limitation.

Suppose Server A is paused for a long time due to a process stall or a network issue.

Its lock expires.

Server B acquires the lock.

Server A resumes and continues executing.

Even though renewal helps reduce the chance of expiration during normal execution, it cannot guarantee that a stale process will never continue working.

8. Fencing Tokens: Protecting the Resource

For stronger correctness, we need to protect the resource itself.

This is where fencing tokens become useful.

A fencing token is a monotonically increasing number assigned to each lock acquisition.

Server A acquires lock → Token 41

Server B acquires lock → Token 42

Server C acquires lock → Token 43

The resource being protected remembers the highest accepted token.

If Server A attempts to write using token 41 after Server B has already written using token 42, the resource rejects the stale write.

Server A writes with token 41
        |
        v
Resource accepts token 41

Server B writes with token 42
        |
        v
Resource accepts token 42

Server A tries again with token 41
        |
        v
Resource rejects stale token

The important part is that the resource must enforce the fencing token.

Simply generating a number in the application doesn't provide protection.

The validation and write must be implemented atomically at the resource.

This is how we protect against stale lock holders.

9. What Have We Actually Learned?

Let's summarize the different mechanisms.

Each mechanism solves a different problem.

A distributed lock is not a single magic command.

It is a coordination mechanism with specific guarantees and limitations.

10. The Mental Model

The most important distinction is between two states.

Redis lock:
Who currently owns the resource?

Application process:
Who is actually executing the work?

These are not the same thing.

A process can still be alive and executing code after its lock expires.

That is the fundamental reason distributed locking is more complicated than setting a Redis key.

When we design a distributed lock, we must consider:

• How ownership is acquired
• How ownership expires
• How ownership is renewed
• How the lock is safely released
• What happens when a stale process continues working
• How the protected resource handles concurrent writes

The next time we write:

SET lock:report:123 token NX EX 30

we should remember:

We have acquired a temporary lease. We have not acquired control over the execution of every other process.

And that distinction is what makes distributed locking interesting.