[Proposal]: Lock statement pattern (VS 17.10, .NET 9)

@kouvel

Lock statement pattern

Proposed
Prototype: Not Started
Implementation: Add Lock object feature roslyn#71716
Specification: https://github.com/dotnet/csharplang/blob/main/proposals/lock-object.md

(This proposal comes from @kouvel. I've populated this issue primarily with text he wrote in a separate document and augmented it with a few more details.)

Summary

Enable types to define custom behaviors for entering and exiting a lock when an instance of the type is used with the C# “lock” keyword.

Motivation

.NET 9 is likely to introduce a new dedicated System.Threading.Lock type. Along with other custom locks, the presence of the lock keyword in C# might lead developers to think they can use it in conjunction with this new type, but doing so won't actually lock according to the semantics of the lock type and would instead treat it as any arbitrary object for use with Monitor.

Detailed design

Example

A type would expose the following to match the proposed pattern:

class Lock : ILockPattern
{
    public Scope EnterLockScope();

    public ref struct Scope
    {
        public void Dispose();
    }
}

public interface ILockPattern { }

EnterLockScope() would enter the lock and Dispose() would exit the lock. The behaviors of entering and exiting the lock are defined by the type.

The ILockPattern interface is a marker interface that indicates that usage of values of this type with the lock keyword would override the normal code generation for arbitrary objects. Instead, the compiler would lower the lock to use the lock pattern, e.g.:

class MyDataStructure
{
    private readonly Lock _lock = new();

    void Foo()
    {
        lock (_lock)
        {
            // do something
        }
    }
}

would be lowered to the equivalent of:

class MyDataStructure
{
    private readonly Lock _lock = new();

    void Foo()
    {
        using (_lock.EnterLockScope())
        {
            // do something
        }
    }
}

Lock pattern and behavior details

Consider a type L (Lock in this example) that may be used with the lock keyword. If L matches the lock pattern, it would meet all of the following criteria:

L implements interface ILockPattern
L has an accessible instance method S EnterLockScope() that returns a value of type S (Lock.Scope in this example). The method must be at least as visible as L. Extension methods don't qualify for the pattern.
A value of type S qualifies for use with the using keyword

A marker interface ILockPattern is used to opt into the behaviors below, including through inheritance, and so that S may be defined by the user (for instance, as a ref struct). For a type L that implements interface ILockPattern:

If L does not fully match the lock pattern, it would result in an error
If a value of type S may not be used in the context, it would result in an error
If a value of type L is implicitly or explicitly casted to another type that does not match the lock pattern (including generic types), it would result in a warning
- This includes implicit casts of this to a base type when calling an inherited method
- This is intended to prevent accidental usage of Monitor with values of type L that may be masked under a different type and used with the lock keyword, such as with casts to base types, interfaces, etc.
If a value of type L is used with the lock keyword, or a value of type S is used with the using keyword, and the block contains an await, it would result in an error
- The warning would only be issued if in-method analysis can detect it
- This is intended to prevent the enter and exit from occurring on different threads. Monitor and Lock have thread affinity.
- SpinLock is optionally thread-affinitized. It can opt into the pattern, but then usage of it with the lock or using keywords would still disallow awaits inside the block statically.

SpinLock example

System.Threading.SpinLock (a struct) could expose such a holder:

struct SpinLock : ILockPattern
{
    [UnscopedRef]
    public Scope EnterLockScope();

    public ref struct Scope
    {
        public void Dispose();
    }
}

and then similarly be usable with lock whereas today as a struct it's not. When a variable of a struct type that matches the lock pattern is used with the lock keyword, it would be used by reference. Note the reference to SpinLock in the previous section.

Drawbacks

If an existing reference type opted-in to using the pattern, it would change the meaning of lock(objectsOfThatType) in existing code, and in ways that might go unnoticed due to the general unobservability of locks (when things are working). This also means if System.Threading.Lock is introduced before the language support, it likely couldn't later participate.
- Also if one library L1 that exposes the type is updated to opt into the lock pattern, another library L2 that references L1 and was compiled for the previous version of L1 would have different locking behavior until it is recompiled with the updated version of L1.
The same thing is achievable today with only a little bit more code, using using instead of lock and writing out the name of the enter method manually (i.e. writing out the lowered code in the first example).
Having the lock semantics vary based on the static type of the variable is brittle.
ILockPattern itself doesn't match the lock pattern, though an interface could be created to match the lock pattern.

Alternatives

Use an attribute instead of a marker interface. A marker interface was used because it works more naturally with inheritence. Although AttributeUsageAttribute.Inherited = true could be used for an attribute, it doesn't seem to work with interfaces.
Use a new syntax like using lock(x) or other alternatives instead, which would eliminate any possibility of misusing values of the type with Monitor.
System.Threading.Lock could be special-cased by the compiler rather than having it be a general pattern.
Various naming options in the pattern itself.
Instead of having a holder type, instead have the compiler explicitly recognize certain enter/exit methods by naming convention or attribution.
Potentially allow/require the ref keyword for certain uses, e.g. in the SpinLock example.
The pattern as suggested doesn't support being hardened against thread aborts (which are themselves obsoleted). It could be extended to now or in the future.

Unresolved questions

Design meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-05-01.md#lock-statement-improvements
https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-10-16.md#lock-statement-pattern
https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-12-04.md#lock-statement-pattern

If a System.Threading.Lock lo becomes typed as Object at compile-time for whatever reason, lock(lo) would break silently. For example:

Dictionary<string, object> dic = new();
Lock lo = new();
lock(dic["my lock"] = lo) {}//this does not work.

This pattern match design creates potential problems for codes which implement custom locks and want to take advantage of this new language feature. For example, suppose I implement my own lock:

class MyLock
{
    public LockHolder EntersLockWithHolder() {...}

    public struct LockHolder
    {
        public void Dispose() {...}
    }
}

...

static readonly MyLock lo = new();

static void MyMethod()
{
    lock(lo)
    {
        ...
    }
}

Everything compiles even though the spelling EntersLockWithHolder is wrong and lock(lo) does not do what I intended.

This may also be problematic if MyLock is first written against .NET 8 and is later ported to .NET 7 or .NET Framework for whatever reason. In these cases, the codes would also break silently.

This problem could be solved by introducing a custom attribute only to be found in .NET 8 BCL that must be present on such lock types. The compiler would then detect any misspelling, and backporting to earlier version would result in a compilation error.

This new design creates inconsistency between newer lock types that should be used with lock(...) and older lock types like Mutex that should not be used with lock(...).
If this new design is implemented in C#, it should also be implemented in VB.NET, as VB.NET has SyncLock which is basically equivalent to lock of C#.

If we consider implementing this feature, shouldn’t we consider its async counterpart which is lowered to await using, too?

@vladd

@vladd

If we consider implementing this feature, shouldn’t we consider its async counterpart which is lowered to await using, too?

I believe that should follow a completely different pattern as locks should have to opt-in to being "async"-friendly.

@tfenise

@tfenise

If a System.Threading.Lock lo becomes typed as Object at compile-time for whatever reason, lock(lo) would break silently.

I have to agree, I think there's enough danger of these newer-style locks being accidentally treated as monitor locks by the compiler in any case where the pattern just happens to not match, with no warning to the developer. I think that needs to be added to the list of "Drawbacks" for this proposal.

Is the plan to also extend this to the myriad of other locks or synchronization primitives that exist in the BCL?

Would it be feasible to introduce the ability of a type to forbid using Monitor on it into the runtime? Let's say this feature would be enabled by annotating the type as [ForbidMonitor]. Then the new Lock could look like this:

[ForbidMonitor]
class Lock
{
    public LockHolder EnterLockWithHolder();

    public struct LockHolder
    {
        public void Dispose();
    }
}

And used like this:

private readonly Lock _lock = new();

void Foo()
{
    lock (_lock) // lowers to EnterLockWithHolder()
    {
    }

    lock ((object)_lock) // lowers to Monitor.Enter, but throws at runtime
    {
    }
}

@svick

@svick

How would that be enforced? If you upcast the Lock it's just a normal object and the compiler wouldn't know to not use Monitor.

@HaloFour

@HaloFour As per the comment, it would be runtime-enforced. Presumably, if [ForbidMonitor] is applied to the type, some bit in the method table would be set to indicate that any Monitor methods called on the object should throw.

@HaloFour

@HaloFour I meant that the compiler would use Monitor, but the implementation of Monitor.Enter would detect that the type of the object is one of the annotated ones and would throw an exception.

But I don't know if that could be implemented without making every existing use of lock slower.

@alexrp

@alexrp @svick

Sorry, I see what you meant after another re-reading of the comment. It'd prevent a subtle break, but you could still end up with exceptions at runtime if you're not careful.

I'd love to see more language support around locking types. It's always annoying me that none of the lock types in the BCL even offer support for using, and I've written a bunch of extension methods specifically to enable that. It's nice to see that they're considering it for this new Lock class. I, personally, think that's sufficient. The problem with lock is that it's always going fallback to Monitor, and unless we want to make generic support for these locks a part of the runtime itself it feels a bit too dangerous to me to try to expand on that, at least without some novel guardrails. Honestly, I wish .NET could deprecate Monitor in general.

@vladd

@vladd

If we consider implementing this feature, shouldn’t we consider its async counterpart which is lowered to await using, too?

I believe that should follow a completely different pattern as locks should have to opt-in to being "async"-friendly.

Perhaps await lock?

Also, though I have seen await using used for an async lock before, I don't think it makes much sense. The popular AsyncEx library just uses using. So await lock (_lock) would lower to using (await _lock.EnterLockWithHolderAsync()).

Hmm, my initial reaction is, not only we shouldn't add to lock() {} in C#, but we should discourage its use, and instead recommend the using(...) {} approach, even for the Monitor-based locking. This seems to be too much of a specific type of feature to be part of the language. And I also don't like the lock suddenly doing different things depending on the type's surface area.
However, I don't think the Monitor locking is bad idea by itself, using something like using(Monitor.Lock(obj)) { ... } is perfectly fine with me and I use lock currently in many places, it's just that it's really not a language-level feature in my opinion.
PS. To clarify, from a clean slate perspective, I wouldn't have made all .NET objects "lockable" with Monitor; it should be a feature offered by a special type, similar to Mutex/etc. But given the legacy, the Monitor just isn't going anywhere any time soon.

Having thought about it, I would have liked in a "perfect world" to have lock (and await lock) statement in C#, which would do something similar to what the OP proposes, as syntactic sugar, and then a few different types could be offered in the BCL to address various common usage scenarios. However, given how lock is now tightly related to Monitor, it's not a good idea to expand it. Perhaps, we can introduce a new keyword here? Like synch and await synch? Then lock and Monitor can both be obsoleted over time with suggestions/warnings, auto-fixers, etc.

If a System.Threading.Lock lo becomes typed as Object at compile-time for whatever reason, lock(lo) would break silently. For example:

Correct and that is an explicit design choice. This proposal breaks compat for any type which is being locked on today that just happens to fit the new locking pattern. Casting to object in a lock is the mechanism for restoring the existing behavior. Essentially, it's a key part of the proposal that it be allowed and have this behavior.

lock(dic["my lock"] = lo) {}//this does not work.

This particular case is not very realistic to me. In practice, it's very uncommon for the expression inside a lock to be anything other than a variable (local or field) so the chance of this type of problem feels very low. I think the more realistic case is a Lock instance getting converted to object as a part of a more vanilla call and the receiver later decides to lock on it. Even that seems fairly unlikely cause locking on instances you don't own is a dangerous behavior.

To be specific, the new System.Threading.Lock type - and its special-cased interaction with the lock keyword - is expected in C#13.

The actual proposal - a lock pattern - is not coming, at least not in C# 13

Is there any reason why we still use the lock keyword now that the language has using disposal blocks? Wouldn't it just be better to encourage developers to use the slightly more explicit using approach and mark the lock keyword as deprecated or something along these lines?

I assume when lock was introduced it made more sense because there was no "similarly simple" construct that would support the same logic, but now (unless I'm missing something obvious) it feels superfluous.

Is there any reason why we still use the lock keyword now that the language has using disposal blocks?

Yes. The lock statement clearly and concisely indicates the intent. It is a region of mutual exclusion. A using block does not indicate that.

Wouldn't it just be better to encourage developers to use the slightly more explicit using approach and mark the lock keyword as deprecated or something along these lines?

We do not think so. The language has supported locking since v1. It is a widely used construct. We see no reason to move away from that

I assume when lock was introduced it made more sense because there was no "similarly simple" construct that would support the same logic, but now (unless I'm missing something obvious) it feels superfluous.

The language supported both lock and using Since v1. We have never felt that the latter supplants the former.

Is there any reason why we still use the lock keyword now that the language has using disposal blocks?

Yes. The lock statement clearly and concisely indicates the intent. It is a region of mutual exclusion. A using block does not indicate that.

A using block indicates whatever the call indicates, IMHO.

using (lock.BeginMutualExclusionSection())
{
    ...
}

Would be even more explicit and obvious to me than a custom lock (lock) (inherently confusing and redundant) call.

But I guess this aspect is still subjective, so fair enough.

Wouldn't it just be better to encourage developers to use the slightly more explicit using approach and mark the lock keyword as deprecated or something along these lines?

We do not think so.

Fair enough.

The language has supported locking since v1.

Just because something is supported since X, doesn't mean it is inherently good.

It is a widely used construct.

Just because something is widely used doesn't mean it is good or that it should never be reconsidered.

We see no reason to move away from that

Again, fair enough... to me it's a shame, as it makes the language unnecessarily more complex/less orthogonal. It introduces a special case for something that clearly didn't need it.

I assume when lock was introduced it made more sense because there was no "similarly simple" construct that would support the same logic, but now (unless I'm missing something obvious) it feels superfluous.

The language supported both lock and using Since v1. We have never felt that the latter supplants the former.

Oh, my bad on this one. I thought the using block was introduced after C# 1. I stand corrected. In that case, looks like we just missed an opportunity to avoid introducing the lock keyword altogether. But I can see people used to that word in other languages could be put off at the time.

A using block indicates whatever the call indicates, IMHO.

That's more complex and less clear to me than just the lock statement. A bread and butter statement with clear and simple meaning since 1.0.

Just because something is supported since X, doesn't mean it is inherently good.

I disagree. This is a good part of hte langauge that people have been using for 25 years. Removing support is just adding friction and complexity. Foolish consistencies, hobgoblins, and all that.

looks like we just missed an opportunity to avoid introducing the lock keyword altogether.

We didn't miss an opportunity. The possibility of using using was well understood. After all, these are both about nicer patterns around try/finally. It's an intentional and continued view that lock has significant value for the language and moving away from that would be a net negative.

It introduces a special case for something that clearly didn't need it.

We didn't need using in the first place either. It was a special case for having a try/finally with a particular null+dispose check. But we still like it.

There are lots of special one-off lang features that are just sugar over other things. The vast majority of hte language is actually just that. Yes, we could go the route of other languages and whittle that away to a core kernel that you use for everything. But we actually view that as a negative. We think this features and sugar are the value and niceness that makes our language more pleasant and more desirable for many users.

It introduces a special case for something that clearly didn't need it.

I would also point out that lock does have additional semantics that are important, beyond what using offers. For example, in async methods you can't include await calls within the lock block because locks are thread-specific, but you can use using statements.

Isn't it just a matter of convention? Some will prefer a more technical focus "I want lock and await lock to use different verbs, because they will work differently and I want this to be explicit" while others will prefer a more functional focus "I want lock and await lock to use the same verb, because I want to protect a critical section of my code regardless of technical differences between sync and async version". Personally I dislike the await using used for locking a critical section. Even if the former convention should stay in place, I would prefer some await lockasync to await using, because this is about locking a critical section - using a critical section single-use-lock is not as straightforward.

var obj = new Lock();
using(obj)
{

}
lock(obj)
{
}

I don't quite understand the meaning of such a construction. Essentially, the compiler will just replace using with lock, but a lot of additional information is needed.

Navigation Menu

Repository navigation

[Proposal]: Lock statement pattern (VS 17.10, .NET 9) #7104

Lock statement pattern

Summary

Motivation

Detailed design

Example

Lock pattern and behavior details

SpinLock example

Drawbacks

Alternatives

Unresolved questions

Design meetings

78 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Proposal]: Lock statement pattern (VS 17.10, .NET 9) #7104

Description

Lock statement pattern

Summary

Motivation

Detailed design

Example

Lock pattern and behavior details

SpinLock example

Drawbacks

Alternatives

Unresolved questions

Design meetings

Activity

tfenise commented on Apr 7, 2023

vladd commented on Apr 8, 2023

HaloFour commented on Apr 8, 2023

HaloFour commented on Apr 8, 2023

svick commented on Apr 8, 2023

HaloFour commented on Apr 8, 2023

alexrp commented on Apr 8, 2023

svick commented on Apr 8, 2023

HaloFour commented on Apr 8, 2023

timcassell commented on Apr 9, 2023

TahirAhmadov commented on Apr 9, 2023

TahirAhmadov commented on Apr 10, 2023

jaredpar commented on Apr 10, 2023

78 remaining items

KennethHoff commented on Jul 19, 2024

julealgon commented on Jul 19, 2024

CyrusNajmabadi commented on Jul 19, 2024

julealgon commented on Jul 19, 2024

CyrusNajmabadi commented on Jul 19, 2024

CyrusNajmabadi commented on Jul 19, 2024

brantburnett commented on Jul 19, 2024

marchewek commented on Oct 8, 2024

Mr0N commented on Oct 13, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions