Opening this as requested by @astearns

In #8738 we resolved to stop hoisting interleaved declarations and introduce an @nest rule that means "exactly the same thing as the parent" instead of wrapping in :is(), which is how interleaved declarations will be represented in the CSS OM. Since we were not able to get consensus on the specifics, but we had consensus that any solution along these lines is better than the status quo, we agreed that Tab would spec whatever (commit here), and we'd discuss the details later, since fixing the specifics is more web compatible than changing the current behavior after even longer.

The issues around which we could not reach consensus were:

1. If authors have no reason to write @nest and it’s only introduced to represent interleaved declarations and rules, should they even be able to?
2. If @nest rules are magically added around interleaved declarations should they also be removed during serialization?
   1. If they are removed during serialization, does this happen always, or only when part of a larger rule (e.g. as part of .cssText)?
3. Do we even need a new @nest rule? What if we simply use the existing CSSStyleDeclaration object to represent interleaved rules? (proposed by @mdubet)
4. How does setProperty() work if we go with one of the designs that involve more magic?
5. What happens when a rule is removed and thus two sets of interleaved declarations become adjacent?
   1. Similar issue not brought up in the call: what about the case when these rules are first? Should they be merged with rule.style?

These are not orthogonal decisions: it seems clear that if @nest serializes to include an actual @nest {} rule, that @nest rule needs to also be valid author code. So essentially there are three possible designs:

1. Magic-minimizing @nest (proposed by @tabatkins, supported by @emilio @andruud @Loirooriol): The rule is automatically added around interleaved declarations, but there is no more magic besides that.
2. Author-exposure minimizing @nest (proposed by @LeaVerou, supported by @fantasai @astearns): The rule becomes a CSS OM detail, with no corresponding CSS syntax, and is removed on serialization (regardless of how serialization happens).
3. No @nest, just CSSStyleDeclaration in the CSSOM (proposed by @mdubet, supported by @LeaVerou @fantasai).
   • Criticism: That means .cssRules will also return non-rules? Would .insertRule() also accept CSSStyleDeclaration?

For 2 and 3, there are also design variations based on the answer to 4 and 5 above.

My position:

• As a design principle, I don't think CSS should have author-facing syntax that provides no benefit to authors. Either we should come up with actual use cases for @nest or make it an internal detail. That said, there could conceivably be use cases for it. E.g. one of the problems with IACVT is that fallbacks are thrown away by the time the declaration becomes invalid. What if this was a way to preserve fallbacks?
• Despite proposing 2, I actually now support 3 as I realized this version of @nest is functionally equivalent to CSSStyleDeclaration and we should not be introducing new interfaces with philosophical purity being the only motivation (i.e. "but otherwise .cssRules would be returning a non-rule?!?")
• Some voices are arguing for less magic as a goal in itself. I don't think avoiding magic at all costs should be a goal; the right magic can make a huge usability difference (at the cost of implementation complexity). It’s the wrong kind of magic that becomes problematic: magic that rarely predicts user intent well, and especially when they cannot opt-out of it. Neither seems to be the case here, at least if we design this well.
• I think authors should be able to interact with the CSS OM predictably without having to care whether there are interleaved declarations or not. This means that setProperty() on the base rule should just work, without them having to do tree walking to find the last nested rule. Like, setProperty() is an incredibly common operation, and an API where calls to setProperty() have no effect are exactly the kind of dreadful APIs that make utility libraries proliferate.
• I strongly disagree with Tab that reading and modifying the CSS OM only affects about a hundred developers in the entire world (!). Reading and modifying the CSS OM is at the core of a ton of very widespread libraries (e.g. jQuery, D3, etc). Even if most authors do not use it directly, I assure you almost every developer reads and modifies the CSS OM regularly, it's just done through several layers of abstractions.

So I would propose a design that would minimize author exposure to all of this, and would just try to do what's reasonable when reading and modifying the CSS OM:

• .cssRules would contain CSSStyleDeclaration objects with interleaved declarations
• insertRule() would accept CSSStyleDeclaration objects
• No nonsensical or redundant structures:
  • Adjacent CSSStyleDeclaration objects would be merged.
  • A CSSStyleDeclaration object cannot be at the start of cssRules. Inserting one should simply merge the declarations with rule.style.

Should rule.style be magic?

One thing I'm ambivalent about is whether rule.style should be magic too.
This would mean:

• rule.style returns the union of all interleaved declarations (we can introduce another property to only get the non-interleaved ones)
• rule.style.setProperty() (and accessors) adds to the last interleaved CSSStyleDeclaration (if any are present). The third argument is turned into a dictionary with a priority property and another property (name TBD) to reverse this behavior and actually add it to the first block of declarations.

Pros & Cons:

• 👍🏼 Max compatibility with existing code
• 👍🏼 Fewer footguns for authors (minimize bugs if authors forget to handle the case of interleaved declarations)
• 👍🏼 More sensible API for when inline styles also support nesting.
• 👀 Would this work or would it end up not predicting author intent well? What cases am I missing?

If we decide to avoid magic here, we can make the API more palatable by:

• Introducing a new rule property that returns a CSSStyleDeclaration for the union of rule.style and interleaved declarations
• Introducing a rule.setProperty() method that would add the property at the end of the last interleaved declaration. rule.style.setProperty() would continue to do what it currently does. Same for removeProperty(), getPropertyValue() etc.