Does this issue occur when all extensions are disabled?: Yes

• VS Code Version: 1.61.2
• OS Version: macOS Big Sur 11.6

I've recently been trying to optimize an extension that decorates [[...]] wikilinks in markdown documents. Every time I switched between editors a very noticeable flickering happened because of the syntax rehighlighting of the editor and, not so shortly after that, a delayed repainting of decorations. I checked other similar extensions and the pattern was always the same. It's even suggested in this "official" example: https://github.com/microsoft/vscode-extension-samples/blob/main/decorator-sample/src/extension.ts. There are two problems:

1. The suggested delay of 500ms might be reasonable for in-editor changes but for editor switching it produces this: Long delay in link highlighting foambubble/foam#805 (comment).
2. Even with no throttling at all and with precalculation and caching of document decorations (that I roughly implemented here perf: Avoid recomputing decorations svsool/memo#479) there is the same repainting glitch, although it's less obvious.

The second point is likely the hard one because it might be due to a more structural limit of the design. Why is VSCode triggering document-wise updates of decorations in unchanged documents? From the perspective of the extension author, is there any way to do that update incrementally, some knowledge of damaged/exposed chunks?

N.B. in the above cases extending the syntax was also problematic, for example because of the need to distinctively decorate valid and invalid links. Even if the syntax extension approach were effective here, the point still stands that decorations seem ill-suited for anything but small documents and even in those documents they produce visual artifacts on editor switching.