Extracted from #327

At the moment, the Polkadot PR is expected to fail until the previously-explained lockfile update happens, but then it is expected to work after the lockfile update happens - which, in practice, is an iffy expectation because we're not running the whole Polkadot CI ahead of time.

This whole workflow could be enhanced with the following approaches

Alternative 1: Predictable References

1. Create a Substrate PR which has breaking changes for Polkadot
2. Set up Git ref for the Substrate PR (a "Predictable Reference") such as pr-123

• Note: this is already provided by GitHub as refs/pull/123/head as refs/pull/123/merge

3. Create a companion PR on Polkadot

4. When bot merge is called, the bot will update the Polkadot PR's Substrate reference in Cargo.toml and Cargo.lock via commit, e.g.

   -sp-io = { git = "https://github.com/paritytech/substrate", branch = "master" }
   +sp-io = { git = "https://github.com/paritytech/substrate", rev = "refs/pull/123/merge" }

   By doing the above we expect that the Polkadot PR's CI will pass because it would be already referencing the Substrate PR, which is supposed to solve its breaking changes. Doing so will viabilize checking that both sides are passing before going through with the merges - at the moment it's not viable because the Polkadot PR is expected to be failing at first - therefore we'll overcome "Risk of desynchronization for merge failures" because all CIs should be green before the merge.

5. Wait for all CIs to pass

6. Merge all pull requests simultaneously

For the record: when @joao-paulo-parity explained this to @bkchr on Matrix DMs, the idea of merging the rev = "refs/pull/123/merge" to master was not well received, which motivated the design of Alternative 2.

Alternative 2: Predictable References, but don't commit the updated references

This approach would work similar to Alternative 1 except that we don't end up merging the changes to Cargo.lock and Cargo.toml (related to step 4 of Alternative 1) into master. In this alternative the reference changes of rev = "refs/pull/123/merge" to branch = "master" are not committed to the companion branches, instead they are tested on temporary GitLab branches.

Explained in https://github.com/paritytech/ci_cd/issues/234#issuecomment-1160141699.