What would you like to be added?

I would like to discuss how etcd v3.6 should behave with regards to v2 store API.

Background

v2 API was deprecated in etcd v3.4, but could be still used as long as you provided --enable-v2 flag. Didn't change in v3.5, however for v3.6 we are planning for total removal. Expected behavior is that when upgrading to v3.6, etcd will panic if there is any v2 data still left. More in #12913

User can do two things:

• Halt the upgrade and reconsider what to do with v2 state.
• Run etcd with --v2-deprecation=write-only-drop-data which is expected to delete the local v2 data.

Problem

What happens if in HA clusters during upgrade/downgrade (v3.6 supports downgrade to v3.5), if user forgets that etcd v3.5 member still uses --enable-v2 and introduces a v2 change to cluster. This is worrying as a single member could take down whole cluster. Fixing this would require to reconfigure whole cluster to run with --v2-deprecation=write-only-drop-data

Options:

• Check the snapshot and WAL for v2 data only on bootstrap, skip it later. It will lead to inconsistency on v2 state.
• Have v3.6 members reject v2 proposals. Not sure this is possible, we as v3.5 can still become a leader. I would be careful about changing logic for leader eligibility.
• Do nothing

Options rejected:

• Downgrades require enabling --v2-deprecation=write-only-drop-data. Doesn't solve upgrade case.
• Document that users should double and triple check that they are not using --enable-v2. Doesn't seem user friendly.

Why is this needed?

Want to make sure this is properly discussed, understood and documented.