This happens because when you install the Flutter extension, it first installs the Dart extension. A few months ago VS Code started activating extensions with workspaceContains automatically at install:
microsoft/vscode@7304f38
This means the Dart extension activates, and since you don't have a Flutter project open it tries to get you to provide a Dart SDK.
I've opened microsoft/vscode#75284 but we may want to add a workaround in the meantime.