The failing tests don't appear to be due to a fluke, but I don't have time to dig into the underlying cause this week

Error: provider.random: Incompatible API version with plugin. Plugin version: 5, Core version: 4

This looked very familiar, so I looked into it. Something changed outside this repo and its tests.

It's related to this one-time release pipeline error: hashicorp/terraform-provider-random#663. As a result, the Registry API returns an empty list of protocol versions for terraform-provider-random v3.7.0-alpha1.

The last Protocol v4 version of the random provider is v2.3.1. I reconstructed the failing test locally and observed that Terraform 0.11.15 installs v3.7.0-alpha1.

To solve this: I'd prefer to backfill the right data in the Registry -- if possible -- rather than work around it here. My team maintains the random provider, and I will look into this.

While the linked issue was closed, we continue seeing the failures, such as

Error: provider.null: Incompatible API version with plugin. Plugin version: 5, Core version: 4

all linked to Terraform v0.11.15.

My theory is that this is connected to the latest version of the null provider being a pre-release, which then trips up the filename parsing logic in the old Terraform version somehow. I did not get the chance to confirm this theory though.

What is unclear to me is how exactly Terraform 0.11 so far managed to ignore all the 3.0.0 versions practically without provider constraints. I assume this is a detail hidden somewhere in the Registry API?

Some possible solutions:

1. Remove support for 0.11 altogether in terraform-exec, i.e. remove 0.11 from the test matrix here + document 0.12 as a minimum required version in Readme.md
2. serve different configuration to 0.11 tests with the appropriate legacy provider constraint, e.g. below

provider "null" {
  version = ">= 2"
}

@bbasata do you have any preferences? I'm inclined to (1), so I'll raise a PR that does that.

Since the tests are parametrised anyway, anyone can still run them on demand if they wish to, they just won't run on every PR anymore and so we may end up introducing changes that break terraform-exec's 0.11 compatibility. I don't think that is a problem since tfexec never explicitly stated compatibility for Terraform versions. We just assumed that people use the library in automation in contexts where older versions are far more likely to occur.

@bbasata do you have any preferences? I'm inclined to (1), so I'll raise a PR that does that.

I'm happy with this choice 😃

@dependabot rebase

Looks like this PR has been edited by someone other than Dependabot. That means Dependabot can't rebase it - sorry!

If you're happy for Dependabot to recreate it from scratch, overwriting any edits, you can request @dependabot recreate.

@dependabot rebase