Skip to content

!terraform.state not using Atmos authentication when planning/applying the entire stack #2081

@leoagueci

Description

@leoagueci

Describe the Bug

When I run a plan for the entire stack

atmos terraform plan --all -s test

I get an error on reading the backend on S3

WARN  Failed to read Terraform state after all retries exhausted file=test/vpc.terraform.tfstate bucket=terraform-tfstate attempts=3 error_code=unknown error="operation error S3: GetObject, exceeded maximum number of attempts, 3, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send failed, Get \"http://169.254.169.254/latest/meta-data/iam/security-credentials/\": dial tcp 169.254.169.254:80: connect: connection refused"

   Error 

   Error: failed to read Terraform state for component  vpc  in stack  test 
   in YAML function:  !terraform.state vpc ".aws_region // ""eu-west-1""" 
   failed to get object from S3: operation error S3: GetObject, exceeded maximum number of attempts, 3, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send failed, Get "http://169.254.169.254/latest/meta-data/iam/security-credentials/": dial tcp 169.254.169.254:80: connect: connection refused

No issues instead when planning a single components, for example

atmos terraform plan vpn -s test

Expected Behavior

!terraform.state uses Atmos authentication even when planning/applying the entire stack

Steps to Reproduce

  1. Use the following configuration in atmos.yaml
# Auth with AWS SSO
auth:
  providers:
    sso:
      kind: aws/iam-identity-center
      region: eu-central-1
      start_url: <sso-url>
  identities:
    platformops:
      kind: aws/permission-set
      default: true
      via:
        provider: sso
      principal:
        name: admin
        account:
          name: test-account

# Stacks configuration  
stacks:
  # Path to stack configurations
  base_path: "stacks"
  # Include patterns for stack files
  included_paths:
    - "deploy/**/*"
  # Exclude patterns
  excluded_paths:
    - "**/_defaults.yaml"
  # Naming pattern for stack files
  name_template: "{{ .vars.vpc }}"
  1. Create a stack in deploy/stack.yaml
terraform:
  backend_type: s3
  backend:
    s3:
      encrypt: true
      key: "{{ .vars.vpc }}/{{ .atmos_component }}.terraform.tfstate"
      bucket: "terraform-tfstate"
      region: "eu-central-1"
      use_lockfile: true
  providers:
    aws:
      region: "eu-central-1"
  
vars:
  vpc: "test"
  aws_region: "eu-central-1"
  environment: "testing"

components:
  terraform:
    vpc:
      metadata:
        component: vpc
      vars:
        name: "{{ .vars.vpc }}"
        cidr: 10.0.0.0/16
    vpn:
      metadata:
        component: vpn
      settings:
        depends_on:
          1:
            component: vpc
      vars:
        region: !terraform.state vpc ".aws_region // ""eu-west-1"""

Screenshots

No response

Environment

No response

Additional Context

Logs when running single component

DEBU  Found component 'vpc' in the stack 'test' in the stack manifest 'deploy/stack'
DEBU  Resolved component path type=terraform component=v1 resolved_path=/test/components/terraform/vpc base_path=/test/components/terraform env_override=false
 DEBU  Using Atmos auth context for AWS SDK profile=platformops credentials=/root/.config/atmos/aws/sso/credentials config=/root/.config/atmos/aws/sso/config

Logs when running the entire stack

DEBU  Found component 'vpc' in the stack 'test' in the stack manifest 'deploy/stack'
 DEBU  Resolved component path type=terraform component=v1 resolved_path=/test/components/terraform/vpc base_path=/test/components/terraform env_override=false
 DEBU  Using standard AWS SDK credential resolution (no auth context provided)

Metadata

Metadata

Assignees

Labels

bug🐛 An issue with the system

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions