Skip to content

Various fixes and improvements. Better error messages. Improve validate stacks and describe stacks commands#149

Merged
aknysh merged 33 commits intomasterfrom
update-docs-2
May 16, 2022
Merged

Various fixes and improvements. Better error messages. Improve validate stacks and describe stacks commands#149
aknysh merged 33 commits intomasterfrom
update-docs-2

Conversation

@aknysh
Copy link
Member

@aknysh aknysh commented May 14, 2022 •
edited
Loading

what

  • Various fixes and improvements
  • Better error handling and error messages
  • Improve validate stacks command
  • Improve describe stacks command

why

  • In this configuration
components:
  terraform:
    "test/test-component-override":
      # The `component` attribute specifies that `test/test-component-override` inherits from the `test/test-component` base component,
      # and points to the `test/test-component` Terraform component in the `components/terraform` folder
      component: "test/test-component"

don't require that the base component test/test-component has to be defined in the YAML config (even with an empty vars section) unless it's by itself has its own base YAML component. The base component test/test-component must exist in the components/terraform folder, but not in YAML config.

  • Fix this (allow naming YAML components not following the folder structure in the components folder)
components:
  terraform:
    # Note that you can name this YAML component anything that fits your needs.
    # For example, it can be named `test/test-component-override-3` or `test-test-component-override-3`
    # This affects only the `atmos` command you use to provision the component:
    # For example:
    # atmos terraform plan test/test-component-override-3 -s xxx (if you name the YAML component `test/test-component-override-3`)
    # atmos terraform plan test-test-component-override-3 -s xxx (if you name the YAML component `test-test-component-override-3`)
    # The terraform component (terraform code) it points to is defined by using the `metadata.component` attribute
    "test/test-component-override-3":
      vars: {}

  • atmos validate stacks command now honors ATMOS_BASE_PATH ENV var (and all other settings from atmos.yaml)

  • atmos validate stacks command now validates:

    • all YAML files for any YAML errors and inconsistencies
    • all imports (if they are configured correctly, have correct data types, and if all imports point to existing files)
    • schema - check if all sections in all YAML files are correctly configured and have the correct data types (Note that this is still not a full schema validation, it just check the sections like vars, settings, metadata, env at all levels: global, component type, component)
atmos validate stacks

Validating all YAML files in the 'examples/complete/stacks' folder and all subfolders

Invalid import in the file 'catalog/invalid-yaml-and-schema/invalid-import-1.yaml'
No matches found for the import 'globals/tenant1-globals-does-not-exist'

Invalid import in the file 'catalog/invalid-yaml-and-schema/invalid-import-2.yaml'
The file imports itself in 'catalog/invalid-yaml-and-schema/invalid-import-2'

Invalid import in the file 'catalog/invalid-yaml-and-schema/invalid-import-3.yaml'
The import '[globals/tenant1-globals-invalid]' is not a valid string

Invalid 'import' section in the file 'catalog/invalid-yaml-and-schema/invalid-import-4.yaml'
The 'import' section must be a list of strings

Invalid 'import' section in the file 'catalog/invalid-yaml-and-schema/invalid-import-5.yaml'
The 'import' section must be a list of strings

Invalid 'import' section in the file 'catalog/invalid-yaml-and-schema/invalid-import-6.yaml'
The 'import' section must be a list of strings

Invalid import in the file 'catalog/invalid-yaml-and-schema/invalid-import-7.yaml'
The import is an empty string

Invalid 'import' section in the file 'catalog/invalid-yaml-and-schema/invalid-import-8.yaml'
The 'import' section must be a list of strings

Invalid 'import' section in the file 'catalog/invalid-yaml-and-schema/invalid-import-9.yaml'
The 'import' section must be a list of strings

Invalid 'vars' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-1'

Invalid 'components.helmfile.bad-component' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-10'

Invalid 'components.terraform.vpc.vars' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-11'

Invalid 'components.helmfile.echo-server.vars' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-12'

Invalid 'components.terraform.vpc.metadata' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-13'

Invalid 'components.terraform.vpc.settings' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-14'

Invalid 'components.terraform.vpc.settings.spacelift' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-15'

Invalid 'terraform.vars' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-2'

Invalid 'helmfile.vars' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-3'

Invalid 'components.terraform' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-4'

Invalid 'components.helmfile' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-5'

Invalid 'components' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-6'

Invalid 'settings' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-7'

Invalid 'env' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-8'

Invalid 'components.terraform.bad-component' section in the file 'catalog/invalid-yaml-and-schema/invalid-schema-9'

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-1.yaml'
yaml: line 15: found unknown directive name

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-2.yaml'
yaml: line 16: could not find expected ':'

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-3.yaml'
yaml: line 13: did not find expected key

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-4.yaml'
yaml: block sequence entries are not allowed in this context

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-5.yaml'
yaml: mapping values are not allowed in this context

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-6.yaml'
yaml: line 2: block sequence entries are not allowed in this context

Invalid YAML file 'catalog/invalid-yaml-and-schema/invalid-yaml-7.yaml'
yaml: line 4: could not find expected ':'

atmos describe stacks --components=test/test-component --sections=none

tenant1/ue2/dev:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}
tenant1/ue2/prod:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}
tenant1/ue2/staging:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}
tenant1/ue2/test1:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}
tenant2/ue2/dev:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}
tenant2/ue2/prod:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}
tenant2/ue2/staging:
  components:
    terraform:
      test/test-component: {}
      test/test-component-override: {}
      test/test-component-override-2: {}
      test/test-component-override-3: {}

NOTE: test/test-component-override-3, test/test-component-override-2, test/test-component-override - all have test/test-component as the base Terraform component (via component or metadata.component attributes)

related

@aknysh aknysh added the patch A minor, backward compatible change label May 14, 2022
@aknysh aknysh self-assigned this May 14, 2022
@aknysh aknysh requested review from a team as code owners May 14, 2022 20:20
Nuru
Nuru previously approved these changes May 15, 2022
View reviewed changes
@aknysh aknysh changed the title Various fixes and improvements. Better error messages. Improve validate stacks command Various fixes and improvements. Better error messages. Improve validate stacks and describe stacks commands May 16, 2022
@aknysh aknysh requested a review from Nuru May 16, 2022 05:50
Nuru
Nuru previously approved these changes May 16, 2022
View reviewed changes
Copy link
Contributor

@Nuru Nuru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, with one style comment/suggestion.

@aknysh aknysh requested a review from Nuru May 16, 2022 16:25
@aknysh aknysh merged commit 6d53c9c into master May 16, 2022
@aknysh aknysh deleted the update-docs-2 branch May 16, 2022 16:53
@aknysh aknysh mentioned this pull request May 16, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Nuru Nuru Nuru approved these changes

@osterman osterman Awaiting requested review from osterman

@mcalhoun mcalhoun Awaiting requested review from mcalhoun

@nitrocode nitrocode Awaiting requested review from nitrocode

Assignees

@aknysh aknysh

Labels

patch A minor, backward compatible change

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Include derived components when filtering on a base component Filter out empty stacks from describe stacks

2 participants

@aknysh @Nuru