Skip to content

DEV-3110: Improve Workflow DX#1229

Merged
milldr merged 47 commits intomainfrom
DEV-3110/workflow-failure
Jun 4, 2025
Merged

DEV-3110: Improve Workflow DX#1229
milldr merged 47 commits intomainfrom
DEV-3110/workflow-failure

Conversation

@milldr
Copy link
Member

@milldr milldr commented May 2, 2025

what

  • Moved workflow utils and tests into the pkg/workflow directory This wont work because we depend on shell utils in internal/exec. Reverted
  • Update all log messages for workflow_utils to use the charmbracelet log handling
  • Add TUI message when workflow kicks off an atmos command to report the command being executed
$ atmos workflow pass --file test
Executing command: `atmos terraform plan mock -s nonprod`     <---- this is new
Initializing the backend...
Initializing provider plugins...

Terraform has been successfully initialized!
Switched to workspace "nonprod".

Changes to Outputs:
  + bar   = "bar nonprod override"
  + baz   = "baz catalog default"
  + foo   = "foo nonprod override"
  + stage = "nonprod"

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
  • Add a pattern for all workflow errors:
# Workflow Error

<static error message>

## Explanation

<what and why here>

why

  • For all other commands, we already have the convention of using a directory for a utilities for a command, such as pkg/workflow. Plus we already had that directory, but it only had incomplete tests
  • We should always see the command being executed before it runs and see the command that fails when it fails.
  • Clear error messages on workflows to explain the true issue in a consistent format

references

screenshots

CleanShot 2025-05-31 at 19 29 24@2x

CleanShot 2025-05-31 at 19 29 40@2x

Summary by CodeRabbit

  • New Features

    • Introduced comprehensive workflow testing scenarios and fixtures for CLI workflows, including production and non-production deployments, catalog imports, and multiple workflow step types.
    • Added detailed error handling and user-friendly, markdown-formatted error messages for workflow execution failures, invalid configurations, and missing files.
    • Enhanced workflow step naming by automatically generating names for unnamed steps.
  • Bug Fixes

    • Improved error reporting for invalid workflow manifests, missing workflows, and unsupported step types, providing clearer guidance to users.
  • Tests

    • Added extensive test cases and snapshot outputs covering successful and failing workflow executions, invalid step types, missing steps, and shell command errors.
    • Refactored and expanded test suites to cover workflow execution logic, step naming, and workflow description features.
  • Chores

    • Updated configuration and fixture files to support new workflow testing scenarios and error cases.

@milldr milldr requested a review from a team as a code owner May 2, 2025 16:35
@github-actions github-actions bot added the size/m label May 2, 2025
@milldr milldr added minor New features that do not break anything and removed size/m labels May 2, 2025
@github-actions github-actions bot added the size/m label May 2, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented May 2, 2025

📝 Walkthrough

Walkthrough

This change overhauls Atmos workflow error handling and test coverage. Workflow step failure errors now display detailed information about what failed, including the specific command or step. Tests and test fixtures were expanded to cover various workflow error scenarios, and markdown-formatted error messages were introduced for clarity.

Changes

Files/Groups Change Summary
internal/exec/workflow_utils.go, cmd/workflow.go Refactored error handling for workflows: added structured error variables, markdown error messages, and detailed failure info.
internal/exec/workflow.go Improved error reporting for workflow selection and manifest validation; uses new error handling approach.
pkg/config/const.go Added AtmosCommand constant.
internal/exec/describe_workflows_test.go Added new tests for describing workflows, including environment setup helpers.
internal/exec/workflow_test.go, pkg/workflow/workflow_test.go Replaced and expanded workflow execution tests; added tests for step name generation and error scenarios.
tests/fixtures/scenarios/workflows/** Added new workflow test fixtures, manifests, and configuration files.
tests/test-cases/workflows.yaml Added comprehensive workflow CLI test cases for success and failure scenarios.
tests/snapshots/TestCLICommands_atmos_workflow_.stderr.golden, tests/snapshots/TestCLICommands_atmos_workflow_.stdout.golden Added new and updated CLI workflow test output snapshots for error and success cases.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant Workflow Engine
    participant TUI

    User->>CLI: Run workflow command
    CLI->>Workflow Engine: ExecuteWorkflow(...)
    Workflow Engine->>TUI: Print step execution info
    alt Step fails
        Workflow Engine->>TUI: Print markdown error with step name and failure details
        Workflow Engine->>CLI: Return structured error
        CLI->>User: Exit with error (no duplicate error print)
    else Step succeeds
        Workflow Engine->>TUI: Print step success info
        Workflow Engine->>CLI: Return success
        CLI->>User: Exit 0
    end
Loading

Assessment against linked issues

Objective Addressed Explanation
Show failure details in Atmos workflow step errors (DEV-3110)

Suggested reviewers

  • osterman

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 49348e8 and 6bc543b.

📒 Files selected for processing (4)
  • internal/exec/workflow.go (4 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_file_not_found.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_manifest.stderr.golden (1 hunks)
  • tests/test-cases/workflows.yaml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_manifest.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_file_not_found.stderr.golden
  • internal/exec/workflow.go
  • tests/test-cases/workflows.yaml
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary or @auto-summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @auto-title anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (4)
tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (2)

1-7: Consider adding explicit step names.
Adding a name field to each step in the "pass" workflow can make snapshots and error messages more descriptive:

 steps:
-  - command: terraform deploy mock -s nonprod
+  - name: step1
+    command: terraform deploy mock -s nonprod
-  - command: terraform deploy mock -s prod
+  - name: step2
+    command: terraform deploy mock -s prod

9-15: Consider adding explicit step names.
Similarly, in the "fail" workflow, name keys for each step would help map failures to human-friendly identifiers rather than relying on auto-generated ones.

tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden (1)

3-5: Trim trailing whitespace for robustness.
Lines containing # Error and the blank separators include extra spaces which may cause snapshot mismatches. Please remove trailing spaces so the golden file matches the actual stderr output exactly.

Also applies to: 9-11

tests/fixtures/scenarios/workflows/atmos.yaml (1)

10-10: Remove trailing space.

There's a trailing space on this line that should be removed for code cleanliness.

-  
+
🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 10-10: trailing spaces

(trailing-spaces)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c87cd51 and 4365009.

📒 Files selected for processing (12)
  • internal/exec/workflow_utils.go (5 hunks)
  • tests/fixtures/scenarios/workflows/.gitignore (1 hunks)
  • tests/fixtures/scenarios/workflows/atmos.yaml (1 hunks)
  • tests/fixtures/scenarios/workflows/stacks/catalog/mock.yaml (1 hunks)
  • tests/fixtures/scenarios/workflows/stacks/deploy/nonprod.yaml (1 hunks)
  • tests/fixtures/scenarios/workflows/stacks/deploy/prod.yaml (1 hunks)
  • tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stdout.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_success.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_success.stdout.golden (1 hunks)
  • tests/test-cases/workflows.yaml (1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/workflows/atmos.yaml

[error] 10-10: trailing spaces

(trailing-spaces)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (17)
tests/fixtures/scenarios/workflows/.gitignore (1)

1-10: Ignore Terraform artifacts in workflow fixtures

The ignore patterns correctly exclude Terraform state, lock, plan, and cache files for this test scenario, preventing transient files from polluting the repository.

tests/snapshots/TestCLICommands_atmos_workflow_success.stderr.golden (1)

1-1: Snapshot updated to reflect Info-level workflow start log

The Info Executing the workflow line matches the new logging level and format from workflow_utils.go. This aligns with the PR objectives.

tests/fixtures/scenarios/workflows/stacks/deploy/nonprod.yaml (1)

1-15: New nonprod stack manifest for workflow tests

The manifest defines stage: nonprod, imports the mock catalog, and correctly overrides foo and bar. Schema header is valid.

tests/fixtures/scenarios/workflows/stacks/catalog/mock.yaml (1)

1-10: Catalog component default variables defined

The mock catalog provides default values for foo, bar, and baz under the correct schema. This supports environment-specific overrides downstream.

tests/fixtures/scenarios/workflows/stacks/deploy/prod.yaml (1)

1-15: New prod stack manifest for workflow tests

The prod manifest mirrors nonprod.yaml with stage: prod and overrides for foo and bar. Appears consistent and ready for testing.

tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (1)

1-16: YAML test fixture is valid.
The workflows "pass" and "fail" are clearly defined with the expected Terraform commands and align with the new test scenarios.

tests/snapshots/TestCLICommands_atmos_workflow_failure.stdout.golden (1)

1-32: Snapshot looks accurate.
ANSI escape codes and Terraform output align with the expected failure run. Ensure there’s no extraneous trailing whitespace and that the leading blank line is intentional.

tests/test-cases/workflows.yaml (1)

18-36: Failure test case is correctly defined.
The "atmos workflow failure" block properly specifies the expected stderr patterns and exit code for the failing scenario.

tests/snapshots/TestCLICommands_atmos_workflow_success.stdout.golden (1)

1-63: Success snapshot is comprehensive.
The output matches the expected Terraform initialization, planning, warnings, and outputs for both "nonprod" and "prod" workspaces.

tests/fixtures/scenarios/workflows/atmos.yaml (1)

22-24: Good log level choice for improved visibility.

Setting the log level to Info aligns perfectly with the PR objectives of making workflow execution more visible. This ensures that workflow start and failure messages will be properly displayed.

internal/exec/workflow_utils.go (7)

13-13: Good addition of the charmbracelet log package.

This change aligns with the PR objective of standardizing logging across workflows.


36-40: Good implementation of conditional logging.

The approach of using different log levels based on dry run status is clean and effective. This allows for appropriate visibility during both actual execution and dry runs.


45-45: Improved workflow visibility through Info level logging.

Changing this log from Debug to Info level ensures that workflow execution is always visible to users, fulfilling a key objective of the PR.


69-69: Better structured logging for step execution.

Using the structured logging approach with key-value pairs provides clearer context about which step is being executed.


102-102: Clear stack identification in logs.

The structured logging for stack information makes it easier to track which stack is being used for each step.


111-117: Comprehensive error logging with structured fields.

The detailed debug log with all relevant workflow information will help with troubleshooting failures. Good use of structured logging here.


119-119: Improved error message clarity.

The error message now more clearly indicates both the step name and the command that failed, which will help users understand what went wrong.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
internal/exec/workflow_utils.go (2)

119-123: Create a constant for 'atmos' string literal

The string literal "atmos" appears multiple times in the codebase. Consider creating a named constant for it.

+const atmosCommand = "atmos"

// Later in the code
 if commandType == "atmos" {
-    failedCmd = fmt.Sprintf("atmos %s", command)
+    failedCmd = fmt.Sprintf("%s %s", atmosCommand, command)
 }
🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 121-121:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it


135-135: Use wrapped static errors instead of dynamic errors

Instead of using dynamic errors with fmt.Errorf, consider using wrapped static errors for better error handling.

+var (
+    ErrWorkflowStepFailed = errors.New("workflow step failed")
+)

// Later in the code
-return fmt.Errorf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg)
+return errors.Wrap(ErrWorkflowStepFailed, fmt.Sprintf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg))
🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 135-135:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg)"

🧹 Nitpick comments (1)
tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (1)

24-24: Add newline at end of file

YAML files should end with a newline character to comply with standard formatting practices.

 echo "This should fail"
 exit 1
 type: shell
+
🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 24-24: no new line character at the end of file

(new-line-at-end-of-file)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4365009 and 75a1977.

📒 Files selected for processing (6)
  • internal/exec/workflow_utils.go (6 hunks)
  • tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stdout.golden (1 hunks)
  • tests/test-cases/workflows.yaml (1 hunks)
✅ Files skipped from review due to trivial changes (2)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stdout.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden
🧰 Additional context used
🪛 GitHub Check: golangci-lint
internal/exec/workflow_utils.go

[failure] 121-121:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it


[failure] 135-135:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg)"

🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml

[error] 24-24: no new line character at the end of file

(new-line-at-end-of-file)

⏰ Context from checks skipped due to timeout of 90000ms (3)
  • GitHub Check: Acceptance Tests (macos-latest, macos)
  • GitHub Check: Acceptance Tests (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (9)
tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (1)

1-24: Well-structured test workflows for different scenarios

This test fixture provides a comprehensive set of test workflows covering success and failure cases. The three workflows (pass, fail, and shell-failure) will effectively test the enhanced logging improvements in the workflow execution.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 24-24: no new line character at the end of file

(new-line-at-end-of-file)

internal/exec/workflow_utils.go (5)

13-13: Good switch to structured logging package

Adopting the charmbracelet/log package for structured logging aligns with the PR objectives to standardize log handling across workflows.


36-40: Appropriate log level selection based on dry run

Setting the log function based on the dryRun flag ensures commands are always visible before execution when needed, which directly addresses the PR's motivation.


45-45: Improved workflow execution logging

Changing from debug to info level for workflow execution messages ensures better visibility, as specified in the PR objectives.


69-69: Consistent structured logging for workflow steps

Using the conditional log function with structured fields improves the clarity and consistency of log messages during workflow execution.

Also applies to: 102-102


111-117: Enhanced error context in debug logs

The structured debug log provides comprehensive context about workflow failures, which will be valuable for troubleshooting.

tests/test-cases/workflows.yaml (3)

19-37: Comprehensive test for workflow failure scenario

This test effectively verifies the improved error messaging for workflow failures, checking that the failing command is clearly displayed along with instructions to resume from the failed step.


39-59: Well-defined shell command failure test

This test ensures proper error handling for shell command failures, which completes the test coverage for the different types of workflow failure scenarios.


1-17:

Details

✅ Verification successful

Complete tests for workflow success scenario

The test for successful workflow execution is well-defined, but it's missing an stderr snapshot file.

Since the snapshot: true flag is set, both stdout and stderr snapshot files are expected. Please ensure you have a TestCLICommands_atmos_workflow_success.stderr.golden file in your snapshot directory to avoid CI failures.


🏁 Script executed:

#!/bin/bash
# Check if the stderr snapshot file exists
find tests/snapshots -name "TestCLICommands_atmos_workflow_success.stderr.golden"

Length of output: 150


All required snapshot files are present

The stderr snapshot file TestCLICommands_atmos_workflow_success.stderr.golden was found under tests/snapshots/. No further action is needed.

coderabbitai[bot]
coderabbitai bot previously approved these changes May 2, 2025
@codecov
Copy link

codecov bot commented May 2, 2025

Codecov Report

Attention: Patch coverage is 56.62651% with 36 lines in your changes missing coverage. Please review.

Project coverage is 49.93%. Comparing base (c86b34f) to head (03b1603).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
internal/exec/workflow.go 14.28% 23 Missing and 1 partial ⚠️
internal/exec/workflow_utils.go 86.00% 7 Missing ⚠️
cmd/workflow.go 0.00% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1229      +/-   ##
==========================================
+ Coverage   49.60%   49.93%   +0.33%     
==========================================
  Files         235      235              
  Lines       25568    25608      +40     
==========================================
+ Hits        12683    12788     +105     
+ Misses      11242    11166      -76     
- Partials     1643     1654      +11     
Flag Coverage Δ
unittests 49.93% <56.62%> (+0.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
internal/exec/workflow_utils.go (2)

113-117: Create a constant for the "atmos" string literal

The string literal "atmos" appears multiple times in the codebase. Creating a named constant would improve maintainability.

+const atmosCmd = "atmos"

 failedMsg := fmt.Sprintf("\nStep '%s' failed:\n", step.Name)
 failedCmd := command
 if commandType == "atmos" {
-  failedCmd = fmt.Sprintf("atmos %s", command)
+  failedCmd = fmt.Sprintf("%s %s", atmosCmd, command)
 }

Then update other occurrences at lines 66, 73, and 99 with this constant.

🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 115-115:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it


129-129: Consider using wrapped static errors instead of dynamic errors

The current error formatting uses dynamic errors with fmt.Errorf. Consider using wrapped static errors for better error handling and consistency.

+var (
+  ErrWorkflowStepFailed = errors.New("workflow step failed")
+)

-return fmt.Errorf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg)
+return errors.Wrap(ErrWorkflowStepFailed, fmt.Sprintf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg))
🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 129-129:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg)"

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7c3fa76 and 0154f45.

📒 Files selected for processing (1)
  • internal/exec/workflow_utils.go (6 hunks)
🧰 Additional context used
🪛 GitHub Check: golangci-lint
internal/exec/workflow_utils.go

[failure] 115-115:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it


[failure] 129-129:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("%s\n%s\n%s", failedMsg, failedCmd, resumeMsg)"

⏰ Context from checks skipped due to timeout of 90000ms (3)
  • GitHub Check: Build (macos-latest, macos)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (4)
internal/exec/workflow_utils.go (4)

13-13: Good update to use structured logging package

The import of the charmbracelet log package aligns with the PR objective to standardize logging across workflows.


39-39: Improved visibility with Info level logging

Elevating the workflow execution notification from Debug to Info level ensures users can always see when a workflow starts, enhancing visibility as intended in the PR objectives.


63-63: Well-structured debug logging for steps

The structured logging with fields for step index and command provides good context for debugging.


96-96: Clear stack context logging

Stack information is properly logged with a structured field providing helpful context.

coderabbitai[bot]
coderabbitai bot previously approved these changes May 6, 2025
coderabbitai[bot]
coderabbitai bot previously approved these changes May 22, 2025
coderabbitai[bot]
coderabbitai bot previously approved these changes May 31, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
internal/exec/workflow_utils.go (2)

50-50: Log level should be Info instead of Debug.

Based on the PR objectives, workflow start messages should be changed from Debug to Info level to ensure commands are always visible before execution.

Apply this change:

-l.Debug(fmt.Sprintf("Executing the workflow: workflow=%s file=%s", workflow, workflowPath))
+l.Info("Executing workflow", "workflow", workflow, "file", workflowPath)

127-128: Log level should be Info instead of Debug.

The past review comments specifically mentioned that workflow failure messages should be at Info level, not Debug. This ensures failures are visible without debug mode.

Apply this change:

-l.Debug(fmt.Sprintf("Workflow failed: workflow=%s path=%s step=%s command=%s error=%v",
-    workflow, workflowPath, step.Name, command, err))
+l.Info("Workflow failed", "workflow", workflow, "path", workflowPath, "step", step.Name, "command", command, "error", err)
🧹 Nitpick comments (3)
internal/exec/workflow_utils.go (2)

80-80: Consider using Info level for step execution.

Since the PR objectives emphasize visibility of commands before execution, step execution logs might benefit from Info level rather than Debug.

-l.Debug(fmt.Sprintf("Executing workflow step: step=%d command=%s", stepIdx, command))
+l.Info("Executing workflow step", "step", stepIdx, "command", command)

141-141: Add constant for repeated "atmos" string.

Static analysis correctly identifies that "atmos" appears multiple times and should be a named constant.

Add this constant at the package level:

+const AtmosCommand = "atmos"

Then use it in the code:

-failedCmd = "atmos " + command
+failedCmd = AtmosCommand + " " + command
🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 141-141:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it

tests/test-cases/workflows.yaml (1)

120-120: Add missing newline at end of file.

Static analysis correctly identifies the missing newline character at the end of the file.

Add a newline at the end:

       exit_code: 1
+
🧰 Tools
🪛 YAMLlint (1.37.1)

[error] 120-120: no new line character at the end of file

(new-line-at-end-of-file)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 46157c4 and 6395e2d.

📒 Files selected for processing (8)
  • internal/exec/workflow_utils.go (4 hunks)
  • tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_from_step.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_step_type.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_no_steps.stderr.golden (1 hunks)
  • tests/test-cases/workflows.yaml (1 hunks)
✅ Files skipped from review due to trivial changes (3)
  • tests/snapshots/TestCLICommands_atmos_workflow_no_steps.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_step_type.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_from_step.stderr.golden
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden
  • tests/fixtures/scenarios/workflows/stacks/workflows/test.yaml
🧰 Additional context used
🪛 YAMLlint (1.37.1)
tests/test-cases/workflows.yaml

[error] 120-120: no new line character at the end of file

(new-line-at-end-of-file)

🪛 golangci-lint (1.64.8)
internal/exec/workflow_utils.go

[error] 36-36: S1039: unnecessary use of fmt.Sprintf

(gosimple)

🪛 GitHub Check: golangci-lint
internal/exec/workflow_utils.go

[failure] 35-35:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("Workflow %s does not have any steps defined", workflow)"


[failure] 36-36:
S1039: unnecessary use of fmt.Sprintf


[failure] 69-69:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("Invalid --from-step flag. Workflow %s does not have a step with the name %s", workflow, fromStep)"


[failure] 120-120:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("Invalid workflow step type %s", commandType)"


[failure] 141-141:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it


[failure] 147-147:
error-strings: error strings should not be capitalized or end with punctuation or a newline

⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Acceptance Tests (ubuntu-latest, linux)
  • GitHub Check: Acceptance Tests (windows-latest, windows)
  • GitHub Check: Acceptance Tests (macos-latest, macos)
  • GitHub Check: [localstack] demo-localstack
  • GitHub Check: Summary
🔇 Additional comments (7)
internal/exec/workflow_utils.go (6)

14-14: Good migration to custom logger.

The import change to use the custom logger from atmos/pkg/logger aligns well with the PR objectives for standardizing log handling across workflows.


33-38: Excellent error handling improvement.

Using PrintErrorMarkdownAndExit provides much better user experience with formatted error messages and clear guidance. This aligns perfectly with the PR objectives for enhanced error clarity.

🧰 Tools
🪛 golangci-lint (1.64.8)

[error] 36-36: S1039: unnecessary use of fmt.Sprintf

(gosimple)

🪛 GitHub Check: golangci-lint

[failure] 35-35:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("Workflow %s does not have any steps defined", workflow)"


[failure] 36-36:
S1039: unnecessary use of fmt.Sprintf


44-48: Logger initialization looks solid.

Creating the logger from CLI config is the right approach for consistent logging behavior.


66-72: Clean error handling with helpful context.

The error message now includes available steps which is much more user-friendly than the previous approach.

🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 69-69:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("Invalid --from-step flag. Workflow %s does not have a step with the name %s", workflow, fromStep)"


145-149: Great improvement in error messaging.

The detailed error format with failed command and resume instructions provides excellent user guidance. The markdown formatting makes it very readable.

🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 147-147:
error-strings: error strings should not be capitalized or end with punctuation or a newline


157-164: Useful helper function for consistent formatting.

The formatList function ensures consistent markdown bullet formatting across error messages.

tests/test-cases/workflows.yaml (1)

1-120: Comprehensive test coverage for workflow scenarios.

The test cases cover all the key workflow scenarios including success, various failure modes, and error conditions. This provides excellent validation of the enhanced error handling and logging behavior.

🧰 Tools
🪛 YAMLlint (1.37.1)

[error] 120-120: no new line character at the end of file

(new-line-at-end-of-file)

coderabbitai[bot]
coderabbitai bot previously approved these changes May 31, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (2)
internal/exec/workflow_utils.go (1)

157-164: Useful helper function with minor improvement opportunity.

The formatList function is a nice addition for consistent markdown formatting.

Consider adding a comment about the trailing newline behavior:

// formatList formats a list of strings into a markdown bullet list.
+// Returns a string with each item as a markdown bullet point, ending with a newline.
func formatList(items []string) string {
tests/test-cases/workflows.yaml (1)

108-130: Fix missing newline at end of file.

Good test case but needs the required newline at EOF to satisfy static analysis.

        - "• step1"
      exit_code: 1
+
🧰 Tools
🪛 YAMLlint (1.37.1)

[error] 130-130: no new line character at the end of file

(new-line-at-end-of-file)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6395e2d and 9a169fe.

📒 Files selected for processing (7)
  • internal/exec/workflow_utils.go (4 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_from_step.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_step_type.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_no_steps.stderr.golden (1 hunks)
  • tests/test-cases/workflows.yaml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_from_step.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_no_steps.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_step_type.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden
🧰 Additional context used
🧬 Code Graph Analysis (1)
internal/exec/workflow_utils.go (4)
pkg/utils/markdown_utils.go (1)
  • PrintErrorMarkdownAndExit (87-89)
pkg/logger/logger.go (1)
  • NewLoggerFromCliConfig (44-50)
pkg/schema/schema.go (2)
  • WorkflowStep (487-492)
  • Command (568-578)
internal/exec/shell_utils.go (1)
  • ExecuteShellCommand (21-75)
🪛 GitHub Check: golangci-lint
internal/exec/workflow_utils.go

[failure] 35-35:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("workflow has no steps defined")"


[failure] 69-69:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("invalid from-step flag")"


[failure] 120-120:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("invalid workflow step type")"


[failure] 141-141:
add-constant: string literal "atmos" appears, at least, 4 times, create a named constant for it


[failure] 146-146:
add-constant: string literal "Workflow Error" appears, at least, 4 times, create a named constant for it


[failure] 147-147:
do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf("workflow step execution failed")"

🪛 YAMLlint (1.37.1)
tests/test-cases/workflows.yaml

[error] 130-130: no new line character at the end of file

(new-line-at-end-of-file)

⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Acceptance Tests (macos-latest, macos)
  • GitHub Check: [localstack] demo-localstack
  • GitHub Check: Acceptance Tests (ubuntu-latest, linux)
  • GitHub Check: Acceptance Tests (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (8)
internal/exec/workflow_utils.go (4)

14-14: Good import change for centralized logging.

The switch from the charmbracelet log package to the custom logger package aligns well with standardizing logging across the codebase.


44-48: Solid logger initialization approach.

Creating the logger from CLI config ensures consistent logging behavior across the application.


80-80: Good step execution logging.

The debug message provides helpful context for troubleshooting workflow execution.


113-113: Consistent debug logging pattern.

Stack logging follows the same pattern as other debug messages, maintaining consistency.

tests/test-cases/workflows.yaml (4)

1-16: Comprehensive workflow success test.

Good baseline test for successful workflow execution with appropriate snapshot configuration.


17-38: Excellent failure scenario coverage.

The test properly validates the new error handling with detailed expectations for the markdown-formatted error output.


40-63: Good shell command failure testing.

Properly tests shell command failures and verifies the resume command suggestion functionality.


65-83: Solid edge case coverage.

Tests the important edge case of workflows with no steps, validating the new error handling.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (6)
internal/exec/workflow_utils.go (6)

60-60: Use structured logging for consistency.

The codebase uses structured logging patterns elsewhere. Replace fmt.Sprintf with key-value pairs for better consistency and performance.

-	l.Info(fmt.Sprintf("Executing the workflow %s from %s", workflow, workflowPath))
+	l.Info("Executing the workflow", "workflow", workflow, "file", workflowPath)

90-90: Use structured logging consistently.

Replace sprintf-based logging with structured logging for better consistency with the codebase patterns.

-	l.Debug(fmt.Sprintf("Executing workflow step %d: %s, %s", stepIdx, step.Name, command))
+	l.Debug("Executing workflow step", "index", stepIdx, "step", step.Name, "command", command)

123-123: Use structured logging consistently.

Maintain consistency with structured logging patterns used elsewhere in the codebase.

-	l.Debug(fmt.Sprintf("Stack: %s", finalStack))
+	l.Debug("Using stack", "stack", finalStack)

137-137: Use structured logging and consider log level.

Replace sprintf-based logging with structured logging for consistency. Also consider whether this should be Info level instead of Debug given the PR objectives about making failures visible.

-	l.Debug(fmt.Sprintf("Workflow failed: %s", err))
+	l.Info("Workflow step failed", "step", step.Name, "command", command, "error", err)

43-48: 🛠️ Refactor suggestion

Remove unreachable return statement.

The return statement after PrintErrorMarkdownAndExit will never be executed since that function exits the program. This creates unreachable code.

 		u.PrintErrorMarkdownAndExit(
 			WorkflowErrTitle,
 			ErrWorkflowNoSteps,
 			fmt.Sprintf("\n## Explanation\nWorkflow `%s` is empty and requires at least one step to execute.", workflow),
 		)
-		return nil // This line will never be reached due to PrintErrorMarkdownAndExit

128-133: 🛠️ Refactor suggestion

Remove unreachable return statement.

The return statement after PrintErrorMarkdownAndExit is unreachable and should be removed.

 		u.PrintErrorMarkdownAndExit(
 			WorkflowErrTitle,
 			ErrInvalidWorkflowStepType,
 			fmt.Sprintf("\n## Explanation\nStep type `%s` is not supported. Each step must specify a valid type. \n### Available types:\n%s", commandType, formatList([]string{"atmos", "shell"})),
 		)
-		return nil // This line will never be reached due to PrintErrorMarkdownAndExit
🧹 Nitpick comments (1)
internal/exec/workflow_utils.go (1)

21-21: Fix comment formatting.

The comment should end with a period for consistency with Go documentation standards.

-// Static error definitions
+// Static error definitions.
🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 21-21:
Comment should end in a period

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9a169fe and b45c11c.

📒 Files selected for processing (6)
  • internal/exec/workflow_utils.go (4 hunks)
  • pkg/config/const.go (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_from_step.stderr.golden (1 hunks)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_step_type.stderr.golden (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • pkg/config/const.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_step_type.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_invalid_from_step.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_failure_on_shell_command.stderr.golden
  • tests/snapshots/TestCLICommands_atmos_workflow_failure.stderr.golden
🧰 Additional context used
🧬 Code Graph Analysis (1)
internal/exec/workflow_utils.go (5)
pkg/utils/markdown_utils.go (1)
  • PrintErrorMarkdownAndExit (87-89)
pkg/logger/logger.go (1)
  • NewLoggerFromCliConfig (44-50)
pkg/schema/schema.go (2)
  • WorkflowStep (487-492)
  • Command (568-578)
internal/exec/shell_utils.go (1)
  • ExecuteShellCommand (21-75)
pkg/config/const.go (1)
  • AtmosCommand (4-4)
🪛 GitHub Check: golangci-lint
internal/exec/workflow_utils.go

[failure] 21-21:
Comment should end in a period

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (2)
internal/exec/workflow_utils.go (2)

167-174: Well-implemented helper function.

The formatList function provides clean markdown formatting for error messages and enhances user experience. Good implementation!


21-28: Excellent static error implementation.

Great job addressing the static analysis warnings by implementing proper static error variables. This follows Go best practices and improves error handling consistency.

🧰 Tools
🪛 GitHub Check: golangci-lint

[failure] 21-21:
Comment should end in a period

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
internal/exec/helmfile_test.go (2)

47-97: Simplify directory validation logic.

The extensive directory validation and logging makes this test quite verbose and harder to maintain. Consider simplifying this logic or extracting it to a helper function if the validation is truly necessary.

For a helmfile version test, basic directory change should suffice:

-		// Get absolute path to work directory
-		workDirAbs, err := filepath.Abs(tt.workDir)
-		if err != nil {
-			t.Fatalf("Failed to get absolute path for work directory: %v", err)
-		}
-		log.Debug("Source work directory", "dir", workDirAbs)
-
-		// Change to the work directory
-		if err := os.Chdir(workDirAbs); err != nil {
-			t.Fatalf("Failed to change directory to %q: %v", workDirAbs, err)
-		}
-
-		// Log current working directory for debugging
-		currentDir, err := os.Getwd()
-		if err != nil {
-			t.Fatalf("Failed to get current working directory: %v", err)
-		}
-		log.Debug("Current working directory", "dir", currentDir)
-
-		// List contents of current directory
-		entries, err := os.ReadDir(currentDir)
-		if err != nil {
-			t.Fatalf("Failed to read directory contents: %v", err)
-		}
-		log.Debug("Directory contents", "files", func() []string {
-			var names []string
-			for _, entry := range entries {
-				names = append(names, entry.Name())
-			}
-			return names
-		}())
-
-		// Verify the stacks directory exists
-		stacksDir := filepath.Join(currentDir, "stacks")
-		if _, err := os.Stat(stacksDir); os.IsNotExist(err) {
-			t.Fatalf("Stacks directory does not exist at %q", stacksDir)
-		}
-
-		// List contents of stacks directory
-		stackEntries, err := os.ReadDir(stacksDir)
-		if err != nil {
-			t.Fatalf("Failed to read stacks directory contents: %v", err)
-		}
-		log.Debug("Stacks directory contents", "files", func() []string {
-			var names []string
-			for _, entry := range stackEntries {
-				names = append(names, entry.Name())
-			}
-			return names
-		}())
+		// Change to the work directory
+		if err := os.Chdir(tt.workDir); err != nil {
+			t.Fatalf("Failed to change directory to %q: %v", tt.workDir, err)
+		}

14-131: Consider the long-term maintainability of extensive debugging code.

The test has grown significantly with debugging infrastructure. If this level of detail is needed for troubleshooting, consider extracting common debugging utilities to reduce duplication across tests.

Is this extensive debugging code intended to be permanent, or is it temporary for troubleshooting specific issues?

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9b35cf8 and 7bc3ee8.

📒 Files selected for processing (1)
  • internal/exec/helmfile_test.go (4 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: milldr
PR: cloudposse/atmos#1229
File: internal/exec/workflow_test.go:0-0
Timestamp: 2025-06-02T14:12:02.698Z
Learning: In the atmos codebase, workflow error handling was refactored to use `PrintErrorMarkdown` followed by returning specific error variables (like `ErrWorkflowNoSteps`, `ErrInvalidFromStep`, `ErrInvalidWorkflowStepType`, `ErrWorkflowStepFailed`) instead of `PrintErrorMarkdownAndExit`. This pattern allows proper error testing without the function terminating the process with `os.Exit`, enabling unit tests to assert on error conditions while maintaining excellent user-facing error formatting.
Learnt from: milldr
PR: cloudposse/atmos#1229
File: internal/exec/workflow_test.go:0-0
Timestamp: 2025-06-02T14:12:02.698Z
Learning: In the atmos codebase, workflow error handling was refactored to use `PrintErrorMarkdown` followed by returning the error instead of `PrintErrorMarkdownAndExit`. This pattern allows proper error testing without the function terminating the process with `os.Exit`, enabling unit tests to assert on error conditions.
⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Build (windows-latest, windows)
  • GitHub Check: Summary
🔇 Additional comments (1)
internal/exec/helmfile_test.go (1)

124-124: Good addition for debugging test failures.

Logging the command output before assertions is helpful for troubleshooting failed tests.

coderabbitai[bot]
coderabbitai bot previously approved these changes Jun 4, 2025
@milldr milldr merged commit c36d6f7 into main Jun 4, 2025
50 checks passed
@milldr milldr deleted the DEV-3110/workflow-failure branch June 4, 2025 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

minor New features that do not break anything

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants