Skip to content

test: refactor test setup#3712

Merged
reubenmiller merged 6 commits intothin-edge:mainfrom
reubenmiller:feat-system-test-setup-improvements
Jun 27, 2025
Merged

test: refactor test setup#3712
reubenmiller merged 6 commits intothin-edge:mainfrom
reubenmiller:feat-system-test-setup-improvements

Conversation

@reubenmiller
Copy link
Copy Markdown
Contributor

@reubenmiller reubenmiller commented Jun 26, 2025

Proposed changes

Refactoring the system tests to improve the setting

  • Use the Cumulocity CA by default
  • Move the tedge configuration and connection logic from the the bootstrap.sh script to the ThinEdgeIO RobotFramework library
  • Setup keyword improvements
    • Add more setup options, register, register_using and connect
    • Support passing additional bootstrap arguments via bootstrap_args=
  • Add additional keywords so that users can also run the same device registration steps manually without using Setup (if users need to customize everything)

Example usage of Setup

Below shows some common usages

# don't run the bootstrap.sh script, register or connect the device
Setup    skip_bootstrap=${True}

# setup the device, but don't register or set the c8y.url
Setup    register=${False}

# setup the device, register it and set the c8y.url, but don't connect it to Cumulocity (e.g. don't run `tedge connect c8y`)
Setup    connect=${False}

# setup the device and pass some additional arguments to the script
Setup    bootstrap_args=--no-secure

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#3708

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
  • I ran just format as mentioned in CODING_GUIDELINES
  • I used just check as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

@codecov
Copy link
Copy Markdown

codecov bot commented Jun 26, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 26, 2025

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
652 0 3 652 100 1h50m28.155910999s

@reubenmiller reubenmiller force-pushed the feat-system-test-setup-improvements branch from 5bf9572 to 7193c86 Compare June 27, 2025 06:37
@reubenmiller reubenmiller temporarily deployed to Test Pull Request June 27, 2025 06:37 — with GitHub Actions Inactive
@reubenmiller reubenmiller force-pushed the feat-system-test-setup-improvements branch from 7193c86 to 2be3e1b Compare June 27, 2025 08:27
@reubenmiller reubenmiller force-pushed the feat-system-test-setup-improvements branch from 2be3e1b to 3487271 Compare June 27, 2025 08:28
@reubenmiller reubenmiller force-pushed the feat-system-test-setup-improvements branch from 3487271 to 47f237a Compare June 27, 2025 08:38
@reubenmiller reubenmiller changed the title tests: refactor test setup test: refactor test setup Jun 27, 2025
Signed-off-by: reubenmiller <reuben.d.miller@gmail.com>
Signed-off-by: reubenmiller <reuben.d.miller@gmail.com>
Signed-off-by: reubenmiller <reuben.d.miller@gmail.com>
Signed-off-by: reubenmiller <reuben.d.miller@gmail.com>
Signed-off-by: reubenmiller <reuben.d.miller@gmail.com>
… due to removing the local-ca option

Signed-off-by: reubenmiller <reuben.d.miller@gmail.com>
@reubenmiller reubenmiller force-pushed the feat-system-test-setup-improvements branch from 47f237a to fcc2009 Compare June 27, 2025 10:47
@reubenmiller reubenmiller temporarily deployed to Test Pull Request June 27, 2025 10:47 — with GitHub Actions Inactive
@reubenmiller reubenmiller marked this pull request as ready for review June 27, 2025 10:56
@Bravo555 Bravo555 self-assigned this Jun 27, 2025
Copy link
Copy Markdown
Contributor

@didier-wenzek didier-wenzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. This is a really nice improvement. Moving away from an all-or-nothing setup in favor of explicit steps to skip, it's a great idea.

Test Setup
${DEVICE_SN}= Setup skip_bootstrap=True
Execute Command ./bootstrap.sh
${DEVICE_SN}= Setup
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this skip-then-execute pattern has been a consequence of a lack of understanding of the bootstrap related features. To be honest, I did the same, i.e. blinding copy-pasting tests to build my own setup.

I do think the revised Setup will really help read and write test specific setup.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though it could of also developed over time, since we changed some of the defaults for using a secure local MQTT broker setup. But all of this is fairly normal, so good to clean things up from time to time.

@Bravo555 Bravo555 removed their assignment Jun 27, 2025
@reubenmiller reubenmiller added this pull request to the merge queue Jun 27, 2025
Merged via the queue into thin-edge:main with commit 70331cd Jun 27, 2025
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

theme:testing Theme: Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants