Skip to content

chore(glue): timeout and worker type validation for Ray jobs#32119

Merged
mergify[bot] merged 3 commits intoaws:mainfrom
badmintoncryer:ray-job-validation
Nov 20, 2024
Merged

chore(glue): timeout and worker type validation for Ray jobs#32119
mergify[bot] merged 3 commits intoaws:mainfrom
badmintoncryer:ray-job-validation

Conversation

@badmintoncryer
Copy link
Copy Markdown
Contributor

@badmintoncryer badmintoncryer commented Nov 14, 2024

Issue # (if applicable)

Closes #29612.

Reason for this change

AWS Glue Ray job has some restriction.

  • must use Z.2X worker type
CREATE_FAILED [...] Worker type cannot be null and only [Z.2X] worker types are supported for glueray jobs
  • must not specify timeout
UPDATE_FAILED [...] Timeout not supported for Ray jobs

Description of changes

Add validation for above restriction.

    if (executable.type.name === JobType.RAY.name) {
      if (props.workerType !== WorkerType.Z_2X) {
        throw new Error(`WorkerType must be Z_2X for Ray jobs, got: ${props.workerType}`);
      }
      if (props.timeout !== undefined) {
        throw new Error('Timeout cannot be set for Ray jobs');
      }
    }

Description of how you validated changes

Add unit test.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team November 14, 2024 00:36
@github-actions github-actions bot added bug This issue is a bug. effort/medium Medium work item – several days of effort p2 distinguished-contributor [Pilot] contributed 50+ PRs to the CDK labels Nov 14, 2024
@badmintoncryer badmintoncryer marked this pull request as draft November 14, 2024 00:38
@badmintoncryer badmintoncryer changed the title chore(glue): add validation for ray job chore(glue): timeout and worker type validation for ray job Nov 14, 2024
@codecov
Copy link
Copy Markdown

codecov bot commented Nov 14, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.17%. Comparing base (c768554) to head (52be7e4).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main   #32119   +/-   ##
=======================================
  Coverage   77.17%   77.17%           
=======================================
  Files         105      105           
  Lines        7169     7169           
  Branches     1315     1315           
=======================================
  Hits         5533     5533           
  Misses       1455     1455           
  Partials      181      181
Flag Coverage Δ
suite.unit 77.17% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk 77.17% <ø> (ø)
---- 🚨 Try these New Features:

@badmintoncryer badmintoncryer marked this pull request as ready for review November 14, 2024 06:43
@badmintoncryer badmintoncryer changed the title chore(glue): timeout and worker type validation for ray job chore(glue): timeout and worker type validation for Ray jobs Nov 14, 2024
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 14, 2024
mazyu36
mazyu36 approved these changes Nov 16, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@mazyu36 mazyu36 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
However, this fix will not be reflected once #30833 is merged.
I'll leave the decision on how to proceed to the maintainers.

@aws-cdk-automation aws-cdk-automation added pr/needs-maintainer-review This PR needs a review from a Core Team Member and removed pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. labels Nov 16, 2024
@badmintoncryer
Copy link
Copy Markdown
Contributor Author

badmintoncryer commented Nov 16, 2024

@mazyu36 Thanks for the information. I don't know this PR. I'll wait for the maintainer's opinion.

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Nov 20, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Nov 20, 2024
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Nov 20, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

aws-cdk-automation commented Nov 20, 2024

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 52be7e4
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit be33610 into aws:main Nov 20, 2024
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Nov 20, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Nov 20, 2024

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@mazyu36 mazyu36 mazyu36 approved these changes

@GavinZZ GavinZZ GavinZZ approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

bug This issue is a bug. distinguished-contributor [Pilot] contributed 50+ PRs to the CDK effort/medium Medium work item – several days of effort p2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

(glue): Timeout and workerType missing validation for Ray job types

4 participants

@badmintoncryer @aws-cdk-automation @mazyu36 @GavinZZ