feat(glue-alpha): add job run queuing to Glue job#31830
feat(glue-alpha): add job run queuing to Glue job#31830mergify[bot] merged 7 commits intoaws:mainfrom jdebuseamazon:jdebuseamazon/glue-job-run-queuing-enabled
Conversation
aws-cdk-automation
left a comment
There was a problem hiding this comment.
The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.
A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.
|
Exemption Request (for |
|
@jdebuseamazon Thank you for your contribution! I think it would be helpful to add a README description for this feature. The descriptions for maxCapacity and minCapacity are necessary, but this should not be addressed in this PR. |
✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.
badmintoncryer
left a comment
There was a problem hiding this comment.
Thank you for your contribution!
I would appreciate it if you could provide links regarding the need for a max retry of 0 and the fact that queueing cannot be enabled with the FLEX ExecutionClass.
| describe('enabling job run queuing', () => { | ||
| beforeEach(() => { | ||
| job = new glue.Job(stack, 'Job', { | ||
| ...defaultProps, | ||
| jobRunQueuingEnabled: true, | ||
| }); | ||
| }); | ||
|
|
||
| test('should set JobRunQueuingEnabled', () => { | ||
| Template.fromStack(stack).hasResourceProperties('AWS::Glue::Job', { | ||
| JobRunQueuingEnabled: true, | ||
| }); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
This description may be sufficient.
| describe('enabling job run queuing', () => { | |
| beforeEach(() => { | |
| job = new glue.Job(stack, 'Job', { | |
| ...defaultProps, | |
| jobRunQueuingEnabled: true, | |
| }); | |
| }); | |
| test('should set JobRunQueuingEnabled', () => { | |
| Template.fromStack(stack).hasResourceProperties('AWS::Glue::Job', { | |
| JobRunQueuingEnabled: true, | |
| }); | |
| }); | |
| }); | |
| test('enabling job run queuing', () => { | |
| job = new glue.Job(stack, 'Job', { | |
| ...defaultProps, | |
| jobRunQueuingEnabled: true, | |
| }); | |
| Template.fromStack(stack).hasResourceProperties('AWS::Glue::Job', { | |
| JobRunQueuingEnabled: true, | |
| }); | |
| }); |
| describe('validation for jobRunQueuingEnabled and maxRetries', () => { | ||
| test('jobRunQueuingEnabled with maxRetries > 0 should throw', () => { |
There was a problem hiding this comment.
If there is only one test within a describe block, I think a single test block is sufficient.
| throw new Error('Both workerType and workerCount must be set'); | ||
| } | ||
|
|
||
| if (props.jobRunQueuingEnabled === true && props.maxRetries !== undefined && props.maxRetries > 0) { |
There was a problem hiding this comment.
Please add condition !Token.isUnresolved(props.maxRetries) to the if statement.
This will enable support when maxRetries is token.
| if (props.jobRunQueuingEnabled === true && props.maxRetries !== undefined && props.maxRetries > 0) { | |
| if (props.jobRunQueuingEnabled === true && props.maxRetries !== undefined && Token.isUnresolved(props.maxRetries) && props.maxRetries > 0) { |
No problem at all, very happy to contribute, and many thanks for reviewing. The restrictions on max retry and FLEX execution class are documented here (in the Considerations section) |
paulhcsun
left a comment
There was a problem hiding this comment.
Thanks for the contribution @jdebuseamazon! And thank you for helping to review @badmintoncryer!
|
@Mergifyio update |
☑️ Nothing to doDetails
|
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
|
Comments on closed issues and PRs are hard for our team to see. |
Issue # (if applicable)
Closes #31826
Reason for this change
Job within @aws-cdk/aws-glue-alpha does not currently include the jobRunQueuingEnabled property of the CfnJob within aws-cdk-lib/aws-glue. Setting this property currently requires a raw override.
Description of changes
Added
jobRunQueuingEnabledto construction properties forJob, along with validation that this is not enabled when execution class is flexible and/ormaxRetriesexceeds zero (see).Description of how you validated changes
Unit tests and an integration test.
Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license