-
Notifications
You must be signed in to change notification settings - Fork 4.5k
(glue): Timeout and workerType missing validation for Ray job types #29612
Copy link
Copy link
Closed
Labels
@aws-cdk/aws-glueRelated to AWS GlueRelated to AWS GluebugThis issue is a bug.This issue is a bug.effort/mediumMedium work item – several days of effortMedium work item – several days of effortp3
Description
Describe the bug
When developing Ray jobs via CDK, I bumped into the fact that specifying a timeout or not specifying a worker type on Job for Ray job types will succeed in synthesis, but fail deployment.
Expected Behavior
I'd expect an error to be thrown if workerType is not provided or timeout is provided, as both of these will cause deployment failures for Ray job types.
Current Behavior
Synthesis passes and failure occurs on deployment:
UPDATE_FAILED [...] Timeout not supported for Ray jobs
or
CREATE_FAILED [...] Worker type cannot be null and only [Z.2X] worker types are supported for glueray jobs
Reproduction Steps
#!/usr/bin/env node
import * as cdk from 'aws-cdk-lib';
import { Code, GlueVersion, Job, JobExecutable, PythonVersion, Runtime, WorkerType } from '@aws-cdk/aws-glue-alpha';
const app = new cdk.App();
const stack = new cdk.Stack(app, 'glue-job-issue-stack');
const job = new Job(stack, 'MyJob', {
jobName: 'glue-issue-test-job',
executable: JobExecutable.pythonRay({
glueVersion: GlueVersion.V4_0,
pythonVersion: PythonVersion.THREE_NINE,
runtime: Runtime.RAY_TWO_FOUR,
script: Code.fromAsset('lib/test-job.py')
}),
workerType: WorkerType.G_025X,
workerCount: 1,
timeout: cdk.Duration.minutes(10),
});
Possible Solution
Additional validation in packages/@aws-cdk/aws-glue-alpha/lib/job.ts would address this and provide a slightly smoother developer experience when working with ray job types. I would be interested in contributing a fix if triage determines this is valid bug.
Additional Information/Context
No response
CDK CLI Version
2.133.0
Framework Version
2.133.0
Node.js Version
20.10.0
OS
macOS
Language
TypeScript
Language Version
5.3.3
Other information
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
@aws-cdk/aws-glueRelated to AWS GlueRelated to AWS GluebugThis issue is a bug.This issue is a bug.effort/mediumMedium work item – several days of effortMedium work item – several days of effortp3