Skip to content

(glue): Timeout and workerType missing validation for Ray job types #29612

@haljarrett

Description

@haljarrett

Describe the bug

When developing Ray jobs via CDK, I bumped into the fact that specifying a timeout or not specifying a worker type on Job for Ray job types will succeed in synthesis, but fail deployment.

Expected Behavior

I'd expect an error to be thrown if workerType is not provided or timeout is provided, as both of these will cause deployment failures for Ray job types.

Current Behavior

Synthesis passes and failure occurs on deployment:

 UPDATE_FAILED [...] Timeout not supported for Ray jobs

or

CREATE_FAILED [...] Worker type cannot be null and only [Z.2X] worker types are supported for glueray jobs

Reproduction Steps

#!/usr/bin/env node
import * as cdk from 'aws-cdk-lib';
import { Code, GlueVersion, Job, JobExecutable, PythonVersion, Runtime, WorkerType } from '@aws-cdk/aws-glue-alpha';

const app = new cdk.App();
const stack = new cdk.Stack(app, 'glue-job-issue-stack');
const job = new Job(stack, 'MyJob', {
  jobName: 'glue-issue-test-job',
  executable: JobExecutable.pythonRay({
    glueVersion: GlueVersion.V4_0,
    pythonVersion: PythonVersion.THREE_NINE,
    runtime: Runtime.RAY_TWO_FOUR,
    script: Code.fromAsset('lib/test-job.py')
  }),
  workerType: WorkerType.G_025X,
  workerCount: 1,
  timeout: cdk.Duration.minutes(10),
});

Possible Solution

Additional validation in packages/@aws-cdk/aws-glue-alpha/lib/job.ts would address this and provide a slightly smoother developer experience when working with ray job types. I would be interested in contributing a fix if triage determines this is valid bug.

Additional Information/Context

No response

CDK CLI Version

2.133.0

Framework Version

2.133.0

Node.js Version

20.10.0

OS

macOS

Language

TypeScript

Language Version

5.3.3

Other information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    @aws-cdk/aws-glueRelated to AWS GluebugThis issue is a bug.effort/mediumMedium work item – several days of effortp3

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions