Describe the bug
When trying to deploy Ray Glue Jobs through CDK, they will fail because it is now required to provide the "Runtime" parameter, which, as far as I know, is currently not mapped (as of 2.81.0).
Expected Behavior
We should be able to set the "Runtime" parameter on Ray Glue Jobs so they can be properly deployed.
Current Behavior
❌ ETLStack failed: Error: The stack named ETLStack failed to deploy: UPDATE_ROLLBACK_COMPLETE: Runtime parameter is required for GlueRay command (Service: AWSGlue; Status Code: 400; Error Code: InvalidInputException; Request ID: fd617e84-6f10-4068-bfe9-aebd7fce26f0; Proxy: null)
at FullCloudFormationDeployment.monitorDeployment (/home/ec2-user/agent1/_work/17/s/node_modules/aws-cdk/lib/index.js:397:10236)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Object.deployStack2 [as deployStack] (/home/ec2-user/agent1/_work/17/s/node_modules/aws-cdk/lib/index.js:400:149977)
at async /home/ec2-user/agent1/_work/17/s/node_modules/aws-cdk/lib/index.js:400:135508
Reproduction Steps
Trying to deploy any kind of Ray Glue Job will do.
Just cdk init in an empty folder, paste the entire code inside the app.py file created, create a hello_world.py file inside the root folder and cdk deploy
import os
from os import path
import aws_cdk as cdk
from aws_cdk import Stack
from aws_cdk import aws_glue_alpha as glue
from constructs import Construct
class RayStack(Stack):
def __init__(self, scope: Construct, id: str, **kwargs) -> None:
super().__init__(scope, id, **kwargs)
# Create a Glue job with Ray engine
glue_job = glue.Job(
self,
"ray-job",
job_name='ray-job',
worker_count=2,
worker_type=glue.WorkerType.Z_2_X,
executable=glue.JobExecutable.python_ray(
glue_version=glue.GlueVersion.V4_0,
python_version=glue.PythonVersion.THREE_NINE,
script=glue.Code.from_asset(path.join(path.dirname(__file__), 'hello_world.py'))
)
)
app = cdk.App()
env = cdk.Environment(
account=os.getenv('CDK_DEFAULT_ACCOUNT'),
region=os.getenv('CDK_DEFAULT_REGION')
)
RayStack(app, "rayStack", env=env)
app.synth()
Possible Solution
Map the new "Runtime" parameter to the python executable props for Ray Jobs
Additional Information/Context
It was apparently a very recent change, since yesterday everything was fine.
CDK CLI Version
2.81.0
Framework Version
No response
Node.js Version
8.1.0
OS
Amazon Linux 2
Language
Python
Language Version
Python 3.9.14
Other information
No response
Describe the bug
When trying to deploy Ray Glue Jobs through CDK, they will fail because it is now required to provide the "Runtime" parameter, which, as far as I know, is currently not mapped (as of 2.81.0).
Expected Behavior
We should be able to set the "Runtime" parameter on Ray Glue Jobs so they can be properly deployed.
Current Behavior
Reproduction Steps
Trying to deploy any kind of Ray Glue Job will do.
Just
cdk initin an empty folder, paste the entire code inside the app.py file created, create ahello_world.pyfile inside the root folder andcdk deployPossible Solution
Map the new "Runtime" parameter to the python executable props for Ray Jobs
Additional Information/Context
It was apparently a very recent change, since yesterday everything was fine.
CDK CLI Version
2.81.0
Framework Version
No response
Node.js Version
8.1.0
OS
Amazon Linux 2
Language
Python
Language Version
Python 3.9.14
Other information
No response