Skip to content

feat(stepfunctions): allow intrinsic functions for json path#15320

Merged
mergify[bot] merged 7 commits intoaws:masterfrom
msambol:sfn-json-path-intrinsic-functions
Jul 26, 2021
Merged

feat(stepfunctions): allow intrinsic functions for json path#15320
mergify[bot] merged 7 commits intoaws:masterfrom
msambol:sfn-json-path-intrinsic-functions

Conversation

@msambol
Copy link
Copy Markdown
Contributor

@msambol msambol commented Jun 25, 2021

QueryString can start with an intrinsic function.

Here is an example of a working QueryString:

"QueryString.$": "States.Format('select contact_id from interactions.conversation where case when \\'{}\\' is not null then bu=\\'{}\\' else 1=1 end and case when \\'{}\\' is not null then channel=\\'{}\\' else 1=1 end and case when {} is not null then year={} else 1=1 end and case when {} is not null then month={} else 1=1 end and case when {} is not null then day={} else 1=1 end limit 1;', States.JsonToString($.completed), $.partitions.bu, $.partitions.bu, $.partitions.channel, $.partitions.channel, $.partitions.year, $.partitions.year, $.partitions.month, $.partitions.month, $.partitions.day, $.partitions.day)"

@gitpod-io
Copy link
Copy Markdown

gitpod-io bot commented Jun 25, 2021

@msambol msambol force-pushed the sfn-json-path-intrinsic-functions branch 5 times, most recently from 5f7d61d to df78631 Compare June 28, 2021 22:20
@moofish32
Copy link
Copy Markdown
Contributor

@BenChaimberg - is this something you can look at? Without this feature it seems hard to have any dynamic behavior with Query Tasks?

@BenChaimberg BenChaimberg added the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Jun 29, 2021
@BenChaimberg BenChaimberg self-assigned this Jun 29, 2021
});
```

You can also use [intrinsic functions](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html) with `JsonPath.stringAt()`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's definitely a good thing that we're unblocking the use of intrinsic functions, I'm just hesitant to advertise JsonPath.stringAt as the entry point. However, I'm not sure if there's a better way without a full design of how to model intrinsic functions properly within the CDK. Any thoughts on a better way to present this to the user as a stopgap until that design takes place?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BenChaimberg right, this doesn't feel like the correct long-term solution but I think it's important to unblock this. Can we add a todo somewhere in the code or docs?

Comment on lines +222 to +225
&& !path.startsWith('States.Format')
&& !path.startsWith('States.StringToJson')
&& !path.startsWith('States.JsonToString')
&& !path.startsWith('States.Array')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an array-based condition might be better long term, something like

Suggested change
&& !path.startsWith('States.Format')
&& !path.startsWith('States.StringToJson')
&& !path.startsWith('States.JsonToString')
&& !path.startsWith('States.Array')
&& ['Format', 'StringToJson', 'JsonToString', 'Array'].every(fn => !path.startsWith(`States.${fn}`))

@msambol msambol force-pushed the sfn-json-path-intrinsic-functions branch from eacfea3 to 59cd4c5 Compare July 23, 2021 00:29
@mergify mergify bot dismissed BenChaimberg’s stale review July 23, 2021 00:29

Pull request has been modified.

@BenChaimberg BenChaimberg changed the title fix(aws-stepfunctions): allow intrinsic functions for json path feat(stepfunctions): allow intrinsic functions for json path Jul 26, 2021
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Jul 26, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
  • Commit ID: 5e802c5
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit d9285cb into aws:master Jul 26, 2021
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Jul 26, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

TikiTDO pushed a commit to TikiTDO/aws-cdk that referenced this pull request Aug 3, 2021
`QueryString` can start with an [intrinsic function](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html).

Here is an example of a working `QueryString`:
```
"QueryString.$": "States.Format('select contact_id from interactions.conversation where case when \\'{}\\' is not null then bu=\\'{}\\' else 1=1 end and case when \\'{}\\' is not null then channel=\\'{}\\' else 1=1 end and case when {} is not null then year={} else 1=1 end and case when {} is not null then month={} else 1=1 end and case when {} is not null then day={} else 1=1 end limit 1;', States.JsonToString($.completed), $.partitions.bu, $.partitions.bu, $.partitions.channel, $.partitions.channel, $.partitions.year, $.partitions.year, $.partitions.month, $.partitions.month, $.partitions.day, $.partitions.day)"
```
@msambol msambol deleted the sfn-json-path-intrinsic-functions branch August 5, 2021 18:16
@Joe-Zer0
Copy link
Copy Markdown

Is there concern that the fix being unintuitive? Would it be better to create a new function for this?
something like
stepf.IntrinsicFunction.stringAt("States.JsonToString($)")
instead of
stepf.JsonPath.stringAt("States.JsonToString($)")

@BenChaimberg
Copy link
Copy Markdown
Contributor

Yes, absolutely, we should support this in a more fluent way. However, this change unblocks usage which is more important at this stage

hollanddd pushed a commit to hollanddd/aws-cdk that referenced this pull request Aug 26, 2021
`QueryString` can start with an [intrinsic function](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html).

Here is an example of a working `QueryString`:
```
"QueryString.$": "States.Format('select contact_id from interactions.conversation where case when \\'{}\\' is not null then bu=\\'{}\\' else 1=1 end and case when \\'{}\\' is not null then channel=\\'{}\\' else 1=1 end and case when {} is not null then year={} else 1=1 end and case when {} is not null then month={} else 1=1 end and case when {} is not null then day={} else 1=1 end limit 1;', States.JsonToString($.completed), $.partitions.bu, $.partitions.bu, $.partitions.channel, $.partitions.channel, $.partitions.year, $.partitions.year, $.partitions.month, $.partitions.month, $.partitions.day, $.partitions.day)"
```
kichik added a commit to kichik/aws-cdk that referenced this pull request Oct 20, 2022
This adds support for States.JsonMerge, States.ArrayPartition, States.ArrayContains, States.ArrayRange, States.ArrayGetItem, States.ArrayLength, States.ArrayUnique, States.Base64Encode, States.Base64Decode, States.Hash, States.MathRandom, States.MathAdd, States.StringSplit, and States.UUID
Builds on top of aws#15320
@singlow
Copy link
Copy Markdown

singlow commented Nov 14, 2023

Does this change allow me to use listAt with intrinsic functions or only stringAt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

@aws-cdk/aws-stepfunctions Related to AWS StepFunctions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants