Skip to content

fix(stepfunctions): add docs and warning DistributedMap Class ignores executionType in the ProcessorConfig#30301

Merged
mergify[bot] merged 8 commits intoaws:mainfrom
mazyu36:sfn-distributed-map-execution-type-30194
Jun 13, 2024
Merged

fix(stepfunctions): add docs and warning DistributedMap Class ignores executionType in the ProcessorConfig#30301
mergify[bot] merged 8 commits intoaws:mainfrom
mazyu36:sfn-distributed-map-execution-type-30194

Conversation

@mazyu36
Copy link
Copy Markdown
Contributor

@mazyu36 mazyu36 commented May 22, 2024

Issue # (if applicable)

Closes #30194

Reason for this change

In #27913, the ItemProcessor was introduced for use with the Map Class. With the executionType in the ProcessorConfig, it was possible to specify the executionType for the Map.
On the other hand, in #28821, the DistributedMap Class was introduced. The mapExecutionType of the DistributedMap class always overwrites the executionType of the ProcessorConfig.
Therefore, when using the DistributedMap class, the implementation ignores the executionType of the ProcessorConfig. However, this behavior cannot be inferred from the documentation.

Description of changes

  • Added to the docs that when using the DistributedMap Class, the executionType in the ProcessorConfig is ignored.
  • Also added a warning.

Description of how you validated changes

Add unit tests and integ tests

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added bug This issue is a bug. valued-contributor [Pilot] contributed between 6-12 PRs to the CDK effort/small Small work item – less than a day of effort p2 labels May 22, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team May 22, 2024 04:10
*/
public toStateJson(): object {
let rendered: any = super.toStateJson();
if (this.mapExecutionType) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this conditional statement as it is meaningless, because in the following code, mapExecutionType is always set, making the condition always evaluate to true.

this.mapExecutionType = props.mapExecutionType ?? StateMachineType.STANDARD;

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label May 22, 2024
Copy link
Copy Markdown
Contributor

@gracelu0 gracelu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mazyu36 , thanks for working on this! Could you add a note in the docstring for the mapExecutionType property and update the README file as well so users know they should use that property to set the execution type for ItemProcessor. Thanks!

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label May 31, 2024
Co-authored-by: Grace Luo <54298030+gracelu0@users.noreply.github.com>
@mergify mergify bot dismissed gracelu0’s stale review June 1, 2024 05:13

Pull request has been modified.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jun 1, 2024
@mazyu36
Copy link
Copy Markdown
Contributor Author

mazyu36 commented Jun 1, 2024

@gracelu0
Thank you for the review!
I have updated the documentation and README.​​​​​​​​​​​​​​​​

distributedMap.itemProcessor(new sfn.Pass(this, 'Pass State'));
```

If you want to specify the execution type for the DistributedMap, you must set the `mapExecutionType` property in the `DistributedMap` class. When using the `DistributedMap` class, the `ProcessorConfig.executionType` property is ignored.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we mean 'execution type for ItemProcessor'? I don't see a mapExecutionType property in the docs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review.
As you correctly understood, I have revised the expression to 'ItemProcessor in the DistributedMap' for more accurate phrasing.


If you want to specify the execution type for the DistributedMap, you must set the `mapExecutionType` property in the `DistributedMap` class. When using the `DistributedMap` class, the `ProcessorConfig.executionType` property is ignored.

In the following example, the execution type for the DistributedMap is set to `EXPRESS` based on the value specified for `mapExecutionType`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, itemProcessor instead of DistributedMap?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the same revision as the one above.

@mazyu36 mazyu36 force-pushed the sfn-distributed-map-execution-type-30194 branch from d631069 to c5b8b0c Compare June 4, 2024 01:49
@gracelu0
Copy link
Copy Markdown
Contributor

@Mergifyio update

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Jun 13, 2024

update

✅ Branch has been successfully updated

Copy link
Copy Markdown
Contributor

@gracelu0 gracelu0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contributing!

@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 5f27336
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Jun 13, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 0499adf into aws:main Jun 13, 2024
@mazyu36 mazyu36 deleted the sfn-distributed-map-execution-type-30194 branch June 18, 2024 14:37
sarangarav pushed a commit to sarangarav/aws-cdk that referenced this pull request Jun 21, 2024
… executionType in the ProcessorConfig (aws#30301)

### Issue # (if applicable)

Closes aws#30194 

### Reason for this change
In aws#27913, the ItemProcessor was introduced for use with the Map Class. With the executionType in the ProcessorConfig, it was possible to specify the executionType for the Map.
On the other hand, in aws#28821, the DistributedMap Class was introduced. The mapExecutionType of the DistributedMap class always overwrites the executionType of the ProcessorConfig.
Therefore, when using the DistributedMap class, the implementation ignores the executionType of the ProcessorConfig. However, this behavior cannot be inferred from the documentation.

### Description of changes
* Added to the docs that when using the DistributedMap Class, the executionType in the ProcessorConfig is ignored.
* Also added a warning.


### Description of how you validated changes
Add unit tests and integ tests



### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
mazyu36 added a commit to mazyu36/aws-cdk that referenced this pull request Jun 22, 2024
… executionType in the ProcessorConfig (aws#30301)

### Issue # (if applicable)

Closes aws#30194 

### Reason for this change
In aws#27913, the ItemProcessor was introduced for use with the Map Class. With the executionType in the ProcessorConfig, it was possible to specify the executionType for the Map.
On the other hand, in aws#28821, the DistributedMap Class was introduced. The mapExecutionType of the DistributedMap class always overwrites the executionType of the ProcessorConfig.
Therefore, when using the DistributedMap class, the implementation ignores the executionType of the ProcessorConfig. However, this behavior cannot be inferred from the documentation.

### Description of changes
* Added to the docs that when using the DistributedMap Class, the executionType in the ProcessorConfig is ignored.
* Also added a warning.


### Description of how you validated changes
Add unit tests and integ tests



### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.

@aws aws locked as resolved and limited conversation to collaborators Jul 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug This issue is a bug. effort/small Small work item – less than a day of effort p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. valued-contributor [Pilot] contributed between 6-12 PRs to the CDK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aws-stepfunctions: Distributed Map State Item Processor executionType doesn't recognize ProcessorType.EXPRESS

3 participants