Skip to content

chore(assertions): snippets in readme are now rosetta compilable#16801

Merged
mergify[bot] merged 8 commits intomasterfrom
nija-at/assertions-readme
Oct 11, 2021
Merged

chore(assertions): snippets in readme are now rosetta compilable#16801
mergify[bot] merged 8 commits intomasterfrom
nija-at/assertions-readme

Conversation

@nija-at
Copy link
Copy Markdown
Contributor

@nija-at nija-at commented Oct 5, 2021

Rosetta automatic snippet translation is doing a poor
job of snippet translation. This is mainly because the
snippets are not compilable.

Update snippets to make them compilable and translate
correctly.

Additionally, make the rosetta compilation strict and run
during the build step.

There are still a few more translation errors, but these are
arising from bugs in rosetta.
aws/jsii#3026
aws/jsii#3029


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@nija-at nija-at requested review from a team and rix0rrr October 5, 2021 12:23
@nija-at nija-at self-assigned this Oct 5, 2021
@gitpod-io
Copy link
Copy Markdown

gitpod-io bot commented Oct 5, 2021

@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Oct 5, 2021
@nija-at
Copy link
Copy Markdown
Contributor Author

nija-at commented Oct 5, 2021

The README in Python after translation looks like so -

Assertions

Functions for writing test asserting against CDK applications, with focus on CloudFormation templates.

The Template class includes a set of methods for writing assertions against CloudFormation templates. Use one of the Template.fromXxx() static methods to create an instance of this class.

To create Template from CDK stack, start off with:

from aws_cdk.core import Stack
from aws_cdk.assertions import Template

stack = Stack()
# ...
template = Template.from_stack(stack)

Alternatively, assertions can be run on an existing CloudFormation template -

template_json = "{ \"Resources\": ... }"# The CloudFormation template as JSON serialized string.
template = Template.from_string(template_json)

Full Template Match

The simplest assertion would be to assert that the template matches a given
template.

expected = {
    "Resources": {
        "Type": "Foo::Bar",
        "Properties": {
            "Baz": "Qux"
        }
    }
}

template.template_matches(expected)

By default, the templateMatches() API will use the an 'object-like' comparison,
which means that it will allow for the actual template to be a superset of the
given expectation. See Special Matchers for details on how
to change this.

Snapshot testing is a common technique to store a snapshot of the output and
compare it during future changes. Since CloudFormation templates are human readable,
they are a good target for åßsnapshot testing.

The toJSON() method on the Template can be used to produce a well formatted JSON
of the CloudFormation template that can be used as a snapshot.

See Snapshot Testing in Jest and Snapshot
Testing in Java
.

Counting Resources

This module allows asserting the number of resources of a specific type found
in a template.

template.resource_count_is("Foo::Bar", 2)

Resource Matching & Retrieval

Beyond resource counting, the module also allows asserting that a resource with
specific properties are present.

The following code asserts that the Properties section of a resource of type
Foo::Bar contains the specified properties -

expected = {
    "Foo": "Bar",
    "Baz": 5,
    "Qux": ["Waldo", "Fred"]
}
template.has_resource_properties("Foo::Bar", expected)

Alternatively, if you would like to assert the entire resource definition, you
can use the hasResource() API.

expected = {
    "Properties": {"Foo": "Bar"},
    "DependsOn": ["Waldo", "Fred"]
}
template.has_resource("Foo::Bar", expected)

Beyond assertions, the module provides APIs to retrieve matching resources.
The findResources() API is complementary to the hasResource() API, except,
instead of asserting its presence, it returns the set of matching resources.

By default, the hasResource() and hasResourceProperties() APIs perform deep
partial object matching. This behavior can be configured using matchers.
See subsequent section on special matchers.

Output and Mapping sections

The module allows you to assert that the CloudFormation template contains an Output
that matches specific properties. The following code asserts that a template contains
an Output with a logicalId of Foo and the specified properties -

expected = {
    "Value": "Bar",
    "Export": {"Name": "ExportBaz"}
}
template.has_output("Foo", expected)

If you want to match against all Outputs in the template, use * as the logicalId.

expected = {
    "Value": "Bar",
    "Export": {"Name": "ExportBaz"}
}
template.has_output("*", expected)

findOutputs() will return a set of outputs that match the logicalId and props,
and you can use the '*' special case as well.

expected = {
    "Value": "Fred"
}
result = template.find_outputs("*", expected)
expect(result.Foo).to_equal({"Value": "Fred", "Description": "FooFred"})
expect(result.Bar).to_equal({"Value": "Fred", "Description": "BarFred"})

The APIs hasMapping() and findMappings() provide similar functionalities.

Special Matchers

The expectation provided to the hasXxx(), findXxx() and templateMatches()
APIs, besides carrying literal values, as seen in the above examples, also accept
special matchers.

They are available as part of the Match class.

Object Matchers

The Match.objectLike() API can be used to assert that the target is a superset
object of the provided pattern.
This API will perform a deep partial match on the target.
Deep partial matching is where objects are matched partially recursively. At each
level, the list of keys in the target is a subset of the provided pattern.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Fred": {
#           "Wobble": "Flob",
#           "Bob": "Cat"
#         }
#       }
#     }
#   }
# }

# The following will NOT throw an assertion error
expected = {
    "Fred": Match.object_like(
        Wobble="Flob"
    )
}
template.has_resource_properties("Foo::Bar", expected)

# The following will throw an assertion error
unexpected = {
    "Fred": Match.object_like(
        Brew="Coffee"
    )
}
template.has_resource_properties("Foo::Bar", unexpected)

The Match.objectEquals() API can be used to assert a target as a deep exact
match.

Presence and Absence

The Match.absent() matcher can be used to specify that a specific
value should not exist on the target. This can be used within Match.objectLike()
or outside of any matchers.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Fred": {
#           "Wobble": "Flob",
#         }
#       }
#     }
#   }
# }

# The following will NOT throw an assertion error
expected = {
    "Fred": Match.object_like(
        Bob=Match.absent()
    )
}
template.has_resource_properties("Foo::Bar", expected)

# The following will throw an assertion error
unexpected = {
    "Fred": Match.object_like(
        Wobble=Match.absent()
    )
}
template.has_resource_properties("Foo::Bar", unexpected)

The Match.anyValue() matcher can be used to specify that a specific value should be found
at the location. This matcher will fail if when the target location has null-ish values
(i.e., null or undefined).

This matcher can be combined with any of the other matchers.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Fred": {
#           "Wobble": ["Flob", "Flib"],
#         }
#       }
#     }
#   }
# }

# The following will NOT throw an assertion error
expected = {
    "Fred": {
        "Wobble": [Match.any_value(), "Flip"]
    }
}
template.has_resource_properties("Foo::Bar", expected)

# The following will throw an assertion error
unexpected = {
    "Fred": {
        "Wimble": Match.any_value()
    }
}
template.has_resource_properties("Foo::Bar", unexpected)

Array Matchers

The Match.arrayWith() API can be used to assert that the target is equal to or a subset
of the provided pattern array.
This API will perform subset match on the target.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Fred": ["Flob", "Cat"]
#       }
#     }
#   }
# }

# The following will NOT throw an assertion error
expected = {
    "Fred": Match.array_with(["Flob"])
}
template.has_resource_properties("Foo::Bar", expected)

# The following will throw an assertion error
unexpected = Match.object_like(
    Fred=Match.array_with(["Wobble"])
)
template.has_resource_properties("Foo::Bar", unexpected)

Note: The list of items in the pattern array should be in order as they appear in the
target array. Out of order will be recorded as a match failure.

Alternatively, the Match.arrayEquals() API can be used to assert that the target is
exactly equal to the pattern array.

Not Matcher

The not matcher inverts the search pattern and matches all patterns in the path that does
not match the pattern specified.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Fred": ["Flob", "Cat"]
#       }
#     }
#   }
# }

# The following will NOT throw an assertion error
expected = {
    "Fred": Match.not(["Flob"])
}
template.has_resource_properties("Foo::Bar", expected)

# The following will throw an assertion error
unexpected = Match.object_like(
    Fred=Match.not(["Flob", "Cat"])
)
template.has_resource_properties("Foo::Bar", unexpected)

Serialized JSON

Often, we find that some CloudFormation Resource types declare properties as a string,
but actually expect JSON serialized as a string.
For example, the BuildSpec property of AWS::CodeBuild::Project,
the Definition property of AWS::StepFunctions::StateMachine,
to name a couple.

The Match.serializedJson() matcher allows deep matching within a stringified JSON.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Baz": "{ \"Fred\": [\"Waldo\", \"Willow\"] }"
#       }
#     }
#   }
# }

# The following will NOT throw an assertion error
expected = {
    "Baz": Match.serialized_json(
        Fred=Match.array_with(["Waldo"])
    )
}
template.has_resource_properties("Foo::Bar", expected)

# The following will throw an assertion error
unexpected = {
    "Baz": Match.serialized_json(
        Fred=["Waldo", "Johnny"]
    )
}
template.has_resource_properties("Foo::Bar", unexpected)

Capturing Values

This matcher APIs documented above allow capturing values in the matching entry
(Resource, Output, Mapping, etc.). The following code captures a string from a
matching resource.

# Given a template -
# {
#   "Resources": {
#     "MyBar": {
#       "Type": "Foo::Bar",
#       "Properties": {
#         "Fred": ["Flob", "Cat"],
#         "Waldo": ["Qix", "Qux"],
#       }
#     }
#   }
# }

fred_capture = Capture()
waldo_capture = Capture()
expected = {
    "Fred": fred_capture,
    "Waldo": ["Qix", waldo_capture]
}
template.has_resource_properties("Foo::Bar", expected)

fred_capture.as_array()# returns ["Flob", "Cat"]
waldo_capture.as_string()

@nija-at nija-at requested a review from kaizencc October 5, 2021 12:29
assert.hasResourceProperties('Foo::Bar', {
const unexpected = {
Fred: Match.objectLike({
Brew: 'Coffee',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this evaluate to Brew="Coffee" in the Python translation? I feel like it should be a :.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's cuz the translator is misinterpreting the types. aws/jsii#3029

I'll fix them up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed but not happy with the output.

@rix0rrr - I've had to cast many of these structs to Record<string, any> to get the transcoding to work correctly.
Any suggestions on anything I missed, or maybe a cleaner way to do this? This looks a bit ugly on the typescript readme (can even be confusing for customers looking for copy-pasta code snippets)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, don't do that. It's better to fix it upstream.

Copy link
Copy Markdown
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this will be sooo helpful if we can get the Python translation to a usable state. Another thing you'll have to change (even though theres no merge conflict) is that now that absentProperty() -> absent() is merged, the examples with absentProperty() must be renamed as well.

(I meant to start a review and instead clicked "add single comment", so sorry for the weird formatting.)

Niranjan Jayakar added 2 commits October 6, 2021 09:57
@nija-at nija-at force-pushed the nija-at/assertions-readme branch from f59362d to 1bd74a7 Compare October 6, 2021 08:58
@nija-at nija-at changed the title chore(assertions): snippets in README.md is rosetta compatible chore(assertions): snippets in readme are now rosetta compatible Oct 6, 2021
@nija-at nija-at requested a review from kaizencc October 6, 2021 16:26
Copy link
Copy Markdown
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this PR changes this already but the current docs for C# say to use new Struct {... everywhere and that is incorrect. I did a lot of research and found that new Dictionary<string,object>{... works. Not sure if that is an intuitive solution for C# devs, as I am not one. I tried explaining myself in depth in my csharp lambda-cron example here but I fear I did not do an adequate job. Happy to explain my concerns synchronously if they are convoluted in writing.

@nija-at nija-at changed the title chore(assertions): snippets in readme are now rosetta compatible chore(assertions): snippets in readme are now rosetta compilable Oct 7, 2021
This reverts commit 64b02d5.
@kaizencc kaizencc self-requested a review October 7, 2021 16:10
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Oct 7, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@nija-at
Copy link
Copy Markdown
Contributor Author

nija-at commented Oct 11, 2021

Not sure if this PR changes this already but the current docs for C# say to use new Struct {... everywhere and that is incorrect. I did a lot of research and found that new Dictionary<string,object>{... works. Not sure if that is an intuitive solution for C# devs, as I am not one. I tried explaining myself in depth in my csharp lambda-cron example here but I fear I did not do an adequate job. Happy to explain my concerns synchronously if they are convoluted in writing.

This is because the rosetta samples are not compilable. It should get fixed in this PR.

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Oct 11, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 8b99fd3 into master Oct 11, 2021
@mergify mergify bot deleted the nija-at/assertions-readme branch October 11, 2021 12:57
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Oct 11, 2021

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
  • Commit ID: af95162
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

TikiTDO pushed a commit to TikiTDO/aws-cdk that referenced this pull request Feb 21, 2022
…#16801)

Rosetta automatic snippet translation is doing a poor
job of snippet translation. This is mainly because the
snippets are not compilable.

Update snippets to make them compilable and translate
correctly.

Additionally, make the rosetta compilation strict and run
during the build step.

There are still a few more translation errors, but these are
arising from bugs in rosetta.
aws/jsii#3026
aws/jsii#3029

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution/core This is a PR that came from AWS.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants