I did not include an integ test since Spark did not write to s3 even when enabled (for a trivial script). Non trivial scripts usually failed since we did not have the set up for them, or Spark did not write anything.

I verified that the set up in the console is correct, but could not get anything to be written by Spark through manual testing.

Since this is an alpha module (and this issue has gotten very little attention), I think it is fine to wait to add the integ test until a user either says this does not work or provides a script that shows Spark writing logs. There is already an integ test that has Spark enabled, so there is almost no value add for an additional test with the prefix set up, since it is currently functionally the same.

(If we feel strongly that we need one, I have one written and can throw it in!)

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 7189ab7
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).