Skip to content

chore: slsa-framework/slsa-github-generator@v2.0.0: add testdata#758

Merged
ramonpetgrave64 merged 3 commits intoslsa-framework:mainfrom
ramonpetgrave64:v2.0.0-testdata
Apr 23, 2024
Merged

chore: slsa-framework/slsa-github-generator@v2.0.0: add testdata#758
ramonpetgrave64 merged 3 commits intoslsa-framework:mainfrom
ramonpetgrave64:v2.0.0-testdata

Conversation

@ramonpetgrave64
Copy link
Copy Markdown
Contributor

@ramonpetgrave64 ramonpetgrave64 commented Apr 23, 2024 •
edited
Loading

slsa-framework/slsa-github-generator#3576

Next step in
https://github.com/slsa-framework/slsa-github-generator/blob/main/RELEASE.md#update-verifier

Creating new test data for slsa-github-generator@v2.0.0

Instructions:

diff to download-artifacts.sh

diff --git a/download-artifacts.sh b/download-artifacts.sh
old mode 100644
new mode 100755
index e5e218e8..49257ea6
--- a/download-artifacts.sh
+++ b/download-artifacts.sh
@@ -88,6 +88,10 @@ unzip_files() {
         rm -rf "${tmp_dir}"
         ;;
 
+    ./*.zip)
+        unzip -o "${zip_path}" -d "${output_path}"
+        ;;
+
     *)
         echo "unexpected file path: ${zip_path}"
         exit 1
@@ -167,7 +171,7 @@ rename_java_files "test-java-project-" "maven"
 rename_java_files "workflow_dispatch-" "gradle"
 
 # Files downloaded. Now copy them
-repo_path="../.."
+repo_path="/path/to/slsa-verifier"
 
 # Go builder files.
 copy_files "gha_go-binary-linux-amd64-" "${repo_path}/cli/slsa-verifier/testdata/gha_go/${version}"

download the artifacts

../slsa-verifier/download-artifacts.sh 8791212155 v2.0.0
../slsa-verifier/download-artifacts.sh 8791219359 v2.0.0
../slsa-verifier/download-artifacts.sh 8791219514 v2.0.0
../slsa-verifier/download-artifacts.sh 8791219607 v2.0.0

docker github auth

gh auth login --scopes=read:packages
echo `gh auth token` | docker login ghcr.io -u ramonpetgrave64 --password-stdin
cosign save \
    --dir ./cli/slsa-verifier/testdata/gha_generic_container/v2.0.0/container_workflow_dispatch \
    ghcr.io/slsa-framework/example-package.verifier-e2e.all.tag.main.default.slsa3@sha256:55aee984fd6b1d0e0a19a55265d10d40063a2212bdbabd75b202b1728236548d

@ramonpetgrave64
Copy link
Copy Markdown
Contributor Author

ramonpetgrave64 commented Apr 23, 2024 •
edited
Loading

Test failing
https://github.com/slsa-framework/slsa-verifier/actions/runs/8801653756/job/24155532842?pr=758#step:6:6

2024-04-23T14:07:06.0587137Z Verified build using builder "https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/tags/v1.9.0" at commit 2bcaa7495e1cbd11fbd4f598d857b3a6f18df933
2024-04-23T14:07:06.0588248Z --- FAIL: Test_runVerifyGHAArtifactImage (0.00s)
2024-04-23T14:07:06.0589075Z     --- FAIL: Test_runVerifyGHAArtifactImage/versioned_tag_no_match_empty_tag_workflow_dispatch_>_v1.9.0 (0.11s)
2024-04-23T14:07:06.0590280Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0591552Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0592595Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0593057Z               )
2024-04-23T14:07:06.0593410Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0594628Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0595643Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0596275Z               )
2024-04-23T14:07:06.0596608Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0597812Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0598825Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0599256Z               )
2024-04-23T14:07:06.0599595Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0600780Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0601796Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0602235Z               )
2024-04-23T14:07:06.0602726Z     --- FAIL: Test_runVerifyGHAArtifactImage/valid_main_branch_default (0.43s)
2024-04-23T14:07:06.0603295Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0604502Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0605250Z               )
2024-04-23T14:07:06.0605568Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0606747Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0607480Z               )
2024-04-23T14:07:06.0607799Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0608982Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0609714Z               )
2024-04-23T14:07:06.0610174Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0611367Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0612102Z               )
2024-04-23T14:07:06.0612719Z     --- FAIL: Test_runVerifyGHAArtifactImage/tag_no_match_empty_tag_workflow_dispatch_>_v1.9.0 (0.12s)
2024-04-23T14:07:06.0613367Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0614543Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0615546Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0615975Z               )
2024-04-23T14:07:06.0616326Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0617525Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0618545Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0618980Z               )
2024-04-23T14:07:06.0619307Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0620496Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0621497Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0621926Z               )
2024-04-23T14:07:06.0622252Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0623429Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0624434Z             + 	e"tag used to generate the binary does not match provenance",
2024-04-23T14:07:06.0625117Z               )
2024-04-23T14:07:06.0625598Z     --- FAIL: Test_runVerifyGHAArtifactImage/wrong_branch_master (0.58s)
2024-04-23T14:07:06.0626149Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0627514Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0628542Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0628986Z               )
2024-04-23T14:07:06.0629314Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0630504Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0631527Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0631974Z               )
2024-04-23T14:07:06.0632299Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0633487Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0634516Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0634967Z               )
2024-04-23T14:07:06.0635293Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0636473Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0637498Z             + 	e"branch used to generate the binary does not match provenance",
2024-04-23T14:07:06.0637941Z               )
2024-04-23T14:07:06.0638404Z     --- FAIL: Test_runVerifyGHAArtifactImage/valid_main_branch_set (0.52s)
2024-04-23T14:07:06.0638951Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0640306Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0641055Z               )
2024-04-23T14:07:06.0641381Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0642599Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0643335Z               )
2024-04-23T14:07:06.0643654Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0644836Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0645566Z               )
2024-04-23T14:07:06.0645882Z         main_regression_test.go:869:   any(
2024-04-23T14:07:06.0647058Z             - 	e"expected hash 'a3e4bf251423a455ad90c3d706f95f133ed11a8e81e3f34e6fa6d056a1c15529' not found: artifact hash does not match provenance subject",
2024-04-23T14:07:06.0647794Z               )

@ramonpetgrave64
add testdata
b8f6ecd 
Signed-off-by: Ramon Petgrave <ramon.petgrave64@gmail.com>
@ramonpetgrave64
revert from main cli/slsa-verifier/testdata/gha_generic_container/v1.…
a82387e 
…10.0/container_workflow_dispatch.digest

Signed-off-by: Ramon Petgrave <ramon.petgrave64@gmail.com>
@ramonpetgrave64
fix the hash
ac533a9 
Signed-off-by: Ramon Petgrave <ramon.petgrave64@gmail.com>
@ramonpetgrave64
Copy link
Copy Markdown
Contributor Author

ramonpetgrave64 commented Apr 23, 2024

Now passing. I had recorded the incorrect hash.

@ramonpetgrave64 ramonpetgrave64 marked this pull request as ready for review April 23, 2024 14:49
@ramonpetgrave64
Copy link
Copy Markdown
Contributor Author

ramonpetgrave64 commented Apr 23, 2024

@laurentsimon @kpk47

laurentsimon
laurentsimon approved these changes Apr 23, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@laurentsimon laurentsimon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks!

@ramonpetgrave64 ramonpetgrave64 merged commit 637b07f into slsa-framework:main Apr 23, 2024
ramonpetgrave64 added a commit to ramonpetgrave64/slsa-verifier that referenced this pull request Apr 25, 2024
@ramonpetgrave64
Revert "chore: slsa-framework/slsa-github-generator@v2.0.0: add testd…
e6d63c6 
…ata (slsa-framework#758)"

This reverts commit 637b07f.
@ramonpetgrave64 ramonpetgrave64 mentioned this pull request Feb 25, 2025
Merged
ramonpetgrave64 added a commit that referenced this pull request Feb 27, 2025
@ramonpetgrave64
chore: update test files for v2.1.0 (#836)
b53bd94 
Similar to #758, we are updating the test files.

Errors for checking the tag in attestations are slightly different. Unit
tests are adjusted with the new test cases.

---------

Signed-off-by: Ramon Petgrave <ramon.petgrave64@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@laurentsimon laurentsimon laurentsimon approved these changes

@kpk47 kpk47 Awaiting requested review from kpk47

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ramonpetgrave64 @laurentsimon