Use dependency cooldown, pinned actions, and more code scanning#2337
Use dependency cooldown, pinned actions, and more code scanning#2337EliahKagan merged 15 commits intoGitoxideLabs:mainfrom
Conversation
This causes CodeQL to scan for more problems in both Rust code and in GitHub Actions workflows. `security-extended` includes the `default` pack, so this is only adding queries, not removing any. Using `security-extended` will create about 30 new alerts for GitHub Actions `uses:` keys whose values are unpinned actions, i.e., (non-immutable) tags rather than OIDs. Most of these will go away in a forthcoming change that pins everything that is currently feasible to pin easily (and thus may or may not ever be raised for the main branch), but a few will remain since we have some use of unpinned actions that may not be immediately pinnable. Using `security-and-quality` will create 1 new alert of the "note" severity identifying an unused variable. This might be a false positive (the variable is unused, but it is marked as deliberately unused), though we should verify that it doesn't identify an area of possible improvement before dismissing it. But since it's just one false positive, the `security-and-qualty` suite is probably reasonable to enable at this time. No CodeQL alerts due to `security-extended` are anticipated in Rust code due to this change, and no CodeQL alerts due to `security-and-quality` are expected in GitHub Actions workflows due to this change. These predictions are based on recent testing where these query suites were enabled in a fork.
To scan and report using GitHub Advanced Security, so alerts appear in the Security tab, as with CodeQL. This is the workflow for that setup currently shown in the readme for https://github.com/zizmorcore/zizmor-action. Subsequent commits will customize it.
- Top-level empty permissions in the CodeQL workflow with a comment about them being overridden. This slightly improves clarity but the main benefit is to avoid unnecessary inconsistency between the CodeQL and Zizmor workflows. - Comment the existing top-level empty permissions in the Zizmor workflow, so it's clear the job runs with greater capabilities. - Comment to explain why extra read permissions that aren't needed in public repositories are *not* being conferred in the CodeQL workflow. - Comment to explain why those same extra read permissions that aren't needed in public repositories *are* being conferred in the Zizmor workflow.
This runs in more branches, but on fewer PRs (since it runs on the branches themselves, this isn't necessary for PRs other than with `main` as the base branch). This is more consistent with our other workflows. It also matches our CodeQL workflow's event triggers, except this does not add a scheduled job, which may not be needed because Zizmor queries don't come from extra packs where they are added even within the exact same Zizmor version. There could be a benefit to including a scheduled run, since Zizmor does scan for some conditions that can emerge in the absence of any change to the code in the repository or to its queries: some of its scans may examine actions configuration set in repo configuration rather than repo contents. That's a thing to look into in the future, but it's probably not essential since Zizmor is doing full workflow scans every time it runs and it will run at least as often as every push to main (and this repository is active).
To somewhat decrease the risk of supply chain attacks, this pins a number of actions used in CI workflows, so that they give full SHA-1 OIDs rather than tags. This was done with `pinact`, by running `pinact run`. Dependabot will manage updates to them, bumping hashes and version comments after a new version tag is available. In most cases these are major.minor.patch tags, which update much more often than major version tags. (In contravention of usual tagging practices, major version tags for GHA actions are usually *moved* whenever a new minor or patch version is released.) This could significantly increase the frequency of Dependabot version updates. Therefore, the cadence is also changed from weekly to monthly. This applies only to GitHub Actions version updates; Rust (`cargo`) version updates already had a monthly cadence configured. Not all versions are pinned here. Throughout our CI jobs, we are using multiple strategies for installing Rust toolchains and build targets, chosen according to the specifics of the job. But we often use `dtonlay/rust-toolchain`. This has `master` and `stable` branches, a well as branches corresponding to specific Rust versions, but it does not offer up-to-date tags (nor any tags corresponding to Rust versions). See dtolnay/rust-toolchain#160 on this limitation and its effect on GitHub Actions pinning. Various other actions exist, and we can also install toolchains manually. However, the functionality impact and supply chain security risks associated with those approaches should be examined before making a change. Therefore, while I intend to move away from using `dtolnay/rust-toolchain` soon, I'm deferring that slightly. (The benefit of pinning OIDs for `actions/checkout` is probably small compared to using a full major.minor.patch version tag, because that action uses immutable version tags and is maintained by GitHub. However, we weren't using major.minor.version tags for it before, and doing so has the same update cadence consquences as pinning this way. Furthermore, more tools are able to recognize that OID pinning is safe than are able to recognize that immutable release tags may be acceptably safe.) In addition to its modest security benefit, this change resolves or avoids new security scanning alerts from CodeQL, due to enabling `security-extended` queries; and from Zizmor, which was recently added. The remaining unpinned actions alerts track areas that should be improved. The fuzzing workflow uses the `master` branch for its actions. It's unclear if this can or should be improved, but we'll have alerts for that too, to allow it to be tracked. Some discussions in the `oss-fuzz` repository suggest this cannot reasonably be done; for example, see google/oss-fuzz#6836. The OID for `actions/checkout` in the Zizmor workflow itself is deliberately not bumped manually from 6.0.0 to 6.0.1. This is to allow the ability of Dependabot to update actions specified in this form to be immediately verfied upon integrating these changes. All other pinned action versions are intended to be at their latest non-prerelease versions, as of the time of this change. Most comments beginning after code on the same line in CI workflows in this repository have been written with two spaces before the `#` character. That continues to be followed for YAML comments *except* for the version comments that immediately follow pinned OIDs. This is because the exact format of an OID folllowed by one space followed by `#` followed by one space followed by a specific verson tag name is near-universally used for this purpose, and some automated tools expect that exact pattern. Whenever making manual changes to pinned actions OIDs or reviewing changes to them (if the changes may be from an untrusted party), it is essential to verify that the OID corresponds to the exact tag it purports to, and also that the tag is in the intended repo rather than (only) in another repo in its fork network. All repos in a fork network effectively share an object database. A commit in a malicious fork could therefore be referenced by upstream OIDs.
Zizmor heuristically attempts to identify CI workflows that may be vulnerable to cache poisoning attacks, by checking for the presence of known caching steps together with indications that a workflow may be used for purposes such as cutting new releases. Workflows whose event triggers specify tags are often used for releases. Zizmor currently takes the presence of either a `tags` or `tags-ignore` subkey of `on.push` as an indication that this is being done. Prior to the change in this commit, that resulted in 7 false positive reports from Zizmor for cache poisoning. The `tags-ignore` key in `ci.yml` was introduced in 42e0487. It has no effect (at least currently, though perhaps also when it was introduced). This is because that workflow also has a `branches` key. The presence of `branches` key without a `tags` key causes a workflow not to be triggered on tag pushes, even without `tags-ignore`. See https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onpushbranchestagsbranches-ignoretags-ignore which notes that: > If you define only `tags`/`tags-ignore` or only > `branches`/`branches-ignore`, the workflow won't run for events > affecting the undefined Git ref. If you define neither > `tags`/`tags-ignore` or `branches`/`branches-ignore`, the > workflow will run for events affecting either branches or tags.
This configures Dependabot to attempt to refrain from upgrading dependencies to versions that have been released less than a week before. This is a measure to somewhat decrease the supply chain attack surface. The hope is that, in the intervening time, someone may hopefully have noticed the introduction of malicious code and raised alarm. Even when implemented robustly, cooldown periods are only one piece of a strategy for mitigating supply chain attacks. This configuration is only expected to achieve a 7-day cooldown period for some dependencies. It is introduced based on the belief that is is good enough to be worth doing, but with the hope of further improvement in the future. The main limitation is that `cargo` does not yet support a feature to specify a mininum age for dependencies resolved in `Cargo.lock`. This feature will likely be implementd in the future; see rust-lang/cargo#15973. Due to its absence, Dependabot may not be able to limit locked depdenencies to those that are at least a week (or other cooldown period) old. However, Dependabot will most likely manage to enforce the cooldown period for dependency specifications it updates in `Cargo.toml` files during Dependabot version updates. This provides a meaningful amount of protection; for example, it makes it less likely that if we do not manage to avoid a supply chain attack then we may still avoid passing on a requirement to use a malicious dependency to our own dependencies. Another significant limitation is that only some ways of resolving dependencies in GitHub Actions workflows are protected even in principle: - Only actions pinned with OIDs, or exact version tags corresponding to immutable releases, are well protected. Those specifying branch names are not protected at all. (Many of these are eliminated, but we continue to use `dtolnay/rust-toolchain` in a number of places, which doesn't currently offer tags usable for this purpose.) - The GitHub Actions ecosystem lacks lockfiles and most other supply chain security features common to other dependency ecosystems. See: https://nesbitt.io/2025/12/06/github-actions-package-manager.html. - We ourselves specify software versions in some of our workflows in ways that don't use pinning, such as installing the latest versions of some software by downloading scripts from unversioned URLs and running them, and in how we currently reference Docker images. Nonetheless, I think a best-effort dependency cooldown, as configured here, is an overall improvement. This change is the primary motivation for the changes related to supply chain security in recent commits. Specifically: - Pinning actions facilitates cooldown periods, as described above. - Enabling more CodeQL queries includes queries that flag unpinned actions. - Adding Zizmor scanning flags unpinned actions and also flags the absence of a Dependabot cooldown period of at least 7 days. Note that the cooldown period (intentionally) applies only to Dependabot version updates, not to Dependabot security updates, and also not to updates performed in ways not involving Dependabot.
This switches from the implicit `regular` persona to the `pedantic` persona, for Zizmor scanning. Some of the new diagnostics from the `pedantic` persona may help improve security, but two fo them don't seem to be necessary or helpful anywhere in this repository given the style and usage of our workflows: `anonymous-definition` and `concurrency-limits`. So this turns off those two checks.
This replaces the expansion of `${{ }}` GitHub Actions templates in
shell commands with assignment to environment variables and shell
expansion of the environment variables. This has a few benefits,
though the third is marginal:
1. It satisfies `template-injection` diagonistics from Zizmor while
allowing them to continue to be run. None of the templates used
here were vulnerable to template injection attacks, because they
never placed untrusted data (nor data originating from anywhere
other than a nearby hard-coded source) into a command. However,
changing the style is preferable over forgoing the diagnostic
or ignoring it, and clearer than selectively suppressing it.
2. Whether splitting, globbing, or neither is intendended is now
clear. When neither is intended, the shell expansion is double
quoted. Otherwise the expansion is unquoted and the reason for it
being unquoted is given in a comment. Although it's possible to
place quotation marks around `${{ }}` strings, this is only
occasionally correct, since quotation marks injected inside them
are not neutralized; in contrast, in shell expansions so long as
neither `eval` nor a similar feature is used, injected quotation
marks are neutralized automatically.
(There's in principle a subtlety here: field splitting, also
called word splitting, occurs on the result of shell expansions;
in contrast, the form of splitting that happens on `${{ }}`
template text, whose value is inserted directly into a script
file on the runner, is the parsing step of breaking up a command
into words and operators. The latter has never been equivalent
to the former, and in all modern Bourne-style shells it is
moreover unaffected by `$IFS`. But the distinction between the
two is not important to any expansions being modified here.)
3. For security reasons, this style was already followed in the
jobs in the release workflow that perform steps that can affect
the content of releases or the way they are presented. So doing
it elsewhere improves consistency across jobs and workflows.
This also breaks up a few affected or nearby long lines to make
them more readable.
This is equivalent to what we had before, but it is more scrutable to Zizmor, fixing a false positive in `pedantic` mode, and maybe slightly eaiser for human readers to understand too.
This satsifies the Zizmor diagnositc `undocumented-permissions` and makes clearer what is going on (other than in the case of why the annoucement job needs `discussions: write`, but giving the obvious explanation in a comment is fine and clearer than suppressinfg the check).
But not for `dtolnay/rust-toolchain`, since there are alternatives that should be explored first (including both other actions and the option of doing it manually with our own shell commands). This only affects Zizmor. CodeQL will still report these, though they could be dismissed in the Security tab.
|
This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation. |
| - name: Build Fuzzers | ||
| id: build | ||
| uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master | ||
| uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master # zizmor: ignore[unpinned-uses] |
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium
|
|
||
| - name: Run Fuzzers | ||
| uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master | ||
| uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master # zizmor: ignore[unpinned-uses] |
Check warning
Code scanning / CodeQL
Unpinned tag for a non-immutable Action in workflow Medium
GitHub Actions uses the `RUNNER_ARCH` variable, described in: https://docs.github.com/en/actions/reference/workflows-and-actions/variables That causes the attempt to use a `RUNNER_ARCH` variable to hold the value of the `runner-arch` matrix variable in the `test-32bit` jobs to fail, because the GitHub Actions value is used instead. These are conceptually related, but in a different form. For example, an effect of the clash is for `X64` to be set instead of `arm64`. The `dpkg` and `apt-get` commands in the container expect the latter. This fixes the bug by renaming the variable `RUNNER_ARCHITECTURE`. To avoid confusion, related matrix variables `runner-arch` (from (which `RUNNER_ARCH` was populated) and `container-arch` are also renamed (to `runner-architecture` and `container-architeture).
The `TEST_ARGS` environment variable needs to be split. It was
introduced to avoid a `${{ matrix.test-args }}` interpolation
directly into a shell script (where the intention for how many
arguments to pass would be unclear, and where automated tooling
has trouble identifying that it's not a template injection
vulnerability). The problem is that the syntax used, unquoted
`$TEST_ARGS`, works on `bash` but not `pwsh`. Likewise, the
line-contination introduced along with it was written with a
backslash, but PowerShell uses a backtick.
This "fixes" the problem by writing shell-specific code, but in one
step that means duplicating the step. This therefore needs to be
refactored, by replacing this fix with a completely different one.
The purpose of fixing it this way is really just to verify the bug.
A better fix might entail going back to interpolating `${{ }}` into
shell scripts in the few cases that this is by far the simplest way
to do get the needed effect portably across shells.
Usually the solution is to just force the same shell on all OSes,
such as with `shell: bash`, and use the syntax of the chosen shell.
But `bash` and `pwsh` can give subtly different environments, which
is why we allow the default to be used in `nextest` runs,
especially on Windows. Thus, that usual solution is not available.
Tooling has a harder time figuring out that this does not produce a template injection vulnerability, but this is much simpler than any other readily available approach for doing the splitting in both shells. It's important that we run the tests on `pwsh` in Windows, and preferable to not run them in `pwsh` on other systems. See the preceding commit for context. This suppreses the Zizmor diagonstic inline. It may be possible to make this more elegant in the future.
|
There were a couple unexpected bugs here, due to me making mistakes when editing #2338 probably doesn't need to block this PR (nor its sequel, nor integrating changes from #2336). I can make a test branch that modifies I haven't done that quite yet, though, so this remains a draft for the moment. |
EliahKagan
left a comment
There was a problem hiding this comment.
Modifying the release workflow on a separate testing branch from this one, with an added commit that comments out all the powerpc64le builds, succeeeds.
I don't think that actual change, which was just for testing, is a good solution to #2338, because even if we end up having to release fewer binaries, I don't think all such builds break; it's probably just max-pure. (I'm not 100% certain due to fail-fast.)
But for the purpose of assessing whether the changes in this PR have additionally broken release.yml, I think this is sufficient evidence that they haven't. WIth that taken together with how CI here is passing, I think this is ready to merge.
I had failed to remove an `if` guard that was meant to be removed in a refactoring, in c1a3a99 (GitoxideLabs#2337). That made it so `test-fast` skips running the tests on non-Windows systems and just reports success! This fixes that serious CI bug by removing the left-over `if:`.
This sets a cooldown period (minimum relese age) of one week for Dependabot updates. This can be helpful as one piece of a strategy for mitigating supply chain attacks. This policy may not be fully effective even in preventing all such updates in `Cargo.lock` until rust-lang/cargo#15973 is implemented. This is analogous to a recent Dependabot configuration change in GitoxideLabs/gitoxide#2337. See details and further caveats in: GitoxideLabs/gitoxide@6e0dde1
Dependency cooldowns
This configures a 7-day cooldown period for Dependabot version updates, to try to refrain from upgrading dependencies until at least week after they are been published. This is as a partial defense against supply chain attacks. See 6e0dde1 for details, including limitations.
The limitations are considerable. They include that
cargodoes not yet implement this functionality for dependency resolution in lockfiles, as tracked in rust-lang/cargo#15973. Thus, I expect the cooldown period will not even be effective, for locked dependencies. Still, I think it is worthwhile to enable the feature at this time.This configuration change intentionally does not affect Dependabot security updates.
GitHub Actions pinning
This PR also makes a number of other changes, which are mostly to support dependency cooldowns and help them work better (but which carry some other benefits).
The most closely related is that it pins a number of our actions to full OIDs, with comments indicating major.minor.patch versions, which Dependabot is able to update; this both defends against some kinds of supply chain attacks in its own right, wherein an attacker who gains control of an action dependency would force-push a tag, and allows Dependabot cooldowns to work better since there is a specific version being awaited.
This accordingly adjusts the cadence of Dependabot version updates for actions so we don't get overwhelmed by them in the face of that change.
More static analysis
This also expands the use of static analyis. It enables more CodeQL queries, for both Rust and GitHub Actions code, than we had been using before. It sets up Zizmor, another static analyzer also focusing on identifying security vulnerabilities, to scan GitHub Actions workflows and
dependabot.yml.Both of these alert us about more GitHub Actions weaknesses we could have it our workflows than we had tooling alerting us about before. This includes informing us of any actions we're using without pinned OIDs (both CodeQL and Zizmor), checking that a cooldown period is configured (Zizmor), and various other checks.
This also modifies workflows for clarity as well as to avoid patterns that are hard for automated tools to tell apart from vulnerabilities (to allow more scanning to be enabled).
Testing and outscoping
This PR deliberately sets one version (and corresponding OID) to a previous patch version (of an acton whose latest patch release is more than a week old), to allow us to verify that Dependabot is actually able to update our actions with them specified in this different way. It also defers eliminating
dtolnay/rust-toolchain, which doesn't have specific version tags and thus can't be pinned and updated in the above-described way, as tracked in dtolnay/rust-toolchain#160.All our current uses of
dtolnay/rust-toolchain, as well as all near-future uses I foresee, are actually readily replaceable with a small number ofrustupcommands, and I've verified in my fork that this seems to work. But I'm deferring that change to the next PR (which I hope to open shortly after this one), so I can make sure alerts from Zizmor and from newly configured CodeQL queries work (we should get alerts now for each use of that action, as we are using it), and to make it easier to revert the change in case there is some subtlety I haven't identified that keeps the simplerustup-based alternative from working.Plan
After merging this PR and the forthcoming PR to replace uses of
dtolnay/rust-toolchain, I plan to integrate changes corresponding to those in #2336. The Dependabot reconfiguration in this PR will cause that PR to be closed and a new Dependabot PR to be opened. The new PR won't be exactly equivalent, becauese it will attempt to upgrade most but not all of the dependencies #2336 upgrades, due to the cooldown period being in effect.Integrating the changes in this order lets me inspect the effect of the cooldown period on a large Dependabot PR where the effects should be observable. The commmits we've added there will still be relevant, since they adapt to changes introduced by versions that are more than a week old; I'll move them over to the newly created Dependabot PR.
This PR remains a draft until I've tested the release workflow in my fork to make sure I didn't accidentally introduce typos that wouild somehow break releasing.