Add DIFC integrity audit tests: detection failure rate, audit trail, safe_outputs blocking by Copilot · Pull Request #2514 · github/gh-aw-mcpg

Copilot · 2026-03-25T15:49:11Z

The 2026-03-25 integrity filtering audit found two warnings: 8/30 workflow runs (26.7%) had detection failures that caused safe_outputs writes to be skipped, and DIFC audit log access was restricted by secret-scope policy limiting retrospective analysis.

New tests

Detection failure rate (`internal/difc/evaluator_test.go`)

TestEvaluator_FilterCollection_DetectionFailureRate — exercises FilterCollection at the exact 8/30 ratio from the audit; asserts accessible + filtered == total as an invariant across multiple filter rates; includes explicit mode assertion (EnforcementStrict).
TestEvaluator_FilterCollection_FilteredItemsHaveReasons — verifies every filtered item carries a non-empty denial reason for both secrecy and integrity violations (required for audit trail continuity).

Safe_outputs blocking via accumulated secrecy (`internal/difc/agent_test.go`)

TestAgentLabels_AccumulateFromMultipleReads_WriteEvaluation — models the core failure pattern: agent calls AccumulateFromRead on two private repos, accumulating secrecy tags from both; write to a safe_outputs sink is rejected in strict mode when one tag falls outside the accept patterns. Includes passing cases (single covered repo, wildcard accept).

// Agent reads from primary + restricted repo
agent.AccumulateFromRead(primaryRepo)   // adds "private:github/gh-aw"
agent.AccumulateFromRead(secondaryRepo) // adds "private:github/internal-restricted"

// Sink only accepts gh-aw scoped data
safeOutputsResource.Secrecy.Label.Add("private:github/gh-aw")
result := eval.Evaluate(agent.Secrecy, agent.Integrity, safeOutputsResource, OperationWrite)
assert.False(t, result.IsAllowed()) // blocked — internal-restricted not covered

JSONL audit trail (`internal/logger/jsonl_logger_test.go`)

TestLogDifcFilteredItem_NilEntryDoesNotPanic — nil-safety guard.
TestLogDifcFilteredItem_WritesAuditEntryToJSONL — verifies rpc-messages.jsonl entries contain Type=DIFC_FILTERED, Timestamp, Reason, SecrecyTags, and identifying metadata.
TestLogDifcFilteredItem_MultipleEntriesAuditTrail — verifies no entries are dropped when logging multiple filtered items in sequence.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

example.com
- Triggering command: /tmp/go-build4268354515/b329/launcher.test /tmp/go-build4268354515/b329/launcher.test -test.testlogfile=/tmp/go-build4268354515/b329/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true go gbLLs2zQN u/13/cc1 (dns block)
- Triggering command: /tmp/go-build1439825310/b333/launcher.test /tmp/go-build1439825310/b333/launcher.test -test.testlogfile=/tmp/go-build1439825310/b333/testlog.txt -test.paniconexit0 -test.timeout=10m0s -I /opt/hostedtoolcache/go/1.25.8/x64/src/net -I iginal --gdwarf-5 --64 -o iginal 8354�� pkg/mod/github.com/segmentio/asm@v1.1.3/base64/decode_amd64.s pkg/mod/github.com/segmentio/asm@v1.1.3/base64/encode_amd64.s (dns block)
invalid-host-that-does-not-exist-12345.com
- Triggering command: /tmp/go-build4268354515/b314/config.test /tmp/go-build4268354515/b314/config.test -test.testlogfile=/tmp/go-build4268354515/b314/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true -c=4 -nolocalimports -importcfg /tmp/go-build4268354515/b286/importcfg -pack /home/REDACTED/go/pkg/mod/github.com/modelcontextprotocol/go-sdk@v1.4.1/auth/auth.go /home/REDACTED/go/pkg/mod/github.com/modelcontextprotocol/go-sdk@v1.4.1/auth/client.go ortc�� /unix/asm_linux_amd64.s 64/src/internal/asan/doc.go 64/pkg/tool/linux_amd64/compile (dns block)
- Triggering command: /tmp/go-build1439825310/b318/config.test /tmp/go-build1439825310/b318/config.test -test.testlogfile=/tmp/go-build1439825310/b318/testlog.txt -test.paniconexit0 -test.timeout=10m0s (dns block)
nonexistent.local
- Triggering command: /tmp/go-build4268354515/b329/launcher.test /tmp/go-build4268354515/b329/launcher.test -test.testlogfile=/tmp/go-build4268354515/b329/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true go gbLLs2zQN u/13/cc1 (dns block)
- Triggering command: /tmp/go-build1439825310/b333/launcher.test /tmp/go-build1439825310/b333/launcher.test -test.testlogfile=/tmp/go-build1439825310/b333/testlog.txt -test.paniconexit0 -test.timeout=10m0s -I /opt/hostedtoolcache/go/1.25.8/x64/src/net -I iginal --gdwarf-5 --64 -o iginal 8354�� pkg/mod/github.com/segmentio/asm@v1.1.3/base64/decode_amd64.s pkg/mod/github.com/segmentio/asm@v1.1.3/base64/encode_amd64.s (dns block)
slow.example.com
- Triggering command: /tmp/go-build4268354515/b329/launcher.test /tmp/go-build4268354515/b329/launcher.test -test.testlogfile=/tmp/go-build4268354515/b329/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true go gbLLs2zQN u/13/cc1 (dns block)
- Triggering command: /tmp/go-build1439825310/b333/launcher.test /tmp/go-build1439825310/b333/launcher.test -test.testlogfile=/tmp/go-build1439825310/b333/testlog.txt -test.paniconexit0 -test.timeout=10m0s -I /opt/hostedtoolcache/go/1.25.8/x64/src/net -I iginal --gdwarf-5 --64 -o iginal 8354�� pkg/mod/github.com/segmentio/asm@v1.1.3/base64/decode_amd64.s pkg/mod/github.com/segmentio/asm@v1.1.3/base64/encode_amd64.s (dns block)
this-host-does-not-exist-12345.com
- Triggering command: /tmp/go-build4268354515/b338/mcp.test /tmp/go-build4268354515/b338/mcp.test -test.testlogfile=/tmp/go-build4268354515/b338/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/runtime/c-errorsas /yaml.v3@v3.0.1/-ifaceassert .13/x64/bin/as (dns block)
- Triggering command: /tmp/go-build1439825310/b342/mcp.test /tmp/go-build1439825310/b342/mcp.test -test.testlogfile=/tmp/go-build1439825310/b342/testlog.txt -test.paniconexit0 -test.timeout=10m0s pkg/�� -unreachable=false /tmp/go-build4268354515/b145/vet.cfg 95da714248d8dc69-d /tmp/go-build426/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -dumpbase _x006.c 64/pkg/tool/linux_amd64/vet -uns�� -unreachable=false /difc/difc_test.go 8354515/b334/vet.cfg go --local 64/pkg/tool/linu--version /opt/hostedtoolcache/go/1.25.8/x-extld=gcc (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.

…blocking Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/50865aa5-dd53-4d62-9b5d-7083dc44330d

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/50865aa5-dd53-4d62-9b5d-7083dc44330d

Copilot

Pull request overview

Adds targeted test coverage for DIFC auditing/integrity scenarios, ensuring filtered-item audit trails and label-accumulation behaviors are exercised and regression-protected.

Changes:

Add JSONL logger tests for DIFC filtered-item audit entries (nil safety + single/multi-entry continuity).
Add DIFC evaluator tests for detection-failure-rate invariants and “filtered items always have reasons”.
Add agent label-accumulation tests validating write acceptance/denial based on accumulated secrecy tags.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
internal/logger/jsonl_logger_test.go	Adds tests validating `LogDifcFilteredItem` JSONL output and continuity across multiple entries.
internal/difc/evaluator_test.go	Adds tests for FilterCollection count invariants and non-empty denial reasons on filtered items.
internal/difc/agent_test.go	Adds tests for secrecy accumulation across reads and subsequent write evaluation to safe outputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T16:08:45Z

internal/difc/agent_test.go

+	// Writes are blocked by DIFC in all enforcement modes (strict, filter, propagate).
+	// We use strict mode here to make the intent explicit.


The comment says “Writes are blocked by DIFC in all enforcement modes”, but writes are only denied when a write violates integrity/secrecy flow checks; valid writes are still allowed. Please reword to something like “Write violations are blocked in all modes (no propagation for writes)” to avoid misleading future readers.

Suggested change

// Writes are blocked by DIFC in all enforcement modes (strict, filter, propagate).

// We use strict mode here to make the intent explicit.

// Write violations are blocked by DIFC in all enforcement modes (strict, filter, propagate);

// valid writes are still allowed, and there is no propagation for writes. We use strict mode

// here to make the intent explicit.

Copilot · 2026-03-25T16:08:45Z

internal/difc/evaluator_test.go

+		// Agent has integrity "approved"; resource has no integrity → resource ⊄ agent
+		// so the read is denied in strict mode.


This integrity comment uses an incorrect set relation (“resource ⊄ agent”). In evaluateRead, the check is resource.Integrity.CheckFlow(agentIntegrity), meaning the resource must include all integrity tags the agent requires (resource ⊇ agent requirement). Please adjust the comment to reflect the actual integrity-flow semantics so the test remains self-explanatory.

Suggested change

// Agent has integrity "approved"; resource has no integrity → resource ⊄ agent

// so the read is denied in strict mode.

// Agent requires integrity "approved"; resource has no integrity → resource ⊅ agent requirement,

// so the read is denied in strict mode (resource lacks required integrity tags).

Initial plan

c3dc4b6

Copilot AI assigned Copilot and lpcox Mar 25, 2026

Copilot started work on behalf of lpcox March 25, 2026 15:49 View session

Copilot AI linked an issue Mar 25, 2026 that may be closed by this pull request

[integrity-audit] Integrity Filtering Audit — github/gh-aw (2026-03-25) #2502

Closed

Add DIFC integrity audit tests: filter rate, audit log, safe_outputs …

384778a

…blocking Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/50865aa5-dd53-4d62-9b5d-7083dc44330d

lpcox marked this pull request as ready for review March 25, 2026 16:04

Copilot AI review requested due to automatic review settings March 25, 2026 16:04

Address code review: use strict mode and add mode assertion

8e7d860

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw-mcpg/sessions/50865aa5-dd53-4d62-9b5d-7083dc44330d

Copilot started reviewing on behalf of lpcox March 25, 2026 16:05 View session

Copilot AI changed the title ~~[WIP] Audit integrity filtering and resolve warnings~~ Add DIFC integrity audit tests: detection failure rate, audit trail, safe_outputs blocking Mar 25, 2026

Copilot AI requested a review from lpcox March 25, 2026 16:06

Copilot finished work on behalf of lpcox March 25, 2026 16:06

lpcox merged commit dabbd6e into main Mar 25, 2026
24 checks passed

lpcox deleted the copilot/integrity-audit-filters-analysis branch March 25, 2026 16:07

Copilot AI reviewed Mar 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DIFC integrity audit tests: detection failure rate, audit trail, safe_outputs blocking#2514

Add DIFC integrity audit tests: detection failure rate, audit trail, safe_outputs blocking#2514
lpcox merged 3 commits intomainfrom
copilot/integrity-audit-filters-analysis

Copilot AI commented Mar 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// Writes are blocked by DIFC in all enforcement modes (strict, filter, propagate).
		// We use strict mode here to make the intent explicit.

-	// Writes are blocked by DIFC in all enforcement modes (strict, filter, propagate).
-	// We use strict mode here to make the intent explicit.
+	// Write violations are blocked by DIFC in all enforcement modes (strict, filter, propagate);
+	// valid writes are still allowed, and there is no propagation for writes. We use strict mode
+	// here to make the intent explicit.

		// Agent has integrity "approved"; resource has no integrity → resource ⊄ agent
		// so the read is denied in strict mode.

Conversation

Copilot AI commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New tests

Detection failure rate (internal/difc/evaluator_test.go)

Safe_outputs blocking via accumulated secrecy (internal/difc/agent_test.go)

JSONL audit trail (internal/logger/jsonl_logger_test.go)

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 25, 2026 •

edited

Loading

Detection failure rate (`internal/difc/evaluator_test.go`)

Safe_outputs blocking via accumulated secrecy (`internal/difc/agent_test.go`)

JSONL audit trail (`internal/logger/jsonl_logger_test.go`)