Skip to content

Commit 7532d22

Browse files
CopilotMossaka
Copilot
and
Mossaka
committed
chore: merge main and change behavior to block all when no domains
- Merged latest changes from main branch (v0.6.0) including log management features - Changed behavior: when no domains specified, all traffic is now BLOCKED (option 2) - Updated warning messages to reflect blocking behavior - Updated tests to match new blocking behavior - All 350 tests passing Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
1 parent 0305e94 commit 7532d22

23 files changed

Lines changed: 2666 additions & 61 deletions

‎README.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,24 @@ sudo awf \
5252

5353
**Note:** Always use the `--` separator to pass commands and arguments. This ensures proper argument handling and avoids shell escaping issues.
5454

55+
### Log Viewing
56+
57+
View Squid proxy logs from current or previous runs:
58+
59+
```bash
60+
# View recent logs with pretty formatting
61+
awf logs
62+
63+
# Follow logs in real-time
64+
awf logs -f
65+
66+
# View logs in JSON format for scripting
67+
awf logs --format json
68+
69+
# List all available log sources
70+
awf logs --list
71+
```
72+
5573
## Domain Whitelisting
5674

5775
Domains automatically match all subdomains:

‎docs-site/src/content/docs/reference/cli-reference.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,59 @@ Docker's embedded DNS (127.0.0.11) is always allowed for container name resoluti
156156
| `130` | Interrupted by SIGINT (Ctrl+C) |
157157
| `143` | Terminated by SIGTERM |
158158

159+
## Subcommands
160+
161+
### `awf logs`
162+
163+
View Squid proxy logs from current or previous runs.
164+
165+
```bash
166+
awf logs [options]
167+
```
168+
169+
#### Options
170+
171+
| Option | Type | Default | Description |
172+
|--------|------|---------|-------------|
173+
| `-f, --follow` | flag | `false` | Follow log output in real-time |
174+
| `--format <format>` | string | `pretty` | Output format: `raw`, `pretty`, `json` |
175+
| `--source <path>` | string | auto | Path to log directory or `running` for live container |
176+
| `--list` | flag | `false` | List available log sources |
177+
178+
#### Output Formats
179+
180+
| Format | Description |
181+
|--------|-------------|
182+
| `pretty` | Colorized, human-readable output (default) |
183+
| `raw` | Logs as-is without parsing |
184+
| `json` | Structured JSON for scripting |
185+
186+
#### Examples
187+
188+
```bash
189+
# View recent logs with pretty formatting
190+
awf logs
191+
192+
# Follow logs in real-time
193+
awf logs -f
194+
195+
# View logs in JSON format
196+
awf logs --format json
197+
198+
# List available log sources
199+
awf logs --list
200+
201+
# Use a specific log directory
202+
awf logs --source /tmp/squid-logs-1234567890
203+
204+
# Stream from running container
205+
awf logs --source running -f
206+
```
207+
208+
:::note
209+
Log sources are auto-discovered in this order: running containers, `AWF_LOGS_DIR` environment variable, then preserved log directories in `/tmp/squid-logs-*`.
210+
:::
211+
159212
## See Also
160213

161214
- [Quick Start Guide](/gh-aw-firewall/quickstart) - Getting started with examples

‎docs/mitmweb-debugging.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Debugging AWF Traffic with mitmweb
2+
3+
This guide shows how to chain Squid proxy through mitmweb to inspect HTTP/HTTPS traffic from Copilot CLI.
4+
5+
## Prerequisites
6+
7+
- mitmproxy installed (`pip install mitmproxy`)
8+
- AWF built and working
9+
10+
## Setup
11+
12+
### 1. Start mitmweb on host
13+
14+
```bash
15+
mitmweb --listen-port 8000
16+
```
17+
18+
Web UI available at http://127.0.0.1:8081
19+
20+
### 2. Run AWF with `--keep-containers`
21+
22+
```bash
23+
sudo -E awf --keep-containers \
24+
--add-host host.docker.internal:host-gateway \
25+
--tty --env-all \
26+
--allow-domains 'api.github.com,registry.npmjs.org,api.enterprise.githubcopilot.com' \
27+
--log-level debug \
28+
-- echo "setup done"
29+
```
30+
31+
Note the workDir from output (e.g., `/tmp/awf-XXXXX`).
32+
33+
### 3. Stop containers
34+
35+
```bash
36+
docker stop awf-agent awf-squid
37+
```
38+
39+
### 4. Edit Squid config to chain through mitmweb
40+
41+
```bash
42+
# Add cache_peer after http_port line
43+
sudo sed -i '/^http_port 3128$/a \
44+
cache_peer host.docker.internal parent 8000 0 no-query no-digest default\
45+
never_direct allow all' /tmp/awf-XXXXX/squid.conf
46+
```
47+
48+
### 5. Add host.docker.internal to Squid container
49+
50+
```bash
51+
sudo sed -i '/image: ghcr.io\/githubnext\/gh-aw-firewall\/squid:latest/a\ extra_hosts:\n - host.docker.internal:host-gateway' /tmp/awf-XXXXX/docker-compose.yml
52+
```
53+
54+
### 6. Change agent command to stay alive
55+
56+
```bash
57+
sudo sed -i "s/echo 'setup done'/sleep infinity/" /tmp/awf-XXXXX/docker-compose.yml
58+
```
59+
60+
### 7. (Optional) Mount additional directories
61+
62+
```bash
63+
# Add volume mount
64+
sudo sed -i '/mcp-config.json:ro$/a\ - /path/to/dir:/path/to/dir:rw' /tmp/awf-XXXXX/docker-compose.yml
65+
66+
# Change working directory
67+
sudo sed -i 's|working_dir: .*|working_dir: /path/to/dir|' /tmp/awf-XXXXX/docker-compose.yml
68+
```
69+
70+
### 8. Restart containers
71+
72+
```bash
73+
docker rm -f awf-squid awf-agent
74+
cd /tmp/awf-XXXXX && docker compose up -d
75+
```
76+
77+
### 9. Disable npm SSL verification (required for mitmproxy interception)
78+
79+
```bash
80+
docker exec awf-agent bash -c "echo 'strict-ssl=false' >> ~/.npmrc"
81+
```
82+
83+
### 10. Run commands in agent container
84+
85+
```bash
86+
docker exec -it awf-agent /bin/bash -c "npx -y '@github/copilot@0.0.365' --prompt 'hello'"
87+
```
88+
89+
## Verify Traffic
90+
91+
- Check mitmweb UI at http://127.0.0.1:8081
92+
- Test connectivity: `docker exec awf-agent curl -k https://api.github.com/zen`
93+
94+
## Cleanup
95+
96+
```bash
97+
docker stop awf-agent awf-squid
98+
docker rm awf-agent awf-squid
99+
sudo rm -rf /tmp/awf-XXXXX
100+
```
101+
102+
## Troubleshooting
103+
104+
| Error | Solution |
105+
|-------|----------|
106+
| `ERR_CANNOT_FORWARD` | Squid can't reach mitmweb. Add `extra_hosts` to Squid container |
107+
| `SELF_SIGNED_CERT_IN_CHAIN` | Set `strict-ssl=false` in ~/.npmrc |
108+
| Container exits immediately | Change command to `sleep infinity` |
109+
| Stale PID file | `docker rm -f` containers before restart |

‎docs/usage.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -353,6 +353,99 @@ docker logs awf-squid
353353
# /tmp/awf-<timestamp>/squid-logs/
354354
```
355355

356+
## Viewing Logs with `awf logs`
357+
358+
The `awf logs` command provides an easy way to view Squid proxy logs from current or previous runs.
359+
360+
### Basic Usage
361+
362+
```bash
363+
# View recent logs with pretty formatting (default)
364+
awf logs
365+
366+
# Follow logs in real-time (like tail -f)
367+
awf logs -f
368+
```
369+
370+
### Output Formats
371+
372+
The command supports three output formats:
373+
374+
```bash
375+
# Pretty: colorized, human-readable output (default)
376+
awf logs --format pretty
377+
378+
# Raw: logs as-is without parsing or colorization
379+
awf logs --format raw
380+
381+
# JSON: structured output for programmatic consumption
382+
awf logs --format json
383+
```
384+
385+
Example JSON output:
386+
```json
387+
{"timestamp":1760987995.318,"clientIp":"172.20.98.20","clientPort":"55960","domain":"example.com","destIp":"-","destPort":"-","httpVersion":"1.1","method":"CONNECT","statusCode":403,"decision":"TCP_DENIED:HIER_NONE","url":"example.com:443","userAgent":"curl/7.81.0","isAllowed":false}
388+
```
389+
390+
### Log Source Discovery
391+
392+
The command auto-discovers log sources in this order:
393+
1. Running `awf-squid` container (live logs)
394+
2. `AWF_LOGS_DIR` environment variable (if set)
395+
3. Preserved log directories in `/tmp/squid-logs-<timestamp>`
396+
397+
```bash
398+
# List all available log sources
399+
awf logs --list
400+
401+
# Output example:
402+
# Available log sources:
403+
# [running] awf-squid (live container)
404+
# [preserved] /tmp/squid-logs-1760987995318 (11/27/2024, 12:30:00 PM)
405+
# [preserved] /tmp/squid-logs-1760987890000 (11/27/2024, 12:28:10 PM)
406+
```
407+
408+
### Using Specific Log Sources
409+
410+
```bash
411+
# Stream from a running container
412+
awf logs --source running -f
413+
414+
# Use a specific preserved log directory
415+
awf logs --source /tmp/squid-logs-1760987995318
416+
417+
# Use logs from AWF_LOGS_DIR
418+
export AWF_LOGS_DIR=/path/to/logs
419+
awf logs
420+
```
421+
422+
### Combining Options
423+
424+
```bash
425+
# Follow live logs in JSON format
426+
awf logs -f --format json
427+
428+
# View specific logs in raw format
429+
awf logs --source /tmp/squid-logs-1760987995318 --format raw
430+
```
431+
432+
### Troubleshooting with Logs
433+
434+
**Find blocked requests:**
435+
```bash
436+
awf logs --format json | jq 'select(.isAllowed == false)'
437+
```
438+
439+
**Filter by domain:**
440+
```bash
441+
awf logs --format json | jq 'select(.domain | contains("github"))'
442+
```
443+
444+
**Count blocked vs allowed:**
445+
```bash
446+
awf logs --format json | jq -s 'group_by(.isAllowed) | map({allowed: .[0].isAllowed, count: length})'
447+
```
448+
356449
## Log Analysis
357450

358451
Find all blocked domains:

‎package-lock.json‎

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@github/agentic-workflow-firewall",
3-
"version": "0.5.1",
3+
"version": "0.6.0",
44
"description": "Network firewall for agentic workflows with domain whitelisting",
55
"main": "dist/cli.js",
66
"bin": {

‎src/cli-workflow.ts‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@ export interface WorkflowDependencies {
44
ensureFirewallNetwork: () => Promise<{ squidIp: string }>;
55
setupHostIptables: (squidIp: string, port: number, dnsServers: string[]) => Promise<void>;
66
writeConfigs: (config: WrapperConfig) => Promise<void>;
7-
startContainers: (workDir: string, allowedDomains: string[]) => Promise<void>;
7+
startContainers: (workDir: string, allowedDomains: string[], proxyLogsDir?: string) => Promise<void>;
88
runAgentCommand: (
99
workDir: string,
10-
allowedDomains: string[]
10+
allowedDomains: string[],
11+
proxyLogsDir?: string
1112
) => Promise<{ exitCode: number }>;
1213
}
1314

@@ -50,11 +51,11 @@ export async function runMainWorkflow(
5051
await dependencies.writeConfigs(config);
5152

5253
// Step 2: Start containers
53-
await dependencies.startContainers(config.workDir, config.allowedDomains);
54+
await dependencies.startContainers(config.workDir, config.allowedDomains, config.proxyLogsDir);
5455
onContainersStarted?.();
5556

5657
// Step 3: Wait for agent to complete
57-
const result = await dependencies.runAgentCommand(config.workDir, config.allowedDomains);
58+
const result = await dependencies.runAgentCommand(config.workDir, config.allowedDomains, config.proxyLogsDir);
5859

5960
// Step 4: Cleanup (logs will be preserved automatically if they exist)
6061
await performCleanup();

0 commit comments

Comments
 (0)