Skip to content

Fix Cloudflare 403 on Aver llms.txt fetch#58

Merged
aallan merged 2 commits into
mainfrom
fix/cloudflare-user-agent
Apr 16, 2026
Merged

Fix Cloudflare 403 on Aver llms.txt fetch#58
aallan merged 2 commits into
mainfrom
fix/cloudflare-user-agent

Conversation

@aallan

@aallan aallan commented Apr 16, 2026

Copy link
Copy Markdown
Owner

Summary

  • Python's default urllib User-Agent (Python-urllib/3.14) is blocked by Cloudflare on averlang.dev, causing --language aver to crash with HTTP 403
  • Extracts a shared _fetch_url() helper that sends vera-bench/0.0.9 as User-Agent
  • Fixes both SKILL.md and llms.txt fetches (veralang.dev could hit the same issue any time)

Test plan

  • load_aver_llms_txt() returns 11702 chars successfully
  • load_skill_md() returns 72260 chars successfully

Generated with Claude Code

Summary by CodeRabbit

  • Refactor
    • Internal HTTP fetching now sends a consistent user-agent to improve server compatibility and request tracking. Behaviour for local files and existing error handling is unchanged, so user-visible functionality remains the same.

Python's default urllib User-Agent ("Python-urllib/3.14") is blocked
by Cloudflare on averlang.dev. Extract a shared _fetch_url() helper
that sends "vera-bench/0.0.9" as User-Agent. Fixes both SKILL.md
and llms.txt fetches.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 16, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Added a module-level _USER_AGENT constant and a _fetch_url(url, *, timeout=10) -> str helper in vera_bench/prompts.py. load_skill_md and load_aver_llms_txt now use _fetch_url() for HTTP sources; local file handling and existing exception behaviour are unchanged.

Changes

Cohort / File(s) Summary
HTTP fetching refactor
vera_bench/prompts.py
Added _USER_AGENT = "vera-bench/0.0.9" and _fetch_url(url, *, timeout=10) -> str to perform UTF-8-decoded HTTP GET with that User-Agent. Updated load_skill_md and load_aver_llms_txt to call _fetch_url() for HTTP URLs; preserved existing urllib.error.URLError / OSError catch-and-reraise behaviour.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

Suggested labels

harness

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly addresses the main change: adding a User-Agent header to fix Cloudflare 403 errors when fetching from averlang.dev.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/cloudflare-user-agent

Comment @coderabbitai help to get the list of available commands and usage tips.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@vera_bench/prompts.py`:
- Around line 24-28: Add a regression test that verifies _fetch_url() sends the
custom User-Agent header: write a unit test that patches or mocks
urllib.request.urlopen (or urllib.request.Request) and captures the Request
passed into urlopen when calling _fetch_url(url). Assert that the captured
Request.headers (or Request.get_header("User-Agent")) equals the module constant
_USER_AGENT; reference the _fetch_url function and the _USER_AGENT symbol to
locate where to call and what to assert, and ensure the mock returns an object
with a read().decode("utf-8") result to simulate a normal response.
- Line 21: The constant _USER_AGENT lacks an explicit type hint; update its
declaration (the _USER_AGENT symbol) to include a Python type annotation (str)
per repository typing rules (Python 3.11+), e.g., annotate _USER_AGENT: str so
the constant is explicitly typed as a string.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 101f37f2-ca63-43a5-9817-bd3af7be5e57

📥 Commits

Reviewing files that changed from the base of the PR and between 0ee89fa and 9b33b03.

📒 Files selected for processing (1)
  • vera_bench/prompts.py

Comment thread vera_bench/prompts.py
Comment thread vera_bench/prompts.py
@codecov

codecov Bot commented Apr 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.27%. Comparing base (60187c4) to head (9b33b03).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #58      +/-   ##
==========================================
- Coverage   83.30%   83.27%   -0.04%     
==========================================
  Files          10       10              
  Lines        1360     1363       +3     
==========================================
+ Hits         1133     1135       +2     
- Misses        227      228       +1     
Flag Coverage Δ
python 83.27% <100.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aallan aallan merged commit efcfd3a into main Apr 16, 2026
10 checks passed
@aallan aallan deleted the fix/cloudflare-user-agent branch April 16, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant