Skip to content

Allow running tests with specific response format#10418

Merged
abnegate merged 2 commits into1.8.xfrom
feat-version-tests
Sep 1, 2025
Merged

Allow running tests with specific response format#10418
abnegate merged 2 commits into1.8.xfrom
feat-version-tests

Conversation

@abnegate
Copy link
Copy Markdown
Member

@abnegate abnegate commented Sep 1, 2025

What does this PR do?

(Provide a description of what this PR does and why it's needed.)

Test Plan

(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work. Screenshots may also be helpful.)

Related PRs and Issues

  • (Related PR or issue)

Checklist

  • Have you read the Contributing Guidelines on issues?
  • If the PR includes a change to an API's metadata (desc, label, params, etc.), does it also include updated API specs and example docs?

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Sep 1, 2025

📝 Walkthrough

Walkthrough

  • .github/workflows/tests.yml: Adds workflow_dispatch with an optional response_format input (string, default '') and injects it as the environment variable _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" into all docker compose exec commands used for unit and E2E tests.
  • tests/e2e/Client.php: Adds public function setResponseFormat(string $value): self which calls addHeader('X-Appwrite-Response-Format', $value) and returns $this.
  • tests/e2e/Scopes/Scope.php: Imports Utopia\System\System; in setUp() reads _APP_E2E_RESPONSE_FORMAT via System::getEnv, and if non-empty validates it matches a numeric semantic version pattern and is <= APP_VERSION_STABLE; on success calls $this->client->setResponseFormat($format), otherwise throws E2E response format must be ' . APP_VERSION_STABLE . ' or lower..
  • No other public signatures changed.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 PHPStan (2.1.17)

Note: Using configuration file /phpstan.neon.
Invalid entry in excludePaths:
Path "/app/sdks" is neither a directory, nor a file path, nor a fnmatch pattern.

If the excluded path can sometimes exist, append (?)
to its config entry to mark it as optional. Example:

parameters:
excludePaths:
analyseAndScan:
- app/sdks (?)

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat-version-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Sep 1, 2025

Security Scan Results for PR

Docker Image Scan Results

Package Version Vulnerability Severity
golang.org/x/crypto v0.31.0 CVE-2025-22869 HIGH
golang.org/x/oauth2 v0.24.0 CVE-2025-22868 HIGH
stdlib 1.22.10 CVE-2025-47907 HIGH

Source Code Scan Results

🎉 No vulnerabilities found!

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
tests/e2e/Client.php (1)

111-123: Trim and ignore empty values to harden the API.

Guard against accidental whitespace or empty strings so callers don’t have to.

Apply this diff:

 public function setResponseFormat(string $value): self
 {
-    $this->addHeader('X-Appwrite-Response-Format', $value);
+    $value = trim($value);
+    if ($value === '') {
+        return $this;
+    }
+    $this->addHeader('X-Appwrite-Response-Format', $value);
 
     return $this;
 }
tests/e2e/Scopes/Scope.php (1)

28-31: Avoid PHP’s empty('0') pitfall; trim input.

Using empty() can skip valid “0”-like values. Prefer strict check against empty string after trim.

Apply this diff:

-        $format = System::getEnv('_APP_E2E_RESPONSE_FORMAT');
-        if (!empty($format)) {
+        $format = trim((string) System::getEnv('_APP_E2E_RESPONSE_FORMAT'));
+        if ($format !== '') {
             $this->client->setResponseFormat($format);
         }
.github/workflows/tests.yml (2)

111-114: Use non-interactive exec (-T) for CI consistency.

Most other execs use -T; align unit tests to avoid tty allocation quirks.

Apply this diff:

-          docker compose exec \
+          docker compose exec -T \
             -e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
             appwrite test /usr/src/code/tests/unit

521-521: Add trailing newline at EOF.

Fix YAMLlint “no new line at end of file”.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between aafc98f and fb82707.

📒 Files selected for processing (3)
  • .github/workflows/tests.yml (10 hunks)
  • tests/e2e/Client.php (1 hunks)
  • tests/e2e/Scopes/Scope.php (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e/Scopes/Scope.php (1)
tests/e2e/Client.php (1)
  • setResponseFormat (118-123)
🪛 YAMLlint (1.37.1)
.github/workflows/tests.yml

[error] 521-521: no new line character at the end of file

(new-line-at-end-of-file)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Check if utopia-php/database changed
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: scan
🔇 Additional comments (11)
tests/e2e/Client.php (2)

111-123: New setter is consistent and chainable.

Matches the existing pattern (e.g., setMode) and cleanly scopes the header.


111-123: Confirm server-side handling of X-Appwrite-Response-Format header value

Client::addHeader lowercases both header names and values (tests/e2e/Client.php 161–163); verify that the server’s code reading X-Appwrite-Response-Format treats lowercase values the same way (e.g. when matching against semver or _APP_SYSTEM_RESPONSE_FORMAT). If parsing is case-sensitive, normalize or canonicalize the header value before comparison.

tests/e2e/Scopes/Scope.php (1)

10-10: Import looks correct.

System::getEnv is the right choice here for parity with other tests.

.github/workflows/tests.yml (8)

11-19: workflow_dispatch input wiring LGTM.

Optional string input with sensible default; safe on pull_request runs (expands to empty).


146-150: Propagating response format to General E2E tests is correct.

Non-interactive (-T) used; good.


222-226: Env passthrough for Services matrix looks good.

Consistent with other jobs; won’t affect runs when input is empty.


311-315: Shared-mode Services wiring LGTM.

Variable present across both table modes.


353-357: Dev Keys job uses the env consistently.

No issues.


409-413: Shared-mode Dev Keys wiring LGTM.

Matches Services job pattern.


452-456: Screenshots job wiring LGTM.

Environment propagation consistent.


509-513: Shared-mode Screenshots wiring LGTM.

No changes needed.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Sep 1, 2025

✨ Benchmark results

  • Requests per second: 1,143
  • Requests with 200 status code: 205,695
  • P99 latency: 0.16873812

⚡ Benchmark Comparison

Metric This PR Latest version
RPS 1,143 915
200 205,695 164,664
P99 0.16873812 0.21303785

Comment thread .github/workflows/tests.yml
Comment thread tests/e2e/Scopes/Scope.php
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
tests/e2e/Scopes/Scope.php (1)

28-28: Add discoverability for the new env var.
Please add _APP_E2E_RESPONSE_FORMAT to the .env example and docs.

🧹 Nitpick comments (1)
tests/e2e/Scopes/Scope.php (1)

28-37: Harden env parsing and error messaging.
Trim the value, avoid empty() edge cases, and surface the invalid value with a more specific exception type.

-        $format = System::getEnv('_APP_E2E_RESPONSE_FORMAT');
-        if (!empty($format)) {
+        $format = \trim((string) System::getEnv('_APP_E2E_RESPONSE_FORMAT'));
+        if ($format !== '') {
             if (
                 !\preg_match('/^\d+\.\d+\.\d+$/', $format) ||
                 !\version_compare($format, APP_VERSION_STABLE, '<=')
             ) {
-                throw new \Exception('E2E response format must be ' . APP_VERSION_STABLE . ' or lower.');
+                throw new \InvalidArgumentException(
+                    "Invalid E2E response format '{$format}'. Must be <= " . APP_VERSION_STABLE . '.'
+                );
             }
             $this->client->setResponseFormat($format);
         }
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between fb82707 and 6e5fe9c.

📒 Files selected for processing (1)
  • tests/e2e/Scopes/Scope.php (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e/Scopes/Scope.php (1)
tests/e2e/Client.php (1)
  • setResponseFormat (118-123)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (18)
  • GitHub Check: E2E Service Test (VCS)
  • GitHub Check: E2E Service Test (Webhooks)
  • GitHub Check: E2E Service Test (Messaging)
  • GitHub Check: E2E Service Test (Functions)
  • GitHub Check: E2E Service Test (Realtime)
  • GitHub Check: E2E Service Test (FunctionsSchedule)
  • GitHub Check: E2E Service Test (Projects)
  • GitHub Check: E2E Service Test (GraphQL)
  • GitHub Check: E2E Service Test (Account)
  • GitHub Check: E2E Service Test (Databases/TablesDB)
  • GitHub Check: E2E Service Test (Avatars)
  • GitHub Check: E2E Service Test (Databases/Legacy)
  • GitHub Check: E2E Service Test (Site Screenshots)
  • GitHub Check: E2E Service Test (Dev Keys)
  • GitHub Check: E2E General Test
  • GitHub Check: Unit Test
  • GitHub Check: Benchmark
  • GitHub Check: scan
🔇 Additional comments (2)
tests/e2e/Scopes/Scope.php (2)

10-10: Import is appropriate and scoped correctly.
No concerns with introducing Utopia\System\System here.


31-34: Ensure APP_VERSION_STABLE is loaded in your test bootstrap
APP_VERSION_STABLE is defined in app/init/constants.php (via app/init.php) but I didn’t find a test‐specific bootstrap under tests/. Add a require/import of app/init.php (or the constants file) in your test bootstrap to avoid undefined‐constant errors in CI.

@abnegate abnegate merged commit 4766d42 into 1.8.x Sep 1, 2025
41 checks passed
@abnegate abnegate deleted the feat-version-tests branch September 1, 2025 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants