Skip to content

Phase 0 Completion: Add Baseline Performance Metrics Documentation#25

Merged
kcenon merged 6 commits into
mainfrom
phase-0-completion
Oct 7, 2025
Merged

Phase 0 Completion: Add Baseline Performance Metrics Documentation#25
kcenon merged 6 commits into
mainfrom
phase-0-completion

Conversation

@kcenon

@kcenon kcenon commented Oct 7, 2025

Copy link
Copy Markdown
Owner

Summary

Complete Phase 0 (Task 0.2: Baseline Performance Benchmarking) by adding comprehensive performance metrics documentation template for network_system.

Changes

Added

  • benchmarks/BASELINE.md: Comprehensive network performance metrics template
    • Connection establishment performance
    • Session management and pooling
    • Data transfer throughput
    • Round-trip latency
    • Concurrent connection scalability
    • Protocol overhead (TCP vs UDP)
    • Event loop performance
    • Buffer management and zero-copy
    • All metrics marked as TBD, ready for data collection

Performance Targets

  • Regression threshold: <5% performance degradation
  • Connection latency: <100μs (localhost)
  • Throughput: >1 GB/s (localhost, 64KB packets)
  • RTT: <50μs (0-byte ping)
  • Event throughput: >1M events/sec
  • Memory per connection: <50KB
  • Concurrent connections: 10,000+ supported

Phase 0 Status

Infrastructure: ✅ Complete

  • CI/CD with sanitizers: ✅
  • Coverage workflows: ✅
  • Static analysis: ✅
  • Benchmark infrastructure: ✅
  • Baseline documentation: ✅ (this PR)

Next Steps:

  1. Run benchmarks on localhost (loopback)
  2. Test concurrent connection limits
  3. Measure zero-copy performance (platform-dependent)
  4. Update BASELINE.md with actual values
  5. Proceed to Phase 1 (Thread Safety)

Part of Phase 0 completion as documented in NEED_TO_FIX.md

Add comprehensive BASELINE.md template for network metrics:
- Connection establishment performance (<100μs localhost target)
- Session management and pooling
- Data transfer throughput (>1GB/s localhost target)
- Round-trip latency (<50μs for 0-byte ping)
- Concurrent connection scalability (10k+ target)
- Protocol overhead (TCP vs UDP)
- Event loop performance (>1M events/sec)
- Buffer management and zero-copy operations
- Template ready for data collection

Establishes Phase 0 infrastructure for network performance tracking.
Target: <5% regression, >1GB/s throughput on localhost

Part of Phase 0 completion (Task 0.2: Baseline Performance Benchmarking)
@github-actions

github-actions Bot commented Oct 7, 2025

Copy link
Copy Markdown
Contributor

Performance Comparison

Base Branch Results

No base results

PR Branch Results

No PR results

Add benchmarks.yml to pull_request path filter to ensure workflow
changes trigger benchmark execution in PRs.

Previously, the workflow only triggered on changes to benchmarks/,
src/, or include/ directories in PRs, causing it to be skipped when
only the workflow file itself was modified.

This ensures consistency between push and pull_request triggers.
@github-actions

github-actions Bot commented Oct 7, 2025

Copy link
Copy Markdown
Contributor

Performance Comparison

Base Branch Results

No base results

PR Branch Results

No PR results

This commit improves the Performance Regression Check workflow to gracefully
handle PRs that only modify documentation files (like BASELINE.md).

Changes:
- Add NETWORK_BUILD_BENCHMARKS=ON flag with fallback for base/PR builds
- Check if benchmark executable exists before attempting to run it
- Add continue-on-error to benchmark run steps
- Provide informative messages for documentation-only changes

This prevents the workflow from being marked as 'skipped' when benchmark
executables don't exist, making the CI status clearer.
@github-actions

github-actions Bot commented Oct 7, 2025

Copy link
Copy Markdown
Contributor

Performance Comparison

Base Branch Results

No base results

PR Branch Results

No PR results

kcenon added 2 commits October 7, 2025 22:35
Add concurrency group to prevent both push and pull_request events
from creating redundant workflow runs for the same commit.

This eliminates the confusing 'skipped' Performance Regression Check
status that appears from the push event when a PR exists.
Split the Performance Regression Check from test-integration.yml into
a separate workflow file that only runs on pull_request events.

Benefits:
- Eliminates confusing 'skipped' status from push events
- Clearer separation of concerns
- Performance checks only run when needed (PRs only)
- No duplicate workflow runs for the same commit

Changes:
- Remove performance-regression job from test-integration.yml
- Create new performance-regression.yml workflow
- Configure to run only on pull_request events
@github-actions

github-actions Bot commented Oct 7, 2025

Copy link
Copy Markdown
Contributor

Performance Comparison

Base Branch Results

No base results

PR Branch Results

No PR results

Remove backslash escapes from shell variables. GitHub Actions YAML
only processes ${{ }} syntax, while regular shell variables should
use $ without escaping.

Fixes: syntax error near unexpected token '('
@github-actions

github-actions Bot commented Oct 7, 2025

Copy link
Copy Markdown
Contributor

Performance Comparison

Base Branch Results

No base results

PR Branch Results

No PR results

@kcenon kcenon merged commit 86b6a55 into main Oct 7, 2025
54 checks passed
@kcenon kcenon deleted the phase-0-completion branch October 7, 2025 14:08
kcenon added a commit that referenced this pull request Apr 13, 2026
* feat(benchmarks): add baseline performance metrics documentation

Add comprehensive BASELINE.md template for network metrics:
- Connection establishment performance (<100μs localhost target)
- Session management and pooling
- Data transfer throughput (>1GB/s localhost target)
- Round-trip latency (<50μs for 0-byte ping)
- Concurrent connection scalability (10k+ target)
- Protocol overhead (TCP vs UDP)
- Event loop performance (>1M events/sec)
- Buffer management and zero-copy operations
- Template ready for data collection

Establishes Phase 0 infrastructure for network performance tracking.
Target: <5% regression, >1GB/s throughput on localhost

Part of Phase 0 completion (Task 0.2: Baseline Performance Benchmarking)

* fix(ci): add workflow file to benchmark trigger paths

Add benchmarks.yml to pull_request path filter to ensure workflow
changes trigger benchmark execution in PRs.

Previously, the workflow only triggered on changes to benchmarks/,
src/, or include/ directories in PRs, causing it to be skipped when
only the workflow file itself was modified.

This ensures consistency between push and pull_request triggers.

* fix(ci): handle documentation-only PRs in performance regression check

This commit improves the Performance Regression Check workflow to gracefully
handle PRs that only modify documentation files (like BASELINE.md).

Changes:
- Add NETWORK_BUILD_BENCHMARKS=ON flag with fallback for base/PR builds
- Check if benchmark executable exists before attempting to run it
- Add continue-on-error to benchmark run steps
- Provide informative messages for documentation-only changes

This prevents the workflow from being marked as 'skipped' when benchmark
executables don't exist, making the CI status clearer.

* fix(ci): prevent duplicate workflow runs for PRs

Add concurrency group to prevent both push and pull_request events
from creating redundant workflow runs for the same commit.

This eliminates the confusing 'skipped' Performance Regression Check
status that appears from the push event when a PR exists.

* refactor(ci): separate performance regression into dedicated workflow

Split the Performance Regression Check from test-integration.yml into
a separate workflow file that only runs on pull_request events.

Benefits:
- Eliminates confusing 'skipped' status from push events
- Clearer separation of concerns
- Performance checks only run when needed (PRs only)
- No duplicate workflow runs for the same commit

Changes:
- Remove performance-regression job from test-integration.yml
- Create new performance-regression.yml workflow
- Configure to run only on pull_request events

* fix(ci): correct shell variable syntax in performance regression

Remove backslash escapes from shell variables. GitHub Actions YAML
only processes ${{ }} syntax, while regular shell variables should
use $ without escaping.

Fixes: syntax error near unexpected token '('
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant