Skip to content

docs(examples): Add comprehensive ADMM optimization examples#67

Merged
noahgift merged 1 commit into
mainfrom
claude/research-optimization-techniques-01LWS5ZwqVEHQ13NbShwH7Ls
Nov 23, 2025
Merged

docs(examples): Add comprehensive ADMM optimization examples#67
noahgift merged 1 commit into
mainfrom
claude/research-optimization-techniques-01LWS5ZwqVEHQ13NbShwH7Ls

Conversation

@noahgift

Copy link
Copy Markdown
Contributor

Add admm_optimization.rs demonstrating distributed and federated learning applications of the Alternating Direction Method of Multipliers.

Examples (~620 lines, 4 comprehensive demos):

  1. Distributed Lasso Regression

    • Consensus form for sparse regression
    • Separates smooth (least squares) and non-smooth (L1) parts
    • Demonstrates adaptive ρ parameter adjustment
    • Recovers sparse coefficients with < 0.1 RMSE
  2. Consensus Optimization (Federated Learning)

    • Simulates 3 distributed workers with local data
    • ADMM enforces global consensus across workers
    • Demonstrates federated averaging
    • Workers converge to global solution
  3. Quadratic Programming with ADMM

    • Box-constrained QP: minimize ½xᵀQx + cᵀx s.t. x ≥ 0
    • Consensus form separates objective from constraints
    • Adaptive penalty parameter for faster convergence
  4. ADMM vs FISTA Comparison

    • Side-by-side on same Lasso problem
    • Shows tradeoffs: ADMM for distributed, FISTA for centralized
    • Performance metrics and convergence analysis
    • Guidance on when to use each method

Key Features Demonstrated:

  • Adaptive ρ adjustment (Boyd et al. 2011)
  • Consensus constraints for distributed optimization
  • Separation of smooth/non-smooth objectives
  • Proximal operators (soft-thresholding)
  • Cholesky solver for x-subproblems

Applications:

  • Distributed data across workers
  • Federated learning scenarios
  • Model parallelism
  • Constrained optimization via consensus

Follows pattern from batch_optimization.rs and convex_optimization.rs with detailed mathematical background and practical ML applications.

Related: ADMM implementation (commit 9f30ffe)

Add admm_optimization.rs demonstrating distributed and federated learning
applications of the Alternating Direction Method of Multipliers.

Examples (~620 lines, 4 comprehensive demos):

1. **Distributed Lasso Regression**
   - Consensus form for sparse regression
   - Separates smooth (least squares) and non-smooth (L1) parts
   - Demonstrates adaptive ρ parameter adjustment
   - Recovers sparse coefficients with < 0.1 RMSE

2. **Consensus Optimization (Federated Learning)**
   - Simulates 3 distributed workers with local data
   - ADMM enforces global consensus across workers
   - Demonstrates federated averaging
   - Workers converge to global solution

3. **Quadratic Programming with ADMM**
   - Box-constrained QP: minimize ½xᵀQx + cᵀx s.t. x ≥ 0
   - Consensus form separates objective from constraints
   - Adaptive penalty parameter for faster convergence

4. **ADMM vs FISTA Comparison**
   - Side-by-side on same Lasso problem
   - Shows tradeoffs: ADMM for distributed, FISTA for centralized
   - Performance metrics and convergence analysis
   - Guidance on when to use each method

Key Features Demonstrated:
- Adaptive ρ adjustment (Boyd et al. 2011)
- Consensus constraints for distributed optimization
- Separation of smooth/non-smooth objectives
- Proximal operators (soft-thresholding)
- Cholesky solver for x-subproblems

Applications:
- Distributed data across workers
- Federated learning scenarios
- Model parallelism
- Constrained optimization via consensus

Follows pattern from batch_optimization.rs and convex_optimization.rs
with detailed mathematical background and practical ML applications.

Related: ADMM implementation (commit 9f30ffe)
@noahgift noahgift merged commit f3afa13 into main Nov 23, 2025
4 of 10 checks passed
@noahgift noahgift deleted the claude/research-optimization-techniques-01LWS5ZwqVEHQ13NbShwH7Ls branch November 23, 2025 21:01
noahgift added a commit that referenced this pull request Apr 19, 2026
Re-publish of v0.31.1 with the aprender-mcp publish fix (PR #910).
v0.31.1 was yanked because aprender-mcp/build.rs panicked on an
out-of-crate path at `cargo install` time. v0.31.2 ships with:

- aprender-mcp contract YAML bundled inside the crate tarball
- build.rs reads CARGO_MANIFEST_DIR/contracts/ (no `..` escape)
- drift-guard test + static check_build_rs_paths.sh in CI

Verified: `cargo package -p aprender-mcp` (which runs the same
unpack+compile verify as `cargo install`) completes cleanly on v0.31.2.
On v0.31.1 the same command failed with the documented build.rs panic.

Refs: task #67

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
noahgift added a commit that referenced this pull request Apr 19, 2026
Re-publish of v0.31.1 with the aprender-mcp publish fix (PR #910).
v0.31.1 was yanked because aprender-mcp/build.rs panicked on an
out-of-crate path at `cargo install` time. v0.31.2 ships with:

- aprender-mcp contract YAML bundled inside the crate tarball
- build.rs reads CARGO_MANIFEST_DIR/contracts/ (no `..` escape)
- drift-guard test + static check_build_rs_paths.sh in CI

Verified: `cargo package -p aprender-mcp` (which runs the same
unpack+compile verify as `cargo install`) completes cleanly on v0.31.2.
On v0.31.1 the same command failed with the documented build.rs panic.

Refs: task #67

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants