Resolve conflicts in verl/experimental/agent_loop/agent_loop.py introduced by
PR verl-project#6129 (refactor: move LLMServerManager out of AgentLoopManager):

  * Imports - keep the function_tool import while accepting main's removal
    of prometheus_utils, teacher_loop, single_controller.ray.base imports.
  * AgentLoopWorker.__init__ - keep both the new "Online policy distillation"
    block (from main) and the "Load function-based tools once per worker"
    block (from this PR); ordering is irrelevant since they touch disjoint
    state.

The function_tools=FunctionToolListWrap(self.function_tools) kwarg in
_run_agent_loop auto-merged cleanly next to main's renamed
server_manager=self.llm_client.

Co-authored-by: Claude
Made-with: Cursor

…factoring

…ckaging bug

The pinned verl commit (a512e90) ships a wheel that is missing
verl/experimental/reward_loop/router/ because the upstream directory had
no __init__.py at that commit and setuptools' default package discovery
silently drops it. This breaks the FlowGRPO trainer at runtime with
"ModuleNotFoundError: No module named 'verl.experimental.reward_loop.router'".

Switch the verl install in docs/start/install.md from a wheel install
(uv pip install git+…@<commit>) to a clone-and-editable install pinned
at the same commit. An editable install exposes the source tree on
sys.path, so router/ is picked up as a PEP 420 implicit namespace
package and the import works without any per-venv patching.

CI workflows are intentionally not touched because they don't exercise
the broken codepath. The pin will be bumped past
verl-project/verl#5209 once verl-omni is also adapted to the breaking
LLMServerClient refactor in verl-project/verl#6129 (tracked separately).

Adapt verl-omni's diffusion agent loop and ray trainer to verl-project/verl#6129,
which removed AsyncLLMServerManager and made AgentLoopManager / AgentLoopWorker
consume an LLMServerClient produced by a separately-owned LLMServerManager.

verl-omni changes:
- DiffusionAgentLoopWorker.__init__ now takes (config, llm_client, teacher_client,
  reward_loop_worker_handles), matching the positional contract that
  AgentLoopManager.create() uses when spawning workers. _get_rollout_and_model_config
  was also dropped upstream, so the config slicing is inlined to keep the diff
  minimal.
- ray_diffusion_trainer now creates an LLMServerManager first, hands its client to
  AgentLoopManager.create(), and uses llm_server_manager.get_replicas() (instead of
  async_rollout_manager.rollout_replicas) to wire the CheckpointEngineManager. This
  mirrors the new pattern in upstream verl/trainer/ppo/ray_trainer.py.
- tests/agent_loop/test_diffusion_agent_loop.py is updated for the new API; in
  standalone test mode LLMServerManager spins up its own replicas via
  rollout.nnodes / n_gpus_per_node.

Pin / docs / CI:
- Bump the pinned verl commit to a4351480 (the merge commit of #5209), which is
  the first commit that ships verl/experimental/reward_loop/router/ in the wheel
  AND contains the #6129 refactor that this change adapts to. With this commit,
  the workaround in PR verl-project#51 (clone + editable install) is no longer required.
- Restore the simple `uv pip install git+...@<commit>` install line in
  docs/start/install.md.
- Bump the same pin in .github/workflows/{cpu_unit_tests,sanity,type-coverage-check}.yml.

This is a BREAKING change because DiffusionAgentLoopWorker.__init__ signature changed.
Any downstream code that subclasses or directly instantiates DiffusionAgentLoopWorker
must switch from (servers, load_balancer_handle, teacher_servers, teacher_load_balancer_handle)
to (llm_client, teacher_client). No public CLI/config surface is affected.

Signed-off-by: samithuang <285365963@qq.com>

* [BREAKING][rollout] feat: adapt to verl LLMServerClient refactor

Adapt verl-omni's diffusion agent loop and ray trainer to verl-project/verl#6129,
which removed AsyncLLMServerManager and made AgentLoopManager / AgentLoopWorker
consume an LLMServerClient produced by a separately-owned LLMServerManager.

verl-omni changes:
- DiffusionAgentLoopWorker.__init__ now takes (config, llm_client, teacher_client,
  reward_loop_worker_handles), matching the positional contract that
  AgentLoopManager.create() uses when spawning workers. _get_rollout_and_model_config
  was also dropped upstream, so the config slicing is inlined to keep the diff
  minimal.
- ray_diffusion_trainer now creates an LLMServerManager first, hands its client to
  AgentLoopManager.create(), and uses llm_server_manager.get_replicas() (instead of
  async_rollout_manager.rollout_replicas) to wire the CheckpointEngineManager. This
  mirrors the new pattern in upstream verl/trainer/ppo/ray_trainer.py.
- tests/agent_loop/test_diffusion_agent_loop.py is updated for the new API; in
  standalone test mode LLMServerManager spins up its own replicas via
  rollout.nnodes / n_gpus_per_node.

Pin / docs / CI:
- Bump the pinned verl commit to a4351480 (the merge commit of #5209), which is
  the first commit that ships verl/experimental/reward_loop/router/ in the wheel
  AND contains the #6129 refactor that this change adapts to. With this commit,
  the workaround in PR #51 (clone + editable install) is no longer required.
- Restore the simple `uv pip install git+...@<commit>` install line in
  docs/start/install.md.
- Bump the same pin in .github/workflows/{cpu_unit_tests,sanity,type-coverage-check}.yml.

This is a BREAKING change because DiffusionAgentLoopWorker.__init__ signature changed.
Any downstream code that subclasses or directly instantiates DiffusionAgentLoopWorker
must switch from (servers, load_balancer_handle, teacher_servers, teacher_load_balancer_handle)
to (llm_client, teacher_client). No public CLI/config surface is affected.

Signed-off-by: samithuang <285365963@qq.com>