Skip to content

[diffusion] It is may be not a good design to settle_port dynamic #21284

@JohnHerry

Description

@JohnHerry

self.scheduler_port = self.settle_port(initial_scheduler_port)

Sglang[Diffusion]. Here, The server using a port that is bias from the user specified in command line [or yaml config] argument. that may introduce some chaos.
For example, I had made a server process on local tcp [instead of the default http proto] with launch_server based on a server_args. but because of the settle_port, I do not know which port is the truly port finally used, and I can not write my client code to connect to this port.
and, because of the settle_port will select an unused port in the machine, so , when the server based on a server_args launched, the configed port may be taken and go into used state. And so I can not use another server_args object base on the same config.yaml to write the client program, to connect to the server, because the internel scheduler_port of the client program must have been different.

All in all, Although the settle_port make sure the server can be launched when the desired port is taken [by using another unused port which is not configed], but it makes some chaos in the situation when you need to seperate server process and client processes. OR some other situations when we need to accurately control the scheduler_port finally used.

Maybe there had been some other design in sglang[diffusion] that can solve my situations, then a relative help is welcome. thanks. I know I can reset this property after obeject was created, it seems argly when you had specified the port in your config file and use the file to create a server_args obj, but then, you should set it again.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions