Skip to content

worker-server node gets task performance problem(worker-server节点获取任务性能问题) #242

@ivivi

Description

@ivivi

I saw the source code and found the following question. I don’t know if it is correct.
In the worker-server node, when task_queue is empty, FetchTaskThread will loop all the time to get the lock, then take the task from task_queue, and then release the lock, but most of the time, the task_queue may be empty, which will cause the zookeeper distribution to be created continuously. Lock

  1. When there are more worker nodes, will there be too much overhead for zookeeper?
  2. I don't know if there is any other intention in this design.
  3. Is it possible to add a timeout in taskQueue.poll, block it all the timeout, wait for task

看了源码,发现下面这个问题,不知理解是否正确
worker-server节点中,当task_queue为空时,FetchTaskThread会一直循环,去获得锁,然后从task_queue里面取task,然后释放锁,但大部分时间,可能task_queue为空,这会造成不断的创建zookeeper分布式锁
1.当worker节点比较多时,对zookeeper会不会造成过多的开销
2.不知道这种设计是否还有其它用意
3.是否可以在taskQueue.poll时加一个timeout,在timeout时间内一直block着,等待task

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions