-
Notifications
You must be signed in to change notification settings - Fork 5k
worker-server node gets task performance problem(worker-server节点获取任务性能问题) #242
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or request
Milestone
Description
I saw the source code and found the following question. I don’t know if it is correct.
In the worker-server node, when task_queue is empty, FetchTaskThread will loop all the time to get the lock, then take the task from task_queue, and then release the lock, but most of the time, the task_queue may be empty, which will cause the zookeeper distribution to be created continuously. Lock
- When there are more worker nodes, will there be too much overhead for zookeeper?
- I don't know if there is any other intention in this design.
- Is it possible to add a timeout in taskQueue.poll, block it all the timeout, wait for task
看了源码,发现下面这个问题,不知理解是否正确
worker-server节点中,当task_queue为空时,FetchTaskThread会一直循环,去获得锁,然后从task_queue里面取task,然后释放锁,但大部分时间,可能task_queue为空,这会造成不断的创建zookeeper分布式锁
1.当worker节点比较多时,对zookeeper会不会造成过多的开销
2.不知道这种设计是否还有其它用意
3.是否可以在taskQueue.poll时加一个timeout,在timeout时间内一直block着,等待task
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request