cache lookup of importlib metadata in Node action#365
Conversation
Signed-off-by: William Woodall <william@osrfoundation.org>
Signed-off-by: William Woodall <william@osrfoundation.org>
|
IIUC, this means that if one installs packages (that add extensions) while launch is still spinning up nodes, the nodes that start after the installation is done won't reflect the changes, right ? If yes, that seems like a weird use case to me, as when the installation completes is not predictable, and nor does launch guarantee the order in which nodes start. So yes a +1 to this change, looks good ! Would this cache also affect nested actions or launch files ? i.e. if a launch file includes another launch file will this cache be shared ? |
|
Right, the first time a node asks for the list of extensions it is cached, so if you install something new in parallel, it will be a race to see if it is included or not and with which nodes. I can't imagine a use case for that anyway, but I wanted to point it out. |
adityapande-1995
left a comment
There was a problem hiding this comment.
Looks good with green CI !
fujitatomoya
left a comment
There was a problem hiding this comment.
it will prevent extensions from changing after launch has started
i believe this is not the user expectation or assumption. lgtm.
* cache lookup of importlib metadata in Node action Signed-off-by: William Woodall <william@osrfoundation.org> * style fixup Signed-off-by: William Woodall <william@osrfoundation.org> --------- Signed-off-by: William Woodall <william@osrfoundation.org>
* cache lookup of importlib metadata in Node action Signed-off-by: William Woodall <william@osrfoundation.org> * style fixup Signed-off-by: William Woodall <william@osrfoundation.org> --------- Signed-off-by: William Woodall <william@osrfoundation.org>
* cache lookup of importlib metadata in Node action Signed-off-by: William Woodall <william@osrfoundation.org> * style fixup Signed-off-by: William Woodall <william@osrfoundation.org> --------- Signed-off-by: William Woodall <william@osrfoundation.org>
The short version of this is that we use
importlib_metadatato load extensions to theNodeaction via Python'sentry_pointpattern. This lets new packages extendNodewithout changinglaunch_rositself. The problem is that we load these extensions every time a node needs them and that can take as long as 1 second on some machines. If you have a launch file with hundreds of nodes, like I did, then you get pauses of 30 seconds to a minute before the launch file starts launching things.I used the profiler to find out that this
launch_ros.actions.Node.get_extensions()was being called some 296 times in my launch file and each time it was taking ~0.4 seconds, leading to a delay of almost 30 seconds:After the change in this pr, the profiler shows what I experience on the command-line, which is much better behavior, taking almost no time to start launching things, since we only look this up once:
Some potential issues with this pr:
For the first item, I don't think the rest of the code is written to assume this is possible in the first place, and I'm not sure what the use case for this would look like. I suppose you could have launch running, and while running install new python packages that add extensions, then cause launch to start a new node after this, and in that case it would not see new extensions. However, I don't know when you'd do that in practice. And I think this change benefits the existing users at the cost of a potential use case that I do not believe folks are using today, but I'm open to hear otherwise.
For the second, I don't think we necessarily need to cache more, based on what I see in the second profiler.