Speeding up simplify_neuron#472
Conversation
* may want to think about making pruned_edges return what it receives
* but actually it turns out that it does not do the same as simplify_neuron because the first computed path does not use the root as an origin
* seems to match simplify_neuron now * still slower
* this does mean that some distances will be double computed but perhaps that is for the best when the memory requirements get so large. * I think we could get some further efficiencies by dropping leaves.
* when finding a simple path between two points across a neuron it is much faster *not* to use weights
|
I benchmarked your latest commits: # large neuron
benchmark("simplify_neuron_old" = {
simplify_neuron_old(nrn[[1]], n=2)
},"simplify_neuron" = {
simplify_neuron(nrn[[1]], n=2)
},"simplify_neuron2" = {
simplify_neuron2(nrn[[1]], n=2)
}, replications = 25)
test replications elapsed relative user.self sys.self
2 simplify_neuron 25 52.759 1.000 49.262 3.017
1 simplify_neuron_old 25 202.476 3.838 124.206 63.886
3 simplify_neuron2 25 113.827 2.157 112.653 0.965
# small neuron
benchmark("simplify_neuron_old" = {
simplify_neuron_old(Cell07PNs[[11]], n=2)
},"simplify_neuron" = {
simplify_neuron(Cell07PNs[[11]], n=2)
},"simplify_neuron2" = {
simplify_neuron2(Cell07PNs[[11]], n=2)
}, replications = 25)
test replications elapsed relative user.self sys.self
2 simplify_neuron 25 0.209 1.035 0.202 0.007
1 simplify_neuron_old 25 0.202 1.000 0.184 0.018
3 simplify_neuron2 25 0.232 1.149 0.228 0.004Looks like we get almost 4x gain and are slightly faster than simplify_neuron2. Some outstanding comments:
|
|
Thanks @dokato. I added another commit, which changes things quite a bit more (still for the better I hope in most cases). |
|
|
Great, that looks really good - almost 8x gain on complex cases. See updated benchmarks below: |
Codecov Report
@@ Coverage Diff @@
## master #472 +/- ##
==========================================
- Coverage 76.89% 76.85% -0.04%
==========================================
Files 47 47
Lines 5856 5825 -31
==========================================
- Hits 4503 4477 -26
+ Misses 1353 1348 -5
Continue to review full report at Codecov.
|
|
@dokato I'm going to merge this now. Thanks for getting it going. The codecov/project diff failure is a false positive (the missing lines are warnings that are never triggered because they shouldn't be!). Is there a way to adjust the config for codecov to be a bit more forgiving? |
|
Not sure, but I'll take a look into that. I'm still confused of why coveralls hasn't worked for us. |
* was initially part of #472 but later superseded
Apart from suggestions @jefferis listed in #471 I noticed (following profiling) that calling apply on columns is much longer than just running dijkstra from start node again, compare:
This already reduces time by half:
Said neuron below to compare:
large_neuron.rds.zip