Skip to content
This repository was archived by the owner on Dec 22, 2021. It is now read-only.

Add groupMap and groupMapReduce operations on Iterable#253

Merged
julienrf merged 1 commit intoscala:masterfrom
julienrf:group-map-reduce
Sep 28, 2017
Merged

Add groupMap and groupMapReduce operations on Iterable#253
julienrf merged 1 commit intoscala:masterfrom
julienrf:group-map-reduce

Conversation

@julienrf
Copy link
Copy Markdown
Contributor

@julienrf julienrf commented Sep 27, 2017 •
edited
Loading

Fixes #42

I think we should put these methods (at least groupMap) into the core rather than collections-contrib because these methods have been well received by the community in my recent poll: highest rate of accept, no strong reject.

Here are the benchmarks showing a 2x speedup (blue line) over the manual equivalent (using mapValues). The benchmark consists in creating a List of 10000 elements and then calling groupMap or groupMapReduce on it. The size axis shows the size of the groups, not the collection.

groupmap

groupmapreduce

JMH report:

[info] Benchmark                                          (size)  Mode  Cnt       Score        Error  Units
[info] GroupMapBenchmark.groupMap                              2  avgt    8  335414.669 ± 149463.463  ns/op
[info] GroupMapBenchmark.groupMap                              3  avgt    8  306090.197 ±   2474.103  ns/op
[info] GroupMapBenchmark.groupMap                              5  avgt    8  309847.013 ±   7548.141  ns/op
[info] GroupMapBenchmark.groupMap                             16  avgt    8  324650.315 ±   7036.237  ns/op
[info] GroupMapBenchmark.groupMap                             17  avgt    8  329948.213 ±  18392.397  ns/op
[info] GroupMapBenchmark.groupMap                             32  avgt    8  337803.658 ±  15355.710  ns/op
[info] GroupMapBenchmark.groupMap                             33  avgt    8  340989.760 ±  12480.886  ns/op
[info] GroupMapBenchmark.groupMap                            128  avgt    8  359415.774 ±  11359.656  ns/op
[info] GroupMapBenchmark.groupMap                            129  avgt    8  383552.499 ±  74398.982  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                  2  avgt    8  350442.525 ±   5067.738  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                  3  avgt    8  359182.768 ±  25426.713  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                  5  avgt    8  392078.460 ±  99857.930  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                 16  avgt    8  388338.380 ±  31305.495  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                 17  avgt    8  415499.506 ±  88394.145  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                 32  avgt    8  410737.022 ±  71035.695  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                 33  avgt    8  465078.012 ± 130100.131  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                128  avgt    8  425787.566 ±  15421.487  ns/op
[info] GroupMapReduceBenchmark.groupMapReduce                129  avgt    8  429147.647 ±  49699.573  ns/op
[info] GroupMapValuesBenchmark.groupMap                        2  avgt    8  447224.659 ±  56984.903  ns/op
[info] GroupMapValuesBenchmark.groupMap                        3  avgt    8  500995.146 ± 119732.382  ns/op
[info] GroupMapValuesBenchmark.groupMap                        5  avgt    8  584349.929 ± 106424.296  ns/op
[info] GroupMapValuesBenchmark.groupMap                       16  avgt    8  536217.832 ±  68511.857  ns/op
[info] GroupMapValuesBenchmark.groupMap                       17  avgt    8  497226.064 ±  14962.429  ns/op
[info] GroupMapValuesBenchmark.groupMap                       32  avgt    8  524292.653 ±  63944.176  ns/op
[info] GroupMapValuesBenchmark.groupMap                       33  avgt    8  538012.643 ± 108682.615  ns/op
[info] GroupMapValuesBenchmark.groupMap                      128  avgt    8  587212.234 ±  43812.312  ns/op
[info] GroupMapValuesBenchmark.groupMap                      129  avgt    8  503130.133 ±  21871.466  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce            2  avgt    8  545629.681 ±   7655.827  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce            3  avgt    8  560725.966 ±  15911.639  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce            5  avgt    8  628930.675 ±  65039.022  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce           16  avgt    8  633669.088 ±  33263.466  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce           17  avgt    8  575034.651 ±  70962.932  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce           32  avgt    8  549855.035 ±   7105.193  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce           33  avgt    8  697515.328 ± 275448.793  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce          128  avgt    8  582547.064 ±  19308.697  ns/op
[info] GroupMapValuesReduceBenchmark.groupMapReduce          129  avgt    8  690085.695 ±  91630.126  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                   2  avgt    8  404137.920 ±  70377.296  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                   3  avgt    8  412401.580 ±  56434.929  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                   5  avgt    8  460823.333 ±  54533.045  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                  16  avgt    8  593412.725 ± 158315.323  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                  17  avgt    8  527071.588 ± 124666.375  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                  32  avgt    8  585856.941 ± 160497.525  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                  33  avgt    8  527073.124 ±  38716.043  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                 128  avgt    8  540493.416 ±  52187.542  ns/op
[info] ScalaGroupMapValuesBenchmark.groupMap                 129  avgt    8  504313.526 ±  31293.212  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce       2  avgt    8  499198.860 ±  87995.612  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce       3  avgt    8  494546.532 ±  89466.564  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce       5  avgt    8  532590.004 ±  77295.785  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce      16  avgt    8  564595.570 ±  31039.433  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce      17  avgt    8  594484.345 ±  87321.190  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce      32  avgt    8  620246.676 ± 133744.040  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce      33  avgt    8  751221.484 ± 133161.040  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce     128  avgt    8  598792.499 ±  27645.042  ns/op
[info] ScalaGroupMapValuesReduceBenchmark.groupMapReduce     129  avgt    8  644554.710 ±  98040.669  ns/op

@julienrf julienrf requested review from Ichoran and szeiger September 28, 2017 08:11
@julienrf julienrf merged commit 54503b4 into scala:master Sep 28, 2017
@julienrf julienrf deleted the group-map-reduce branch September 28, 2017 12:30
@LPTK LPTK mentioned this pull request Oct 18, 2017
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@szeiger szeiger Awaiting requested review from szeiger

@Ichoran Ichoran Awaiting requested review from Ichoran

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@julienrf @lavrov