Collapsing cubes with weights kwarg very inefficient (full realization of data, twice) #47
Description
two used cases from running a recipe and profiling it via debug and resource file:
First instance example
-----------------------
2019-02-26 14:03:40.811397 7250.1 7576.5 108 2.7 0 0.606 0.006
2019-02-26 14:03:41.897243 7251.2 7577.7 108 3.8 0 0.606 0.006
2019-02-26 14:03:42.984943 7252.3 7578.9 108 5.8 0 0.606 0.006
2019-02-26 14:03:44.071390 7253.4 7580.0 108 6.4 0 0.606 0.006
2019-02-26 14:03:45.156149 7254.5 7581.2 109 7.9 0 0.606 0.006
2019-02-26 14:03:46.243115 7255.5 7582.4 108 9.4 0 0.606 0.006
2019-02-26 14:03:47.341183 7256.6 7583.6 109 11.1 1 0.606 0.006
2019-02-26 14:03:48.465922 7257.8 7584.8 111 11.7 1 0.606 0.006
2019-02-26 14:03:49.567116 7258.9 7586.0 110 13.2 1 0.606 0.006
2019-02-26 14:03:50.656928 7260.0 7587.1 96 0.6 0 0.606 0.007
2019-02-26 14:03:41,277 UTC [16489] DEBUG esmvaltool.preprocessor:197 Running preprocessor step average_region
2019-02-26 14:03:41,280 UTC [16489] DEBUG esmvaltool.preprocessor:187 Running area_average(precipitation_flux / (kg m-2 s-1) (time: 1740; latitude: 360; longitude: 720)
2019-02-26 14:03:41,284 UTC [16489] INFO esmvaltool.preprocessor._area_pp:179 Calculated grid area:(1740, 360, 720)
2019-02-26 14:03:50,494 UTC [16489] DEBUG esmvaltool.preprocessor:197 Running preprocessor step cmor_check_data
Second instance example
-----------------------
2019-02-26 14:36:54.558816 9243.9 9778.3 108 12.4 1 0.708 0.007
2019-02-26 14:36:55.667920 9245.0 9779.5 111 13.5 1 0.708 0.007
2019-02-26 14:36:58.053553 9247.4 9781.9 100 0.5 0 0.708 0.007
2019-02-26 14:17:33,671 UTC [16489] INFO esmvaltool.preprocessor._area_pp:179 Calculated grid area:(1740, 360, 720)
2019-02-26 14:36:57,901 UTC [16489] DEBUG esmvaltool.preprocessor:197 Running preprocessor step cmor_check_data
I am assigning this to me since if I can't optimize it myself then I can talk to Bill next week.