This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. @carolromero & @xabier feel free to chime in.

Keep open!

Hi @Leusev @ivan-mr
Just to be clear how is the review process: have you guys tested this locally? Have you compared the memory usage before/after this branch?

Also, @tramuntanal, do you think that we need to start having performance testing to check for this kind of issue and especially for possible regressions in the future? (That could be developed on another issue to keep small the scope of this one)

Hi @andreslucena ,
I've deployed a backport of this branch to 0.21 to a Heroku staging env of ours with a dump of decidim-barcelona (which has thousands of participants) and, while running participants metrics, it reduces and stabilizes worker memory usage to around 750mb which is not ideal but is far better than having an always growing memory.

Although this problem was a design problem rather a bad performing query, performance testing would be great of course, not only for worker processes but for specially slow loading views. Probably not to be implemented up-front but as regression tests as you said.

I also plan to backport this PR to v0.22 so we can start testing it in real environments. Waiting for your approval in order to merge and backport

I also plan to backport this PR to v0.22 so we can start testing it in real environments. Waiting for your approval in order to merge and backport

I can't review it but if you are confident that this improves what we already have is OK for me

Hi @Leusev @ivan-mr
Just to be clear how is the review process: have you guys tested this locally? Have you compared the memory usage before/after this branch?

This question wasn't answered... Just to understand it, because I'm seeing lots of billed hours on PR reviews from @Leusev and @ivan-mr, but I actually don't see any comment from them in your PRs.

Hi @Leusev @ivan-mr
Just to be clear how is the review process: have you guys tested this locally? Have you compared the memory usage before/after this branch?

Hi @andreslucena
I'll try to explain review process in this case. Here, I just only reviewed modified files in PR.
I realised as the improvements followed the same pattern in all files.
In first place, I saw it was created a common visible_component_ids_from_spaces method in order to collect component ids instead of entire object to reduce memory consumption and increase speed.

• decidim-core/app/queries/decidim/metric_manage.rb

After that, I checked that in almost all other files were replaced its own method's calls by this new one.

as well as was changed the way records are saved at the moment instead of being accumulated into an instance variable

Finally, I saw it was created a shared_example included/changed in _spec.rb's files in order to speed up testing too:

• decidim-core/lib/decidim/core/test/shared_examples/metric_manage_shared_context.rb

In other hand, I also spoke with @tramuntanal and he explained me he tested it in on Heroku environment 'cause many records were needed and in order to have an environment as close to real as possible was needed too, so I accepted it.

Hope this is what you were asking for

I'll try to explain review process in this case. Here, I just only reviewed modified files in PR.

Ok, thanks for the clarification!