Merged
Conversation
* first commit * enclose classmethods in the class * brought in minor improvements to halos * coverage
damonge
reviewed
Apr 18, 2023
Collaborator
damonge
left a comment
There was a problem hiding this comment.
This I can live with (just a couple spurious comments). Ping me for review when it's fully implemented
Collaborator
|
Also, there seem to be conflicts |
Contributor
Author
|
@damonge back to you |
damonge
approved these changes
Apr 21, 2023
damonge
added a commit
that referenced
this pull request
May 3, 2023
* halo profiles in their own directory * concentrations in their own directory * mass functions & halo biases in their own directory * moved CIB stuff to profiles & 2pts * split halo model into 1/2/4-pt * fix imports * remove empty module * fix typo * Baryons in v3 (#1039) * Added new baryons module that will deprecate old BCM * Tracers v3 (#1040) * Tracers in V3 * Background, boltzmann, and cls in v3 (#1041) * Background, boltzmann and cls in v3 * Covariances in v3 (#1046) * Covariances in V3 * Correlations v3 (#1042) * Correlations in v3 * split base.py and restructured into its own dir to simplify * Neutrinos, parameters, power in v3 (#1047) * done * Make `T_ncdm` a cosmological parameter + (v3) (#1049) * first commit * rename TNCDM to T_ncdm but preserve API * T_CMB directly into cosmo struct * Omega_g in ccl_parameters_create; no split between C/Python * physical_constants.T_CMB doesn't mutate anymore! * force mutate T_CMB in benchmarks * define defaults on instantiation just in case constants mutate * A_s & sigma8: don't play with numbers * simplify * remove leftover mallocs * temporarily preserve API for CCLv3 * Unfreeze option for `physical_constants` (#1050) * first commit * update rtd * addressed comments 1 * Remove `T_CMB` and `T_ncdm` as constants (reloaded) (#1058) * first commit * OmNuh2 fix * addressed comments * keep only massive * first step * temp commit * debug neutrinos & deprecate Omnuh2 * ccl_omega_x & ccl_Omeganuh2 consistency --------- Co-authored-by: David Alonso <dam.phys@gmail.com> * RTD: Removed The Docs --------- Co-authored-by: David Alonso <dam.phys@gmail.com> * flaked --------- Co-authored-by: Nick Koukoufilippas <nikfilippas@gmail.com> * Pk2D and Tk3D in v3 (#1048) * tk3d and pk2d in v3 --------- Co-authored-by: Nick Koukoufilippas <nikfilippas@gmail.com> * PT biases in v3 (#1056) * New PT bias framework * Halos v3 (#1043) * ported changes in halos/profiles * changes from star in halos/ (no tests) * refactor concentration, mass_function, halo_bias; improvements in HMCalculator * HaloProfile subclasses & HaloProfile.normprof attribute * simplify imports * remove HaloProfile.name * cleaned up pk_Npt * add extrap_pk argument * FFTLogParams class instead of dictionary in HaloProfile * simplified code * removed mass_def from HMIngredients; comprehensive code review 1 * re-implementation, bugfixes, tests * c_m_relation >> concentration; code improvements; A_SPLINE_MAX in Python * HMIngredients initializers - no repeated code * rogue file * patch for new changes * updated Build Status badge website * fix websites * added a DOI badge & links * added nonbreaking space * removed extra space * FancyRepr out of CCLObject to simplify * update tests to match changes * coverage * simple test for tkkssc * more tests for tkkssc * addressed comments 1 * addressed comments 2: mass_def defaults are name strings etc. * deprecate Gaussian & PowerLaw profiles * bugfix in master: sometimes mass_def_strict=False fails unexpectedly * typo * addressed comments * New feature: Arbitrary function for profile normalization * gain efficiency * comments + deprecate k_min from HMCalculator * renamed initialize_from_input --> create_instance * brought in CCLNamedClass from docs_v3 * brought in from docs_v3: directly callable HMIngredients * prep for Davids input * renamed HMCalculator --> HaloModel * fix typo * brought in from docs_v3: initialize mass_def from any string (e.g. 400c) * Revert "renamed HMCalculator --> HaloModel" This reverts commit 43591ce. * counterterms func inside of 4pt func * replace deprecated abstractproperty * abstract linked methods for HaloProfile, MassFunc, HaloBias, Concentration * fully working implementation * renamed lM --> log10M etc. * minor cosmetic fix * alternative way to include linked abstract methods and declare the template methods * reorder * even simpler * bring back eq_attrs * r -> r_t * Additional halos features (#1068) * first commit * enclose classmethods in the class * brought in minor improvements to halos * coverage * Tests v3 (#1059) * ported changes in halos/profiles * changes from star in halos/ (no tests) * refactor concentration, mass_function, halo_bias; improvements in HMCalculator * HaloProfile subclasses & HaloProfile.normprof attribute * simplify imports * remove HaloProfile.name * cleaned up pk_Npt * add extrap_pk argument * FFTLogParams class instead of dictionary in HaloProfile * simplified code * removed mass_def from HMIngredients; comprehensive code review 1 * re-implementation, bugfixes, tests * c_m_relation >> concentration; code improvements; A_SPLINE_MAX in Python * HMIngredients initializers - no repeated code * rogue file * patch for new changes * updated Build Status badge website * fix websites * added a DOI badge & links * added nonbreaking space * removed extra space * FancyRepr out of CCLObject to simplify * update tests to match changes * coverage * simple test for tkkssc * more tests for tkkssc * addressed comments 1 * addressed comments 2: mass_def defaults are name strings etc. * deprecate Gaussian & PowerLaw profiles * remove all warnings in testing * use pytest as per docs instead of numpy.testing and pyutils.assert_warns * bugfix in master: sometimes mass_def_strict=False fails unexpectedly * typo * addressed comments * New feature: Arbitrary function for profile normalization * gain efficiency * comments + deprecate k_min from HMCalculator * renamed initialize_from_input --> create_instance * brought in CCLNamedClass from docs_v3 * brought in from docs_v3: directly callable HMIngredients * prep for Davids input * renamed HMCalculator --> HaloModel * fix typo * brought in from docs_v3: initialize mass_def from any string (e.g. 400c) * Revert "renamed HMCalculator --> HaloModel" This reverts commit 43591ce. * comments * comments * shortcut for __eq__ methods * Eq. for EulerianPT calculator (#1073) * __eq_attrs__ for nl_pt * No normprof (#1072) * extricated normprof * Cosmology v3 (#1071) * rename core --> Cosmology * renamed core --> cosmology in docs/imports except docstrings * cosmology v2 * first pass * neutrinosneutrinosneutrinos * simplified * yamlyamlyaml * fully fix neutrino mayhem * new arg names in tests * comments * comments on comments on comments * removed tests * no warn --------- Co-authored-by: David Alonso <dam.phys@gmail.com> * avoid warning due to zeros (#1076) * Fix halo inconsistencies v3 (#1069) * ported changes in halos/profiles * changes from star in halos/ (no tests) * refactor concentration, mass_function, halo_bias; improvements in HMCalculator * HaloProfile subclasses & HaloProfile.normprof attribute * simplify imports * remove HaloProfile.name * cleaned up pk_Npt * add extrap_pk argument * FFTLogParams class instead of dictionary in HaloProfile * simplified code * removed mass_def from HMIngredients; comprehensive code review 1 * re-implementation, bugfixes, tests * c_m_relation >> concentration; code improvements; A_SPLINE_MAX in Python * HMIngredients initializers - no repeated code * rogue file * patch for new changes * updated Build Status badge website * fix websites * added a DOI badge & links * added nonbreaking space * removed extra space * FancyRepr out of CCLObject to simplify * update tests to match changes * coverage * simple test for tkkssc * more tests for tkkssc * addressed comments 1 * addressed comments 2: mass_def defaults are name strings etc. * deprecate Gaussian & PowerLaw profiles * bugfix in master: sometimes mass_def_strict=False fails unexpectedly * typo * addressed comments * New feature: Arbitrary function for profile normalization * gain efficiency * comments + deprecate k_min from HMCalculator * renamed initialize_from_input --> create_instance * brought in CCLNamedClass from docs_v3 * brought in from docs_v3: directly callable HMIngredients * prep for Davids input * renamed HMCalculator --> HaloModel * fix typo * brought in from docs_v3: initialize mass_def from any string (e.g. 400c) * Revert "renamed HMCalculator --> HaloModel" This reverts commit 43591ce. * counterterms func inside of 4pt func * replace deprecated abstractproperty * abstract linked methods for HaloProfile, MassFunc, HaloBias, Concentration * fully working implementation * renamed lM --> log10M etc. * minor cosmetic fix * first commit * alternative way to include linked abstract methods and declare the template methods * reorder * even simpler * enclose classmethods in the class * brought in minor improvements to halos * clone of 1064 * bring back eq_attrs * r -> r_t * Additional halos features (#1068) * first commit * enclose classmethods in the class * brought in minor improvements to halos * coverage * removed MassConcentration * full implementation (v3 only) * comments * single-liner * coverage * Final checklist v3 (#1075) * imports * homogenize all imports as per PEP8 and the Python Import System * homogenize all imports as per PEP8 and the Python Import System * instance checks * all exceptions raised correctly * comments * add aliases * reverted changes and introduced proper enums * Preserve API in v2.final (#1077) * first commit * Einasto mass translator + MassDef concentration API * universal api no-break * comments * MassDef vars * Coverage v3 (#1078) * coverage * comments --------- Co-authored-by: David Alonso <dam.phys@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains fixes to some design flaws in halos which can ultimately cause memory leaks and model inconsistencies.
Note: at the moment it breaks API (to make review easier), but once review is complete I will clone it into a non API-breaking one.
(5' read)
Problem 1
When
MassDefis instantiated with a concentration it inadvertently contains a reference to itself:This causes a memory leak because when the object is deleted, the garbage collector doesn't know the order of deletion so the instances hang in there for the rest of the runtime. This can easily be demonstrated:
which outputs
An easy way to fix that is to use a weakref proxy of
selfwhen passing it to theConcentrationconstructor:import weakref ... self.concentration = Concentration(weakref.proxy(self)). This allows the reference count ofmass_defto go to zero and the gbc to purge it. This tactic is not generally advised though.However, there is a deeper design flaw with the interplay of
MassDefandConcentration.Problem 2
Matter profiles require a
Concentrationto initialize. However, this isn't really a parameter; it is akin to setting up the model. As ismass_def(which is why we agreed to deprecate it from calls to subclasses ofHMIngredients). This is how these two are used in the code, in every matter profile:so they aren't used in anything other than finding the comoving virial radius, given
M. These two work together; they should be together, but one is passed at initialization and the other one at calls. What's more, this creates an inconsistency: themass_defofconcentrationmay not be the same as themass_defpassed in_real. Luckily, most implemented concentrations aremass_def-agnostic, and also users have (hopefully) been passing the same one, but this can easily create problems.Solution
_realis called to verify that the mass definitions are consistent. That's a bad solution, no explanation needed.concentrationin__init__and pass it as an extra argument in e.g._real. That's a bad solution because not all profiles use aConcentration, so we'll end up unnecessarily complicating the code.mass_defto__init__and deprecate it from e.g._real. That's okay, but the non-matter profiles still usemass_defto compute the radius, given the mass. That means, in order to be consistent, we'd have to introducemass_defas an__init__argument in all profiles.MassDefandConcentrationare so strongly coupled, it makes sense to combine them into one single object. Then, every profile will accept aMassConcentrationobject in__init__which contains all the info it will ever need: mass definition and an optional concentration. This will simplify things.In practice, the new class would contain
self.mass_defandself.concentrationand would avoid self-references and other caveats. It would initialize both the mass definition and the concentration prescription at the same time.This is what it could look like:
and then, all halo profiles would have:
We could even go as far as implementing a
get_comoving_virial_radiusmethod, to do all of that in a single step and avoid having to repeat that in all matter profiles.It would also make physical sense because the parameters passed in
__init__would uniquely define a profile 'entity', which would then be called via its methods and(r, M, a)(andcosmowhich is an outlier as we've already discussed).For
v2.finalwe can leave things as they are, but add warnings that these will be deprecated in the future, and point users towards usingMassConcentrationobjects instead of separateMassDefandConcentration.The solution would simplify the code - for example, in
benchmarks/test_haloprofile.pywe doso there are two instantiations of a mass definition to get the one we want to use. With the new implementation we won't need that - it would be a single step.
This proposal is a widely used design pattern in OOP called "dependency injection", and is a technique where an object is passed as a dependency to another object, rather than having the dependent object create the dependency itself. It helps to reduce coupling between different objects, and makes the code more modular and testable.