Handle case of n_outputs > n_inputs#12
Conversation
parameter descriptors does not work yet.
…le output models.
… a True-like value, so ensure that we actually get a float when a float result is expected.
…creating new model classes with custom names.
…s. In retrospect it should have been blatantly obvious that was a bad idea, and unnecessary anyways. Turns out Python creates a new method object every time you access a method on an object anyways (this is still reasonably efficient, as it uses a cache of already malloc'd PyMethodObjects and just updates the im_func and im_self pointers on those objects).
…erly support parameterless models here.
…om for improvement here but this is a start.
…y expression. This requires one input which is a mapping of operators to a numerical precedence so that the correct order of operations can be determined. This ended up looking a lot like ExpressionTree.evaluate, which might be able to be used to implement this with a little refactoring.
…afs. This method still needs more work though :/
…il the enhancements to the tree display are complete.
… a new instance is returned, rather than the class itself. If the class is for some reason needed it can still be gotten at, but in most cases it seems to be clearer if we just return an instance.
…om so that it's out of the way while I do a little more reorg.
… process I actually rewrote traverse_postorder() to be, I think, a little clearer as to what's going on. But it still had the same bug (worse even because now it would go into an infinite loop). But simply ensuring that one node does not have two subtrees that are the same *object* seems good enough, I think.
…upport joining models. Now the eval chain is not built up of functions, but tuples of functions and their input/output counts. This information is required to make the join operator work (and could be useful for other validation purposes)
…s into a separate method; this might prove useful later.
…be looked up by their names. Now we also need to add this capability for CompoundModel instances.
…maintain a separate name->submodel mapping just maintain an ordered tuple of the names of the submodels (with ordering corresponding to their order in the ._submodels list). In particular I think this will prove more useful for compound model instances
…s as well (though indexing does not work yet on instances; that's going to take some more reworking
… list) rather than a lambda--turns out lambdas will create problems for pickling)
…his converts the values in Model._param_metrics to a dict rather than a tuple for easier understanding of the code (maybe at some point a namedtuple?) and merged the _param_broadcast_shape attribute into _param_metrics (no reason they should be separate)
…ce--ensure that regardless of whether the compound model's template used model classes or instances a *new* model instance is returned whose parameters are linked to the compound model's parameters. This was done for now by changing Model._from_existing to a sharedmethod. Add a reminder to remove code that is only there to support legacy code, once said legacy code has been removed.
…eventing adding a name to Rotation2D)
…, since it's not very useful in its current state. I have a new implementation of it that is much better and is 90% done, but has a few corner cases to be worked out. I will add it back in when it's ready.
fix model slicing corner cases
…ndex, which is doing the same thing, essentially.
…he print() function.
…n 2. There is still a separate bug that seems to affect both Python versions, but this fixes the case that was currently in the test suite.
…methods; it seems at some point I accidentally reversed the logical sense of this. The doctest is still broken on Python 3, but it's a mystery that it's even running at all (doctests shouldn't be running on Python 3, generally)
compound models created from model instances, brought up in this comment: astropy#3231 (comment) This is still a bit of a hack, but not such a bad one, and far less brittle than the temporary fix that was introduced in 2112c12 (I didn't necessarily intend it to be temporary but I at least deeply hoped that a better way could be found later--for now this is it). This makes it possible to pass in additional class members that should be added to classes generated from _CompoundModelMeta._from_operator, and includes a helper function _model_operator that makes it easy to make replacements for the standard operator functions (operator.add, etc.) that can include additional members in the returned classes. This way, when taking slices, we can include the critical _slice_offset attribute in the class creation as soon as it is needed. Clearly this is only an internal detail right now, but I can envision this being potentially useful to open this up, in some form, to a documented feature for customizing the behavior of model expressions (likely with the help of some context manager in which the expressions are evaluated...)
…aken from compound models.
…Parameter will do since that's where its __init__ parameters are documented.
idea--from a class's point of view, and from the point of view of any code inspecting a class's __dict__ (particularly, API doc generation by sphinx) this makes a sharedmethod indistinguishable from a normal classmethod which is as it should be.
turbulence should still be expected, but not as *much*, at least not for the most common use cases, and I want to give more of a guarantee of backward compat support as much as possible. I don't want to scare people away from getting what they can out of this. [ci skip]
to the original report of this issue, explaining how I solved the problem: As I suspected, this had to do with a concession I had to make way back in this PR: astropy#2634 This concession is mentioned in the section of the PR text "Caveats" under item 3. The rules I came up with for broadcasting arrays work for the vast majority of models, but there are some (particularly ones like AffineTransformation2D that involve multiple coordinates) where they don't quite work. As a temporary workaround I added a `standard_broadcasting = False` flag to these models--this allows them to opt out of the standard assumptions (and, if they wish, implement their own prescriptions by overriding prepare_inputs and prepare_outputs). The trick here was to make sure that a compound model containing AffineTransformation2D (or others like it) propagate the `standard_broadcasting = False` flag. If one model in a compound model has this limitation, then it must, for now, be applied to the entire model. In the near future I hope to do something a little less brute-force here. I envision models being able to provide some sort of broadcast_hints data structure, which I think can be fairly simple. Most models wouldn't need to use this (the defaults are fine). But this could explain to the code exactly what inputs and parameters are used with each other, and how they should fit with each other. (Obviously reimplementing prepare_inputs/outputs could do this too, but I think the realistic cases are still limited enough that this can be done in a declarative manner and avoid having to reimplement lots of code over and over again). In the meantime, the fix I will add shortly should be good, I think, for most cases we care about?
|
I see. Instead of making a PR against my compound modeling branch it would be fine to just send a PR against the main Astropy--while I agree this affects compound models it's still a more general problem. And include a functional test of some sort. I think this is a reasonable workaround for now, though I think maybe this could be done in a way that's a little more clear about what's happening; it's pretty unclear from the code here why or where an |
|
I'm working on some tests for this. |
|
I have a PR with a test that I'll go ahead and submit. It takes the same approach this this fix, but instead tries to make it more explicit that this is supported at least in the basic sense. I tested the above example against the fix too and it works as expected. |
they have inputs, at least in the most basic cases. This sort of works the same as how models with more inputs than they have outputs work, which is that each input is not necessarily directly tied to each output; instead all outputs are just assumed to have the same dimensions, which are determined from the result of broadcasting all inputs with all parameters. In the future there may be more ways for individual model implementations to control this (besides outright overriding prepare_inputs and/or prepare_outputs). This provides an alternate to this PR: #12
0ae8a95 to
8bdb9bc
Compare
@embray I feel the problem with
n_outputs > n_inputsis relevant to this PR. Consider the following failing example:(It may also be relevant to single models but I don't have an example of such a model now.)
This PR suggests a way of fixing the problem, making the assumption that all outputs from a model have the same shape. I think this is a reasonable assumption for now but it certainly may be revisited later.