fix corner cases with slicing#10
Conversation
…ather than just counts of inputs/outputs.
…gs the same code generation used by custom_model to the main Model class itself for __call__ methods. Now it's generally unnecessary for Model subclasses to redefine __call__ even if they want to update the method signature to reflect the names of their inputs. That is now automatic. Also the Model base class no longer makes any assumptions about the number of a model's inputs or outputs. It assumes no inputs/outputs until otherwise specified in a subclass.
…t came up, interestingly, when trying to call help() on a model class.
…h are no longer needed due to the automatic code generation in the _ModelMeta metaclass.
… generating __init__ methods as well. This allows still more boilerplate to be removed from most Model subclasses (though it still allows individual subclasses to define their own __init__ when needed--see for example the polynomial models).
…orator, since that part is now already handled (and somewhat better) by _ModelMeta.__new__
…onger needs to include its own copy of the make_func_with_sig utility. I also renamed make_func_with_sig to make_function_with_signature as suggested here: astropy#2835 (comment) Previously I thought the unabbreviated name was a little on the long side. But on second thought I prefer the clarity here.
…le output models.
… a True-like value, so ensure that we actually get a float when a float result is expected.
…creating new model classes with custom names.
…erly support parameterless models here.
…om for improvement here but this is a start.
…y expression. This requires one input which is a mapping of operators to a numerical precedence so that the correct order of operations can be determined. This ended up looking a lot like ExpressionTree.evaluate, which might be able to be used to implement this with a little refactoring.
…afs. This method still needs more work though :/
…s. In retrospect it should have been blatantly obvious that was a bad idea, and unnecessary anyways. Turns out Python creates a new method object every time you access a method on an object anyways (this is still reasonably efficient, as it uses a cache of already malloc'd PyMethodObjects and just updates the im_func and im_self pointers on those objects).
…il the enhancements to the tree display are complete.
… a new instance is returned, rather than the class itself. If the class is for some reason needed it can still be gotten at, but in most cases it seems to be clearer if we just return an instance.
…om so that it's out of the way while I do a little more reorg.
… process I actually rewrote traverse_postorder() to be, I think, a little clearer as to what's going on. But it still had the same bug (worse even because now it would go into an infinite loop). But simply ensuring that one node does not have two subtrees that are the same *object* seems good enough, I think.
…not that it really matters I just think tuples look better for cases like this
…es, instances of compound models should be treated as single entities when being used in an expression--their tree should not be added to the tree for the expression they're being used in.
Rotation2D model. - Rotation2D was using a pre-computed rotation matrix that was computed when the model is instantiated (regardless if it's ever evaluated). This matrix was never *updated* though, so if the user sets a new angle parameter it would still compute the rotation with the old rotation matrix. In my initial fix for this I thought of using an LRU cache (specificlly the functools.lru_cache from Python 3.3+ which I included a backport of), which would work for single angles, but which does not work for Numpy arrays (and hence arrays of angles, which are technically allowed). I may reintroduce the cache idea again later on (perhaps we could cache only single-valued parameters and just not array parameters which is fine for most cases) but for now it's an unnecessary complication. - I added a workaround for the problem where parameter values stored in radians were being converted from radians to degrees back to radians again in order to perform the evaluation. This is just an ugly workaround for now until I've refactored how parameter values are stored.
…y wound up revealing a number of seemingly unrelated bugs which are now fixed. I also modified the ExpressionTree.evaluate method so that the 'getter' callable now takes two arguments, the first of which is the node's leaf index, which proves to be useful to have available.
…ccept names yet (only integers) but that will be next. Compound model instances can't be sliced yet, but that is also coming soon.
… exactly motivated that, but I don't think it's necessary anymore (undoing it doesn't break any existing tests at least). In any case this is one small necessary step toward the reworked handling of model instances in compound models.
…nything. Fixed that, and then fixed another bug in the subexpression evaluation algorithm that the now fixed tests revealed.
…d in question is defined on some parent class.
…del classes *and* instances, by name, by index, or some combination thereof. Added a few basic tests of the capability.
…de, once said legacy code has been removed.
… list) rather than a lambda--turns out lambdas will create problems for pickling)
…his converts the values in Model._param_metrics to a dict rather than a tuple for easier understanding of the code (maybe at some point a namedtuple?) and merged the _param_broadcast_shape attribute into _param_metrics (no reason they should be separate)
…ce--ensure that regardless of whether the compound model's template used model classes or instances a *new* model instance is returned whose parameters are linked to the compound model's parameters. This was done for now by changing Model._from_existing to a sharedmethod.
|
examples updated |
|
Thanks--incidentally I just found some other issues with slicing that only fail on Python 2, but not on Python 3 (where most of my testing is done anymore). Could you please add a regression test based on this example? Also it looks like this fix might still need some tweaking. Although it did fix the >>> model[1:6]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "astropy/modeling/core.py", line 1834, in __getitem__
index = self.__class__._normalize_index(index)
File "astropy/modeling/core.py", line 1780, in _normalize_index
stop = check_index(stop)
File "astropy/modeling/core.py", line 1758, in check_index
raise IndexError("Model index {0} out of range.".format(index))
IndexError: Model index 6 out of range.
>>> model['shift_x':'rotation']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "astropy/modeling/core.py", line 1834, in __getitem__
index = self.__class__._normalize_index(index)
File "astropy/modeling/core.py", line 1780, in _normalize_index
stop = check_index(stop)
File "astropy/modeling/core.py", line 1758, in check_index
raise IndexError("Model index {0} out of range.".format(index))
IndexError: Model index 3 out of range.In the first case, although 6 is out of range, it should work like Python lists in that an out of range end-point should be equivalent to "take until the end". The second case obviously should also work (it should include 'shift_x' through 'rotation' inclusive). |
e0eb8c2 to
680baba
Compare
|
Ack, sorry, I just rebased my compound modeling branch too, so you'll have to rebase this PR as well. |
|
Well, my intention was to raise an error if the name is not in the model.submodel_names, not to return the model until the end (like lists). The same for indexing. With the WCS use case in mind I consider these a user error. If a user does: |
|
For a non-existent name, sure. I'm just talking about when using integers, I was trying to keep in the interface consistent with the expectations for integer-based slicing. Using names is different. |
|
It's too much hassle to go through all conflicts. On Wed, Dec 17, 2014 at 1:38 PM, Erik Bray notifications@github.com wrote:
|
|
Ah, sorry. Since this was based on an older version of my branch which was since rebased, then a simple rebase doesn't work. Since this is just one commit though the easiest thing to do is reset your branch and then cherry-pick the last commit: I didn't realize when pushing a rebase of my branch that it would screw up existing PRs (it should not be hard for GH to do this automatically). Sorry. |
680baba to
0dda7ee
Compare
|
No problem about the rebase, I'll make a new PR against the updated branch. One more question about slicing using integers vs slicing using labels - with labels it includes the stop label in the sliced models, not so with integer inidices. I wonder if this will be confusing. How shoudl I write the regression tests given that model comparison is not implemented yet? I can compare sliced_model.submodel_names and/or sliced_model._submodels ? Is there a better way? |
|
@nden Okay, no problem. A new PR when you're ready to is fine. I actually noticed that one or two of the slicing-related tests were failing on Python 2 but not on Python 3 (where I'm doing most of my testing now) so I definitely need to look into that. That might be the same problem, but I'm not sure.
I think in a way it's a little confusing, but there's also precedent for it. The fact that normal (with integers) slicing does not include the end point makes sense for doing index arithmetic and zero-based indexing. But none of that logic applies when using strings. Other libraries that have some objects that can be sliced by string labels take the same approach (see for example, Pandas). It's less confusing to have integer slices work just like slicing lists, but for labels the end-point should be inclusive (otherwise there's no way to make a slice that explicitly includes the end-point via labels).
Looking at There isn't otherwise a great API right now to loop over or introspect the models making up a compound model except be looking at |
Make sure that astropy_helpers/src gets installed
@embray
This is a suggested PR to fix a few minor problems I found with slicing. The examples below did not work as expected: