💊 CI failures summary and remediations

As of commit cefc65d (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

since we only have aten::batch_norm the dimension
is only known at runtime

for dimension, are we talking about BatchNorm{1|2|3}d? If so, can we get it from the module, since the majority of users will probably be using the default PT modules?

since we only have aten::batch_norm the dimension
is only known at runtime

for dimension, are we talking about BatchNorm{1|2|3}d? If so, can we get it from the module, since the majority of users will probably be using the default PT modules?

yeah, dim refers to batch_norm 1d/2d/3d, this informations is not available in aten ops, it's available only in module. we can't get it from module, we are dealing with aten ops in the IR.

yeah, dim refers to batch_norm 1d/2d/3d, this informations is not available in aten ops, it's available only in module. we can't get it from module, we are dealing with aten ops in the IR.

makes sense for the current implementation. I'm more asking for my own understanding, this info is in the graph before inlining, and we technically could spend eng time to preserve this info through the passes (by adding a new attribute, etc) if we really needed to (not saying we should do this, just trying to understand). Just curious if I understood that correctly.

(not saying we should do this, just trying to understand). Just curious if I understood that correctly.

the dimension info is encoded in the check function right now: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/batchnorm.py#L190, the check function will be different for each module. we'll need to analyze the code to extract this info.

Another thing is if people use F.batch_norm(https://github.com/pytorch/pytorch/blob/master/torch/nn/functional.py#L1998), there is no way we can get the dimension info, the dimension is known at runtime.

In general this comes from the discrepancy of the APIs between floating point op and quantized op, we have aten::batch_norm in fp but quantized::batch_norm_2d and quantized::batch_norm_3d in quantized.

although in the future I think we should move towards breaking aten::batch_norm to aten::batch_norm_{n}d, this will allow us to handle inheritance.

the dimension info is encoded in the check function right now: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/batchnorm.py#L190, the check function will be different for each module. we'll need to analyze the code to extract this info.

oh, I was asking about parsing it from the module type instead, not from the graph of the function calls. I.e. if someone is running a vanilla nn.BatchNorm{1|2|3}d Is that challenging / not possible to capture that from the module graph before any kind of inlining, and then assign it to the correponding aten node in the inlining pass? To clarify, not suggesting it, more just looking to understand curious on if this possible in the future if it's needed.

the dimension info is encoded in the check function right now: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/batchnorm.py#L190, the check function will be different for each module. we'll need to analyze the code to extract this info.

oh, I was asking about parsing it from the module type instead, not from the graph of the function calls. I.e. if someone is running a vanilla nn.BatchNorm{1|2|3}d Is that challenging / not possible to capture that from the module graph before any kind of inlining, and then assign it to the correponding aten node in the inlining pass? To clarify, not suggesting it, more just looking to understand curious on if this possible in the future if it's needed.

This is certainly possible, but I don't think we'll do this in the future. I think you'll get a better idea as you work more on graph mode.

We would like to restrict the analysis and transformations locally instead of spreading it out, e.g. you mentioned we need to get this information in module and then "pass" it to aten node, we would only do this when it is absolutely necessary I think, since this can be achieved with a much simpler alternative, as shown in the PR, I don't think it make sense to do it.

the dimension info is encoded in the check function right now: https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/batchnorm.py#L190, the check function will be different for each module. we'll need to analyze the code to extract this info.

oh, I was asking about parsing it from the module type instead, not from the graph of the function calls. I.e. if someone is running a vanilla nn.BatchNorm{1|2|3}d Is that challenging / not possible to capture that from the module graph before any kind of inlining, and then assign it to the correponding aten node in the inlining pass? To clarify, not suggesting it, more just looking to understand curious on if this possible in the future if it's needed.

This is certainly possible, but I don't think we'll do this in the future. I think you'll get a better idea as you work more on graph mode.

We would like to restrict the analysis and transformations locally instead of spreading it out, e.g. you mentioned we need to get this information in module and then "pass" it to aten node, we would only do this when it is absolutely necessary I think, since this can be achieved with a much simpler alternative, as shown in the PR, I don't think it make sense to do it.

thanks a ton. This is really helpful to understand the past tradeoffs and design principles taken in this codebase.

This pull request has been merged in 1a388da.