Dear @arogozhnikov,
In this implementation, we have additional computation because of right side padding of the input, which dosen't require in the Casual Convolution. Moreover, when self.__padding is not zero it also introduces an additional copy in compared to conv1d. you can check it in the following

just print(id(result)) is added to the forward method of the @arogozhnikov's implimentation

class CausalConv1d(torch.nn.Conv1d):
    def __init__(self,
                 in_channels,
                 out_channels,
                 kernel_size,
                 stride=1,
                 dilation=1,
                 groups=1,
                 bias=True):
        self.__padding = (kernel_size - 1) * dilation

        super(CausalConv1d, self).__init__(
            in_channels,
            out_channels,
            kernel_size=kernel_size,
            stride=stride,
            padding=self.__padding,
            dilation=dilation,
            groups=groups,
            bias=bias)

    def forward(self, input):
        result = super(CausalConv1d, self).forward(input)
        print(id(result))  # printing object refrence
        if self.__padding != 0:
            return result[:, :, :-self.__padding]
        return result

input = torch.from_numpy(np.ones((3,2,5))).float()
CaConv1d = CausalConv1d(in_channels=2, out_channels=6, kernel_size=2, dilation=1)
out = CaConv1d(input)
print(id(out))

output:

140510012165504
140510011010144

as you see, the objects references are different; thetefore, new object is created.
but when we do not have padding it is correct.

CaConv1d = CausalConv1d(in_channels=2, out_channels=6, kernel_size=1, dilation=1) 
out = CaConv1d(input) 
print(id(out))

output:

140510011660256
140510011660256

so, these are same python object.

Causal Convolution #1333

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions