Seems we need to mark arguments with [OPTIONAL] in THNN.h and THCUNN.h.

Yes, if you mark arguments with [OPTIONAL], then they will accept None in Python too

Hi, I download your Conv2dLocal modules and run it.
There is a bug:

torch.FloatTensor constructor received an invalid combination of arguments - got (float, float, int, int, int, int), but expected one of:

• no arguments
• (int ...)
  didn't match because some of the arguments have invalid types: (!float!, !float!, !int!, !int!, !int!, !int!)
• (torch.FloatTensor viewed_tensor)
• (torch.Size size)
• (torch.FloatStorage data)
• (Sequence data)

The code I run:
a = nn.Conv2dLocal(in_channels=256, out_channels=256, in_height=8, in_width=8, kernel_size=3, stride=1, padding=0)

What's the problems?

@Gwan-Siu

Can you give the full code to reproduce the error?

This module works for me.

a = nn.Conv2dLocal(in_channels=256, out_channels=256, in_height=8, in_width=8, kernel_size=3, stride=1, padding=0) 
input = Variable(torch.randn(1, 256, 8, 8)) 
output = a(input)

OK.

I just simple check:

a = nn.Conv2dLocal(in_width=8, in_height=8, in_channels=256, out_channels=256, kernel_size=3)
Traceback (most recent call last):

  File "<ipython-input-3-e1224e69fb35>", line 1, in <module>
    a = nn.Conv2dLocal(in_width=8, in_height=8, in_channels=256, out_channels=256, kernel_size=3)

  File "/home/gwan-siu/anaconda2/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 642, in __init__
    out_channels, in_channels, *self.kernel_size))

TypeError: torch.FloatTensor constructor received an invalid combination of arguments - got (float, float, int, int, int, int), but expected one of:
 * no arguments
 * (int ...)
      didn't match because some of the arguments have invalid types: (!float!, !float!, !int!, !int!, !int!, !int!)
 * (torch.FloatTensor viewed_tensor)
 * (torch.Size size)
 * (torch.FloatStorage data)
 * (Sequence data)

The code in line 642 is:

self.weight = Parameter(torch.Tensor(
            self.out_height, self.out_width,
            out_channels, in_channels, *self.kernel_size))

I check it, may be the problem is *self.kernel_size

Fixed. It should work now.

OK, let me try.

Yes, It works. Many thanks. wow~~~
How do you learn to modify the source code. I would like to learn the method, so that I can modify by myself for any operators that I need. That's so cool~~

So from what I've seen this module is not featured in the documentation; Also, I cannot access it when running local1 = Conv2dLocal(8, 16, 64, 64). So how do I use this, do I have to download a 'nightly' build or is this included already?

Oh I see now that this is still not accepted as a PR, is that right? And how long is this going to take do you think? Also, thanks a lot for the commit! :)

@yenicelik
I can't say for sure when will this PR be merged into master. But you can create a separate python environment with conda and install my forked version.

These lines may help you use Conv2dLocal.

When will it be merged into master?

Looks good to me, but I think it should be renamed to ConvUntied2d. There's nothing more local in this convolution than in regular Conv2d

I dont think it's realistic to add a neighborhood option in this PR. What you ask needs quite a bit of rewrite of the C/CUDA kernels.

I can't get these auto-generated functions to work in latest version (0.2.0+8262920).

The code from #499 will give the error:

/home/biao/miniconda2/envs/pth/lib/python3.6/site-packages/torch/nn/_functions/thnn/auto.py in forward(ctx, input, *params)
    152                 args += (None,)
    153             else:
--> 154                 raise ValueError("missing required argument '%s'" % param.name)
    155 
    156         args += tuple(additional_args)

ValueError: missing required argument 'bias'

Any suggesstions?

@1zb Not sure, but maybe like this (?)

import torch
from torch.autograd import Variable

inputs=Variable(torch.randn(1,3,32,32)) # N x inC x inH x inW
weight=Variable(torch.randn(30,30,64,3,3,3)) # outH x outW x outC x inC x kH x kW
bias=Variable(torch.randn(64,30,30)) # outC x outH x outW

local = torch.nn.backends.thnn.backend.SpatialConvolutionLocal.apply
## arguments: input, weight, bias, kH, kW, strideH, strideW, padH, padW, inH, inW, outH, outW
output = local(inputs, weight, bias, 3, 3, 1, 1, 0, 0, 32, 32, 30, 30)

I can duplicate the error mentioned above. I believe a quick fix can solve it. Any core developer online?
@soumith @apaszke

This function is potentially useful for solving general occlusion problem in image recognition.

Auto-generated SpatialConvolutionLocal of new style seems not working anymore. It invokes _updateOutput in the following way:

SpatialConvolutionLocal_updateOutput(
            state,
            inputs,
            output,
            finput, # <-- 
            fgradInput, # <-- buffer before weight and bias
            weight,
            bias,
            kW, kH,
            dW, dH,
            padW, padH,
            iW, iH,
            oW, oH
        )

But the correct declaration is

SpatialConvolutionLocal_updateOutput(
            state,
            inputs,
            output,
            weight, # <-- 
            bias, # <-- weight and bias before buffer
            finput,
            fgradInput,
            kW, kH,
            dW, dH,
            padW, padH,
            iW, iH,
            oW, oH
        )

I implemented Conv2dLocal with Im2Col. And it supports bias=False and dilation=x. Could you check if it is ready?

I confirm the new pull request is compatible with the current version of Pytorch. The forward and backward works smoothly. Thank you very much 1zb!

Where should this be downloaded?
Conv2dLocal I can not run.
I have updated to the latest version.But this is not.

http://pytorch.org/

I tested the Im2Col version and it works, and it has an OK batch speedup. I didn't test the non-functional way of using it though. I think you should merge this (there is no other sane way of doing non-shared kernel convolutions currently otherwise).

I build your code from source(conv-layer branch), but I run into GPU memory leak when add Conv2dLocal layer in network. (During training process and loading batch iterations) a possible related issue: #3394 Do you have any ideas? @1zb

I've tested this as well a bit more, and while functionally the unfold solution works fine, it requires gigantonormous intermediary tensors as unfold has to expand the input tensor into a form that can be directly element-wise multiplied with the weights, and if the input tensor is batched, the intermediaries often don't fit in the GPU even if you have 10 GB of VRAM.

For example, I tried a relatively small 128x128 input with a 13x13 kernel, this requires a weight tensor of 128x128x13x13 which is around 10 MBytes. But as soon as you start batching (which is required for speed), even for a very small batch of 32 this is 0.3 GB, and the unfold solution requires more than one such intermediary.

I'm not saying the proposed solution shouldn't be merged as it's much better than nothing, but it would be super awesome if there could be a more native solution that doesn't require the scratchpad RAM. The operation itself is local so this is theoretically possible at least. I'd be happy to contribute it but I'm a rookie at pyTorch and there seems to be so many different interacting backends that I'm not sure where to actually put such an implementation.

By the way, this is not in cuDNN, but it would be interesting to know if Nvidia has an implementation in the works.

Yeah, I know that the kernels will be very large cause it don't share weights.. But the problem is that the memory usage keeps increasing when loading batches, that's strange. I think the memory usage should keep at a constant value when training.

Why still not pass review?

Any progress?

Does this issue have any progress in 2019?

import torch,torch.nn as nn,torch.nn.functional as F
from copy import deepcopy
class LocalLinear(nn.Module):
    def __init__(self,in_features,local_features,kernel_size,stride=1,bias=True):
        super(LocalLinear, self).__init__()
        self.kernel_size = kernel_size
        self.stride = stride

        fold_num = (in_features-self.kernel_size)//self.stride+1
        self.lc = nn.ModuleList([deepcopy(nn.Linear(kernel_size,local_features,bias=bias))
                                 for _ in range(fold_num)])

    def forward(self, x:torch.Tensor):
        x = x.unfold(-1,size=self.kernel_size,step=self.stride)
        fold_num = x.shape[1]
        x= torch.cat([self.lc[i](x[:,i,:]) for i in range(fold_num)],1)
        return x

Does it work with image or only vector ?

In the forward function in the code snippet provided by @sheljoy, the fold_num variable shouldn't be calculated as x.shape[2] after the unfolding operation?
When keeping x.shape[1] I cannot stack LocalLinear layers.

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
Stale pull requests will automatically be closed 30 days after being marked Stale

Hi @1zb!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

Any updates in 2023?