Good catch @ronghanghu -- this definitely looks wrong. I'll leave open for the moment since I haven't read this code before, but we should take care of this immediately.

We should really have better testing for this situation... if this is bugged as it appears, this breaks weight sharing and gradient accumulation.

Yep, this does look like a bug -- when I wrote the initial support on our fork and merged it across, it looks like this got dragged in (it's still present in our code)

Good catch @ronghanghu and sorry for missing this is my review. While there
are cuDNN layer tests and weight sharing + gradient accumulation tests
there are not weight sharing + gradient accumulation tests for cuDNN.
On Wed, Oct 28, 2015 at 05:25 slayton58 notifications@github.com wrote:

Yep, this does look like a bug -- when I wrote the initial support on our
fork and merged it across, it looks like this got dragged in (it's still
present in our code)

—
Reply to this email directly or view it on GitHub
#3254 (comment).

We need to adapt the gradient checker to gradient accumulation, in order to prevent future mistakes like this one and #2532.

👍 I agree that the gradient checker needs to be updated. A simple fixed was proposed earlier by @tnarihi : tnarihi@7d45526

See also #3036.