Rebased and ready for review. (Previously depended on gradient accumulation PR #1663.)

This layer works as a lookup table and could be renamed to LookupTable.
https://github.com/torch/nn/blob/master/doc/convolution.md#nn.LookupTable
https://github.com/torch/nn/blob/master/LookupTable.lua

I am very confused by this Embed layer. My hunch is that no one uses it outside of the RNN/LSTM branch. So I doubt I'll get any answer but let's try just in case.

I've tried to use it in a simple MLP like this:

Here is an example of a typical prototxt section:

layer {
  name: "embed"
  type: "Embed"
  bottom: "data"
  top: "embed_data"
  embed_param {
    input_dim: 5454
    num_output: 200
    weight_filler {
      type: "uniform"
      min: -0.08
      max: 0.08
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "ip0"
  type: "InnerProduct"
  bottom: "embed_data"
  top: "ip0"
  inner_product_param {
    num_output: 200
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  } 
}

Filling up the data elements with the vocabulary indices of words in a sentence, naturally I get an error from the data_transformer since datum channels are now of various sizes. Then I tried padding the remaining elements to 0, as I understand it is done in https://github.com/BVLC/caffe/pull/1873/files

But in this case, there's no memory advantage of doing this vs one-hot vectors since the input dim is the same. Thus I am confused :)

Needless to say, any help is highly appreciated at this point!

Understood, of course the padding is to fix the input sequence length.