Additive Recurrent Neural Net

Inspiration

I was inspired by the ability of recurrent nets to learn cyclical patterns, and I wanted something that could learn a cyclical pattern in a smooth space, like a sin wave over continuous numbers.

What it does

For each frame x_i, the net computes the output f(x_i) and adds it to the state, so that x_{i+1} = x_i + f(x_i).

How I built it

By hand

Challenges I ran into

It turned out Theano didn't help the project at all, because I was just doing an exploration, and didn't need backpropogation. Therefore I ditched it and implemented a neural net by hand.

Accomplishments that I'm proud of

Good proof of concept of cyclical behavior.

Built With

• matplotlib
• python