Add bias to Conv2d Module.

Defaults to false for back compatibility, **this is different from torch
default**.

Towards #57

Add some necessary module components used frequently in Stable
Diffusion's UNet.

Includes fixes to module attribute access from LLM branch and work
arounds for torch weight copying.

Towards #57.

Add graph module for using flash attention and clarify some differences
in flash attention vs torch sdpa.

**Attention: (pun intended)**

Softmax has temperature scaling option. Divides inputs by scalar, good
explanation of numerical effects
[here](https://medium.com/@harshit158/softmax-temperature-5492e4007f71).

Used when softmax inputs QK are too big for float 16 (abs value >
65504). This usually means the numbers are so large that dividing by
small (< 4) scalar has little effect.

Stable diffusion does not use this, as torch spda supports float 32 (or
somehow avoids NaNs from large values). No visual or significant numeric
differences in this output layer noticed.

Towards #57.

Define complete UNet, with forward pass broken into down, mid, and up
sections. Useful diagrams
[here](http://jalammar.github.io/illustrated-stable-diffusion/)

Uses blocks defined in hidet-org#97. Heavily reduced version from diffusers
containing only necessary features for stable diffusion v2-1.

Towards hidet-org#57.

---------

Co-authored-by: vadiklyutiy <156319763+vadiklyutiy@users.noreply.github.com>

Stable diffusion uses fundamentally the same positional embeddings, but
since timesteps change, a cache is not possible.

There's also small changes in tensor layouts and calculation parameters
between the diffusers version and the one from Llama, so I've recreated
it here for now. An abstract version that combines both version is TODO.

Towards hidet-org#57.

Add UNet Down, Up, and Mid block definitions and attention transformer
utility layer.

Modules are designed so that kwargs passed to constructors are all the
same config from huggingface with minimal changes - lots of shared
values and too many parameters to list individually. Same kwargs are
passed to nested objects. Open to other suggestions, although this is a
single use case problem.

Towards hidet-org#57.

Infrastructure for compiled stable diffusion app.

Towards hidet-org#57

Define complete UNet, with forward pass broken into down, mid, and up
sections. Useful diagrams
[here](http://jalammar.github.io/illustrated-stable-diffusion/)

Uses blocks defined in #97. Heavily reduced version from diffusers
containing only necessary features for stable diffusion v2-1.

Towards #57.

---------

Co-authored-by: vadiklyutiy <156319763+vadiklyutiy@users.noreply.github.com>

Stable diffusion uses fundamentally the same positional embeddings, but
since timesteps change, a cache is not possible.

There's also small changes in tensor layouts and calculation parameters
between the diffusers version and the one from Llama, so I've recreated
it here for now. An abstract version that combines both version is TODO.

Towards #57.

Add UNet Down, Up, and Mid block definitions and attention transformer
utility layer.

Modules are designed so that kwargs passed to constructors are all the
same config from huggingface with minimal changes - lots of shared
values and too many parameters to list individually. Same kwargs are
passed to nested objects. Open to other suggestions, although this is a
single use case problem.

Towards #57.

Infrastructure for compiled stable diffusion app.

Towards #57

Define complete UNet, with forward pass broken into down, mid, and up
sections. Useful diagrams
[here](http://jalammar.github.io/illustrated-stable-diffusion/)

Uses blocks defined in #97. Heavily reduced version from diffusers
containing only necessary features for stable diffusion v2-1.

Towards #57.

---------

Co-authored-by: vadiklyutiy <156319763+vadiklyutiy@users.noreply.github.com>

Stable diffusion uses fundamentally the same positional embeddings, but
since timesteps change, a cache is not possible.

There's also small changes in tensor layouts and calculation parameters
between the diffusers version and the one from Llama, so I've recreated
it here for now. An abstract version that combines both version is TODO.

Towards #57.

Add UNet Down, Up, and Mid block definitions and attention transformer
utility layer.

Modules are designed so that kwargs passed to constructors are all the
same config from huggingface with minimal changes - lots of shared
values and too many parameters to list individually. Same kwargs are
passed to nested objects. Open to other suggestions, although this is a
single use case problem.

Towards #57.

Infrastructure for compiled stable diffusion app.

Towards #57