Model description

New LLM from MosaicML, 7B parameters. See: https://www.mosaicml.com/blog/mpt-7b

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

https://huggingface.co/mosaicml/mpt-7b/tree/main

The model is already implemented in a HF/T compatible way, but has multiple source files for its model implementation, some components that aren't used in this current model, and most importantly, dependencies that aren't normally included in HF (e.g. einops, flash_attn).

Do the HF folks have a view on whether we would want to add those dependencies, or implement a vanilla version based only on existing requirements (in which case, it would arguably be easier to modify an existing LM implementation instead, rather than use MosaicML's implemenetation)?