One pain point I've had while maintaining AI Services is to keep up with provider details and their model metadata, especially as the available models and their metadata (e.g. maximum supported input and output tokens, capabilities like function calling) change all the time.

Unfortunately, the majority of providers do not provide structured information about these aspects in their APIs. Most providers barely provide any model information other than the identifier and human readable name - not even something as high level as whether a model is for text generation or image generation, for example.

This means that, the more providers and models we'll eventually support, the worse the maintenance burden becomes. At some point it's probably unreasonable for the maintainers of this project, however many it will be, to be able to keep up with all the model changes across all the supported providers.

When chatting to Jamie and Marcus from the Drupal AI side, they also raised this as a severe problem they've been facing. They shared https://models.litellm.ai/, which might be the only source of granular model information across providers that's publicly available. If we could somehow implement infrastructure to consume that data to help with model discovery based on capabilities, it would severely reduce the maintenance burden.

Let's think about how we could make that a reality. Potentially this is something that could be built into this SDK, or it could be a separate package purely focused on the data which this SDK could consume.