I have attached to this ticket a WIT file that describes a generic interface to large-language models (LLMs). This interface can be implemented by many different providers, either by emulating features which are not present for a given provider, by using the provider's native support for a feature, or by indicating an error if some particular combination is not natively supported by a given provider.

The intent of this WIT specification is to allow developers of WASM components (on wasmCloud, Spin, or Golem) to leverage chat-based AI to build agents and agentic services in a portable and provider-agnostic fashion.

This ticket involves constructing implementations of this WIT interface for the following providers:

• Anthropic
• OpenAI
• Grok
• OpenRouter

These implementations must be written in Rust and must be compilable to WASM Components (WASI 0.2 only, since Golem does not yet support WASI 0.3). In order to do this, the standard Rust toolchain for WASM component development can be employed (see cargo component and the Rust examples of components in this and other Golem repositories).

As an added caveat, however, we wish these implementations of the WIT to be explicitly durable, and therefore, they should use the Golem durability API and the Golem host API to provide custom durability semantics.

Why customize the durability of these components, rather than just relying on Golem's wasi:http durability? It's because wasi:http durability is at a low level. Semantically, it would be much better if durability were at the level of individual calls to the LLM, rather than at the level of individual operations in an HTTP request. This would make the oplog cleaner and clearer (much more high-level) and assist debugging.

The final deliverables associated with this ticket are:

• Anthropic implementation of the WIT, which compiles to a WASM Component (WASI 0.2), called llm-anthropic.wasm, with full test suite, and custom durability implementation at the level of LLM calls
• OpenAI implementation of the WIT, which compiles to a WASM Component (WASI 0.2), called llm-openai.wasm, with full test suite, and custom durability implementation at the level of LLM calls
• Grok implementation of the WIT, which compiles to a WASM Component (WASI 0.2), called llm-grok.wasm, with full test suite, and custom durability implementation at the level of LLM calls
• OpenRouter implementation of the WIT, which compiles to a WASM Component (WASI 0.2), called llm-openrouter.wasm, with full test suite, and custom durability implementation at the level of LLM calls

Note that all of these will require some runtime configuration to work: most notably an API key. For configuring this information, the components can use environment variables for now (in the future, they will use wasi-runtime-config, but Golem does not support this just yet, whereas Golem has good support for environment variables).

Moreover, the Rust components need to be tested within Golem to ensure they are compatible with Golem 1.2.x.

This WIT has been designed in advanced by looking at and comparing the APIs of Anthropic, OpenAI, Grok, and OpenRouter. However, given there are no implementations, it is possible the provided WIT is not the optimal abstraction across all of these providers. Therefore, some deviations can be made from the proposed design. However, to be accepted, any deviation must be fully justified, and must be deemed by Golem core contributors to be an improvement from the original specification.

package golem:llm@1.0.0;

interface llm {
  # --- Roles, Error Codes, Finish Reasons ---

  enum role {
    user,
    assistant,
    system,
    tool,
  }

  enum error-code {
    invalid-request,
    authentication-failed,
    rate-limit-exceeded,
    internal-error,
    unsupported,
    unknown,
  }

  enum finish-reason {
    stop,
    length,
    tool-calls,
    content-filter,
    error,
    other,
  }

  enum image-detail {
    low,
    high,
    auto,
  }

  # --- Message Content ---

  record image-url {
    url: string,
    detail: option<image-detail>,
  }

  variant content-part {
    text(string),
    image(image-url),
  }

  record message {
    role: role,
    name: option<string>,
    content: list<content-part>,
  }

  # --- Tooling ---

  record tool-definition {
    name: string,
    description: option<string>,
    parameters-schema: string,
  }

  record tool-call {
    id: string,
    name: string,
    arguments-json: string,
  }

  record tool-success {
    id: string,
    name: string,
    result-json: string,
    execution-time-ms: option<u32>,
  }

  record tool-failure {
    id: string,
    name: string,
    error-message: string,
    error-code: option<string>,
  }

  variant tool-result {
    success(tool-success),
    error(tool-failure),
  }

  # --- Configuration ---

  record kv {
    key: string,
    value: string,
  }

  record config {
    model: string,
    temperature: option<float32>,
    max-tokens: option<u32>,
    stop-sequences: option<list<string>>,
    tools: list<tool-definition>,
    tool-choice: option<string>,
    provider-options: list<kv>,
  }

  # --- Usage / Metadata ---

  record usage {
    input-tokens: option<u32>,
    output-tokens: option<u32>,
    total-tokens: option<u32>,
  }

  record response-metadata {
    finish-reason: option<finish-reason>,
    usage: option<usage>,
    provider-id: option<string>,
    timestamp: option<string>,
    provider-metadata-json: option<string>,
  }

  record complete-response {
    id: string,
    content: list<content-part>,
    tool-calls: list<tool-call>,
    metadata: response-metadata,
  }

  # --- Error Handling ---

  record error {
    code: error-code,
    message: string,
    provider-error-json: option<string>,
  }

  # --- Chat Response Variants ---

  variant chat-event {
    message(complete-response),
    tool-request(list<tool-call>),
    error(error),
  }

  # --- Streaming ---

  record stream-delta {
    content: option<list<content-part>>,
    tool-calls: option<list<tool-call>>,
  }

  variant stream-event {
    delta(stream-delta),
    finish(response-metadata),
    error(error),
  }

  resource chat-stream {
    get-next: () -> list<stream-event>;
    has-next: () -> boolean;
  }

  # --- Core Functions ---

  send: func(
    messages: list<message>,
    config: config
  ) -> result<chat-event, error>;

  continue: func(
    messages: list<message>,
    tool-results: list<tool-result>,
    config: config
  ) -> result<chat-event, error>;

  stream: func(
    messages: list<message>,
    config: config
  ) -> chat-stream;
}