Model providers

Goal: add a model provider and point agents at it with a Model manifest.

Prerequisites

• An endpoint that speaks the OpenAI chat-completions API, or an OpenRouter account.
• The API key, if the endpoint needs one, in an environment variable.

Steps

1. Pick a provider: openrouter, or openai-compatible for any endpoint that speaks the OpenAI chat-completions API (Ollama, vLLM, OpenAI).

2. Declare a Model manifest. It is an alias, and its provider names a plugin port:

   apiVersion: afe.dev/v1alpha1
   kind: Model
   metadata:
     name: locale
   spec:
     provider: openai-compatible
     model: qwen2.5-coder:7b
     baseUrl: http://127.0.0.1:11434/v1
     pricing: none

3. Name the environment variable that holds the API key with apiKeyEnv. The key value never crosses the port.

4. Run the flow. afe run asks for the keys of the providers a flow actually uses, before the run starts.

Notes

Two providers ship today:

Provider	What it is
openrouter	A cloud endpoint that reports the cost of every call.
openai-compatible	Any endpoint that speaks the OpenAI chat-completions API (Ollama, vLLM, OpenAI). With pricing: none a local model costs nothing.

Troubleshooting

invalid configuration: model 'locale': set OPENROUTER_API_KEY The flow uses a model whose apiKeyEnv variable is not set in the environment afe run or afe worker sees. Export the variable before the run starts. afe serve needs no key value: it runs no harness and only resolves the provider. A --dry run needs none either.

model 'locale': <plugin error> The provider field names a port no installed plugin provides. Check the value against the providers your configuration actually installs (openrouter, openai-compatible, or a plugin you added).

The ledger shows cost 0 for a model you expect to pay for The provider did not report a cost, and the Model manifest has no pricing. Add a numeric pricing (input and output price per token) to the manifest, so the ledger can compute the cost from the tokens used.

See also

• Model
• Harnesses