Cloud models, governed

Role-based model selection, zero data retention by default, and local models for air-gapped sites.

Converge Team, 2026-10-30

Choosing a language model is a decision that changes every few months. Choosing where your data may go should not. In Converge, those two decisions are separated by one component: the AI gateway.

Ask for a role, not a model

Applications in Converge never hard-code a model. They ask the gateway for a role:

RoleUsed for
agenticChat, tool use and agents
ocrReading documents and images
transcribeSpeech to text
voiceSpoken responses
embeddingSearch and retrieval
decisionStructured decisions

The deployment's model catalog maps each role to a model, with ordered fallbacks. Swapping a model is a catalog change; no agent or Logic function has to be edited.

{ "model": "role:agentic", "messages": [{ "role": "user", "content": "Summarise open incidents" }] }

One gateway sits between every application and every model provider.

Cloud models, with zero data retention

When a deployment uses cloud models, the gateway reaches them through OpenRouter. Every request asks for zero data retention and denies data collection, so providers that would keep prompts are not used.

External providers are off until an administrator turns them on. Until then, the gateway refuses the call before a single byte leaves the deployment.

Moving a model from a self-hosted provider to a cloud one is logged, because it changes where data goes.

Local models for air-gapped sites

Some sites cannot send anything outside. For them, every role can be served by models hosted inside the deployment through an OpenAI-compatible endpoint, such as Ollama. Agents, Logic and the Decisions API work the same way; only the catalog differs.

The same rules for every call

Whatever the provider, each call passes the same checks:

  • Classification. Data marked restricted is refused unless the deployment explicitly allows it.
  • Rate limits. Per user and per model, with 429 and Retry-After when exceeded.
  • Attribution. Every call is recorded against a user and a model for usage reporting.
  • Identity. Tools run as the person using them, so a model never sees what its user cannot.

Ensembles follow the rules too: each leg of a K-LLM ensemble is an ordinary gateway call.

The model roles documentation has the details, including the gateway's OpenAI-compatible endpoints.

Converge