Cloud models, governed
Role-based model selection, zero data retention by default, and local models for air-gapped sites.
Converge Team, 2026-10-30
Choosing a language model is a decision that changes every few months. Choosing where your data may go should not. In Converge, those two decisions are separated by one component: the AI gateway.
Ask for a role, not a model
Applications in Converge never hard-code a model. They ask the gateway for a role:
| Role | Used for |
|---|---|
agentic | Chat, tool use and agents |
ocr | Reading documents and images |
transcribe | Speech to text |
voice | Spoken responses |
embedding | Search and retrieval |
decision | Structured decisions |
The deployment's model catalog maps each role to a model, with ordered fallbacks. Swapping a model is a catalog change; no agent or Logic function has to be edited.
{ "model": "role:agentic", "messages": [{ "role": "user", "content": "Summarise open incidents" }] }
Cloud models, with zero data retention
When a deployment uses cloud models, the gateway reaches them through OpenRouter. Every request asks for zero data retention and denies data collection, so providers that would keep prompts are not used.
External providers are off until an administrator turns them on. Until then, the gateway refuses the call before a single byte leaves the deployment.
Moving a model from a self-hosted provider to a cloud one is logged, because it changes where data goes.
Local models for air-gapped sites
Some sites cannot send anything outside. For them, every role can be served by models hosted inside the deployment through an OpenAI-compatible endpoint, such as Ollama. Agents, Logic and the Decisions API work the same way; only the catalog differs.
The same rules for every call
Whatever the provider, each call passes the same checks:
- Classification. Data marked restricted is refused unless the deployment explicitly allows it.
- Rate limits. Per user and per model, with
429andRetry-Afterwhen exceeded. - Attribution. Every call is recorded against a user and a model for usage reporting.
- Identity. Tools run as the person using them, so a model never sees what its user cannot.
Ensembles follow the rules too: each leg of a K-LLM ensemble is an ordinary gateway call.
The model roles documentation has the details, including the gateway's OpenAI-compatible endpoints.