Skip to main content
A provider normally holds one API key, and every request to it uses that key. Configure more than one and GoModel spreads identified sessions across them. Each conversation stays on one key by default, so it can keep hitting the provider’s prompt cache while each key still carries a fraction of the overall traffic. Rotation is off until a second key is configured. One key behaves exactly as it always has.
Requests without a detected session still rotate round robin. See Session Keeping for supported session headers, body fields, and automatic conversation detection.

​
Configure

Add keys with numbered environment variables. <PROVIDER>_API_KEY is the first key; <PROVIDER>_API_KEY_2, _3, and so on add the rest.
Three keys, so different sessions distribute deterministically across the three keys; every request in one session selects the same key. Or in config.yaml:
api_key and api_keys may be combined; api_key is used first. To restore strict per-request round robin, opt out for that provider:
Environment-only provider instances use the same provider prefix and optional suffix as their API keys:
The same setting is the Session-sticky API keys checkbox in the dashboard’s provider editor. It is checked by default. Any provider that authenticates with an API key can rotate — ANTHROPIC_API_KEY_2, GEMINI_API_KEY_2, GROQ_API_KEY_2, and so on. Providers that authenticate another way (Vertex AI and Bedrock, which use Google and AWS credentials) ignore these variables.

​
Rules

  • Numbering starts at 2, since <PROVIDER>_API_KEY is key 1. Spelling out <PROVIDER>_API_KEY_1 instead of the unsuffixed form also works.
  • Gaps are fine. Setting only _API_KEY and _API_KEY_3 gives two keys.
  • Duplicate keys are collapsed, so a key repeated across two variables does not take a double share of the traffic.
  • Keys are used in the order configured: the unsuffixed key first, then ascending by number.
  • Environment variables replace the provider’s whole api_keys list from config.yaml rather than merging into it.
  • A key that does not resolve — an ${UNSET_VAR} placeholder — is dropped. A provider left with no keys at all is not registered.
With session stickiness enabled, retries use the session’s pinned key. With stickiness disabled, every outbound attempt participates in round robin.

​
Rotation on suffixed providers

Suffixed environment variables register a separate provider of the same type. The two mechanisms compose, and the trailing number is what tells them apart:
This is provider openai-eu with two keys. The trailing _2 names a key.
This is provider openai-region-2 with one key. Here the 2 is part of the provider suffix, not a key number.

​
What rotation covers

Every call GoModel makes to the provider uses the same selection policy: chat completions, responses, embeddings, audio, images, batches, provider-native passthrough routes, and realtime sessions. Identified sessions are sticky; sessionless traffic is round robin. Rotation is per gateway process. Counters live in memory and reset on restart, so N replicas each rotate independently — which is fine, since the goal is to spread load rather than to sequence it exactly.

​
Choosing between rotation and separate providers

Both let you use several keys of one provider type. They solve different problems. Separate providers are still useful when keys point at different accounts, regions, base URLs, or model catalogs. They are no longer required merely to preserve prompt caching across several keys.

​
Verify

Rotation is not reported by the API. To confirm it, watch the provider’s own usage dashboard: traffic should divide roughly evenly across the keys.
Last modified on August 1, 2026