Skip to main content

​
Overview

GoModel ships with two response-cache layers for non-streaming requests on:
  • /v1/chat/completions
  • /v1/responses
  • /v1/messages
  • /v1/embeddings
Exact-match cache returns byte-identical responses with:
Semantic cache uses embeddings plus vector search so meaning-equivalent prompts can reuse a stored response:

​
Enable the exact cache

Point response caching at Redis:
You can also configure it with environment variables:
  • REDIS_URL
  • REDIS_KEY_RESPONSES
  • REDIS_TTL_RESPONSES

​
Enable semantic caching

Add a semantic block with an embedder provider and a vector store:
Supported vector stores:
  • qdrant
  • pgvector
  • pinecone
  • weaviate
Both cache layers run after workflow and guardrail patching, so they operate on the final request sent upstream. Use Cache-Control: no-cache or Cache-Control: no-store to bypass caching per request. For the full semantic-cache design and storage options, see ADR-0006.

​
What the exact cache keys on

The exact cache hashes:
  • the request path
  • the resolved workflow context used for execution specifically execution mode, provider type, and resolved model
  • the final request body
This means guardrails and workflows affect cache keys when they change the resolved workflow or the final body sent through execution. JSON object member order and insignificant whitespace are canonicalized before hashing. Requests that differ only in formatting therefore share one exact entry. Concurrent identical misses are also coalesced in-process: one request reaches the provider and the waiters replay its response. Failed or otherwise non-cacheable responses are never fanned out.

​
Provider prompt-cache planning

After routing resolves the concrete provider and model, GoModel adds a cache plan when the stable prefix meets that provider’s minimum size and the client did not already supply a cache directive:
  • OpenAI receives a stable prompt_cache_key; models supporting explicit caching also receive a breakpoint and explicit cache mode.
  • Anthropic receives top-level automatic ephemeral caching.
  • Amazon Bedrock Converse receives a cache point after the stable prefix.
  • Native Gemini AI Studio creates and reuses a five-minute cached-content object, with concurrent creation coalesced by prefix and stable credential.
The plan is applied post-routing so provider-only fields cannot leak through a fallback to another backend. Cache keys include the concrete provider instance and model, and caller-supplied cache controls always win. Provider prompt-cache planning is enabled by default. To disable GoModel’s automatic planning while continuing to pass through compatible client cache directives, set:
The flag accepts Go boolean values. Missing, empty, or invalid values keep the safe default (true) and an invalid value emits a startup warning.

​
user_path behavior

For the exact cache, user_path is not added to the cache key by itself. That is intentional. If two requests end up with the same path, resolved workflow, and final request body, they can share the same exact-cache entry even when they originate from different user_path values.
If you need tenant or path-specific cache behavior, use a scoped workflow or otherwise make the final request differ for that scope. user_path alone is not an exact-cache partition key.
Common patterns:
  • disable cache in a scoped workflow
  • use different scoped workflows for different user_path values
  • include scope-specific context so the final request body differs

​
Cache analytics

When response caching and usage tracking are enabled, the admin API exposes a cached-only overview at:
Cached usage entries are also visible in the regular usage log and summary endpoints.
Last modified on August 1, 2026