AgentCloak Pre-Router
AgentCloak Pre-Router is a privacy gateway for applications that call OpenAI-compatible LLM APIs. It sits between your application and the provider (OpenAI, Azure OpenAI, OpenRouter, vLLM, Ollama, LiteLLM, or any other OpenAI-compatible endpoint), replaces personal data in every outgoing request with placeholders, and puts the original values back into the provider's answer before it reaches your application.
Your users see normal answers with real names and details. The provider sees only placeholders.
Integration is one change: point your client's base URL at the Pre-Router. Your SDK, your provider API key, and your code stay the same.
Test the Pre-Router carefully on your own data before using it in production. Personal data detection is statistical. It is measured at 99% of values replaced on a multilingual test set (see Detection safety nets), not 100%, and results on your traffic depend on its languages, formats, and kinds of personal data. Verify what reaches the provider on a representative sample of your requests, and tune the configuration until it meets your requirements.
How it works
Before: direct calls to the provider
Every prompt, chat history, tool result, and attached file reaches the provider as written, personal data included.
After: calls through AgentCloak Pre-Router
The exact look of placeholders depends on your deployment. They are either realistic
stand-ins (a different name in place of a name) or tags such as PERSON_7f3a91. Either
way, each distinct value gets its own placeholder, and the same value gets the same
placeholder everywhere in a request, so the model can still reason about who is who.
Safety guarantees
- Fails closed. If personal data cannot be detected and replaced, or detection takes too long, the request is rejected and nothing is sent to the provider.
- Your key passes through. Your provider API key is forwarded to the provider as is.
The Pre-Router does not store, log, or compare it. A rejected key comes back as the
provider's own
401or403. - No data returned to the wrong caller. Original values are only ever restored for the caller who sent them (see Conversations).
Use cases
- Customer support assistants. Summarize tickets, draft replies, and classify requests with a hosted model without sending customer names, emails, phone numbers, or addresses to a third party.
- HR and internal workflows. Work with employee records, reviews, and correspondence through an LLM while keeping personal data inside your network.
- Document Q&A. Ask questions about contracts, reports, or PDFs. Attached files are cleaned the same way as message text, and answers quote the real names back.
- Tool-calling agents. Agents that call functions or MCP servers keep working: tool call arguments and tool outputs are protected in both directions.
- One privacy layer for every app. Put the Pre-Router in front of all LLM traffic in your organization to meet GDPR and internal data-handling rules without changing each application.
- Provider independence. Switch between hosted vendors and self-hosted models behind one gateway that behaves the same way for all of them.
Integration
1. Change the base URL
Your Pre-Router administrator gives you a base URL for each provider route, for example
<https://prerouter.example.com/to/v1/openai.> Use it in place of the provider's base URL.
Python:
import os
from openai import OpenAI
client = OpenAI(
base_url="https://prerouter.example.com/to/v1/openai",
api_key=os.environ["OPENAI_API_KEY"], # your provider key, unchanged
)
reply = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Write a thank-you note to Ada Lovelace."}],
)
print(reply.choices[0].message.content) # mentions "Ada Lovelace"
Node.js:
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://prerouter.example.com/to/v1/openai",
apiKey: process.env.OPENAI_API_KEY,
});
curl:
curl https://prerouter.example.com/to/v1/openai/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello, I am Ada Lovelace."}]}'
Query strings are forwarded unchanged, so Azure's api-version works as usual. Azure
clients send their key in the api-key header, which is supported.
2. Authentication
By default the provider decides: send your provider key exactly as you do today. A
request without any API key header is answered 401 missing_credentials right away.
Some deployments add their own sign-in in front of the provider. In that case your
administrator tells you which header carries your sign-in token (usually
Authorization) and which carries the provider key (usually api-key).
Default configuration
This section is for administrators who run the Pre-Router. The configuration is one
YAML file, and the Docker image ships with the one below at
/app/agentcloak-server.yaml. It defines one provider route at /to/v1/openai that:
- forwards to the provider in
OPENAI_BASE_URL(by default<https://api.openai.com/v1;> for example<https://openrouter.ai/api/v1),> and lets the provider check each caller's API key, - detects people, companies, locations, and email addresses in many languages with the multilingual GLiNER2-PII model, backed by three detection safety nets,
- replaces people and companies with realistic stand-in names, and locations and email
addresses with tags such as
EMAIL_3c9e1a, - keeps conversations per provider API key in a database file, so placeholders stay stable across turns; a conversation with no activity for a day expires,
- protects attached files and Files API uploads,
- sends system prompts to the provider as written.
server:
bind: "0.0.0.0:5050"
# Inline file attachments arrive base64-encoded: room for a 10 MiB file.
max_body_size: 16777216
# Upper bound for detecting personal data in one request.
ner_timeout_secs: 60
# Longer than the provider route's timeout, so long generations can finish.
request_timeout_secs: 660
# Callers authenticate with their provider key; the provider checks it.
auth: none
# Conversation storage in a SQLite file, created on first start.
storage:
backend: libsql
tenancy:
mode: single
env_id: 3f0c9a52-6d1e-4b7a-9c28-5e4f1a7b2d90
access:
mode: environment
libsql:
path: "${AGENTCLOAK_DB_PATH}"
create_schema: true
entities:
codec: json
# A conversation untouched for a day expires and is deleted with its messages.
ttl_secs: 86400
pipelines:
- name: chat
stages:
# Finds people, locations, companies, and email addresses.
- type: ner_detector
models:
- type: gliner
model_dir: "/app/models/gliner2-pii-multi"
model_name: "model.onnx"
# Copies kept loaded, each about 1.2 GiB of memory.
# More copies serve more requests in parallel.
sessions: 2
labels: [person, location, company, email]
min_confidence: 0.2
# Spans scored from here up to min_confidence are candidates: replaced
# only where a safety net below points at them.
candidate_confidence: 0.05
# Email addresses the model missed.
- type: static_processor
pattern: "(?i)[a-z0-9._%+\\-]+@[a-z0-9.\\-]+\\.[a-z]{2,}"
label: "email"
# Safety nets: each finds hints of a name and replaces a candidate covering one.
# A hint no candidate covers stays as written (`on_failure: continue`).
# Cue words: the capitalized word after a title, greeting, or naming word.
- type: guard
pattern: >-
\b(?:Mr|Mrs|Ms|Miss|Mx|Dr|Prof|Herr|Frau|Hr|Fr|M|Mme|Mlle|Me|Sr|Sra|Srta|Don|Doña|Dña|Sig|Sig\.ra|Signor|Signora|Dott|Dhr|Mevr|Mw|Dear|Liebe|Lieber|Hallo|Cher|Chère|Bonjour|Estimado|Estimada|Querido|Querida|Hola|Caro|Cara|Gentile|Ciao|Beste|Geachte|Hi|Hello|Hey|named|called|signed|by|von|de|van|di|da)\.?,?\s+(\p{Lu}[\p{L}\p{M}'’-]*)
on_failure: { type: continue }
accept_candidates: true
# Boundary: a capitalized word joined to a detection that stopped short of it.
- type: guard
matcher: { type: unknown_word }
scope: adjacent
joiners: [{ builtin: western_joining }, zu, ten, ter, "'t", "d'", la, le, des]
on_failure: { type: continue }
accept_candidates: true
# Unknown words: a capitalized word no common-word list holds.
- type: guard
matcher:
type: unknown_word
known:
- { path: "/app/models/lexicons/known-en.txt" }
- { path: "/app/models/lexicons/known-de.txt" }
- { path: "/app/models/lexicons/known-fr.txt" }
- { path: "/app/models/lexicons/known-es.txt" }
- { path: "/app/models/lexicons/known-it.txt" }
- { path: "/app/models/lexicons/known-nl.txt" }
on_failure: { type: continue }
accept_candidates: true
# A bare first or last name gets the placeholder of the full name it belongs to.
- type: link
linkers:
preset: default
patch:
names:
categories: [person, company]
scan: true
min_part_length: 2
# How each kind of value is replaced.
- type: anonymizer
tokenizers:
# Locations and email addresses: tags such as EMAIL_3c9e1a.
- tokenizer:
type: keyed_pseudonym
template: "{LABEL}_{id}"
id_length: 6
salt_scope: document
# People and companies: realistic stand-in names.
- tokenizer:
selector:
type: category
categories: [person, company]
type: surrogate
salt_scope: document
proxy:
- id: openai # served at /to/v1/openai
base_url: "${OPENAI_BASE_URL}" # provider URL, version prefix included
api: openai
auth: none
timeout_secs: 600 # long generations stream for minutes
read_timeout_secs: 120
pipeline:
reference: chat
openai:
# A request without one of these headers is refused before any work is done.
credential_headers: [authorization, api-key]
conversation:
# A conversation belongs to one provider API key.
scope: credential
secret_env: AGENTCLOAK_CONVERSATION_SECRET
# Where the conversation id comes from, first match wins.
id:
- header: x-conversation-id
- responses: conversation
- responses: previous_response_id
- fingerprint: [system, first_user]
# Earlier text used to recognise names that refer back to it.
context_window_bytes: 4096
messages:
# System prompts are your own text and go out as written.
- match: { role: [system, developer] }
request: skip
files:
preprocessors:
- type: text
- type: anydoc # Office, OpenDocument, RTF, EPUB, CSV, PDF
postprocessors:
- type: markdown
max_file_bytes: 10485760
uploads:
purposes: [user_data, assistants]
The configuration reads three environment variables:
| Variable | Default in the image | Meaning |
|---|---|---|
AGENTCLOAK_CONVERSATION_SECRET | none, required | Protects the link between an API key and its conversations. Generate it once with openssl rand -hex 32 and keep it stable: changing it starts new conversations for every caller. |
OPENAI_BASE_URL | <https://api.openai.com/v1> | The provider, version prefix included. |
AGENTCLOAK_DB_PATH | /app/data/agentcloak.db | The conversation database file. |
Detection safety nets
The detection model gives every span of text a score. A span scored at
min_confidence (0.2) or above is replaced. A span scored between
candidate_confidence (0.05) and min_confidence is a candidate: on its own it is
left as written, because low scores also land on ordinary words. Three safety nets
(guard stages) look for hints of a name, and a candidate that covers a hint is
replaced:
- Cue words: a capitalized word after a title, greeting, or naming word in
English, German, French, Spanish, Italian, or Dutch (
Mr,Herr,Mme,Sra,Dhr,Dear,Liebe,Hola,Geachte,named,von, and others). - Boundary: a capitalized word right next to a detected name that the detection stopped short of, such as the last part of a double surname.
- Unknown words: a capitalized word that none of the common-word lists holds. The image includes a list of about 20,000 words for each of English, German, French, Spanish, Italian, and Dutch.
A hint that no candidate covers stays as written, so the safety nets never reject a request. Measured on 3,000 texts, 500 each in those six languages, with 4,340 given names, surnames, cities, streets, and email addresses:
| Metric | Without safety nets | With safety nets |
|---|---|---|
| Values fully replaced | 98.2% | 99.0% |
| Over-replaced spans per text | 0.44 | 0.46 |
| Time per text | 35.5 ms | 35.4 ms |
| Language | de | en | es | fr | it | nl |
|---|---|---|---|---|---|---|
| Without safety nets | 97.6% | 99.0% | 98.4% | 98.6% | 98.8% | 97.1% |
| With safety nets | 98.2% | 99.7% | 99.2% | 99.0% | 99.3% | 98.5% |
To tune them:
- List the languages you serve. Remove the word lists of languages you do not
serve. For another language, mount a word list (one lowercase word per line) and add
its path under
known. - Fewer false detections: raise
min_confidence. Fewer missed values: lower it. Below 0.2, the share of replaced values stops rising while false detections keep growing. - Remove a safety net by deleting its
guardstage. Each one costs no measurable time.
Custom processors and guards
Add your own stages to the stages: list of the chat pipeline, after the
ner_detector and before the link stage. Use them for values only your organisation
knows, such as customer numbers or project code names, for words that must stay
readable, and for checks a request must pass. Stages run in the order they are listed.
Word list files go in a folder you mount next to the configuration, for example:
-v "$PWD/custom:/app/custom:ro" \
A relative path in the configuration is resolved against the configuration file's
folder (/app), so custom/projects.txt and /app/custom/projects.txt name the same
file.
Processors: values the model does not know
A processor finds values with a pattern or a word list and gives them a category
(label). There are two types:
| Type | Adds a detection for | Changes existing detections |
|---|---|---|
static_detector | every match not already detected at the same place with the same label | No |
static_processor | every match no detection fully covers | Yes: a detection whose text contains a match gets the processor's label |
When two detections overlap, they become one replacement, and the longer one keeps its
category. If the model has found the same text under another category, a
static_detector match loses to it. For example, the model may read a customer number
as a company, and the number then gets a made-up company name. Use static_processor
whenever the category must be yours.
Customer numbers and phone numbers:
# Customer numbers such as CUST-004211, always tagged as customer_id.
- type: static_processor
pattern: '\bCUST-\d{6}\b'
label: customer_id
# Phone numbers, which the model is not asked to look for.
- type: static_detector
pattern: '\+?\d[\d ()-]{7,}\d'
label: phone
called from +49 30 1234 5678 about CUST-004211 goes out as
called from PHONE_68a8db about CUSTOMER_ID_78e67d. A new category uses the tokenizer
without a selector, unless you list it under one (see
Placeholders (tokenizers)).
Project code names from a word list:
- type: static_processor
label: project
matcher:
type: dictionary
entries:
- { path: custom/projects.txt } # one name per line
- "Blue Heron" # or written inline
case: insensitive
Words that must stay readable, such as your own company or product names, which the model may take for companies:
- type: static_processor
label: keep
pattern: '(?i)^(?:InCountry|Kubernetes|Terraform)(?:\s+(?:Inc|Ltd|GmbH)\.?)?$'
# ...
- type: anonymizer
ignore_categories: [keep]
tokenizers:
# ...
ignore_categories leaves detections of the listed categories as written. Anchor the
pattern with ^ and $: it then relabels only a detection that is exactly one of
these names. Without the anchors, a detection such as Joe Miller of InCountry would
be relabeled too, and the person's name would go out readable. A detection to keep
that overlaps one to replace is dropped, so the replacement always wins. A category
cannot be both ignored and listed under a tokenizer selector.
Guards: checks on what was detected
A guard looks for a pattern or a word list, and every match must lie fully inside a
detection. With pattern:, a capture group narrows the check: when the pattern has
one, only group 1 must be covered, as in the name after a cue word. What happens when
a match is not covered is up to you: go on, detect again with other settings, or
refuse the request.
| Option | Default | Meaning |
|---|---|---|
pattern or matcher | one is required | What to look for (see Matchers). |
on_failure | { type: retry } | What to do when a match is not covered (see below). |
retry_limit | 1 | How many times detection runs again before on_exhausted applies. |
on_exhausted | abort | Once the retries are used up: abort refuses the request; continue keeps the last result. |
scope | anywhere | anywhere: every match is checked. adjacent: only a match joined to a detection, with nothing but spaces, a hyphen, or one joiners word between them, or overlapping it. |
joiners | [{ builtin: western_joining }] | Only with scope: adjacent: the words that join a match to a detection, such as von, de, van. |
accept_candidates | false | Replace a candidate (a span scored from the model's candidate_confidence up to its min_confidence) that covers the match, instead of failing on it. |
on_failure types:
| Type | Effect |
|---|---|
continue | Go on. The match stays as written, unless accept_candidates replaced it. The request is never refused. |
retry | Detect again with the same settings. The model gives the same answer to the same text, so this ends at on_exhausted. |
retry_with_confidence | Detect again, also replacing spans scored from min_confidence (0.0 to 1.0, written next to type). |
retry_with_escalation | Detect again with the next model in the ner_detector's models list. The image includes one model, so this needs a second model you mount and add; without one it is a startup error. |
abort | Refuse the request at once. |
A refused request is answered with 422 cloak_failed, and nothing is sent to the
provider.
A field that must always be protected, failing closed:
# The name after "Patient:" must be replaced, or the request is refused.
- type: guard
pattern: '\b[Pp]atient:\s*(\p{Lu}[\p{L}''-]+)'
on_failure: { type: retry_with_confidence, min_confidence: 0.05 }
# retry_limit: 1 and on_exhausted: abort are the defaults.
Patient: Zbigniew Brzezinski called goes out with a stand-in name. If detection
still leaves the word after Patient: uncovered at 0.05, the request is refused. This
also catches text where the word is not a name, such as Patient: Stable, discharged.,
so test a guard like this on your own data before relying on it.
The unknown-word safety net with your own vocabulary, so product and tool names are not taken as hints of a name:
- type: guard
matcher:
type: unknown_word
known:
- { path: "/app/models/lexicons/known-en.txt" }
- { path: custom/known-domain.txt } # one word per line
- { words: [Grafana, Prometheus] }
on_failure: { type: continue }
accept_candidates: true
A word in known is never a hint, so a low-scored candidate on it stays as written.
This does not stop the model from replacing a word it scores above min_confidence;
to keep such a word readable, use the keep processor above.
The boundary safety net with joining words for more naming traditions:
- type: guard
matcher: { type: unknown_word }
scope: adjacent
joiners: [{ builtin: western_joining }, bin, ibn, al]
on_failure: { type: continue }
accept_candidates: true
Matchers
pattern: '...' is short for matcher: { type: regex, pattern: '...' }. Three
matcher types are available to processors and guards:
| Type | Options | Matches |
|---|---|---|
regex | pattern; group: the capture group to report (default: the whole match) | The pattern. |
dictionary | entries; case: sensitive (default) or insensitive | Every listed word or phrase, as whole words. |
unknown_word | known (compared case-insensitively); min_length (default 2) | Every capitalized word (letters, apostrophes, hyphens) that known does not hold. |
entries, known, and joiners are lists of word sources, which can be mixed:
"Red Kite": one word or phrase, written inline.{ words: [Grafana, Prometheus] }: several, inline.{ path: custom/projects.txt }: a file with one entry per line; blank lines and lines starting with#are skipped.{ builtin: countries }(inentries) or{ builtin: western_joining }(injoiners): a list included in the server.
Patterns use the Rust regex syntax,
which has no look-around and no back-references. \\p{Lu} is an upper-case letter and
\\p{L} any letter, in every script. Under (?i), \\p{Lu} matches lower-case letters
too, so leave (?i) out of a pattern that relies on capitals. In YAML, write patterns
in single quotes so backslashes stay as written, and double a single quote inside
them ('').
Placeholders (tokenizers)
The anonymizer stage decides what each detected value is replaced with. Its
tokenizers: list is required: the Pre-Router has no built-in default, because the
choice decides what the provider can learn from the placeholders.
- type: anonymizer
tokenizers:
# No selector: the default for every category no other entry claims.
- tokenizer:
type: keyed_pseudonym
salt_scope: document
# A selector claims the listed categories.
- tokenizer:
selector:
type: category
categories: [person, company]
type: surrogate
Rules for the list:
- Exactly one entry has no
selector. It handles every category the others do not claim, so every detected value always has a placeholder. selectorsits insidetokenizer:and hastype: categorywith a list ofcategories. The categories are the detector'slabels(person,location,company,emailin the default configuration) and thelabelof eachstatic_processorandstatic_detector. They match whole names, case-insensitively:persondoes not claimperson_name.- A category is claimed at most once. Naming it under two selectors is a startup error.
Placeholders are chosen per value, not per occurrence: the same value gets the same placeholder everywhere in a request, and different values get different ones. Within a conversation, a value keeps its placeholder across turns.
keyed_pseudonym: tags such as PERSON_7f3a91
A short keyed hash of the value. The most reliable choice for restoring answers: the tag never occurs in normal text, and the model tends to copy it exactly.
| Parameter | Default | Meaning |
|---|---|---|
template | "{LABEL}_{id}" | The placeholder text. Must contain {id}. {LABEL} is the category in upper case, {label} in lower case; characters other than letters and digits become _. |
id_length | 6 | Hex characters in {id}, from 4 to 64. Longer ids make two values less likely to look alike. |
salt_scope | tenant | document: every request draws fresh placeholders (a conversation keeps the ones it already has), so two conversations cannot be linked through them. tenant: a value gets the same placeholder in every request and conversation. |
key | none | Required with salt_scope: tenant, refused as empty. Write it as an environment variable, for example "${AGENTCLOAK_TOKEN_KEY}". |
Write salt_scope explicitly. With tenant, the placeholder is a lasting identifier:
anyone who sees two requests can tell they concern the same person, so the text the
provider receives is pseudonymised, not anonymised. Treat the key as a secret, and
change it only knowing that every placeholder changes with it.
Keep templates to letters, digits, and _, such as {LABEL}_{id} or ID_{id}, so the
placeholder survives the model's answer unchanged.
surrogate: realistic stand-ins such as Jane Doe
A made-up value of the same kind, so the provider's model reads natural text. People,
companies, and first and last names are replaced word by word: "Ada" becomes the same
stand-in word in every name it appears in, and articles, legal suffixes such as Inc,
and titles stay as written.
| Parameter | Default | Meaning |
|---|---|---|
salt_scope | document | As for keyed_pseudonym. With tenant, a person keeps one stand-in name everywhere, which is a lasting identifier that also looks like a real person. |
key | none | Required with salt_scope: tenant, as for keyed_pseudonym. |
The kind of stand-in follows the category name:
| Category contains | Stand-in |
|---|---|
person, name | Full name |
first and name, given | First name |
last and name, surname, family | Last name |
compan, org, employer | Company name |
email | Email address |
phone, mobile, fax | Phone number |
city, location, address, place | City |
street | Street name |
country, state, province, region | Country, state |
postal, zip | Postal code |
username, url, ip, credit and card, ssn, date, birth | A value of that kind |
Any other category gets a stand-in shaped like the original value (email-like,
phone-like, or words). A stand-in word that also appears elsewhere in the request
cannot be told apart from it, so that value is left as the stand-in in the answer
rather than restored to a wrong one. Use keyed_pseudonym where every value must
come back.
counter: numbered tags such as PERSON_1
Numbers each category's values in the order they appear: PERSON_1, PERSON_2. Easy
to read, and within a conversation a new value never reuses an earlier number. With
ten or more values of a category, numbers are zero-padded (PERSON_01) so no tag is
the start of another.
| Parameter | Default | Meaning |
|---|---|---|
template | "{LABEL}_{n}" | The placeholder text. Must contain {n}, the number. {LABEL} and {label} as for keyed_pseudonym. |
counter cannot be used while the Files API is enabled (openai.files.uploads): an
uploaded file is protected on its own and later joins conversations, where its numbers
would refer to other values.
template: fixed text such as [redacted.person]
The same text for every value of a category, with format (default
"[redacted.{label}]"). Every person looks alike, so answers cannot be restored, and
the Pre-Router refuses it at startup. It exists for other AgentCloak uses where text is
removed for good.
File attachments
The files block protects files attached to chat and responses requests and enables
the Files API: uploads are protected before they reach the provider, and their names
and content are restored on the way back. Remove the block to refuse file inputs with
400 unsupported_content instead. The Files API (uploads) needs conversations
enabled.
Optional: your own sign-in in front of the provider
To require users to sign in with your identity provider before they reach the LLM,
define a policy under the top-level auth: and reference it on the route. Callers then
send their sign-in token in Authorization and the provider key in api-key:
auth:
providers:
- id: m365
type: microsoft
# tenant and client settings
policies:
users: { providers: [m365] }
proxy:
- id: openai
# ...
auth: { reference: users }
openai:
credential_headers: [api-key]
conversation:
scope: principal # a conversation belongs to one signed-in user
id:
- header: x-conversation-id
Installation and running with Docker
AgentCloak Pre-Router is published on Docker Hub as
incountrycom/agentcloak-pre-router, tagged with the version (1.0.0) and latest.
The commands below use 1.0.0.
The image is ready to run. It contains:
| Container path | Contents |
|---|---|
/app/agentcloak-server.yaml | The default configuration |
/app/models/gliner2-pii-multi/ | The GLiNER2-PII detection model |
/app/models/lexicons/ | The word lists of the safety nets |
/app/data/ | A volume for the conversation database |
/app/licenses/NOTICE | Licenses of the included model and word lists |
The server listens on port 5050 inside the container and runs as a non-root user.
Images are published for linux/amd64 and linux/arm64.
This is a preview release. It works through 31 January 2027 (UTC), and logs that date at startup. After that date the server no longer starts, and a running one refuses every request. The next version will offer several license options, which you will be able to obtain from the InCountry portal.
Test on your own data before production. Detection quality depends on your languages, writing style, and the kinds of personal data in your traffic. Before you send production traffic through the Pre-Router, run a representative sample of your own requests through it, check what the provider receives, and confirm that every value you must protect is replaced.
1. Pull the image
docker pull incountrycom/agentcloak-pre-router:1.0.0
2. Run with docker run
Generate the conversation secret once and keep it, for example in a file:
openssl rand -hex 32 > conversation-secret
Start the container:
docker run -d --name agentcloak-pre-router \
-p 5050:5050 \
-e AGENTCLOAK_CONVERSATION_SECRET="$(cat conversation-secret)" \
-e OPENAI_BASE_URL="https://api.openai.com/v1" \
-v agentcloak-data:/app/data \
--memory 4g --cpus 2 \
--read-only --tmpfs /tmp \
--cap-drop ALL --security-opt no-new-privileges:true \
--restart unless-stopped \
incountrycom/agentcloak-pre-router:1.0.0
agentcloak-data is a named volume that keeps conversations across restarts and
image updates. To keep the database in a host folder instead, mount a folder writable
by user id 65532:
mkdir -p data && sudo chown 65532:65532 data
# then: -v "$PWD/data:/app/data"
Set AGENTCLOAK_DB_PATH to keep the database file somewhere else, for example on a
different mounted volume.
2 (alternative). Run with Docker Compose
Create .env next to the configuration:
OPENAI_BASE_URL=https://api.openai.com/v1
AGENTCLOAK_CONVERSATION_SECRET=<output of: openssl rand -hex 32>
Create docker-compose.yml:
services:
agentcloak-pre-router:
image: incountrycom/agentcloak-pre-router:1.0.0
ports:
- "5050:5050"
environment:
OPENAI_BASE_URL: ${OPENAI_BASE_URL:-https://api.openai.com/v1}
AGENTCLOAK_CONVERSATION_SECRET: ${AGENTCLOAK_CONVERSATION_SECRET:?set AGENTCLOAK_CONVERSATION_SECRET in .env}
volumes:
- agentcloak-data:/app/data
mem_limit: 4g
cpus: 2
read_only: true
tmpfs:
- /tmp
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
stop_grace_period: 30s
restart: unless-stopped
volumes:
agentcloak-data:
Start it:
docker compose up -d
3. Check that it runs
Loading the model takes a few seconds. A configuration problem, such as a missing
AGENTCLOAK_CONVERSATION_SECRET, stops the server at startup with an error naming the
setting. Check the logs:
docker logs agentcloak-pre-router # or: docker compose logs agentcloak-pre-router
GET / is the health check. It answers "status": "healthy" once the server is
ready, and works as a liveness probe for orchestrators:
curl http://localhost:5050/
Send a request through it with your provider key:
curl http://localhost:5050/to/v1/openai/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Say hello to Ada Lovelace."}]}'
The answer mentions Ada Lovelace by name, while the provider received a stand-in name.
Clients now use <http://<host>>:5050/to/v1/openai as their base URL.
Changing the configuration
Copy the default configuration out of the image:
id=$(docker create incountrycom/agentcloak-pre-router:1.0.0)
docker cp "$id:/app/agentcloak-server.yaml" ./agentcloak-server.yaml
docker rm "$id"
Edit it, then mount it over the default by adding this to the docker run command:
-v "$PWD/agentcloak-server.yaml:/app/agentcloak-server.yaml:ro" \
or this under volumes: in docker-compose.yml:
- ./agentcloak-server.yaml:/app/agentcloak-server.yaml:ro
Keep server.bind at 0.0.0.0:5050 so the server is reachable from outside the
container, and keep the /app/models/... paths unless you mount your own model or word
lists. The configuration is read at startup: after editing it, restart the container
(docker restart agentcloak-pre-router or docker compose restart).
Production notes
- Test first. Run a representative sample of your own requests through the Pre-Router and check what the provider receives before you switch production traffic to it. Repeat the check after every configuration change or image update.
- HTTPS. Put the Pre-Router behind a reverse proxy that terminates TLS (Caddy, nginx, Traefik, or your cloud load balancer), and give clients the HTTPS URL. Your users' provider keys travel in request headers.
- Resources. Detection runs on the CPU. Each copy of the model (
sessions, 2 by default) takes about 1.2 GiB of memory: allow at least 2 CPUs and 4 GiB with the default configuration. More CPUs and sessions serve more requests in parallel. - Conversation data. The database holds the original values of each conversation
so answers can be restored. Treat the
/app/datavolume like any store of personal data: restrict access to it and include it in your retention policy. A conversation with no activity for a day expires and is deleted (entities.ttl_secs). Clients that resend the full chat history keep working after it expires. - One container per database. Keep the database on a local disk, not a network file system, and do not share one file between containers.
Supported endpoints
| Endpoint | What is protected | Streaming |
|---|---|---|
POST /chat/completions | Message text, refusals, tool call arguments, tool outputs, attached files | Yes |
POST /responses | Instructions, input text, function calls and outputs, reasoning summaries, MCP call data, attached files | Yes |
POST /embeddings | Input text | Not applicable |
GET /models and other requests without a body | Forwarded unchanged | Not applicable |
Files API (/files) | File name and content, when enabled by your administrator | Not applicable |
Model names, roles, tool names, JSON schemas, ids, and every other parameter are forwarded exactly as you sent them, so requests stay valid for the provider.
Requests with a body on any other path are answered 404 unsupported_endpoint. The
Pre-Router never forwards a body it has not protected.
Content that cannot be protected
Image and audio content parts, token-id embedding inputs, and file URLs are rejected
with 400 unsupported_content, because the personal data inside them cannot be
replaced. The error's param field points at the exact part, for example
messages[2].content[1]. Your administrator can allow specific messages (for example
system prompts) to pass through as written, in which case such parts are accepted there.
Behaviour details
Responses
Every text in the provider's answer is checked for placeholders, including tool call arguments (which stay valid JSON) and provider-specific extra fields. Placeholders are replaced with the original values. Error responses from the provider are returned unchanged, with the provider's status code.
Streaming
stream: true works for chat completions and responses. Because a placeholder can be
split across two stream events, the Pre-Router holds back a small amount of text and
releases it in short bursts once it is safe. In practice:
- Text arrives slightly behind the provider's pace, in bursts of a few hundred bytes.
- Some events carry an empty text delta while text is held. Roles, finish reasons, and usage still arrive on time.
- A burst may end in the middle of a word. Placeholders are never released half-way.
- The full text, concatenated, is exactly the restored answer.
If a problem occurs after streaming has started, the stream ends with one error event in the OpenAI error shape, which the OpenAI SDKs raise as an API error.
Files
When file support is enabled, files attached to chat or responses requests (as
file_data) are converted to text, protected together with the surrounding message, and
delivered to the provider as text or as a Markdown file. Supported formats depend on
your deployment and can include plain text, Markdown, Office documents, OpenDocument,
RTF, EPUB, CSV, and PDF.
With the Files API enabled, POST /files uploads are protected before they reach the
provider, and later requests that reference the file by file_id restore its values in
the answer. Listing, retrieving, and downloading files through the Pre-Router return
the original file names and content. Uploads accept the assistants and user_data
purposes.
Conversations
By default each request stands alone: values are restored only from what was sent in the same request. This works for any client that sends the full chat history on every turn.
Your deployment can also keep conversations. Then:
- A value keeps the same placeholder for the whole conversation, across turns.
- A placeholder the model repeats from an earlier turn is restored even if your application did not resend that turn.
- Stateful Responses API features work:
previous_response_idandconversation.
To tell turns of one conversation apart, send a conversation id in the header your
administrator configures (typically X-Conversation-Id). The header is not forwarded
to the provider. Ids are up to 200 characters of letters, digits, and -_.:@.
Depending on configuration, the Pre-Router can also recognise a conversation from a
field in the request body, from previous_response_id, or automatically from the
opening messages when your client resends the full history.
Conversations are private to their owner: depending on configuration, to one provider API key, to one signed-in user, or to one organization. A different key or user cannot reach another's conversation, even with the same conversation id.
Without conversations enabled, requests using previous_response_id or conversation
are answered 400 stateful_conversation.
Administrator policies
Your administrator can set rules for specific parts of a request, for example:
- send system prompts or public search results as written,
- refuse requests that would call a specific MCP server (
400 refused_by_rule), - return tool call arguments with placeholders, for agents that work entirely on protected data.
Errors
Errors produced by the Pre-Router use the OpenAI error shape, so SDKs raise them as
regular API errors. The type is always agentcloak_error:
{
"error": {
"message": "image content cannot be cloaked",
"type": "agentcloak_error",
"code": "unsupported_content",
"param": "messages[2].content[1]"
}
}
| Status | code | Meaning |
|---|---|---|
| 400 | invalid_request_body | The body is not a JSON object, or a field has the wrong type. |
| 400 | unsupported_content | A part that cannot be protected (image, audio, file URL). param names it. |
| 400 | refused_by_rule | An administrator rule refuses this part. param names it. |
| 400 | stateful_conversation | previous_response_id or conversation used without conversations enabled. |
| 400 | missing_conversation_id | Your deployment requires a conversation id and none was sent. |
| 400 | invalid_conversation_id | The conversation id is too long or contains other characters. |
| 400 | unknown_previous_response | previous_response_id names a response not made through the Pre-Router by you. |
| 400 | unknown_file | file_id names a file not uploaded through the Pre-Router by you. |
| 400 | unsupported_purpose | File upload with a purpose other than assistants or user_data. |
| 401 | missing_credentials | No API key header was sent. |
| 404 | unsupported_endpoint | A request with a body on an endpoint the Pre-Router does not support. |
| 413 | request_too_large | The request body is too large. |
| 415 | unsupported_encoding | The request body is compressed. Send it uncompressed. |
| 413, 415, 422 | file codes such as unsupported_media_type, encrypted, needs_ocr, unreadable | An attached file could not be converted. Nothing was sent. |
| 422 | cloak_failed | Personal data could not be protected. Nothing was sent. |
| 502 | upstream_unreachable, response_too_large, uncloak_failed | The provider could not be reached, its answer was too large, or the answer could not be restored. |
| 503 | storage_unavailable | Conversation storage is temporarily unavailable. Nothing was sent. |
| 504 | cloak_timeout, upstream_timeout | Protection or the provider took too long. |
Errors from the provider itself keep the provider's status and body. They may contain placeholders where they echo your request.
Best practices
- Send one conversation id per chat when conversations are enabled, so placeholders stay stable and the model's references to earlier turns are restored.
- Prefer text over images for content that contains personal data.
- Allow for a short delay in streams. Render text as it arrives; do not assume each event carries text.
- Handle
422 cloak_failedand504 cloak_timeoutlike any transient API error: retry, or tell the user the request could not be processed safely. - Keep your provider key in the same place as today. Nothing about key management changes.