The securest way to AI — now open source.
Detect PII, block jailbreaks, and guard every AI call — before data ever reaches a provider. Xilos is the open source layer that sits between your applications and the LLMs. Self-host. Audit the code. Your infrastructure, your data, your rules.
Solo (single user) · Org (team, RBAC) · Xilos Cloud (managed)
cd xilos-oss
./install.sh
# Pick Solo or Org mode
# Open http://localhost:3001
Open Source
BSL 1.1 Licensed
Docker
One-command setup
PII Protection
Real-time detection
17ms
Cache Response Time
One line change. Everything inherits.
Point your existing agents at Xilos. One base_url change gets routing, caching, guardrails, and full cost tracking.
export ANTHROPIC_BASE_URL="https://api.xilos.ai/api/v1"
export ANTHROPIC_API_KEY=sk-xilos-...
# Claude Code now routes through Xilos:
# → Smart routing to best model
# → Semantic cache (50% lower costs)
# → PII detection & guardrails
# → MCP tools with audit trails
Run your AI through Xilos and it changes everything.
Protect your data with real-time PII detection and local guardrails. Enhance every query with prompt improvement and contextual knowledge. Save money by caching questions and answers. All open source, all on your infrastructure.
Multi-Model Routing
Route every query to the best model based on cost, latency, or capability. Define rules in natural language — no code required.
Semantic Cache
Cache responses for semantically similar queries. Cache hits bypass LLMs entirely — lower cost, faster responses, smarter over time.
Agent Guardrails
Local PII detection, jailbreak prevention, and content filters. No data leaves your infrastructure.
Workflow Orchestration
Chain LLMs sequentially or in parallel. Build multi-stage pipelines — search with one model, reason with another, summarize with a third.
Cost Visibility
Per-model, per-user, per-org cost tracking. Know exactly what your agents are spending, in real time.
Context Engine
Every answered query becomes a searchable knowledge asset. Your agents' collective experience compounds over time.
MCP Tool Integration
Connect any Model Context Protocol server. Web search, file access, code execution — all routed through Xilos with audit trails.
Audit & Compliance
Complete immutable trail of every AI interaction. Per-request logs with prompt, response, model, tokens, and cost.
Self-Host or Cloud
Run on your own infrastructure with Docker, or use Xilos Cloud for zero-ops managed hosting.
Self-host Solo or Org. Scale to Cloud.
Start with Solo for personal use — no login, no auth wall. Upgrade to Org for teams with RBAC and shared governance. Or let us host it with Cloud.
For personal use
- Single user, no login required
- Self-host with Docker
- All routing features
- Semantic caching
- Basic guardrails
For teams
- Multi-user with RBAC
- Email/password or SSO
- Teams and departments
- Shared governance
- Invite flows and audit
We host it. Zero-ops.
- Fully managed by MPR
- Unlimited users
- Enterprise SSO (SAML)
- 99.9% SLA
- Priority support
Audit the code. Trust nothing else.
Every AI provider wants you to trust their security promises. Xilos is different — the code is open. PII detection, jailbreak prevention, content filtering — read every line. Self-host on your infrastructure. Zero telemetry. Zero call-home. Free for Solo and Org use.
View on GitHubBSL 1.1
License
Solo + Org
Free to use
No Telemetry
No call-home
How Xilos compares
The only open-source AI gateway that gives you total control.
| Feature | Xilos | OpenRouter | Portkey | Helicone | LiteLLM | Cloudflare | Palo Alto | Microsoft | OpenAI | Anthropic |
|---|---|---|---|---|---|---|---|---|---|---|
| Open source | — | — | — | — | — | — | — | — | ||
| Self-hosted | — | — | — | — | — | — | — | — | ||
| Multi-model routing | — | — | — | — | — | |||||
| Semantic cache | — | — | — | — | — | — | — | — | — | |
| Local guardrails | — | — | — | — | — | — | — | — | — | |
| Workflow engine | — | — | — | — | — | — | — | — | — | |
| MCP tool integration | — | — | — | — | — | — | — | — | — | |
| Cost tracking | — | — | — | |||||||
| Audit trails | — | — | — | — | ||||||
| Context Engine | — | — | — | — | — | — | — | — | — |
✓ = native feature · — = not available · Xilos is the only open-source gateway combining all of the above
Don't want to run it yourself?
Xilos Cloud — the same gateway, hosted and managed by Mill Pond Research. Zero-ops, auto-scaling, enterprise controls.
- Zero-ops, auto-scaling, updates handled
- Teams, orgs, and enterprise controls
- SOC 2, SLAs, priority support
Frequently asked questions
How is Xilos different from Portkey?
Can I self-host Xilos?
./install.sh from the repo, pick Solo or Org mode, and you're up in minutes. Everything runs in Docker on your infrastructure. No telemetry, no call-home. Full docs at docs.xilos.ai.What LLM providers do you support?
What about data privacy?
Can I migrate from self-host to Cloud?
What is the BSL license?
Secure your AI. No provider gets your data.
Clone the repo, run the installer, and your AI has a security layer. Free. Open source. Audit the code yourself.