Kagi currently offers multiple search/answer engines (e.g., Kagi Search, FastGPT, Universal Summarizer, various LLM endpoints). This feature introduces an intelligent auto-router that:
Analyses the incoming query — intent, complexity, domain, required reasoning depth, latency sensitivity, and cost profile
Routes to the optimal engine from a user-configurable pool — not a fixed preset
Provides transparency — shows which engine was selected, why (rationale), and allows one-click override
This mirrors the NeMo Switchyard pattern (NVIDIA's open-source Apache 2.0 library): a provider-agnostic orchestration layer that sits between the client and a pool of model backends, selecting the best model per request based on capability, cost, and latency — with session-aware routing and full observability (selected model, decision rationale, token usage, latency, outcomes).
Additional Context — How It Could Work
1. User-Defined Engine Pool (Core Requirement)
- In Settings → Engine Pool, the user sees all available engines (Kagi Search, FastGPT, Universal Summarizer, Claude, GPT-4o, Llama 3.1 405B, etc.)
- User enables/disables each engine with a toggle
- User can pin favourites (e.g., "Always use Kagi Search for navigational queries")
- User can set per-engine caps (e.g., "Max 50 FastGPT calls/day", "Prefer local/private models when available")
2. Routing Intelligence (Auto Mode)
| Query Signal | Routing Heuristic |
|---|
| Navigational / factual lookup | → Kagi Search (fast, authoritative) |
| Complex reasoning / coding / math | → Strongest reasoning model in pool (e.g., GPT-4o, Claude 3.5 Sonnet) |
| Summarisation / extraction | → Universal Summarizer or long-context model |
| Creative / open-ended | → Best creative model in pool |
| Low-latency requirement (user flag) | → Fastest enabled engine |
| Privacy-sensitive (user flag) | → Local/on-prem/private endpoints only |
| Cost-sensitive (user flag) | → Cheapest enabled engine meeting quality threshold |
3. Session-Aware Routing
- Multi-turn conversations carry routing state (like NeMo Switchyard's session awareness)
- If a conversation starts with a coding task routed to GPT-4o, subsequent follow-ups stay on that model unless the user explicitly switches or the task class changes
4. Observability & Override (Every Response)
- Small badge: "Routed to: GPT-4o • Reason: Complex reasoning • Latency: 1.2s • Cost: $0.004"
- One-click "Re-route with…" dropdown to try another enabled engine instantly
- "Why this engine?" expandable panel showing the decision rationale
5. A/B Testing Mode (Explicit User Request)
- "Compare 2 engines" toggle in the UI
- User selects Engine A and Engine B from their enabled pool
- Kagi runs the query against both in parallel, shows side-by-side results with latency/cost/quality metrics
- User picks preferred result; system learns preference for future similar queries
6. API & Programmability
- Expose routing logic via API:
POST /v1/route with { query, pool: ["engine-a", "engine-b"], strategy: "auto" | "fastest" | "best-reasoning" | "cheapest" }
- Returns
{ selected_engine, rationale, estimated_cost, estimated_latency } — caller can accept or override
Priority / Impact
- High — Differentiates Kagi as the only search/answer platform with user-controlled, transparent, observable multi-engine orchestration
- Aligns with power-user workflows (researchers, developers, analysts) who already juggle multiple tools
- Builds on Kagi's existing strength: user agency over the search experience
Related Existing Features
- FastGPT / Universal Summarizer / Kagi Search (current engines)
- Lenses / Bangs (user-customisable routing primitives)
- Privacy settings (local/private model preference)
Suggested Implementation Phases
| Phase | Scope |
|---|
| 1 | Engine Pool toggles + manual per-query engine selector (dropdown in search bar) |
| 2 | Auto-router with heuristics + routing badge + "Why this engine?" panel |
| 3 | Session-aware routing + A/B compare mode + override logging |
| 4 | API exposure + routing analytics dashboard (usage per engine, cost, latency trends) |
This would make Kagi the first search platform to treat model routing as a first-class, user-configurable primitive — exactly the "system-of-models" approach NeMo Switchyard enables, but exposed directly to end users with full control and transparency.
