Context
Context is Kilimanjaro's semantic-intelligence extension. Context generates embeddings across project collections and the object-storage surface, makes them searchable with natural-language queries, and exposes a retrieval API that composes cleanly into agentic workflows. Search the content of every file uploaded to a project — images, PDFs, spreadsheets, documents, audio — and the document records alongside them; plain-language queries return the records and files that mean what was asked, not just the ones that share a keyword.
Context is built on the same extension framework available to every extension class — Anchor and Circuit extensions ship against the same contract.
1. What Context covers
| Surface | Role |
|---|---|
| Embedding pipeline | Generates embeddings for the selected collections and buckets with ai.embed and persists them in the Context-owned schema |
| Query API | Natural-language query endpoint with hybrid keyword + semantic retrieval and optional re-ranking |
| Retrieval API for agents | Structured retrieval surface designed for agentic workflows — returns top-k matches with provenance metadata and policy-evaluated visibility |
| Console UI | Configuration surface for selecting collections and buckets to embed, managing embedding-model choice, monitoring index health |
| Cost controls | Per-project budget caps; re-embedding cadence; embedding-job back-off policy |
Context is a full-surface extension: Console UI + runtime API + background-logic functions.
2. What Context embeds
| Content type | Embedding strategy |
|---|---|
| Document records (any collection) | Field-level chunking on text-typed fields; one embedding per chunk; collection schema determines the chunk boundaries |
| Plain text + Markdown files | Whole-file or chunked depending on length |
| PDFs | OCR-pass first (for image-only PDFs); text-extracted PDFs chunk by page boundary |
| Spreadsheets | Per-row embeddings with column-name context; aggregated whole-sheet embedding for the index |
| Word documents | Whole-file extraction then chunked by heading boundary |
| Images | Visual-language-model captioning; the caption is what gets embedded; image bytes are stored alongside for return-on-retrieval |
| Audio | Transcription first; transcript is what gets embedded; original audio is stored alongside |
Each collection and bucket is opt-in. The Context Console UI shows what's currently indexed, what's pending, and what's been skipped (with reason).
3. Embedding model selection
| Deployment model | Default model | Custom-model support |
|---|---|---|
| Cloud | A platform-hosted general-purpose embedding model | Choose from a curated catalogue of platform-hosted models |
| Sovereign Control / Private Perimeter deployments | Bring-your-own embedding endpoint — self-hosted models running in the same trust zone as the rest of the platform | Full BYO endpoint binding; the embedding pipeline routes through your endpoint instead of the platform's |
Switching models triggers a re-embedding of the index — the prior embeddings remain queryable during the transition; the platform writes the new embeddings alongside; the index cuts over atomically when the re-embedding completes.
4. Querying
4.1 Natural-language query
The query surface accepts a plain-language string, an optional filter clause, and an optional top-k bound. The query goes through:
- Query embedding. The input string is embedded using the same model as the index.
- Vector search. Nearest-neighbour retrieval against the per-collection / per-bucket embedding index.
- Hybrid re-rank. If the query contains terms that resemble keywords, the result set is re-ranked combining the vector score with a keyword score (BM25-style).
- Policy filter. Every candidate result is checked against the calling caller's JSON policies — results the caller cannot see are dropped silently.
The response carries record IDs, similarity scores, and provenance metadata (which collection or bucket, which chunk, which embedding version).
4.2 Retrieval-augmented patterns
For agentic workflows, the retrieval API returns the full payload of the matched records and files alongside scores, so a downstream LLM call can be assembled in one round-trip. The platform applies the policy filter before payloads are returned; an agent invoked under a constrained role sees only the records that role is authorised for.
5. Permissions Context requests at enable
| Permission | Why |
|---|---|
documents/read/* qualifier-bound to the collections selected at Console-config time | Reading source records for embedding + retrieval payload |
collections/read/* | Reading collection schemas to determine chunk boundaries |
files/read/* qualifier-bound to selected buckets | Reading source files for embedding + retrieval payload |
webhooks/subscribe/* | Subscribing to document.created, document.updated, storage.object.created events so the index can be refreshed |
The grant binds at enable time; a PROJECT_ADMIN reviews each line before authorising. Collection-list and bucket-list qualifiers are tightened through the Context Console UI after enable.
6. Cost controls
Embedding is the dominant operational cost. Context surfaces three knobs:
| Knob | Effect |
|---|---|
| Per-project monthly budget | Hard cap on embedding cost per billing period. Context pauses re-embedding when the cap is hit; queries against existing embeddings continue. |
| Re-embedding cadence | How often the pipeline re-embeds records that have changed. Default: real-time on document.updated. Tunable to nightly batches for cost-sensitive projects. |
| Initial-embedding back-off | When Context is enabled on a project with a large existing corpus, the initial-embedding job paces itself per a configurable embeddings-per-minute rate to avoid budget blowout. |
Budget consumption surfaces on the project's billing reference alongside other metered usage.
7. Events
| Event | Fired when |
|---|---|
context.index.ready | A collection's or bucket's initial embedding pass completes |
context.embedding.failed | An embedding job fails (model unreachable, content unparseable, budget cap hit) — context.cause carries the reason |
context.query.completed | A query completes (optional; opt-in for query-analytics workflows) |
8. See also
- Extensions API reference — the framework contract Context installs against.
- API → Compute — the function lifecycle,
ai.embed, and the trigger registry. - API → Data — query the underlying records Context retrieves.
- Paperless — embed Paperless captures and workflow records into Context for semantic search across workspace content.
- Specs → Billing — embedding-cost metering.