Skip to main content

Context

Context is Kilimanjaro's semantic-intelligence extension. Context generates embeddings across project collections and the object-storage surface, makes them searchable with natural-language queries, and exposes a retrieval API that composes cleanly into agentic workflows. Search the content of every file uploaded to a project — images, PDFs, spreadsheets, documents, audio — and the document records alongside them; plain-language queries return the records and files that mean what was asked, not just the ones that share a keyword.

Context is built on the same extension framework available to every extension class — Anchor and Circuit extensions ship against the same contract.

1. What Context covers​

SurfaceRole
Embedding pipelineGenerates embeddings for the selected collections and buckets with ai.embed and persists them in the Context-owned schema
Query APINatural-language query endpoint with hybrid keyword + semantic retrieval and optional re-ranking
Retrieval API for agentsStructured retrieval surface designed for agentic workflows — returns top-k matches with provenance metadata and policy-evaluated visibility
Console UIConfiguration surface for selecting collections and buckets to embed, managing embedding-model choice, monitoring index health
Cost controlsPer-project budget caps; re-embedding cadence; embedding-job back-off policy

Context is a full-surface extension: Console UI + runtime API + background-logic functions.

2. What Context embeds​

Content typeEmbedding strategy
Document records (any collection)Field-level chunking on text-typed fields; one embedding per chunk; collection schema determines the chunk boundaries
Plain text + Markdown filesWhole-file or chunked depending on length
PDFsOCR-pass first (for image-only PDFs); text-extracted PDFs chunk by page boundary
SpreadsheetsPer-row embeddings with column-name context; aggregated whole-sheet embedding for the index
Word documentsWhole-file extraction then chunked by heading boundary
ImagesVisual-language-model captioning; the caption is what gets embedded; image bytes are stored alongside for return-on-retrieval
AudioTranscription first; transcript is what gets embedded; original audio is stored alongside

Each collection and bucket is opt-in. The Context Console UI shows what's currently indexed, what's pending, and what's been skipped (with reason).

3. Embedding model selection​

Deployment modelDefault modelCustom-model support
CloudA platform-hosted general-purpose embedding modelChoose from a curated catalogue of platform-hosted models
Sovereign Control / Private Perimeter deploymentsBring-your-own embedding endpoint — self-hosted models running in the same trust zone as the rest of the platformFull BYO endpoint binding; the embedding pipeline routes through your endpoint instead of the platform's

Switching models triggers a re-embedding of the index — the prior embeddings remain queryable during the transition; the platform writes the new embeddings alongside; the index cuts over atomically when the re-embedding completes.

4. Querying​

4.1 Natural-language query​

The query surface accepts a plain-language string, an optional filter clause, and an optional top-k bound. The query goes through:

  1. Query embedding. The input string is embedded using the same model as the index.
  2. Vector search. Nearest-neighbour retrieval against the per-collection / per-bucket embedding index.
  3. Hybrid re-rank. If the query contains terms that resemble keywords, the result set is re-ranked combining the vector score with a keyword score (BM25-style).
  4. Policy filter. Every candidate result is checked against the calling caller's JSON policies — results the caller cannot see are dropped silently.

The response carries record IDs, similarity scores, and provenance metadata (which collection or bucket, which chunk, which embedding version).

4.2 Retrieval-augmented patterns​

For agentic workflows, the retrieval API returns the full payload of the matched records and files alongside scores, so a downstream LLM call can be assembled in one round-trip. The platform applies the policy filter before payloads are returned; an agent invoked under a constrained role sees only the records that role is authorised for.

5. Permissions Context requests at enable​

PermissionWhy
documents/read/* qualifier-bound to the collections selected at Console-config timeReading source records for embedding + retrieval payload
collections/read/*Reading collection schemas to determine chunk boundaries
files/read/* qualifier-bound to selected bucketsReading source files for embedding + retrieval payload
webhooks/subscribe/*Subscribing to document.created, document.updated, storage.object.created events so the index can be refreshed

The grant binds at enable time; a PROJECT_ADMIN reviews each line before authorising. Collection-list and bucket-list qualifiers are tightened through the Context Console UI after enable.

6. Cost controls​

Embedding is the dominant operational cost. Context surfaces three knobs:

KnobEffect
Per-project monthly budgetHard cap on embedding cost per billing period. Context pauses re-embedding when the cap is hit; queries against existing embeddings continue.
Re-embedding cadenceHow often the pipeline re-embeds records that have changed. Default: real-time on document.updated. Tunable to nightly batches for cost-sensitive projects.
Initial-embedding back-offWhen Context is enabled on a project with a large existing corpus, the initial-embedding job paces itself per a configurable embeddings-per-minute rate to avoid budget blowout.

Budget consumption surfaces on the project's billing reference alongside other metered usage.

7. Events​

EventFired when
context.index.readyA collection's or bucket's initial embedding pass completes
context.embedding.failedAn embedding job fails (model unreachable, content unparseable, budget cap hit) — context.cause carries the reason
context.query.completedA query completes (optional; opt-in for query-analytics workflows)

8. See also​

  • Extensions API reference — the framework contract Context installs against.
  • API → Compute — the function lifecycle, ai.embed, and the trigger registry.
  • API → Data — query the underlying records Context retrieves.
  • Paperless — embed Paperless captures and workflow records into Context for semantic search across workspace content.
  • Specs → Billing — embedding-cost metering.