Skip to main content
POST/compute/llms/{modelId}/invoke-stream

Stream LLM Response

Dispatches an inference request to the specified LLM model and streams the response as Server-Sent Events (SSE), delivering tokens incrementally for low-latency UX.

Requires PROJECT_AUTHOR or PROJECT_ADMIN as an LLM cost gate.

Returns K9O_LLM_MODEL_UNAVAILABLE (404) when the requested model identifier is not in the curated library (every listed model is available on every plan), and K9O_LLM_PROVIDER_UNAVAILABLE (503) when an external provider is unreachable.

Parameters
modelId *path

The model's id, as the model list returns it.

projectSlug *path

The project's slug: the path segment after @.

It is unique within the org.

Request body
prompt *string

The prompt, sent to the model as a single user message. 1 to 100,000 characters.

maxTokensinteger

The most tokens the model may generate, 1 to 32,000.

Default: the model server's own limit.

temperaturenumber

Sampling temperature, 0 to 2.

Lower is more deterministic.

Default: the model server's own.

usageTypestring

A label of your own for this call, echoed back in the response.

Default sdk.direct.

Responses
HTTPK9O CodeDescription
200The reply, streamed as server-sent events.
400K9O_MODEL_NOT_DECLAREDThe model identifier is not declared in the extension's manifest compatibility list.
401K9O_INVALID_CREDENTIALSThe request credentials are missing or invalid.
401K9O_SESSION_EXPIREDYour session has expired; sign in again to continue.
402K9O_OUT_OF_TOKENSThe org's included inference credit is used up and its wallet cannot cover the call (W9-Billing's inference gate). Never raised for your own provider (BYOM).
403K9O_ACCESS_DENIEDYour role does not permit direct LLM streaming inference; PROJECT_AUTHOR or PROJECT_ADMIN is required.
502K9O_UPSTREAM_UNAVAILABLEThe inference credit check is unavailable, so the model was not called; retry.
503K9O_LLM_PROVIDER_UNAVAILABLEThe external LLM provider is temporarily unreachable; please retry with exponential backoff.
Response body (200)
TypeScript
// const k9o = new K9o({ orgSlug: 'playground', projectSlug: 'storefront', apiKey: 'k9o_…' })
const { data } = await k9o.functions.invoke('mistral-7b', {})

Showing example in TypeScript

Request example
{
  "prompt": "string",
  "maxTokens": 0,
  "temperature": 0,
  "usageType": "string"
}
Response example (200)
"data: {\"id\":\"chatcmpl-417\",\"object\":\"chat.completion.chunk\",\"created\":1762937117,\"model\":\"mistral:7b\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"Export\"},\"finish_reason\":null}]}\n\ndata: {\"id\":\"chatcmpl-417\",\"object\":\"chat.completion.chunk\",\"created\":1762937117,\"model\":\"mistral:7b\",\"choices\":[{\"index\":0,\"delta\":{\"content\":\" fails in Safari.\"},\"finish_reason\":\"stop\"}]}\n\ndata: {\"id\":\"chatcmpl-417\",\"object\":\"chat.completion.chunk\",\"created\":1762937117,\"model\":\"mistral:7b\",\"choices\":[],\"usage\":{\"prompt_tokens\":31,\"completion_tokens\":5,\"total_tokens\":36}}\n\ndata: [DONE]\n\n"

More in Compute & Automation