Skip to main content
POST/compute/llms/{modelId}/invoke

Invoke LLM

Dispatches a synchronous inference request to the specified LLM model and returns { completion: string, tokensUsed: int, latencyMs: int }.

Requires PROJECT_AUTHOR or PROJECT_ADMIN as an LLM cost gate.

Returns K9O_LLM_MODEL_UNAVAILABLE (404) when the requested model identifier is not in the curated library (every listed model is available on every plan), and K9O_LLM_PROVIDER_UNAVAILABLE (503) when an external provider is unreachable.

Parameters
modelId *path

The model's id, as the model list returns it.

projectSlug *path

The project's slug: the path segment after @.

It is unique within the org.

Request body
prompt *string

The prompt, sent to the model as a single user message. 1 to 100,000 characters.

maxTokensinteger

The most tokens the model may generate, 1 to 32,000.

Default: the model server's own limit.

temperaturenumber

Sampling temperature, 0 to 2.

Lower is more deterministic.

Default: the model server's own.

usageTypestring

A label of your own for this call, echoed back in the response.

Default sdk.direct.

Responses
HTTPK9O CodeDescription
200Success
400K9O_MODEL_NOT_DECLAREDThe model identifier is not declared in the extension's manifest compatibility list.
401K9O_INVALID_CREDENTIALSThe request credentials are missing or invalid.
401K9O_SESSION_EXPIREDYour session has expired; sign in again to continue.
402K9O_OUT_OF_TOKENSThe org's included inference credit is used up and its wallet cannot cover the call (W9-Billing's inference gate). Never raised for your own provider (BYOM).
403K9O_ACCESS_DENIEDYour role does not permit direct LLM inference invocation; PROJECT_AUTHOR or PROJECT_ADMIN is required.
502K9O_UPSTREAM_UNAVAILABLEThe inference credit check is unavailable, so the model was not called; retry.
503K9O_LLM_PROVIDER_UNAVAILABLEThe external LLM provider is temporarily unreachable; please retry with exponential backoff.
Response body (200)
data *object

The model's reply and token usage.

modelId *string

The model that answered.

content *string

The model's reply.

Empty when the model returned no text.

finishReason *any

Why the model stopped, as the model server reports it: usually stop (it finished) or length (it reached maxTokens).

Null when not reported.

One of: string
or: null
usage *object

Token counts reported by the model server.

Only reported counts are metered.

promptTokens *integer

Tokens in the prompt. 0 when the model server didn't report it.

completionTokens *integer

Tokens generated. 0 when the model server didn't report it.

totalTokens *integer

Prompt and generated tokens together. 0 when the model server didn't report it.

usageType *string

The usageType you sent, or sdk.direct.

meta *any
TypeScript
// const k9o = new K9o({ orgSlug: 'playground', projectSlug: 'storefront', apiKey: 'k9o_…' })
const { data } = await k9o.functions.invoke('mistral-7b', {
  prompt: 'Summarise this ticket in one sentence: the export button does nothing on Safari.',
  maxTokens: 256,
  temperature: 0.2,
  usageType: 'support.summarise'
})

Showing example in TypeScript

Request example
{
  "prompt": "Summarise this ticket in one sentence: the export button does nothing on Safari.",
  "maxTokens": 256,
  "temperature": 0.2,
  "usageType": "support.summarise"
}
Response example (200)
{
  "data": {
    "modelId": "string",
    "content": "string",
    "finishReason": "string",
    "usage": {
      "promptTokens": 0,
      "completionTokens": 0,
      "totalTokens": 0
    },
    "usageType": "string"
  }
}

More in Compute & Automation