Python API reference

What is the exact name of the thing I need?

Generated from axonium.__all__. Every symbol the package exports appears here, and nothing that it does not.

Go and Rust reference documentation is generated by their own ecosystems — pkg.go.dev and docs.rs. Duplicating them here would create a second copy to keep in step.

Covers 99 exported symbols.

Clients

The one object you construct.

`Axonium`

Synchronous client for the Prometheus inference platform.

Usable as a context manager, which closes the underlying connection pools::

with Axonium() as client:
    completion = client.chat.completions.create(
        model="qwen3-0.6b",
        messages=[{"role": "user", "content": "Hello"}],
    )
    print(completion.content)

Axonium(*, retry: RetryPolicy | None = None, token_provider: TokenProvider | None = None, **settings: Any) -> None

Member	Summary
`close(self) -> None`	—

`AsyncAxonium`

Asynchronous client for the Prometheus inference platform. See :class:Axonium.

AsyncAxonium(*, retry: RetryPolicy | None = None, token_provider: AsyncTokenProvider | None = None, **settings: Any) -> None

Member	Summary
`aclose(self) -> None`	—

Configuration

Supplied to the client, or read from AXONIUM_* variables.

`AxoniumConfig`

Resolved client configuration.

Values are taken from explicit arguments first and from the environment otherwise. A missing required setting raises :class:~axonium.errors.ConfigurationError naming both the field and the environment variable that can supply it.

Field	Type	Default
`gateway_base_url`	`<class str>`	`'http://127.0.0.1:8020'`
`client_id`	`str \| None`	`None`
`client_secret`	`pydantic.types.SecretStr \| None`	`None`
`scope`	`str \| None`	`None`
`ca_bundle`	`str \| None`	`None`
`otel_enabled`	`<class bool>`	`False`
`verify_modality`	`<class bool>`	`False`
`refresh_ahead_ratio`	`<class float>`	`0.8`
`refresh_ahead_min_seconds`	`<class float>`	`30.0`
`timeouts`	`<class axonium.config.Timeouts>`	`PydanticUndefined`

Member	Summary
`scopes`	The requested scope as a tuple, empty when no scope was requested.

`Timeouts`

Per-phase HTTP timeouts, in seconds.

Defaults follow the gateway's own limits. Image generation legitimately takes minutes, so the non-streaming read timeout is deliberately long; override it per call for endpoints where a faster failure is preferable.

Field	Type	Default
`connect`	`<class float>`	`10.0`
`read`	`<class float>`	`600.0`
`write`	`<class float>`	`600.0`
`pool`	`<class float>`	`10.0`
`stream_read`	`<class float>`	`180.0`
`auth_read`	`<class float>`	`30.0`

`RetryPolicy`

When to retry, and how long to wait.

retry_upstream_errors is off by default. A 502 upstream-error means the gateway's own three attempts already failed, so an immediate fourth is unlikely to help — and unlike the fast-fail errors, the request may have reached a model, making a retry a second billable generation. Enable it only where that trade is acceptable.

RetryPolicy(max_attempts: int = 3, initial_backoff: float = 1.0, max_backoff: float = 60.0, jitter: bool = True, retry_upstream_errors: bool = False) -> None

Member	Summary
`backoff_for(self, attempt: int) -> float`	Exponential backoff with jitter for attempt `attempt`, 1-based.
`delay_for(self, error: APIError, *, attempt: int) -> float	None`

`DEFAULT_GATEWAY_BASE_URL`

DEFAULT_GATEWAY_BASE_URL = 'http://127.0.0.1:8020'

`version`

__version__ = '1.0.0rc11'

Requests

What you send. Validated client-side against what the contract constrains.

`ChatCompletionRequest`

Body for POST /v1/chat/completions.

Field	Type	Default
`model`	`<class str>`	required
`messages`	`list[axonium.models.requests.Message]`	required
`stream`	`<class bool>`	`False`
`max_tokens`	`int \| None`	`None`
`temperature`	`float \| None`	`None`
`top_p`	`float \| None`	`None`
`stop`	`str \| list[str] \| None`	`None`
`tools`	`list[dict[str, Any]] \| None`	`None`
`response_format`	`dict[str, Any] \| None`	`None`
`tool_choice`	`str \| dict[str, Any] \| None`	`None`
`chat_template_kwargs`	`dict[str, Any] \| None`	`None`
`logprobs`	`bool \| None`	`None`
`top_logprobs`	`int \| None`	`None`

`EmbeddingsRequest`

Body for POST /v1/embeddings.

Field	Type	Default
`model`	`<class str>`	required
`input`	`str \| list[str]`	required

`ImageGenerationRequest`

Body for POST /v1/images/generations.

Field	Type	Default
`model`	`<class str>`	required
`prompt`	`<class str>`	required
`n`	`int \| None`	`None`
`size`	`str \| None`	`None`

`RerankRequest`

Body for POST /v1/rerank.

Field	Type	Default
`model`	`<class str>`	required
`query`	`<class str>`	required
`documents`	`list[str]`	required
`top_n`	`int \| None`	`None`
`raw_scores`	`bool \| None`	`None`

`Message`

One chat message.

Field	Type	Default
`role`	`Literal[system, user, assistant, tool]`	required
`content`	`str \| list[axonium.models.requests.ContentPart] \| None`	`None`
`name`	`str \| None`	`None`
`tool_calls`	`list[axonium.models.chat.ToolCall] \| None`	`None`
`tool_call_id`	`str \| None`	`None`
`reasoning_content`	`str \| None`	`None`

`ContentPart`

One part of a multi-part message, used for vision models.

Field	Type	Default
`type`	`<class str>`	required
`text`	`str \| None`	`None`
`image_url`	`dict[str, Any] \| None`	`None`

`ToolCall`

A tool call, in the one shape both streaming and non-streaming produce.

Streamed calls arrive split across fragments that are individually invalid JSON; the SDK reassembles them into exactly this, so the same caller code handles both.

Field	Type	Default
`id`	`str \| None`	`None`
`type`	`str \| None`	`None`
`function`	`<class axonium.models.chat.FunctionCall>`	`FunctionCall(name=None, arguments='')`

Member	Summary
`name`	The function name, without reaching through :attr:`function`.
`parse_arguments(self) -> dict[str, Any]`	Decode :attr:`FunctionCall.arguments` into an object.

`FunctionCall`

The function a tool call names, and the arguments it was called with.

Field	Type	Default
`name`	`str \| None`	`None`
`arguments`	`<class str>`	`''`

Responses

What comes back. Every one keeps the raw payload, so an unmodelled field stays reachable.

`ChatCompletion`

A non-streaming chat completion.

id, created and system_fingerprint are backend-dependent rather than guaranteed by the gateway contract, so they are optional here.

Field	Type	Default
`id`	`str \| None`	`None`
`object`	`str \| None`	`None`
`created`	`int \| None`	`None`
`model`	`str \| None`	`None`
`choices`	`list[axonium.models.chat.ChatChoice]`	`[]`
`usage`	`axonium.models.common.Usage \| None`	`None`
`system_fingerprint`	`str \| None`	`None`
`timings`	`axonium.models.chat.Timings \| None`	`None`

Member	Summary
`content`	Text of the first choice, or `None` for a tool-call response.
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.
`reasoning`	The first choice's chain of thought, if the model produced one.
`tool_calls`	Tool calls from the first choice.

`ChatChoice`

Base for backend-shaped payloads: tolerant on input, preserving unknown fields.

Field	Type	Default
`index`	`int \| None`	`None`
`message`	`axonium.models.chat.CompletionMessage \| None`	`None`
`finish_reason`	`str \| None`	`None`
`logprobs`	`axonium.models.chat.ChoiceLogprobs \| None`	`None`

`CompletionMessage`

The assistant message in a completion.

Field	Type	Default
`role`	`str \| None`	`None`
`content`	`str \| None`	`None`
`reasoning_content`	`str \| None`	`None`
`tool_calls`	`list[axonium.models.chat.ToolCall] \| None`	`None`

`CreateEmbeddingResponse`

An embeddings response.

usage carries no completion_tokens: embedding has no generation phase.

Field	Type	Default
`object`	`str \| None`	`None`
`data`	`list[axonium.models.inference.Embedding]`	`PydanticUndefined`
`model`	`str \| None`	`None`
`usage`	`axonium.models.common.Usage \| None`	`None`

Member	Summary
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.
`vectors`	Just the vectors, in request order.

`Embedding`

Base for backend-shaped payloads: tolerant on input, preserving unknown fields.

Field	Type	Default
`object`	`str \| None`	`None`
`index`	`int \| None`	`None`
`embedding`	`list[float]`	`PydanticUndefined`

`ImagesResponse`

A top-level response body, carrying the correlation metadata of the response it came from.

meta is a private attribute rather than a field so that it cannot collide with a payload key of the same name, and so it stays out of model_dump() — it describes the HTTP exchange, not the resource.

Field	Type	Default
`created`	`int \| None`	`None`
`data`	`list[axonium.models.inference.GeneratedImage]`	`PydanticUndefined`
`output_format`	`str \| None`	`None`

Member	Summary
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.

`GeneratedImage`

One generated image.

The gateway returns image bytes inline as base64 rather than a URL, so persisting the result is the caller's responsibility.

Field	Type	Default
`b64_json`	`str \| None`	`None`

Member	Summary
`save(self, path: str	Path) -> Path`
`to_bytes(self) -> bytes`	Decode the image to raw bytes.

`RerankResponse`

Documents scored against a query, best first.

usage.total_tokens equals prompt_tokens: a reranker generates nothing, so there are no completion tokens and billing is prompt-only.

Field	Type	Default
`object`	`str \| None`	`None`
`model`	`str \| None`	`None`
`results`	`list[axonium.models.inference.RerankResult]`	`PydanticUndefined`
`usage`	`axonium.models.common.Usage \| None`	`None`

Member	Summary
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.
`ranking`	The indices of your `documents`, best first.

`RerankResult`

One document's score against the query.

Field	Type	Default
`index`	`int \| None`	`None`
`relevance_score`	`float \| None`	`None`

`ModelList`

A catalog response.

Iterating yields :class:Model objects directly, since the object/data envelope is rarely what a caller wants to work with.

Field	Type	Default
`object`	`<class str>`	`'list'`
`data`	`list[axonium.models.catalog.Model]`	`PydanticUndefined`

Member	Summary
`get(self, model_id: str) -> Model	None`
`ids`	—
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.

`Model`

One deployed model.

Field	Type	Default
`id`	`<class str>`	required
`object`	`<class str>`	`'model'`
`owned_by`	`str \| None`	`None`
`context_length`	`int \| None`	`None`
`family`	`str \| None`	`None`
`quantization`	`str \| None`	`None`
`modality`	`Union[Literal[text, vision, embedding, image, rerank, classification, zero_shot, typed_decision], str, NoneType]`	`None`
`payload_schema`	`str \| None`	`None`

`RequestUsage`

The billing record for one request.

Every field is optional. The gateway guarantees the row exists for a billed request, not which columns a given deployment populates — cost_usd comes back None where no price is configured, and new columns are appended over time.

Field	Type	Default
`request_id`	`str \| None`	`None`
`model`	`str \| None`	`None`
`request_kind`	`str \| None`	`None`
`usage`	`axonium.models.common.Usage \| None`	`None`
`image_count`	`int \| None`	`None`
`interrupted`	`bool \| None`	`None`
`termination_reason`	`str \| None`	`None`
`cost_usd`	`float \| None`	`None`
`instance_id`	`str \| None`	`None`
`created_at`	`str \| None`	`None`

Member	Summary
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.

`Usage`

Token accounting for a request.

Embedding responses carry no completion_tokens because there is no generation phase.

Field	Type	Default
`prompt_tokens`	`int \| None`	`None`
`completion_tokens`	`int \| None`	`None`
`total_tokens`	`int \| None`	`None`
`cache_read_tokens`	`int \| None`	`None`
`estimated`	`<class bool>`	`False`

`Timings`

Backend timing figures.

Emitted by llama.cpp-family backends only; MLX, vLLM and SGLang backends do not include it, so never depend on its presence.

Field	Type	Default
`prompt_n`	`int \| None`	`None`
`prompt_ms`	`float \| None`	`None`
`prompt_per_second`	`float \| None`	`None`
`predicted_n`	`int \| None`	`None`
`predicted_ms`	`float \| None`	`None`
`predicted_per_second`	`float \| None`	`None`
`predicted_per_token_ms`	`float \| None`	`None`
`cache_n`	`int \| None`	`None`

`APIObject`

A top-level response body, carrying the correlation metadata of the response it came from.

APIObject(**extra_data: Any) -> None

Member	Summary
`meta`	Request and trace IDs, and the rate-limit budget, as of this response.
`model_post_init(self: BaseModel, context: Any, /) -> None`	This function is meant to behave like a BaseModel method to initialize private attributes.

Streaming

A separate method from create, so the return type stays honest.

`ChatCompletionStream`

A streaming completion.

Iterate it inside a with block::

with client.chat.completions.stream(model=..., messages=...) as stream:
    for chunk in stream:
        print(chunk.content or "", end="")
    print(stream.usage())

ChatCompletionStream(opening: StreamOpen) -> None

Member	Summary
`close(self) -> None`	—

`AsyncChatCompletionStream`

A streaming completion. See :class:ChatCompletionStream.

AsyncChatCompletionStream(opening: StreamOpen, preflight: Callable[[], Awaitable[None]]) -> None

Member	Summary
`aclose(self) -> None`	—

`ChatCompletionChunk`

One chunk of a streamed completion.

Chunks are forwarded from the backend essentially verbatim, so almost everything is optional: what a given backend puts in each chunk varies, and a chunk carrying only a finish_reason or only timings is normal rather than malformed.

Field	Type	Default
`id`	`str \| None`	`None`
`object`	`str \| None`	`None`
`created`	`int \| None`	`None`
`model`	`str \| None`	`None`
`choices`	`list[axonium.models.chat.StreamChoice]`	`[]`
`usage`	`axonium.models.common.Usage \| None`	`None`
`timings`	`axonium.models.chat.Timings \| None`	`None`

Member	Summary
`content`	Text carried by this chunk's first choice, if any.
`finish_reason`	—
`reasoning`	Reasoning carried by this chunk, if the model is still thinking.
`tool_call_fragments`	This chunk's raw tool-call fragments, which are not usable on their own.

`StreamChoice`

Base for backend-shaped payloads: tolerant on input, preserving unknown fields.

Field	Type	Default
`index`	`int \| None`	`None`
`delta`	`axonium.models.chat.ChoiceDelta \| None`	`None`
`finish_reason`	`str \| None`	`None`

`ChoiceDelta`

The incremental part of a streamed choice.

Field	Type	Default
`role`	`str \| None`	`None`
`content`	`str \| None`	`None`
`reasoning_content`	`str \| None`	`None`
`tool_calls`	`list[dict[str, Any]] \| None`	`None`

Correlation and budget

On every response, successes included.

`ResponseMeta`

Correlation and budget information attached to every response.

Carried on successes as well as failures: correlating a slow but successful call with platform traces matters as much as correlating a failed one.

Field	Type	Default
`request_id`	`str \| None`	`None`
`trace_id`	`str \| None`	`None`
`rate_limit`	`axonium.models.common.RateLimitSnapshot \| None`	`None`
`instance`	`str \| None`	`None`
`instance_id`	`str \| None`	`None`
`idempotent_replay`	`<class bool>`	`False`
`idempotent_replay_of`	`str \| None`	`None`
`waited_s`	`<class float>`	`0.0`
`attempts`	`<class int>`	`1`

`RateLimitSnapshot`

Rate-limit budget as of one response, parsed from the X-RateLimit-* headers.

Present on the inference and catalog endpoints, absent on health and metrics routes — in which case every field is None.

The token figures reflect the gateway's post-hoc accounting rather than a pre-flight reservation, so a burst of large requests can still exceed the token budget between header updates. Treat them as a strong signal, not a guarantee against ever seeing a 429.

Field	Type	Default
`scope`	`str \| None`	`None`
`limit_requests`	`int \| None`	`None`
`remaining_requests`	`int \| None`	`None`
`reset_requests`	`int \| None`	`None`
`limit_tokens`	`int \| None`	`None`
`remaining_tokens`	`int \| None`	`None`
`reset_tokens`	`int \| None`	`None`

Member	Summary
`is_empty`	True when the response carried no rate-limit headers at all.
`with_scope_from(self, body: Any) -> RateLimitSnapshot`	Fill in :attr:`scope` from a problem+json body when the header did not carry it.

Authentication

Handled for you. Public for the cases where it is not.

`TokenSet`

A cached access token and everything needed to decide when to replace it.

TokenSet(access_token: str, expires_at: float, expires_in: float, scope: tuple[str, ...] = ()) -> None

Member	Summary
`needs_refresh(self, *, ratio: float, min_seconds: float, now: float	None = None) -> bool`
`remaining(self, *, now: float	None = None) -> float`

`TokenClaims`

Claims read out of the access token.

Decoded without verifying the signature, which is fine because this is introspection for display and diagnostics only. The gateway is the sole authority on what a token may do — never make an access-control decision from these values.

Note that the gateway does not use role for authorization either; only scope governs what a token can actually call.

TokenClaims(subject: str | None = None, client_name: str | None = None, role: str | None = None, scope: tuple[str, ...] = (), expires_at: int | None = None, issued_at: int | None = None, raw: dict[str, Any] = <factory>) -> None

`TokenProvider`

TokenProvider = collections.abc.Callable[[str | None], str]

`AsyncTokenProvider`

AsyncTokenProvider = collections.abc.Callable[[str | None], collections.abc.Awaitable[str]]

Errors

One class per row of the gateway's catalog. OAuthError does not inherit from APIError: different envelope, different semantics.

Class	Meaning
`APIError`	An RFC 9457 problem-details error returned by the gateway.
`AuthTransportError`	The auth-service could not be reached, or answered with something unusable.
`AxoniumError`	Base class for every error raised by this SDK.
`BackendUnavailableError`	The gateway's circuit breaker is open for this backend, or the backend was unreachable.
`BadRequestError`	4xx client error that is not covered by a more specific subclass.
`CapacityExhaustedError`	Every replica of this model is above its pending-work headroom.
`ConfigurationError`	A required setting is missing or invalid.
`ContextExceededError`	The request exceeds the model's context window.
`ForbiddenError`	The token lacks a scope the request requires.
`IdempotencyInProgressError`	The first request with this key is still running.
`IdempotencyKeyReuseError`	The gateway's fingerprint for this key does not match the one it stored.
`IdempotencyResponseNotRetainedError`	The original succeeded, but its response was too large to store.
`InconsistentModelGroupError`	The replicas serving one model disagree about their modality.
`InvalidClientError`	Bad `client_id` or `client_secret`.
`InvalidDateError`	A usage query's `start` or `end` is not a `YYYY-MM-DD` date.
`InvalidIdempotencyKeyError`	The `Idempotency-Key` is malformed — today, longer than 255 characters.
`InvalidRangeError`	A usage export's `end` is before its `start`.
`InvalidRequestError`	A request was rejected by the SDK before it was sent.
`InvalidScopeError`	An unrecognized scope, or one this account is not allowed to request.
`InvalidTokenError`	Signature, algorithm, issuer, audience or `sub` claim validation failed.
`MethodNotAllowedError`	The URL exists, the verb does not. The `Allow` response header lists the ones that do.
`MissingCredentialsError`	No or malformed `Authorization` header, or a token passed as a query parameter.
`ModalityMismatchError`	The model's modality does not match the request.
`ModelNotLoadedError`	The model is registered but not currently deployed.
`NotFoundError`	The addressed resource does not exist, or does not belong to this client.
`OAuthError`	An error from the auth-service token endpoint.
`PredictBackendRejectedError`	The engine behind `predict` refused the request, wrapped rather than forwarded.
`RangeTooLargeError`	A usage export's range exceeds 366 days. Split it into several exports.
`RateLimitError`	The requests-per-minute budget was exceeded.
`RateLimitingUnavailableError`	The rate limiter's backing store is down and the deployment fails closed.
`RerankDialectUnknownError`	The reranker runs, on an engine whose rerank request shape the gateway has not recorded.
`ServerError`	5xx error that is not covered by a more specific subclass.
`SpendCapExceededError`	The client hit its configured monthly spend cap.
`StreamInterruptedError`	The stream failed partway through.
`TimeoutError`	The client gave up waiting for a response.
`TokenEndpointNotConfiguredError`	This deployment has no token endpoint wired up.
`TokenEndpointUnavailableError`	The gateway could not reach the auth-service to issue a token.
`TokenExpiredError`	The token's `exp` has passed.
`TokenRevokedError`	The token or client was explicitly revoked by an admin.
`ToolCallArgumentsError`	A tool call's `arguments` string could not be decoded.
`TransportError`	The request never produced an HTTP response (DNS, connection, or TLS failure).
`UnauthorizedClientError`	The account is deactivated.
`UnauthorizedError`	401 that is not one of the specific token errors.
`UnauthorizedRequestError`	The request carried no verified claims.
`UnknownInstanceError`	The pinned instance does not serve the requested model.
`UnknownModelError`	The model ID is not registered.
`UnknownParameterError`	A request field outside the gateway's accepted subset, refused instead of dropped.
`UnknownRouteError`	No route at that URL — a mistake in the calling code, not a fact about the data.
`UnsupportedFieldWarning`	A request field was dropped because the gateway does not accept it.
`UnsupportedGrantTypeError`	The grant type is not supported. The SDK always uses `client_credentials`.
`UnusedCredentialWarning`	Credentials were found but will not be used, because a token provider takes precedence.
`UpstreamError`	The backend failed and the gateway's own internal retries were exhausted.
`UsageStoreUnavailableError`	A usage-endpoint database read failed.
`ValidationError`	The request body failed schema validation.

Transport

Rarely touched directly.

`CooldownRegistry`

Remembers server-supplied waits so a known-open circuit is not hammered.

When the gateway reports an open circuit breaker it also says when the backend is expected to recover. Ignoring that and sending the next request anyway just buys another 503, so the wait is recorded and later calls for the same backend fail locally until it elapses.

Only ever populated from a wait the platform supplied; this never invents a cooldown of its own, and never guesses that a backend is unhealthy from local failure counts.

CooldownRegistry(_until: dict[str, float] = <factory>, _lock: threading.Lock = <factory>) -> None

Member	Summary
`clear(self, key: str) -> None`	—
`note(self, key: str, error: APIError) -> None`	Record a cooldown if this error came with a server-supplied wait.
`record(self, key: str, seconds: float) -> None`	—
`remaining(self, key: str) -> float`	Seconds still to wait for `key`, or `0.0` if it is clear.

Other

Exported but not yet grouped. This heading is a note for whoever maintains this page: new symbols land here until they are given a home.

`ChoiceLogprobs`

The per-token probabilities for one choice, in OpenAI's shape.

Field	Type	Default
`content`	`list[axonium.models.chat.TokenLogprob]`	`[]`

`PredictResult`

Whatever the engine returned, undecoded, and the correlation metadata beside it.

A dataclass rather than a pydantic model because there is no schema to validate against: the body is the engine's and the gateway forwards it verbatim. A model with extra="allow" would still impose that the top level is an object, and one of the three live shapes is not.

PredictResult(value: Any, meta: ResponseMeta) -> None

`TokenLogprob`

One generated token and how likely the model thought it was.

logprob is a natural logarithm, which is the part worth saying out loud: -0.00054 is about 99.95% and -7.6 is about 0.05%. Read as a probability it looks like a number near zero meaning "unlikely", and the mistake is silent, so :attr:probability exists.

Field	Type	Default
`token`	`str \| None`	`None`
`logprob`	`float \| None`	`None`
`top_logprobs`	`list[axonium.models.chat.TokenLogprob]`	`[]`

Member	Summary
`probability`	`exp(logprob)`, or `None` when the backend sent no `logprob`.