Inference API reference

HTTP routes the inference endpoint serves, with request bodies, responses, and status codes.

The HTTP endpoints served by a deployed inference model. Each route lists its request, response, and errors. In every path, <inference-url> is the base URL of the deployment's inference endpoint.

Endpoints

Method	Path	Route
`POST`	`/v1/endpoints/<endpoint-name>/v1/chat/completions`	Chat completions
`POST`	`/v1/endpoints/<endpoint-name>/v1/audio/transcriptions`	Audio transcriptions
`GET`	`/v1/models`	List models
`GET`	`/v1/models/stats`	Model stats
`GET`	`/v1/models/stats/<endpoint-name>`	Model stats for one endpoint
`GET`	`/v1/health`	Service health
`GET`	`/v1/auth/validate`	Validate an API key

Authentication

CosmicAC reads the API key from the Authorization header as a Bearer token. A key must start with the csm_live_ prefix.

Authorization: Bearer csm_live_xxxxxxxxxxxxxxxxxxxxxxxx

The routes use three authentication modes.

Mode	Behavior	Used by
None	The route runs no authentication.	List models, Model stats, Model stats for one endpoint, Service health
Conditional	CosmicAC enforces authentication only when the endpoint enables `require_auth_header`. Otherwise it records a valid key for usage tracking, and does not block a missing or invalid key.	Chat completions, Audio transcriptions
Required	A valid key is always required.	Validate an API key

Chat completions

Returns a chat completion from the model at the named endpoint. Supports non-streaming and streaming Server-Sent Events (SSE) responses, and multimodal input with image, video, or audio parts.

HTTP request

POST <inference-url>/v1/endpoints/<endpoint-name>/v1/chat/completions

Path parameters

Parameter	Type	Required	Description
`endpoint-name`	string	Yes	Name of the inference endpoint that serves the model.

Request headers

Header	Type	Required	Description
`Content-Type`	string	Yes	Must be `application/json`.
`Authorization`	string	No	API key as a Bearer token, `Bearer csm_live_...`. Required only when the endpoint enables `require_auth_header`.

Request body

{
  "model": "Qwen/Qwen2-VL-2B-Instruct",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "stream": false,
  "temperature": 0.7,
  "max_tokens": 256,
  "top_p": 0.95,
  "frequency_penalty": 0,
  "presence_penalty": 0
}

Field	Type	Required	Description
`messages`	array	Yes	Conversation history. Must contain at least one message.
`messages[].role`	string	Yes	Role of the message author. The route accepts any string.
`messages[].content`	string \| array	Yes	Message text as a string, or an array of typed parts for multimodal models. CosmicAC rejects any other type. An empty string is accepted, `null` is not.
`model`	string	Yes	The model to call. Must match either the model the endpoint serves or the endpoint name.
`stream`	boolean	No	If `true`, the response streams as Server-Sent Events. Defaults to `false`.
`temperature`	number	No	Sampling temperature, `0` to `2`. Higher values make the output more random, lower values more deterministic.
`top_p`	number	No	Nucleus sampling, `0` to `1`. The model considers only the tokens in the top `top_p` probability mass.
`max_tokens`	number	No	Maximum number of tokens to generate in the completion. Minimum `1`.
`frequency_penalty`	number	No	Penalty scaled by how often a token has already appeared, `-2` to `2`. Higher values reduce repetition.
`presence_penalty`	number	No	Penalty applied to tokens that have already appeared, `-2` to `2`. Higher values encourage new topics.
`n`	number	No	Number of completions to generate. Minimum `1`.
`stream_options`	object	No	Streaming options. CosmicAC forwards this object to the model without validating its contents.
`stream_options.include_usage`	boolean	No	CosmicAC sets this field to `true` on every streaming request, overriding any value sent in the request.
`mm_processor_kwargs`	object	No	Multimodal processor settings.
`mm_processor_kwargs.fps`	number	No	Frames per second to sample from video input.
`mm_processor_kwargs.max_frames`	number	No	Maximum number of frames to sample from video input.
`mm_processor_kwargs.max_pixels`	number	No	Maximum number of pixels per image or frame.
`stop`	string \| string[]	No	CosmicAC accepts this field but does not forward it to the model.

Each message uses one form or the other. When messages[].content is an array, each part takes these fields.

Field	Type	Required	Description
`type`	string	Yes	One of `text`, `image_url`, `video_url`, or `audio_url`. CosmicAC accepts any other value without further validation.
`text`	string	When `type` is `text`	The text content of the part.
`image_url.url`	string	When `type` is `image_url`	URL of the image.
`video_url.url`	string	When `type` is `video_url`	URL of the video.
`audio_url.url`	string	When `type` is `audio_url`	URL of the audio.

Response

Non-streaming response.

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "Qwen/Qwen2-VL-2B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 9,
    "total_tokens": 19
  }
}

The model server produces this body and CosmicAC returns it unchanged.

Field	Type	Description
`id`	string	Identifier for the completion.
`object`	string	Always `chat.completion`.
`created`	number	Unix timestamp in seconds.
`model`	string	Model that produced the completion.
`choices`	array	Generated completions. One entry unless `n` is set higher.
`choices[].index`	number	Position of the choice in the array.
`choices[].message`	object	The generated message.
`choices[].message.role`	string	Role of the message author, `assistant`.
`choices[].message.content`	string	Generated text.
`choices[].finish_reason`	string	Why generation stopped, such as `stop` or `length`.
`usage`	object	Token counts for the request.
`usage.prompt_tokens`	number	Tokens in the prompt.
`usage.completion_tokens`	number	Tokens in the completion.
`usage.total_tokens`	number	Sum of prompt and completion tokens.

Streaming response. When stream is true, the response streams as Server-Sent Events with Content-Type: text/event-stream. Each event is a data: line carrying one completion chunk, and the stream ends with data: [DONE].

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Field	Type	Description
`id`	string	Identifier for the completion, the same on every chunk.
`object`	string	Always `chat.completion.chunk`.
`choices`	array	Chunk contents.
`choices[].index`	number	Position of the choice in the array.
`choices[].delta`	object	The increment added by this chunk.
`choices[].delta.role`	string	Present on the first chunk only.
`choices[].delta.content`	string	Text fragment added by this chunk.
`choices[].finish_reason`	string \| null	`null` until the final chunk.
`usage`	object	Token counts, sent on the final chunk because CosmicAC sets `include_usage`.

Errors

Failed requests return an error object. The structure matches the OpenAI error response.

{
  "error": {
    "message": "No available inference agents for endpoint: <endpoint-name>",
    "type": "invalid_request_error",
    "param": "endpointName",
    "code": "model_not_found"
  }
}

Field	Type	Description
`error.message`	string	Human-readable description of the failure.
`error.type`	string	Error category.
`error.param`	string \| null	The request field that caused the failure, when CosmicAC can identify it. `null` otherwise.
`error.code`	string \| null	Machine-readable code. `null` when the failure carries no specific code.

The type field takes one of these values.

`error.type`	Meaning
`invalid_request_error`	CosmicAC rejected the request.
`server_error`	The endpoint or the model server failed.
`error`	A streaming response that already started, then broke. Carries `message` only.

The code field takes one of these values.

`error.code`	Meaning
`invalid_value`	A field failed validation. `param` names the field.
`missing_required_parameter`	A required field is absent. `param` names the field.
`model_mismatch`	`model` matches neither the model the endpoint serves nor the endpoint name.
`model_not_found`	The endpoint has no replicas available.
`null`	The failure carries no specific code.

The route returns these status codes.

Status	Meaning
`400`	The request body failed validation. For example, `model` is missing or does not match the endpoint, `messages` is empty, or `temperature` is outside `0` to `2`.
`401`	The endpoint requires an API key, and the request omitted it or sent an invalid one.
`500`	The model server failed, or CosmicAC failed while handling the request.
`502`	The model server returned a response that is not valid JSON.
`503`	The endpoint has no available capacity to serve the request.

When the model server answers with a status other than 200, CosmicAC returns that same status, with a message of the form vLLM API error: <status> and a code of null. A 4xx from the model server is therefore returned as a 4xx, and a 5xx as a 5xx with the type server_error.

A request with stream: true returns HTTP 200 as soon as the event stream opens. Failures after that point arrive as a single data: event carrying the error object, followed by data: [DONE]. A streaming request that finds no available capacity reports invalid_request_error with model_not_found. A stream that breaks after it starts sending chunks reports the type error and omits param and code.

Audio transcriptions

Returns a transcription of an audio file, supplied as a file upload or as a URL. The endpoint must serve a speech-to-text model, such as Parakeet. A request to a text-only endpoint fails with 400.

HTTP request

POST <inference-url>/v1/endpoints/<endpoint-name>/v1/audio/transcriptions

Path parameters

Parameter	Type	Required	Description
`endpoint-name`	string	Yes	Name of the inference endpoint that serves the model.

Request headers

Header	Type	Required	Description
`Content-Type`	string	Yes	`multipart/form-data` for a file upload or an `audio_url` field, or `application/json` for an `audio_url` field.
`Authorization`	string	No	API key as a Bearer token, `Bearer csm_live_...`. Required only when the endpoint enables `require_auth_header`.

Request body

The route accepts audio in one of these forms.

multipart/form-data with a file upload.
multipart/form-data with an audio_url field.
application/json with an audio_url field.

Field	Type	Required	Description
`file`	file	No	The audio file to transcribe, for a multipart upload. Either `file` or `audio_url` is required. The maximum size is the endpoint's `max_file_size_mb` setting.
`audio_url`	string	No	URL of the audio file to transcribe. Either `file` or `audio_url` is required. The format comes from the URL extension, or from the response content type when the URL has no extension.

Example application/json body with a URL.

{
  "audio_url": "https://example.com/audio.mp3"
}

Request size limits

Limit	Value	Description
Maximum upload size	`max_file_size_mb`	The Parakeet job that serves the endpoint sets this value. A larger upload fails with `400`.

Response

Success response.

{
  "text": "the transcribed audio text",
  "segments": [
    {
      "start": 0.0,
      "end": 2.5,
      "text": "the transcribed",
      "id": 0,
      "seek": 0,
      "tokens": ["the", "transcribed"],
      "word_count": 2,
      "temperature": 0.0,
      "avg_logprob": null,
      "compression_ratio": null,
      "no_speech_prob": null
    }
  ],
  "words": [
    { "word": "the", "start": 0.0, "end": 0.4 }
  ],
  "language": "en",
  "metadata": {
    "total_segments": 1,
    "total_words": 6,
    "audio_duration": 5.2
  },
  "processing_time": 1.234,
  "transcription_time": 1.234,
  "source": "file"
}

Field	Type	Description
`text`	string	The transcribed text.
`segments`	array	Timed segments of the transcription.
`segments[].start`	number	Segment start time in seconds.
`segments[].end`	number	Segment end time in seconds.
`segments[].text`	string	Text of the segment.
`segments[].id`	number	Position of the segment in the array.
`segments[].seek`	number	Placeholder for OpenAI compatibility. Always `0`.
`segments[].tokens`	array	Words in the segment. Falls back to the segment text split on whitespace when the model returns no word timings.
`segments[].word_count`	number	Number of entries in `tokens`.
`segments[].temperature`	number	Placeholder for OpenAI compatibility. Always `0.0`.
`segments[].avg_logprob`	null	Placeholder for OpenAI compatibility. Always `null`.
`segments[].compression_ratio`	null	Placeholder for OpenAI compatibility. Always `null`.
`segments[].no_speech_prob`	null	Placeholder for OpenAI compatibility. Always `null`.
`words`	array	Word-level timings. CosmicAC omits words that carry no timestamp.
`words[].word`	string	The word.
`words[].start`	number	Word start time in seconds.
`words[].end`	number	Word end time in seconds.
`language`	string	Always `en`.
`metadata`	object	Counts describing the transcription.
`metadata.total_segments`	number	Number of entries in `segments`.
`metadata.total_words`	number	Word count of the transcription. CosmicAC bills this as output.
`metadata.audio_duration`	number	Audio duration in seconds. CosmicAC bills this as input.
`processing_time`	number	Seconds spent transcribing.
`transcription_time`	number	Same value as `processing_time`. CosmicAC keeps this field for backward compatibility.
`source`	string	Origin of the audio, `file` or `url`.

Errors

Not every failure on this route returns an error object. Requests that CosmicAC rejects return the error object shown below. Requests the model rejects, such as unsupported or corrupt audio, return the model's own error format instead.

{
  "error": {
    "message": "Endpoint '<endpoint-name>' does not support audio transcriptions. This endpoint's model accepts: text",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Field	Type	Description
`error.message`	string	Human-readable description of the failure.
`error.type`	string	Error category.
`error.param`	string \| null	The request field that caused the failure, when CosmicAC can identify it. `null` otherwise.
`error.code`	string \| null	Machine-readable code. `null` when the failure carries no specific code.

The type field takes one of these values.

`error.type`	Meaning
`invalid_request_error`	CosmicAC rejected the request.
`server_error`	The endpoint has no capacity, or the model server failed.

The code field is always null on this route.

The route returns these status codes.

Status	Meaning
`400`	`Content-Type` is not `multipart/form-data` or `application/json`, the endpoint's model does not support audio input, the upload exceeds `max_file_size_mb`, `audio_url` is malformed, or the audio format cannot be determined.
`401`	The endpoint requires an API key, and the request omitted it or sent an invalid one.
`500`	CosmicAC could not complete the request to the model server, for example the request timed out.
`503`	The endpoint has no available capacity to serve the request.

When the model server answers with a status other than 200, CosmicAC returns that same status and forwards the model server's body and headers unchanged.

File validation

The model server runs these checks, so their responses use the model's error format rather than the error object above.

Validation	Status	Description
File size	`400`	Uploads larger than the endpoint's `max_file_size_mb` setting.
Audio format	`400`	Audio whose format is not one of `.wav`, `.mp3`, `.flac`, `.ogg`, `.m4a`, `.webm`, or `.mp4`.
URL format	`400`	An `audio_url` value that is not a valid URL.

List models

Lists the models available at the inference endpoint, with availability and modality metadata for each.

HTTP request

GET <inference-url>/v1/models

Response

The data array contains one entry per available model endpoint. The example below shows one ASR model and one vLLM model.

{
  "object": "list",
  "status": "healthy",
  "data": [
    {
      "id": "nvidia/parakeet-tdt-0.6b-v3",
      "object": "model",
      "created": 1783083042,
      "owned_by": "nvidia",
      "endpoint_name": "parakeet-prod",
      "agents_available": 1,
      "job_id": "e2a127bf-5f70-4127-8c42-b93d3978b6cc",
      "status": "healthy",
      "last_health_check": {
        "timestamp": 1783084228663,
        "success": true,
        "response_time_ms": 1142,
        "unhealthy_since": null
      },
      "modalities": {
        "input": ["audio"],
        "output": ["text"],
        "pipeline_tag": "automatic-speech-recognition",
        "source": "fallback"
      }
    },
    {
      "id": "Qwen/Qwen2-VL-2B-Instruct",
      "object": "model",
      "created": 1783083126,
      "owned_by": "vllm",
      "root": "Qwen/Qwen2-VL-2B-Instruct",
      "parent": null,
      "max_model_len": 27000,
      "permission": [
        {
          "id": "modelperm-3a3b6d993f074cb59078a9823827ace5",
          "object": "model_permission",
          "created": 1783083126,
          "allow_create_engine": false,
          "allow_sampling": true,
          "allow_logprobs": true,
          "allow_search_indices": false,
          "allow_view": true,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ],
      "endpoint_name": "qwen-2-prod",
      "agents_available": 2,
      "job_id": "69c06b04-caf5-4776-87a0-2df9ff255b36",
      "status": "healthy",
      "last_health_check": {
        "timestamp": 1783084228663,
        "success": true,
        "response_time_ms": 525,
        "unhealthy_since": null
      },
      "modalities": {
        "input": ["text", "image", "video"],
        "output": ["text"],
        "pipeline_tag": "image-text-to-text",
        "source": "fallback"
      }
    }
  ]
}

Field	Type	Description
`object`	string	Always `list`.
`status`	string	Aggregate health across all endpoints. `healthy` when every model is healthy, `down` when every model is down or no models are listed, and `degraded` otherwise.
`data`	array	The list of available model endpoints.
`data[].id`	string	Model identifier reported by the model server.
`data[].object`	string	Always `model`.
`data[].created`	number	Unix timestamp in seconds.
`data[].owned_by`	string	Owner reported by the model server.
`data[].endpoint_name`	string	Endpoint that serves the model.
`data[].agents_available`	number	Number of serving instances available for this model.
`data[].job_id`	string \| null	Job that serves the endpoint. `null` when no agent reports one.
`data[].status`	string	Endpoint health, `healthy`, `degraded`, or `down`.
`data[].last_health_check`	object \| null	Most recent probe result. `null` before the first probe.
`data[].last_health_check.timestamp`	number	Unix timestamp in milliseconds.
`data[].last_health_check.success`	boolean	Whether the probe succeeded.
`data[].last_health_check.response_time_ms`	number \| null	Probe round trip in milliseconds.
`data[].last_health_check.unhealthy_since`	number \| null	Unix timestamp in milliseconds when the current run of failed probes began. `null` while the probe is succeeding.
`data[].modalities`	object	Input and output modality metadata.
`data[].modalities.input`	array	Accepted input modalities, such as `text`, `image`, `video`, or `audio`.
`data[].modalities.output`	array	Produced output modalities.
`data[].modalities.pipeline_tag`	string	Task tag, such as `automatic-speech-recognition`.
`data[].modalities.source`	string	Where the modality data came from, `huggingface`, `fallback`, or `heuristic`.

Each data[] entry also carries any other fields the model server reports, such as root, parent, max_model_len, and permission. CosmicAC passes them through unchanged.

Errors

The route takes no parameters and runs no authentication. It fails only when CosmicAC cannot build the list, and the message carries the underlying failure.

{
  "error": {
    "message": "<underlying failure>",
    "type": "server_error",
    "param": null,
    "code": null
  }
}

Field	Type	Description
`error.message`	string	Human-readable description of the failure.
`error.type`	string	Error category. Always `server_error` on this route.
`error.param`	string \| null	The request field that caused the failure, when CosmicAC can identify it. `null` otherwise.
`error.code`	string \| null	Machine-readable code. `null` when the failure carries no specific code.

The route returns these status codes.

Status	Meaning
`500`	CosmicAC could not build the model list.

Model stats

Returns performance and health statistics for deployed inference endpoints, including per-replica metrics.

HTTP request

GET <inference-url>/v1/models/stats

Query parameters

Parameter	Type	Required	Description
`period`	string	No	Lookback window for traffic, failures, and latency statistics. One of `1h`, `6h`, `24h`, `7d`, or `30d`. Defaults to `1h`.
`overwrite_cache`	boolean	No	If `true`, bypasses the cached response and recomputes the statistics.

CosmicAC caches each response for 10 seconds, keyed on period. Repeat requests inside that window return the cached body unless overwrite_cache is true.

Response

The data array contains one entry per endpoint. The example below shows one single-replica endpoint and one two-replica endpoint.

{
  "object": "list",
  "period": "1h",
  "data": [
    {
      "endpoint_name": "parakeet-prod",
      "job_id": "e2a127bf-5f70-4127-8c42-b93d3978b6cc",
      "status": "healthy",
      "success_rate": 100,
      "traffic": 11,
      "failures": 0,
      "avg_response_time_ms": 339.79,
      "timestamp": 1783079163257,
      "last_health_check": {
        "timestamp": 1783079106559,
        "success": true,
        "response_time_ms": 302,
        "unhealthy_since": null
      },
      "replicas": [
        {
          "replica_id": "r-01",
          "job_id": "e2a127bf-5f70-4127-8c42-b93d3978b6cc",
          "status": "healthy",
          "success_rate": 100,
          "traffic": 11,
          "failures": 0,
          "avg_response_time_ms": 339.79,
          "timestamp": 1783079163256,
          "last_health_check": {
            "timestamp": 1783079106559,
            "success": true,
            "response_time_ms": 302,
            "unhealthy_since": null
          }
        }
      ]
    },
    {
      "endpoint_name": "qwen-2-prod",
      "job_id": "69c06b04-caf5-4776-87a0-2df9ff255b36",
      "status": "healthy",
      "success_rate": 100,
      "traffic": 25,
      "failures": 0,
      "avg_response_time_ms": 233.17,
      "timestamp": 1783079163257,
      "last_health_check": {
        "timestamp": 1783079106559,
        "success": true,
        "response_time_ms": 257,
        "unhealthy_since": null
      },
      "replicas": [
        {
          "replica_id": "r-01",
          "job_id": "69c06b04-caf5-4776-87a0-2df9ff255b36",
          "status": "healthy",
          "success_rate": 100,
          "traffic": 12,
          "failures": 0,
          "avg_response_time_ms": 235.94,
          "timestamp": 1783079163256,
          "last_health_check": {
            "timestamp": 1783079106559,
            "success": true,
            "response_time_ms": 257,
            "unhealthy_since": null
          }
        },
        {
          "replica_id": "r-02",
          "job_id": "69c06b04-caf5-4776-87a0-2df9ff255b36",
          "status": "healthy",
          "success_rate": 100,
          "traffic": 13,
          "failures": 0,
          "avg_response_time_ms": 230.61,
          "timestamp": 1783079163257,
          "last_health_check": {
            "timestamp": 1783079106559,
            "success": true,
            "response_time_ms": 257,
            "unhealthy_since": null
          }
        }
      ]
    }
  ]
}

Field	Type	Description
`object`	string	Always `list`.
`period`	string	The applied lookback window.
`data`	array	One entry per endpoint.
`data[].endpoint_name`	string	Name of the endpoint.
`data[].job_id`	string \| null	Job that serves the endpoint. `null` when no replica reports one.
`data[].status`	string	Endpoint health, `healthy`, `degraded`, or `down`.
`data[].success_rate`	number \| null	Percentage of successful requests in the period. `null` when the endpoint served no requests in the period.
`data[].traffic`	number	Request count in the period.
`data[].failures`	number	Failed request count in the period.
`data[].avg_response_time_ms`	number \| null	Mean response time in milliseconds. `null` when the endpoint served no requests in the period.
`data[].timestamp`	number	Unix timestamp in milliseconds when CosmicAC built the entry.
`data[].last_health_check`	object \| null	Most recent probe across the endpoint's replicas. `null` before the first probe.
`data[].last_health_check.timestamp`	number	Unix timestamp in milliseconds.
`data[].last_health_check.success`	boolean	Whether the probe succeeded.
`data[].last_health_check.response_time_ms`	number \| null	Probe round trip in milliseconds.
`data[].last_health_check.unhealthy_since`	number \| null	Unix timestamp in milliseconds when the current run of failed probes began. `null` while the probe is succeeding.
`data[].replicas`	array	Per-replica breakdown.
`data[].replicas[].replica_id`	string	Identifier of the replica.
`data[].replicas[].job_id`	string \| null	Job that serves the replica.
`data[].replicas[].status`	string	Replica health, `healthy`, `degraded`, or `down`.
`data[].replicas[].success_rate`	number \| null	Percentage of successful requests in the period. `null` when the replica served no requests in the period.
`data[].replicas[].traffic`	number	Request count in the period.
`data[].replicas[].failures`	number	Failed request count in the period.
`data[].replicas[].avg_response_time_ms`	number \| null	Mean response time in milliseconds. `null` when the replica served no requests in the period.
`data[].replicas[].timestamp`	number	Unix timestamp in milliseconds when CosmicAC built the entry.
`data[].replicas[].last_health_check`	object \| null	Most recent probe for this replica, in the same shape as `data[].last_health_check`.

Errors

Failed requests return an error object. The structure matches the OpenAI error response.

{
  "error": {
    "message": "querystring/period must be equal to one of the allowed values",
    "type": "invalid_request_error",
    "param": "period",
    "code": "invalid_value"
  }
}

Field	Type	Description
`error.message`	string	Human-readable description of the failure.
`error.type`	string	Error category.
`error.param`	string \| null	The query parameter that caused the failure, when CosmicAC can identify it. `null` otherwise.
`error.code`	string \| null	Machine-readable code. `null` when the failure carries no specific code.

The route returns these status codes.

Status	Meaning
`400`	`period` is not one of the accepted values.

A failure to build the statistics is not an error. The route returns 200 with an empty data array instead.

Model stats for one endpoint

Returns the same statistics as Model stats, for a single endpoint.

HTTP request

GET <inference-url>/v1/models/stats/<endpoint-name>

Path parameters

Parameter	Type	Required	Description
`endpoint-name`	string	Yes	Name of the inference endpoint.

Query parameters

Parameter	Type	Required	Description
`period`	string	No	Lookback window for traffic, failures, and latency statistics. One of `1h`, `6h`, `24h`, `7d`, or `30d`. Defaults to `1h`.
`overwrite_cache`	boolean	No	If `true`, bypasses the cached response and recomputes the statistics.

CosmicAC caches each response for 10 seconds, keyed on the endpoint name and period. Repeat requests inside that window return the cached body unless overwrite_cache is true.

Response

{
  "object": "model_stats",
  "period": "1h",
  "data": {
    "endpoint_name": "qwen-2-prod",
    "job_id": "69c06b04-caf5-4776-87a0-2df9ff255b36",
    "status": "healthy",
    "success_rate": 100,
    "traffic": 25,
    "failures": 0,
    "avg_response_time_ms": 233.17,
    "timestamp": 1783079163257,
    "last_health_check": {
      "timestamp": 1783079106559,
      "success": true,
      "response_time_ms": 257,
      "unhealthy_since": null
    },
    "replicas": [
      {
        "replica_id": "r-01",
        "job_id": "69c06b04-caf5-4776-87a0-2df9ff255b36",
        "status": "healthy",
        "success_rate": 100,
        "traffic": 25,
        "failures": 0,
        "avg_response_time_ms": 233.17,
        "timestamp": 1783079163256,
        "last_health_check": {
          "timestamp": 1783079106559,
          "success": true,
          "response_time_ms": 257,
          "unhealthy_since": null
        }
      }
    ]
  }
}

Field	Type	Description
`object`	string	Always `model_stats`.
`period`	string	The applied lookback window.
`data`	object \| null	Statistics for the endpoint, in the same shape as one `data[]` entry of Model stats. `null` when the statistics cannot be built.

Errors

Failed requests return an error object.

{
  "error": {
    "message": "No available inference agents for endpoint: <endpoint-name>",
    "type": "invalid_request_error",
    "param": "endpointName",
    "code": "model_not_found"
  }
}

Field	Type	Description
`error.message`	string	Human-readable description of the failure.
`error.type`	string	Error category.
`error.param`	string \| null	The request field that caused the failure, when CosmicAC can identify it. `null` otherwise.
`error.code`	string \| null	Machine-readable code. `null` when the failure carries no specific code.

The route returns these status codes.

Status	Meaning
`400`	`period` is not one of the accepted values. The `code` is `invalid_value`.
`503`	The endpoint has no replicas available. The `code` is `model_not_found`.

Service health

Returns one health value covering every registered endpoint.

HTTP request

GET <inference-url>/v1/health

Response

{
  "status": "healthy",
  "endpoints": 2,
  "timestamp": "2026-08-04T09:12:44.501Z"
}

Field	Type	Description
`status`	string	`healthy` when every endpoint is healthy, `unhealthy` when every endpoint is down or none are registered, and `degraded` otherwise.
`endpoints`	number	Number of endpoints with at least one registered agent. Absent when none are registered.
`error`	string	Reason the check could not report on endpoints. Present only when `status` is `unhealthy` and CosmicAC found no statistics. Either `No inference endpoints registered` or `Health metrics unavailable`.
`timestamp`	string	ISO 8601 time of the check.

This route reports unhealthy where the other routes report down. The statistics come from a fixed 24-hour window.

Errors

The route always returns 200 and never returns an error object. When CosmicAC cannot gather the statistics, it reports the failure in the body instead, as a status of unhealthy with an error field.

{
  "status": "unhealthy",
  "error": "No inference endpoints registered",
  "timestamp": "2026-08-04T09:12:44.501Z"
}

The error field takes one of these values.

`error`	Meaning
`No inference endpoints registered`	No endpoint has at least one registered agent.
`Health metrics unavailable`	CosmicAC could not read the health statistics for the registered endpoints.

The route returns these status codes.

Status	Meaning
`200`	CosmicAC always returns this status, including when the check reports `degraded` or `unhealthy`.

Validate an API key

Confirms that an API key is valid. The CosmicAC CLI uses this route.

HTTP request

GET <inference-url>/v1/auth/validate

Request headers

Header	Type	Required	Description
`Authorization`	string	Yes	API key as a Bearer token, `Bearer csm_live_...`.

Response

{
  "valid": true,
  "message": "API key is valid"
}

Field	Type	Description
`valid`	boolean	Always `true`. An invalid key returns `401` instead.
`message`	string	Always `API key is valid`.

Errors

Failed requests return an error object.

{
  "error": {
    "message": "Invalid or missing API key",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Field	Type	Description
`error.message`	string	Human-readable description of the failure. One of `Invalid or missing API key`, `Invalid API key`, or `Authentication failed`.
`error.type`	string	Error category. Always `invalid_request_error` on this route.
`error.param`	string \| null	Always `null` on this route.
`error.code`	string \| null	Always `null` on this route.

The route returns these status codes.

Status	Meaning
`401`	The API key is missing, malformed, or not valid.

Inference API reference

On this page