# FlareAI Cloud API

Compute orchestration for GPU and ASIC hardware behind one API.
Base URL: `https://api.flare-cloud.com`

# Start here

## Overview

FlareAI Cloud runs models on interchangeable hardware behind one API. You ask for compute; FlareAI decides which backend serves it and tells you why.

There are three layers. **Providers** each wrap one compute backend and implement a single interface. **Orchestration** scores every provider on live cost and capacity and picks one. **The API** is what you talk to, and it never exposes which vendor ran your workload.

> You address workloads only by FlareAI's own `workload_id`. Vendor identifiers stay inside the backend, which is what lets the hardware underneath change without breaking your integration.

## Quickstart

Four calls, start to finish. Replace `WORKLOAD_ID` with the `workload_id` that step 1 returns.

```bash
# 1. Provision. Omit gpu_type to let the orchestrator choose.
curl -X POST https://api.flare-cloud.com/api/v1/workloads \
  -H 'Content-Type: application/json' \
  -d '{"model_id": "Qwen/Qwen2.5-1.5B-Instruct"}'

# 2. Poll until "ready" is true (5–15 min on a cold provision).
curl https://api.flare-cloud.com/api/v1/workloads/WORKLOAD_ID/status

# 3. Run inference.
curl -X POST https://api.flare-cloud.com/api/v1/workloads/WORKLOAD_ID/infer \
  -H 'Content-Type: application/json' \
  -d '{"prompt": "Explain GPU orchestration.", "max_tokens": 128}'

# 4. Release the hardware. It bills by the second.
curl -X DELETE https://api.flare-cloud.com/api/v1/workloads/WORKLOAD_ID
```

> Gate inference on `ready`, not on `state`. A provider reports the machine as running before the model server has finished loading weights.

# Concepts

## Workload lifecycle

Every provider's native status vocabulary is normalised onto one set of states.

| | | |
| --- | --- | --- |
| `PROVISIONING` | not ready | Waiting for the backend to allocate hardware. |
| `STARTING` | not ready | Hardware allocated; model weights still loading. |
| `RUNNING` | ready | Serving inference. |
| `STOPPED` | not ready | Machine halted but not released. |
| `FAILED` | terminal | Provisioning or the model server failed. |
| `TERMINATED` | terminal | Hardware released. |

## How routing works

At provision time the orchestrator asks every routable provider what it can offer for each hardware class, then filters and scores what comes back.

Candidates with no capacity, or priced above your `max_cost_per_hour_usd`, are dropped before scoring. Survivors are scored on normalised cost (70%) and declared capacity (30%); highest wins.

```json
"routing": {
  "provider_id": "runpod",
  "gpu_type": "RTX4090",
  "rationale": "Routed to RunPod - RTX4090 available, $0.74/hr, medium capacity.",
  "hardware_substituted": false,
  "candidates": [
    { "provider_id": "runpod", "gpu_type": "RTX4090", "eligible": true,  "score": 0.889 },
    { "provider_id": "runpod", "gpu_type": "A6000",   "eligible": false, "reason": "No capacity reported." }
  ]
}
```

> The losing candidates ship with the decision. Placement is auditable after the fact rather than something you have to trust.

## Hardware and fallback

Request hardware by class — `B300`, `B200`, `H200`, `H100`, `A100`, `L40S`, `A6000`, `RTX4090` — not by vendor SKU. Each provider maps these onto whatever its own API wants.

Scarce parts are the normal case, not the exception. With `allow_hardware_fallback` on (the default), a class with no capacity anywhere is substituted for the next best, and the response says so via `hardware_substituted` and `requested_gpu_type`. With it off, you get `409` and the reason.

## Errors

```json
{
  "code": "no_eligible_provider",
  "message": "No active provider has capacity for B300 within the given constraints.",
  "detail": "RunPod/B300: no rentable secure-cloud capacity right now."
}
```

| | | |
| --- | --- | --- |
| `workload_not_found` | 404 | Unknown workload id. |
| `provider_not_found` | 404 | Unknown provider id. |
| `no_eligible_provider` | 409 | Nothing could satisfy the request. |
| `capacity_unavailable` | 409 | The backend could not allocate the hardware. |
| `workload_not_ready` | 409 | Still starting. Poll status and retry. |
| `provider_not_configured` | 503 | Backend is registered but missing credentials. |
| `provisioning_failed` | 502 | The backend rejected or failed the request. |
| `inference_failed` | 502 | The model server rejected the request. |

# API reference

### POST /api/v1/workloads

Provision a workload. Allocates hardware and starts a model server. Pin `gpu_type`, or omit it and the orchestrator picks on live cost and capacity. The response carries the full routing decision, including the options that lost.

| Field | Type | Notes |
| --- | --- | --- |
| `model_id` | string | Hugging Face model id to serve. |
| `gpu_type` | string | null | B300, B200, H200, H100, A100, L40S, A6000, RTX4090. Omit to let the orchestrator choose. |
| `gpu_count` | int | 1–8. Default 1. |
| `max_cost_per_hour_usd` | float | null | Hard ceiling. Pricier providers are excluded. |
| `preferred_provider` | string | null | Pin a provider by id. |
| `allow_hardware_fallback` | bool | Substitute other silicon when the requested class is dry. Default true. |
| `max_model_len` | int | Context length passed to vLLM. Default 4096. |
| `disk_gb` | int | Volume size holding the weight cache. Default 60. |

```bash
curl -X POST https://api.flare-cloud.com/api/v1/workloads \
  -H 'Content-Type: application/json' \
  -d '{
      "model_id": "Qwen/Qwen2.5-1.5B-Instruct",
      "gpu_type": "B300",
      "gpu_count": 1,
      "allow_hardware_fallback": true
    }'
```

Response:

```json
{
  "workload_id": "wl_4d642f4583de4247",
  "model_id": "Qwen/Qwen2.5-1.5B-Instruct",
  "provider_id": "runpod",
  "provider_name": "RunPod",
  "gpu_type": "RTX4090",
  "gpu_count": 1,
  "hardware_class": "GPU",
  "created_at": "2026-09-16T22:54:44Z",
  "terminated_at": null,
  "status": {
    "state": "STARTING",
    "ready": false,
    "detail": "Pod is running; model weights are still loading.",
    "cost_per_hour_usd": 0.74,
    "region": "US-KS-2"
  },
  "routing": {
    "provider_id": "runpod",
    "gpu_type": "RTX4090",
    "requested_gpu_type": null,
    "hardware_substituted": false,
    "rationale": "Routed to RunPod - RTX4090 available, $0.74/hr, medium capacity.",
    "cost_per_hour_usd": 0.74,
    "capacity": "MEDIUM",
    "candidates": [ /* every option scored, winners and losers */ ]
  }
}
```

### GET /api/v1/workloads

List workloads. Every workload, newest first, active and historical. Pass `?refresh=false` to read cached status instead of polling each provider.

```bash
curl https://api.flare-cloud.com/api/v1/workloads
```

Response:

```json
[ {
    "workload_id": "wl_4d642f4583de4247",
    "model_id": "Qwen/Qwen2.5-1.5B-Instruct",
    "provider_id": "runpod",
    "provider_name": "RunPod",
    "gpu_type": "RTX4090",
    "gpu_count": 1,
    "hardware_class": "GPU",
    "created_at": "2026-09-16T22:54:44Z",
    "terminated_at": null,
    "status": {
      "state": "STARTING",
      "ready": false,
      "detail": "Pod is running; model weights are still loading.",
      "cost_per_hour_usd": 0.74,
      "region": "US-KS-2"
    },
    "routing": {
      "provider_id": "runpod",
      "gpu_type": "RTX4090",
      "requested_gpu_type": null,
      "hardware_substituted": false,
      "rationale": "Routed to RunPod - RTX4090 available, $0.74/hr, medium capacity.",
      "cost_per_hour_usd": 0.74,
      "capacity": "MEDIUM",
      "candidates": [ /* every option scored, winners and losers */ ]
    }
  } ]
```

### GET /api/v1/workloads/{id}

Get a workload. One workload, including the routing decision recorded at provision time.

```bash
curl https://api.flare-cloud.com/api/v1/workloads/{id}
```

Response:

```json
{
  "workload_id": "wl_4d642f4583de4247",
  "model_id": "Qwen/Qwen2.5-1.5B-Instruct",
  "provider_id": "runpod",
  "provider_name": "RunPod",
  "gpu_type": "RTX4090",
  "gpu_count": 1,
  "hardware_class": "GPU",
  "created_at": "2026-09-16T22:54:44Z",
  "terminated_at": null,
  "status": {
    "state": "STARTING",
    "ready": false,
    "detail": "Pod is running; model weights are still loading.",
    "cost_per_hour_usd": 0.74,
    "region": "US-KS-2"
  },
  "routing": {
    "provider_id": "runpod",
    "gpu_type": "RTX4090",
    "requested_gpu_type": null,
    "hardware_substituted": false,
    "rationale": "Routed to RunPod - RTX4090 available, $0.74/hr, medium capacity.",
    "cost_per_hour_usd": 0.74,
    "capacity": "MEDIUM",
    "candidates": [ /* every option scored, winners and losers */ ]
  }
}
```

### GET /api/v1/workloads/{id}/status

Get live status. Polls the owning provider for a normalised snapshot. Gate inference on `ready` — a provider can report the machine as running while the model server is still loading weights.

| Field | Type | Notes |
| --- | --- | --- |
| `state` | string | PENDING, PROVISIONING, STARTING, RUNNING, STOPPED, FAILED, TERMINATED. |
| `ready` | bool | True only when the model server accepts requests. |

```bash
curl https://api.flare-cloud.com/api/v1/workloads/{id}/status
```

Response:

```json
{
  "state": "RUNNING",
  "ready": true,
  "detail": "vLLM server is accepting requests.",
  "cost_per_hour_usd": 0.74,
  "region": "US-KS-2",
  "updated_at": "2026-09-16T22:58:59Z"
}
```

### POST /api/v1/workloads/{id}/infer

Run inference. Sends a prompt to a running workload. Returns 409 while it is still starting.

```bash
curl -X POST https://api.flare-cloud.com/api/v1/workloads/{id}/infer \
  -H 'Content-Type: application/json' \
  -d '{
      "prompt": "Explain GPU orchestration.",
      "max_tokens": 256,
      "temperature": 0.7
    }'
```

Response:

```json
{
  "workload_id": "wl_4d642f4583de4247",
  "model_id": "Qwen/Qwen2.5-1.5B-Instruct",
  "text": "GPU orchestration refers to...",
  "prompt_tokens": 44,
  "completion_tokens": 80,
  "finish_reason": "stop",
  "latency_ms": 770.7,
  "served_by": "runpod"
}
```

### DELETE /api/v1/workloads/{id}

Tear down. Releases the hardware. Idempotent, and the record stays in history. Pods bill by the second, so tear down what you are done with.

```bash
curl -X DELETE https://api.flare-cloud.com/api/v1/workloads/{id}
```

Response:

```json
204 No Content
```

### GET /api/v1/providers

List providers. Registered backends with declared capacity and cost. `routable` is what the orchestrator acts on — a provider can be registered and fully implemented while not yet taking traffic.

```bash
curl https://api.flare-cloud.com/api/v1/providers
```

Response:

```json
[
  {
    "provider_id": "runpod",
    "display_name": "RunPod",
    "state": "ACTIVE",
    "hardware_class": "GPU",
    "routable": true,
    "availability": [
      { "gpu_type": "B300", "capacity": "LOW", "cost_per_hour_usd": 7.89 }
    ]
  }
]
```

### GET /api/v1/catalog

Models and hardware. The model and hardware catalogue the console renders its pickers from.

```bash
curl https://api.flare-cloud.com/api/v1/catalog
```

Response:

```json
{
  "models": [
    { "id": "Qwen/Qwen2.5-1.5B-Instruct", "parameters": "1.5B", "gated": false }
  ],
  "gpu_types": [ { "id": "B300", "label": "NVIDIA B300 (Blackwell Ultra)" } ],
  "default_model_id": "Qwen/Qwen2.5-1.5B-Instruct"
}
```

### GET /api/v1/health

Health. Liveness plus a count of how many providers are actually routable.

```bash
curl https://api.flare-cloud.com/api/v1/health
```

Response:

```json
{
  "status": "ok",
  "environment": "production",
  "providers_registered": 2,
  "providers_active": 1
}
```