Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions calculating-cost.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ description: How to calculate the cost of your deployment on Cerebrium

Deployment cost is based on the hardware selected and the execution time. <b>Every time code runs or a machine is specified to stay running, compute is billed</b>. GPU, CPU, and Memory usage are charged per second; persistent storage is charged per GB per month. View compute pricing on the [pricing page](https://www.cerebrium.ai/pricing).

Listed rates apply to the default `interruptible` compute tier. Apps configured with the `protected` tier are billed at 2x these rates across GPU, CPU, and memory. See [Compute Tier](/scaling/scaling-apps#compute-tier).

Deploying a model incurs two billable processes:

1. **Build process** — sets up the app environment: a Python environment with the specified parameters, required apt packages, Conda and Python packages, and any model files.
Expand Down
26 changes: 20 additions & 6 deletions scaling/scaling-apps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -256,19 +256,33 @@ Implements the "Power of Two Choices" algorithm: randomly samples two replicas a

<Warning>Requires CLI version 2.1.6 or higher.</Warning>

The `compute_tier` parameter controls whether your replicas are scheduled on spot (interruptible) or on-demand (protected) instances. This directly affects cost and availability.
The `compute_tier` parameter sets the interruption guarantee for an app's instances: whether they run only on capacity that cannot be reclaimed (protected) or on any available capacity, including preemptible instances (interruptible). This directly affects cost and availability.

```toml
[cerebrium.scaling]
compute_tier = "protected" # Use on-demand instances for higher availability
compute_tier = "protected" # No interruptions, billed at 2x the interruptible rate
```

### Available Tiers

| Tier | Description |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `interruptible` | **(Default)** Scheduled on preemptible instances. Lower cost, but may be interrupted by the cloud provider during periods of high demand. |
| `protected` | Scheduled on instances with higher availability with no interruptions, at increased cost. |
| Tier | Description | Price |
| --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- |
| `interruptible` | **(Default)** Runs on any available capacity, preferring preemptible instances. Lower cost, but may be interrupted by the cloud provider or relocated during consolidation. | Base rate |
| `protected` | Runs only on capacity that cannot be reclaimed. No interruptions, excluded from capacity consolidation. | 2x the interruptible rate |

### Pricing

Protected compute is billed at **2x the interruptible rate**. The multiplier applies to all compute on the instance: GPU, CPU, and memory. Persistent storage is priced independently of the compute tier. The rates on the [pricing page](https://www.cerebrium.ai/pricing) are interruptible rates, and the cost estimate shown on the dashboard reflects the configured tier.

The tier is fixed when an instance starts. Changing `compute_tier` applies to instances started after the next deployment; instances already running keep the tier, and the rate, they started with.

Interruptible instances run on preemptible capacity when it is available but can be scheduled on on-demand capacity during shortages. They are billed at the interruptible rate regardless of the underlying capacity.

### Interruptions and Consolidation

Interruptible instances can be interrupted in two ways: the cloud provider can reclaim preemptible capacity during periods of high demand, and the platform periodically consolidates workloads onto fewer nodes to reduce cost, which can relocate interruptible instances regardless of the capacity they run on. In both cases instances are drained gracefully within `response_grace_period`; see [Graceful Termination](/scaling/graceful-termination).

Protected instances are excluded from capacity consolidation and never run on preemptible capacity, so they are not interrupted in either case.

**Choosing a tier:**

Expand Down
4 changes: 2 additions & 2 deletions toml-reference/toml-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ The `[cerebrium.scaling]` section controls auto-scaling behavior.
| scaling_buffer | integer | optional | 2.1.2+ | Additional replica capacity above what scaling metric suggests |
| evaluation_interval_seconds | integer | 30 | 2.1.5+ | Time window in seconds over which metrics are evaluated before scaling decisions (6-300s) |
| load_balancing_algorithm | string | "" | 2.1.5+ | Algorithm for distributing traffic across replicas. Default: round-robin if replica_concurrency > 3, first-available otherwise. Options: round-robin, first-available, min-connections, random-choice-2 |
| compute_tier | string | "interruptible" | 2.1.6+ | Controls pod scheduling on spot vs on-demand instances. Options: interruptible (spot, lower cost), protected (on-demand, higher availability) |
| compute_tier | string | "interruptible" | 2.1.6+ | Sets the interruption guarantee for app instances. Options: interruptible (may be interrupted or relocated, base rate), protected (no interruptions, billed at 2x the interruptible rate) |
| roll_out_duration_seconds | integer | 0 | 2.1.2+ | Gradually send traffic to new revision after successful build. Max 600s. Keep at 0 during development. |

<Warning>
Expand Down Expand Up @@ -252,7 +252,7 @@ scaling_metric = "concurrency_utilization"
scaling_target = 100
evaluation_interval_seconds = 30
# load_balancing_algorithm = "" # Auto-selects based on replica_concurrency
# compute_tier = "interruptible" # Use "protected" for on-demand instances
# compute_tier = "interruptible" # Use "protected" for no interruptions at 2x the interruptible rate
roll_out_duration_seconds = 0

[cerebrium.dependencies.pip]
Expand Down
Loading