Vast.ai: Support spot instances#4025
Conversation
Vast.ai offers interruptible (spot) instances, but the backend filtered
out every spot offer with an `extra_filter` and never bid on one, so only
on-demand instances could be provisioned.
Vast's `PUT /asks/{id}/` endpoint creates an interruptible instance when
the payload includes a `price` (per-machine bid in $/hour); omitting it
creates an on-demand instance. gpuhunt already emits Vast spot offers
(price set to the offer's `min_bid`), so no catalog changes are needed.
* Drop the `extra_filter` that removed spot offers in
`get_offers_by_requirements`.
* In `run_job`, bid the spot offer's price and pass it to
`create_instance`; on-demand offers pass no bid.
* Add the `price` field to the `create_instance` payload.
Interruption detection and retry are backend-agnostic (the server treats
a lost spot instance as `INTERRUPTED_BY_NO_CAPACITY`), so no further
changes are required.
jvstme
left a comment
There was a problem hiding this comment.
Hi @trashhalo. Thank you for contributing this useful feature. Overall, the PR looks good to me.
One issue I found during testing is that if you try to create an instance with an insufficient bid (for example, if someone outbids you just before you create the instance), Vast.ai returns an error but still creates an instance in the stopped state, which then continues to accumulate storage charges. This case requires additional cleanup.
I've already fixed this and added a few related improvements in 5e01c9c on dstackai:vastai-spot. If you agree with these changes, could you push the commits from dstackai:vastai-spot to Hanno-Labs:vastai-spot so they appear in this PR?
For some reason, GitHub won't let me push them to your repo myself, even though you've allowed edits from maintainers. This may be related to this known GitHub limitation.
|
We'd like to include this feature in this week's release, so I'll merge as-is and add my fixes in a separate PR. Thanks again for contributing |
Vast.ai offers interruptible (spot) instances, but the
vastaibackend has never been able to provision them.Currently,
get_offers_by_requirements()strips every spot offer:and
create_instance()never bids, so only on-demand instances are provisioned. gpuhunt already emits Vast spot offers (each on-demand offer is duplicated withpriceset to the machine'smin_bidandspot=True), so these offers reach dstack today and are then discarded.Vast's create endpoint is the same for both instance types.
PUT /asks/{id}/creates an interruptible instance when the payload includes aprice(per-machine bid in $/hour), and an on-demand instance when it is omitted. This matches the officialvast-pythonclient, whosecreate instance --bid_pricesends the samepricefield to the same endpoint.The fix:
extra_filterthat removed spot offers inget_offers_by_requirements(). Spot vs. on-demand selection is already driven by the run'sspot_policythrough the catalog query, so no offers are surfaced that the user did not ask for.run_job(), bid the spot offer's price (which gpuhunt already set tomin_bid) and pass it tocreate_instance(); on-demand offers pass no bid.pricefield to thecreate_instance()payload (Nonefor on-demand preserves the existing behavior).Interruption handling requires no backend changes: the server infers a reclaimed spot instance from the lost runner connection and, because the offer is marked
spot, terminates the job withINTERRUPTED_BY_NO_CAPACITY, which maps to the genericinterruptionretry event. This is the same path RunPod, OCI, and Nebius spot instances already use.Tested against a live Vast.ai account: spot offers carry a
min_bidbelowdph_base(e.g. an RTX 4090 at $0.24/hr on-demand vs. $0.12/hrmin_bid), and the create endpoint accepts thepricefield. Unit tests cover thatrun_job()bids on spot offers and does not bid on on-demand offers.Vast.ai API reference: https://docs.vast.ai/api/create-instance
AI assistance: This PR was written primarily by Claude Code (investigation, implementation, and tests) and reviewed by me before submitting.