Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

- **Fixed** An invalid glob in `--filter` no longer shows its error message twice ([#763](https://github.com/voidzero-dev/vite-task/pull/763)).
- **Changed** The detailed summary from `vp run --verbose` and `vp run --last-details` now shows each underlying cause of an error on its own line ([#761](https://github.com/voidzero-dev/vite-task/pull/761)).
- **Added** Remote caching. Configure an endpoint with the workspace's `cache: { remote: { url } }` or `VP_REMOTE_CACHE_URL`, and choose access with `--remote-cache=off|read|read-write` or `VP_REMOTE_CACHE`. The default is `read` with an endpoint and `off` without one. After a local cache miss, `vp run` looks the task up in the remote cache and, on a hit, restores its outputs and caches it locally. The task output and the run summary show which hits came from the remote cache. A failed read is just a cache miss, with the failure as its reason. In `read-write` mode, `vp run` also uploads the results of successful, cacheable tasks after caching them locally. A failed upload doesn't fail the task; the run summary shows a warning instead. Ctrl-C, or a failing task, stops remote cache requests right away, and a task still being looked up doesn't start. Tasks can opt out with `cache: { remote: false }`. Requests use the proxy environment variables or, on macOS and Windows, the system proxy settings ([#727](https://github.com/voidzero-dev/vite-task/pull/727), [#755](https://github.com/voidzero-dev/vite-task/pull/755), [#756](https://github.com/voidzero-dev/vite-task/pull/756), [#757](https://github.com/voidzero-dev/vite-task/pull/757), [#764](https://github.com/voidzero-dev/vite-task/pull/764), [#771](https://github.com/voidzero-dev/vite-task/pull/771), [#772](https://github.com/voidzero-dev/vite-task/pull/772)).
- **Added** Remote caching. Configure an endpoint with the workspace's `cache: { remote: { url } }` or `VP_REMOTE_CACHE_URL`, and choose access with `--remote-cache=off|read|read-write` or `VP_REMOTE_CACHE`. The default is `read` with an endpoint and `off` without one. After a local cache miss, `vp run` looks the task up in the remote cache and, on a hit, restores its outputs and caches it locally. The task output and the run summary show which hits came from the remote cache. A failed read is just a cache miss, with the failure as its reason. In `read-write` mode, `vp run` also uploads the results of successful, cacheable tasks after caching them locally. A failed upload doesn't fail the task; the run summary shows a warning instead. In a GitHub Actions job granted `id-token: write`, uploads authenticate with a GitHub OIDC token that `vp run` requests for the endpoint, so no token needs to be configured. Once an upload is rejected as unauthorized, or no token can be obtained, `vp run` stops uploading to that endpoint for the rest of the run. Ctrl-C, or a failing task, stops remote cache requests right away, and a task still being looked up doesn't start. Tasks can opt out with `cache: { remote: false }`. Requests use the proxy environment variables or, on macOS and Windows, the system proxy settings ([#727](https://github.com/voidzero-dev/vite-task/pull/727), [#755](https://github.com/voidzero-dev/vite-task/pull/755), [#756](https://github.com/voidzero-dev/vite-task/pull/756), [#757](https://github.com/voidzero-dev/vite-task/pull/757), [#764](https://github.com/voidzero-dev/vite-task/pull/764), [#771](https://github.com/voidzero-dev/vite-task/pull/771), [#772](https://github.com/voidzero-dev/vite-task/pull/772), [#774](https://github.com/voidzero-dev/vite-task/pull/774)).
- **Fixed** On Windows, environment variable names used by `vp run` now match regardless of ASCII letter case. Assignments in task commands override earlier assignments and inherited variables spelled differently, and `FORCE_COLOR`, `VP_RUN_CONCURRENCY_LIMIT`, and variables requested through `@voidzero-dev/vite-task-client` are found under any spelling ([#747](https://github.com/voidzero-dev/vite-task/pull/747)).
- **Changed** A task's cache settings now go inside `cache`, e.g. `cache: { env: ["NODE_ENV"], input: ["src/**"] }`; `cache: true` is the same as `cache: {}`. `env`, `untrackedEnv`, `input`, and `output` are no longer supported at the top level of a task ([#749](https://github.com/voidzero-dev/vite-task/pull/749)).
- **Fixed** Cached tasks on macOS no longer intermittently fail with exit 2 and `oils I/O error (main): No such process` when a fast command finishes before the shell gets scheduled. The bundled shell that runs task commands is updated to Oils 0.38.0, which fixes this race ([#702](https://github.com/voidzero-dev/vite-task/issues/702), [#703](https://github.com/voidzero-dev/vite-task/pull/703)).
Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 9 additions & 6 deletions crates/vt/src/session/cache/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ use vt_plan::{
cache_metadata::{CacheMetadata, ExecutionCacheKey, SpawnFingerprint},
remote_cache::{RemoteCacheAccess, ResolvedRemoteCacheConfig},
};
use vt_remote_cache::StoreAuth;
use vt_str::Str;
use wincode::{
SchemaRead, SchemaReadOwned, SchemaWrite,
Expand Down Expand Up @@ -312,8 +313,10 @@ pub fn cache_schema_dir_name() -> Str {
}

impl ExecutionCache {
/// Open the cache in `path`. Uploads to the remote cache authenticate with
/// `store_auth`.
#[tracing::instrument(level = "debug", skip_all)]
pub fn load_from_path(path: &AbsolutePath) -> anyhow::Result<Self> {
pub fn load_from_path(path: &AbsolutePath, store_auth: StoreAuth) -> anyhow::Result<Self> {
tracing::info!("Creating task cache directory at {}", path.as_path().display());
std::fs::create_dir_all(path)?;

Expand All @@ -337,7 +340,7 @@ impl ExecutionCache {
CREATE TABLE IF NOT EXISTS task_fingerprints (key BLOB PRIMARY KEY, value BLOB);",
)?;
// Lock is released when lock_file is dropped
Ok(Self { conn: Mutex::new(conn), remote_clients: RemoteClients::default() })
Ok(Self { conn: Mutex::new(conn), remote_clients: RemoteClients::new(store_auth) })
}

#[tracing::instrument]
Expand Down Expand Up @@ -711,15 +714,15 @@ mod tests {
fn reopening_preserves_existing_entries() {
let (_tmp, dir) = temp_dir();

drop(ExecutionCache::load_from_path(&dir).unwrap());
drop(ExecutionCache::load_from_path(&dir, StoreAuth::Anonymous).unwrap());
{
let conn = open_raw(&dir.join("cache.db"));
conn.execute("INSERT INTO cache_entries (key, value) VALUES (X'01', X'02')", ())
.unwrap();
}

// Reopening must not recreate or clear the tables.
drop(ExecutionCache::load_from_path(&dir).unwrap());
drop(ExecutionCache::load_from_path(&dir, StoreAuth::Anonymous).unwrap());

let count: u32 = open_raw(&dir.join("cache.db"))
.query_one("SELECT COUNT(*) FROM cache_entries", (), |r| r.get(0))
Expand All @@ -738,8 +741,8 @@ mod tests {
let dir_a = base.join("v13");
let dir_b = base.join("v14");

drop(ExecutionCache::load_from_path(&dir_a).unwrap());
drop(ExecutionCache::load_from_path(&dir_b).unwrap());
drop(ExecutionCache::load_from_path(&dir_a, StoreAuth::Anonymous).unwrap());
drop(ExecutionCache::load_from_path(&dir_b, StoreAuth::Anonymous).unwrap());

assert!(dir_a.join("cache.db").as_path().exists());
assert!(dir_b.join("cache.db").as_path().exists());
Expand Down
Loading
Loading