Repository navigation
Proxy storage: flows table, bodies in blobs.sqlite, settings - #751
Conversation
 [Critical impact] Database schema migration and storage restructuring for proxy flows. The PR should not merge while upgrading deletes saved traffic and Findings
SummaryThe PR moves captured traffic metadata into
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| @@ -0,0 +1,27 @@ | |||
| DROP TABLE http_exchanges; | |||
There was a problem hiding this comment.
Opening an existing proxy.db runs DROP TABLE http_exchanges before any history is copied. This permanently deletes every saved exchange, including both inline bodies. Copy the old rows into flows and move their bodies into blobs.sqlite before dropping the table.
Knowledge Base Used:
| let mut config = ProxyConfig::new(settings.port, Arc::new(ca)); | ||
| config.capture_body_limit = settings.capture_body_limit; | ||
| let mut proxy_handle = yaak_proxy::start_proxy(config)?; |
There was a problem hiding this comment.
Setting systemProxy to true in settings.json has no effect. start() reads only port and capture_body_limit, and neither lifecycle method changes the operating system's proxy settings. Desktop traffic therefore stays outside the proxy. Apply the requested setting on start and restore the previous setting on stop.
| let settings = ProxySettings::load(&self.settings_path())?; | ||
| let ca = CaStore::load_or_create(&self.data_dir.join("ca")) | ||
| .map_err(|e| format!("Failed to load proxy CA: {e}"))?; | ||
| let mut config = ProxyConfig::new(settings.port, Arc::new(ca)); |
There was a problem hiding this comment.
With {"port":8080} in settings.json, the engine listens on 8080 but ProxyLayout still says Running :9090. Users following that label can configure their clients with the wrong port. Return the actual bound port through the proxy-state response and display it instead of the fixed label. This also covers port: 0.
| ProxyEvent::ResponseBodyComplete { id, size, truncated, elapsed_ms, error } => { | ||
| if let Some(mut f) = in_flight.remove(&id) { |
There was a problem hiding this comment.
request_size and request_truncated are set only by RequestBodyComplete, but a response can finish before an upload does, particularly with HTTP/2. Removing the flow on ResponseBodyComplete discards the later request completion. The saved row can report zero request bytes and no truncation despite already having captured request bytes. Keep request counts current from chunk events and retain enough state to save the final request totals.
Knowledge Base Used: Local proxy service
| let mut decoded = Vec::new(); | ||
| match streaming_decoder(Cursor::new(raw), encoding).read_to_end(&mut decoded).await { |
There was a problem hiding this comment.
read_body() puts no limit on decoded output. A compressed body below the capture limit can expand far beyond it when read_to_end() fills decoded. There is no production caller yet, so this is hardening rather than a current exposed failure. Add a decoded-byte limit and a test with highly compressible input before this helper is used for body inspection.
1 participant
Flows move to a
flowstable with the engine's metadata and no body columns; bodies go to the proxy'sblobs.sqliteas<flow id>.requestand<flow id>.responsethrough a chunk writer now shared with the client's request-body capture. Also:resolve_data_dirin yaak-common, a standalone blob-store opener in yaak-models, WAL and a single writer forproxy.db, andsettings.json(port, system proxy, capture limit) feedingProxyConfig.Tests: chunk writer (grouping, empty body); a flow end to end through
ProxyCtx(row, both blobs, gzip decode on read, truncated body over the cap); the migration on an old-schema DB; settings defaults, missing keys, round trip, invalid JSON.