add MockExchange for offline development and testing#107
Conversation
|
Thanks for this — the concept is solid and it covers the issue requirements well. A few things to address before this is ready to merge:
This is the main concern. faker is ~3.5MB and it would ship to every
Option 1 is probably the cleanest here since the data generation doesn't need faker's full feature set. No tests 610 lines of order lifecycle, balance accounting, and position tracking need test coverage. The position math (entry price averaging, PnL) and balance deductions are especially important to verify. Orders always fill immediately
Mutation Position tracking and Minor
The foundation here is good — deterministic seeding, proper BaseExchange contract, |
CCXT-compatible signature: fetchOrderBook(outcomeId, limit?, params?) - Add optional `limit` (depth) and `params` bag to all 14 exchange implementations, Router, and SDK client - Add `datetime` field to OrderBook type (CCXT-compatible) - Migrate `side` from positional arg to `params.side` in Limitless and Router (backwards compatible via params bag) - Prepares for hosted-pmxt to route `params.since` to ClickHouse archive for historical order book snapshots
realfishsam
left a comment
There was a problem hiding this comment.
PR Review: FAIL
What This Does
Adds MockExchange — a fully offline, deterministic prediction market exchange backed by a custom seeded PRNG (SeededRng). It extends PredictionMarketExchange, overrides every unified method with in-memory implementations, and supports a limitOrderMode: 'resting' option to allow testing open/cancel/fill flows. This directly closes #19.
The v2 iteration (this revision) correctly addressed all the points from the earlier review: faker was dropped in favour of a zero-dependency SeededRng, 162 lines of tests were added, resting limit orders are now supported via limitOrderMode, and mutations are fixed with spread operators. The foundation is solid.
Blast Radius
- New files only (
core/src/exchanges/mock/): no existing exchange logic touched. core/src/index.ts:MockExchangeadded to the default export object and named exports — affects every consumer that importspmxt-core.exchange-factory.ts(pre-committed before this PR): already hascase "mock"and routes sidecar HTTP calls tonew MockExchange()— this PR fills in that stub.openapi.yamland both SDK clients: not updated — the source of both CI failures.
Consumer Verification
Before (base branch / without this PR):
exchange-factory.ts already had case "mock" but core/src/exchanges/mock/ didn't exist, so importing pmxt-core would fail at module resolution when the server started.
After (PR branch):
# fetchMarkets — deterministic, realistic market data
$ curl -s -X POST http://localhost:3848/api/mock/fetchMarkets \
-H "x-pmxt-access-token: test123" \
-H "Content-Type: application/json" \
-d '{"args": [{"limit": 2}]}'
# → { "success": true, "data": [ { "marketId": "mock-m0", "title": "Which party wins the 2026 harborland election?", ... } ] }
# createOrder — fills immediately in default mode, returns 'filled'
$ curl -s -X POST http://localhost:3848/api/mock/createOrder \
-H "x-pmxt-access-token: test123" \
-H "Content-Type: application/json" \
-d '{"args": [{"marketId":"mock-m1","outcomeId":"mock-m1-yes","side":"buy","type":"limit","price":0.55,"amount":10}]}'
# → { "success": true, "data": { "id": "mock-order-1-9gqws6", "status": "filled", "filled": 10, "fee": 0.0055, ... } }
# fetchBalance — deducted correctly
$ curl -s -X POST http://localhost:3848/api/mock/fetchBalance -H "x-pmxt-access-token: test123" ...
# → { "success": true, "data": [{ "currency": "USDC", "total": 1000, "available": 994.5, "locked": 0 }] }Market generation, balance accounting, and order fills are all verified working end-to-end through the sidecar.
Test Results
- Build: PASS (
tscclean, no errors) - Unit tests: PASS — 505 passed, 0 failed (8 suites, including the 8 new
MockExchangetests) - Server starts: PASS
- E2E smoke (mock exchange): PASS —
fetchMarkets,fetchBalance,createOrder,fetchBalancepost-order all behave correctly through the sidecar HTTP API - CI on GitHub: 2 FAILURES (see Findings 1 and 2 below)
Findings
1. [BLOCKING] mock missing from ExchangeParam enum in openapi.yaml — CI "Verify exchanges reach all consumer SDKs" is FAILING
core/src/server/openapi.yaml:2459 lists the valid exchange names:
ExchangeParam:
in: path
name: exchange
schema:
type: string
enum:
- polymarket
- kalshi
- kalshi-demo
- limitless
- probable
- baozi
- myriad
- opinion
- metaculus
- smarkets
- polymarket_us
- router
# mock is not heremock must be added to this enum. The server itself dispatches correctly (the runtime doesn't validate against this list), but the CI check validates that every exchange in exchange-factory.ts is declared in the spec. The enum is also what the SDK code generator reads to know which exchanges exist, so until it's added here, SDK consumers can't discover mock through the generated client layer.
2. [BLOCKING] API_REFERENCE.md not regenerated — CI "Verify API_REFERENCE.md files are up-to-date" is FAILING
Adding a new exchange requires regenerating the API reference documentation. This is likely handled by a npm run generate or npm run docs script. The CI check is detecting the stale state.
3. [BUG] fetchMyTrades silently ignores the marketId filter — index.ts:649–653
override async fetchMyTrades(_params?: { outcomeId?: string; marketId?: string }): Promise<UserTrade[]> {
let trades = [...this._myTrades];
if (_params?.outcomeId) trades = trades.filter(t => t.outcomeId === _params.outcomeId);
// marketId filter is never applied
return trades.sort((a, b) => b.timestamp - a.timestamp);
}Confirmed via live test: after placing a buy on mock-m1-yes, calling fetchMyTrades({ marketId: 'mock-m0' }) returns that trade even though it belongs to a different market. A caller filtering by market ID gets silently incorrect results. Fix:
if (_params?.marketId) trades = trades.filter(t => t.outcomeId.startsWith(_params.marketId + '-'));(Or, store marketId on each UserTrade and filter directly — cleaner and more robust.)
4. MockExchangeOptions not configurable via sidecar
exchange-factory.ts always creates new MockExchange() with defaults — marketCount: 50, balance: 1000, limitOrderMode: 'immediate'. There's no way to pass options through the sidecar HTTP API. This means SDK consumers (Python/TypeScript via the sidecar) always get the default configuration, regardless of what the PR description's example shows. This is a documentation gap: the new pmxt.Mock({ marketCount: 20, balance: 5000 }) usage in the PR body only works when using the class directly in TypeScript, not through the sidecar pattern.
5. fillOrder() and reset() are not in openapi.yaml and won't appear in generated SDK clients
These mock-specific methods work via the sidecar (the server dispatches dynamically by method name, so POST /api/mock/fillOrder routes correctly), but they're absent from the OpenAPI spec. Generated Python/TypeScript SDK clients won't expose them. This is probably intentional — they're test utilities — but it should be documented clearly, since fillOrder is the only way to resolve a resting limit order from outside the exchange.
6. Stale PR description references @faker-js/faker
The description still says the exchange is "built on top of @faker-js/faker" and cites it as a dependency. This is no longer true — the v2 revision uses the custom SeededRng. Worth updating to avoid confusion.
7. Minor: redundant non-null assertion — index.ts:487
const p = price; // price: number
// ...
price: p!, // p is already number, ! is unnecessaryPMXT Pipeline Check
- Field propagation (3-layer): N/A —
MockExchangegenerates its own data; it doesn't normalize from a venue API - OpenAPI sync: ISSUE —
mocknot inExchangeParamenum (Finding 1) - Financial precision: OK — uses
round()helper consistently; no floating point accumulation on balances beyond the rounding threshold - Type safety: OK — no
anytypes introduced; spread operators used correctly for immutability - Auth safety: N/A — no credentials, no network calls
Semver Impact
minor — new public class added (MockExchange), no existing API changed.
Risk
Two CI checks are hard-blocking. Finding 3 (fetchMyTrades ignoring marketId) is a correctness bug that will silently mislead any consumer who filters by market, and it's not caught by the current test suite. Everything else is clean and the core mechanics (market generation, balance accounting, resting/immediate order modes, deterministic seeding, reset()) all work correctly.
Generated by Claude Code
…derBook - BaseExchange JSDoc: document `until` param for range queries - SDK client.ts: add JSDoc examples + update return type to OrderBook | OrderBook[] for range queries
- Replace @faker-js/faker with local SeededRng (mulberry32 + string hash) - market orders price from same mid as fetchOrderBook (first float) - limitOrderMode: 'resting' for open/cancel/fill; fillOrder for partial/complete - Buy resting uses locked USDC; immutable position updates - Add core/test/unit/mockExchange.core.test.ts Made-with: Cursor
d2711a9 to
37ae1b3
Compare
closes #19
what this does
adds
MockExchange— a fully offline, zero-network exchange implementation built on top of@faker-js/faker. it lets developers write and test application code against pmxt without real API keys or an internet connection, which is especially useful in CI/CD pipelines and during rapid prototyping.how it works
MockExchangeextendsPredictionMarketExchangeand overrides every relevant method with local in-memory implementations. faker is seeded deterministically from each market/outcome id, so the same call always returns the same data across runs — you can write stable assertions against it.features
fetchMarkets/fetchEvents— generates realistic binary and multi-outcome markets grouped into events, with prices, volumes, slugs, tags, and categoriesfetchOrderBook— produces a realistic CLOB-style bid/ask ladder centred around the mid pricefetchOHLCV— generates candlestick data for any resolution (1m through 1d) and any date rangefetchTrades— returns a list of fake historical trades for an outcomefetchBalance— returns a configurable starting USDC balancecreateOrder— records the order in-memory, updates balance and positions, simulates a configurable latency (default 100ms), always fillscancelOrder,fetchOrder,fetchOpenOrders,fetchClosedOrders,fetchAllOrders,fetchMyTradesfetchPositions— returns live position state derived from all trades placed so farbuildOrder/submitOrder— supportedreset()— clears all orders, positions and trades between test casesusage
made by mooncitydev