Account synchronization and fill ledger

Account imports are read-only, resumable jobs for explicitly selected spot markets. They are independent of public market-data mode. An import never places an order, changes a paper balance, or infers portfolio performance from a preview.

Open Trade → Account history, or Settings → Exchanges → Connected accounts → Account history. Choose an account and one of its exact spot markets, then press Sync market history. Sync again collects new fills from the saved checkpoint. This is user-triggered incremental synchronization, not continuous private-account polling.

Implemented adapters

Client	Pagination	Scope
Demo	(created_at, id) keyset	Existing manual paper fills; timestamp ties are retained
Binance, Binance.US, Binance Testnet	Native fromId, beginning at zero; next highest trade ID plus one	Exact spot market; continues until an empty page
Coinbase Advanced Trade	Native opaque cursor in a fixed time window	Spot fills for the selected product and permissioned portfolio
Coinbase Exchange Sandbox	Native CB-AFTER response header and after parameter	Selected sandbox product
Kraken	with_cursor, cursor.next, fixed time window, exact pair and type=no position	Non-position spot fills; requests individual taker fills
OKX	after with native billId, frozen begin/end	Cash spot fills; requests the last 89 days within three-month retention
Bybit UTA	nextPageCursor inside contiguous seven-day windows	UTA spot fills; requests the last 730 days; pre-upgrade/classic and margin excluded
KuCoin classic	HF endpoint lastId, frozen startAt/endAt	Last seven days; inactive orders have a seven-day retention limit; no abandoned endpoint fallback
Bitget classic	idLessThan using the returned tradeId, frozen startTime/endTime	Last 89 days within 90-day retention; no UTA/margin substitution

Other clients retain balance, order and bounded-fill reads. Durable imports explicitly show Coming soon. CCXT's fetchMyTrades capability alone does not establish safe historical pagination. The provider catalog reports history_import separately.

Subsequent Coinbase, Kraken, OKX, Bybit, KuCoin and Bitget passes overlap one day to pick up late/corrected observations. Binance and Demo resume native identity checkpoints. Native IDs deduplicate or update existing records. Coinbase's custom sort_by is omitted because its documentation warns about unstable pagination for sorted results.

Rescan available history restarts the selected market at its adapter's available-history boundary. Pause active work first. Rescans keep existing evidence, upsert returned corrections and report new/corrected fill counts per completed pass. They do not delete an old fill because the provider omits it. Records and corrections outside current retention can remain unavailable; rescanning is not proof of complete history or final settlement. There is no automatic periodic rescan.

OKX cursors use bill IDs while execution timestamps use reported fillTime, falling back to ts when absent. KuCoin uses the account fill-detail id as ledger identity: both legs of a self-trade can share a match tradeId. Bybit retains only explicitly reported fee currencies instead of the SDK's inference. Unsupported margin, special execution, tax and extra-fee cases reject the page rather than silently dropping charges.

Offline tests exercise actual installed CCXT parsers and native requests with final transports replaced. Authenticated private-account acceptance tests remain outstanding. Provider retention, permissioned portfolio and active/delisted markets constrain coverage.

References: CCXT pagination, Binance account trades, Binance.US account trades, Coinbase fills, Coinbase Exchange cursor headers, Kraken trade history.

Additional native contracts: OKX fill history, Bybit executions, KuCoin classic HF fills, Bitget classic fills.

Ownership and persistence

• exchanges/history.py defines TradePage and original native page adapters; exchanges/spot_history.py implements the four additional reviewed contracts and provider limits. ExchangeClient.fetch_trade_page() is separate from bounded fetch_my_trades().
• services/accounts/ledger.py owns account_fills and account_sync in workspace sibyl.db. Fills retain execution values, source/environment, native fill/order IDs, and fee currencies/signs. Raw provider responses and credentials are excluded.
• services/accounts/sync.py validates selections, schedules pages, sanitizes failures and exposes public progress. Opaque cursors and a short cursor-loop guard stay server-side.
• A separate lifespan worker attempts one page per second, fairly across queued markets. Pages contain at most 100 fills; existing CCXT rate limits and client locks also apply. Run one API process; this is not a distributed scheduler.

Identity is (account_id, exact_symbol, trade_id), so accounts/environments cannot collapse together. Every page's upserts and next checkpoint commit in one transaction. Validation failure rejects the entire page and preserves the prior checkpoint. A crash before commit repeats the page safely. Startup returns interrupted running jobs to queued at their saved checkpoints.

Each new pass has a private revision; a new rescan invalidates an old in-flight success or failure. Duplicate identities within a page, repeated normalized pages and non-advancing cursors reject the page. A 10,000-page budget bounds each pass. Retained ranges show earliest/latest execution timestamps, not continuous coverage. Counts are observations and correction counts, not an archive of previous payload versions.

Pause prevents further pages. An already in-flight successful page may commit its fills and cursor while staying paused. Resume uses that checkpoint. Disconnect removes saved credentials, stops imports and retains local fill history. Retained records do not count as current holdings or reconstruct balances.

Progress and failures

States are Queued, Importing, Waiting to retry, Caught up, Paused, Needs attention, and Disconnected. Progress includes retained fills, cumulative pages, inserted/updated counts, last successful page, last completed pass, safe errors and retry time.

Transport/rate-limit failures retry up to five times after the initial attempt, after 15, 30, 60, 120 and 240 seconds. Authentication, unavailable markets, unsupported operations and malformed pages wait for explicit retry. Non-advancing/recently repeated cursors stop imports. Private provider error text never reaches the UI or agents. Technician can read import status through its existing ProtoLink tool; it cannot mutate imports.

Caught up means the available selected-market range was exhausted. Every ledger/job response retains complete_history: false; this does not establish coverage of every account event, historical instrument or opening lot.

Ledger semantics

Execution timestamps differ from first/last local observation times. Amount is in base units; price/notional are in the quote. Notional may be unavailable. Numeric values use the unified client/CCXT representation, not a tax-grade decimal accounting engine.

Fees keep reported currencies and signs. Negative values are rebates. Missing amounts/currencies remain unavailable with an incomplete-data flag. Totals group by currency over the full filtered selection, independently of pagination. Currencies are never added together or assumed to equal the trade quote.

The UI filters by account, market, side and environment. It reads 50 fills per page; Export this page exports that page with IDs, fee currencies and completeness flags. Previews remain in Executions. Demo imports copy existing manual paper fills into this read-only ledger; they create no new trades/balances. Strategy fills stay in their isolated monitor.

Retained fills feed market FIFO performance, including audited opening inventory and optional balance-checked current marks. Separate funding range imports cover Binance crypto deposits/withdrawals, Kraken deposit/withdrawal/transfer entries and Bybit on-chain withdrawals. Order/fill reconciliation shares these fill identities. Full funding coverage, account-wide balance reconciliation and complete cross-market performance remain pending. Accountant's native ProtoLink account_history tool reads bounded retained fills, import progress and provider limits without private network calls or mutations. See the backend checklist.

HTTP contracts

All paths use /api/v1 and normal workspace access controls.

Method	Path	Purpose
GET	/account-sync	Accounts, import support and public progress; no provider calls
GET	/connections/{id}/markets	Exact active spot symbols at the account's venue/environment
POST	/connections/{id}/sync	Queue/resume {"symbols":["BTC/USDT"],"mode":"incremental"} or restart with "mode":"rescan"; 1–12 canonical symbols; returns 202
POST	/connections/{id}/sync/pause?symbol=BTC/USDT	Pause one market; omit symbol for all account imports
GET	/account-ledger/fills	Retained fills, total count, native-currency fee totals and limits

Ledger filters: account_id, optional exact symbol, side (buy/sell), environment (demo/test/production), limit (1–100, default 50), and offset (0–1,000,000). Sort is execution time and stable local ID, newest first. New imports may shift offset page boundaries. Caller-supplied provider cursors, arbitrary endpoints and CCXT parameters are not accepted.

Run uv run pytest tests/test_account_sync.py tests/test_extended_account_history.py for offline pagination, parser, retention, rescan races, recovery, fee, isolation and API tests.