Reference · July 2026 snapshot

Midnight Preprod · Known issues

Paraphrased notes from the Midnight team on Discord (July 2026). These are moving targets — always cross-check the official support matrix and open a Service Desk ticket with your pinned versions before assuming a workaround still applies.

Preprod fresh-wallet sync never completes

Symptom

7.5h+ without finishing, sometimes OOM on smaller machines. Preview first sync in ~44 min is normal; Preprod is heavier, but hours-without-progress + OOM = struggling, not just slow.

Cause

No documented snapshot / fast-sync path for headless WalletFacade today. Memory scales with chain history during initial sync.

Workaround
  • Use Preview for throughput / latency benchmarking.
  • Align every dep to the Preprod row on the support matrix — version skew often surfaces as sync decode failures.
  • Confirm indexer + node are healthy and at chain tip before sync.
  • NODE_OPTIONS="--max-old-space-size=8192" helps OOM but won't fix decode loops.

Lace shows DUST but SDK reports 0 / unshielded never syncs

Symptom

Lace displays a healthy DUST balance while `DustWallet.balance() = 0` and/or the unshielded leg never finishes syncing from the SDK. If the SDK dust leg doesn't reach tip, deploys fail with "no fee DUST" even though Lace looks funded.

Cause

Confirmed known Preprod pattern (Midnight team, Discord, 30/06/2026). Tracked on their side; wallet-SDK fixes in progress.

Workaround
  1. Pin every package to the current Preprod row on the support matrix.
  2. Preprod indexer v4: https://indexer.preprod.midnight.network/api/v4/graphql · wss://indexer.preprod.midnight.network/api/v4/graphql/ws
  3. Wallet config workaround that helps some setups: batchUpdates: { size: 5000, timeout: 1, spacing: 4 }
  4. NODE_OPTIONS="--max-old-space-size=8192" for OOM during first sync.
  5. Confirm the SDK uses the same seed as Lace — different seed = different wallet.
  6. Only read DUST after full sync on all legs.
  7. Unblock local dev with create-mn-app / local docker network (undeployed) while Preprod sync is rough.

DustSpendProcessed ledger event decode failures

Symptom

Repeated `Could not deserialize Ledger Event` during shielded / DUST sync. Wallet SDK ↔ ledger/indexer event-format mismatch on Preprod.

Cause

Pinned ledger-v8 / wallet-sdk-dust-wallet combo can't decode current Preprod events (e.g. midnight-js 4.0.4 vs matrix 4.1.1, proof-server 8.0.3 vs newer rows).

Workaround
  • Re-pin wallet + ledger + midnight-js + proof-server from the Preprod matrix in one pass.
  • Resync a fresh wallet dir.
  • Wallet config workaround that helps some setups: batchUpdates: { size: 5000, timeout: 1, spacing: 4 }
  • Preprod indexer v4: https://indexer.preprod.midnight.network/api/v4/graphql · wss://indexer.preprod.midnight.network/api/v4/graphql/ws

DUST regeneration caps concurrent settlement

Symptom

`InsufficientFunds: could not balance dust` around round 3 of parallel `balanceTx` / `submitCallTx` from one wallet.

Cause

Expected: DUST is rate-limited from registered NIGHT; concurrent calls compete for the same DUST UTXO pool.

Workaround
  • Serialize submits per wallet (or keep 1–2 in flight).
  • Pre-warm DUST: fund NIGHT, wait for cap fill, run a few serial txs before load-testing.
  • Use multiple wallets for parallel lanes — each has its own DUST generation.

Prove + submit coupled; submit hangs without timeout

Symptom

`submitCallTx` / `callTx` fuse the pipeline. Same-wallet parallel submit is still risky (DUST + wallet lock); SDK doesn't always surface a timeout.

Cause

High-level helpers fuse prove and submit into one long call with no client-side deadline.

Workaround

Decouple with the lower-level pipeline:

createUnprovenCallTx
  → proofProvider.proveTx
  → walletProvider.balanceTx
  → midnightProvider.submitTx

Parallelize proving, serialize balance+submit per wallet. Add app-level timeouts around submit.

1010 Custom error: 170 = InvalidDustSpendProof

Symptom

Node rejects submit with `1010 Custom error: 170` — DUST fee proof invalid or stale.

Cause

Common Preprod causes: wallet/indexer stale (DUST Merkle roots pruned while balance still looks fine), indexer lag behind node tip, or version skew between ledger / proof-server / wallet-sdk.

Workaround
  1. Confirm WS vs HTTP: if you get `1010 Custom error: N`, WS is fine.
  2. Fresh wallet resync (or resync right before submit).
  3. Compare indexer block height vs RPC tip.
  4. Pin the full stack to the current Preprod matrix.
  5. Retry after sync completes — don't submit from a long-idle wallet.

/check 400 bad input on callTx (deploy works, callTx fails)

Symptom

Proof-server `/check` returns `400 bad input` in ~3ms with a ~461-byte body on a complete preimage. Deploy `/prove` works; callTx hits `/check` first and fails.

Cause

Compact 0.31.0 changed the ZKIR representation for Uint downcasts, byte-vector ↔ Field/Uint conversions, and relational comparisons. Local proof-server 8.0.3 / 8.1.0 `/check` parser doesn't accept the reworked ZKIR on the wrapped-ir path — client/server serialization gap, per-circuit.

Workaround
  1. cat managed/<circuit>/compiler/contract-info.json — if it's 0.30.x / 0.22.0, confirms the 0.31 ZKIR rework is the trigger.
  2. Point httpClientProofProvider at the public prover https://lace-proof-pub.preprod.midnight.network. If register_asset /check passes there, your local image is just behind the deployed prover. (Lace's wallet-delegated proving uses this same public backend, which is why Lace succeeds where local Docker rejects.)
  3. Bisect the reworked op: drop decimals Uint<8>, drop the secret-key → owner-id conversion, recompile, retry /check to pin the exact op.
  4. Fix is upstream in a later Compact release — see the toolchain 0.31.0 release notes.

/check 400 — engineering-confirmed 0.31 ZKIR serialization gap

Symptom

`httpClientProofProvider`'s `createCheckPayload(preimage, keyMaterial.ir)` is rejected by `/check` on the 8.0.3 public prover, while Lace's wallet-delegated proving of the exact same callTx succeeds against that same prover.

Cause

Midnight team confirmed (Discord, 03/07/2026): this is a client/server `/check` serialization gap for 0.31 ZKIR, not user error or version drift. Needs the engineering team.

Workaround

Open a Service Desk ticket with two linked issues:

  1. /check bad input. Include: deploy / create_market work, register_asset fails, Uint<64> widen didn't fix (→ second reworked op), check() isn't skippable (stub → WASM unreachable), latest stable provider is 4.1.1. Ask which prover parses 0.31 ZKIR on /check, ETA for the ZKIR-format fix from the 0.31.0 notes, and the precise list of reworked ops to avoid on 8.0.3.
  2. Matrix conflict. ledger-v8 8.0.3 (matrix) vs wallet-sdk-dust-wallet 4.1.0 needing 8.1.0's Transaction.addIntent. Ask for the coherent wallet-sdk set for the 8.0.3 row.

Also note in the ticket that lace-proof-pub.preprod.midnight.network from COMPATIBILITY.md doesn't resolve publicly. See the local-repro workaround section for the Lace-vs-httpClientProofProvider bisect that produced this evidence.

Preprod support matrix (current docs)

Symptom

Mixing SDK rows silently produces the failures above. Align every dep to the SAME row.

Cause

Wallet SDK ships as three packages (facade, dust, shielded) — do NOT mix 4.x + 4.x + 3.x ad-hoc.

Workaround
  • ledger-v8: 8.0.3
  • proof-server: 8.0.3 (must match ledger tag exactly)
  • compact: 0.5.1 · toolchain 0.31.1
  • compact-runtime: 0.16.0 · compact-js: 2.5.1
  • midnight-js-*: 4.1.1
  • onchain-runtime-v3: 3.0.0
  • Wallet SDK: align facade / dust / shielded to the same matrix row.

Preprod public RPC — one official endpoint

Symptom

No second official backup URL. `lace-proof-pub.preprod.midnight.network` (from COMPATIBILITY.md) does not resolve publicly.

Cause

One official Midnight-hosted Preprod RPC exists today.

Workaround
  • HTTP: https://rpc.preprod.midnight.network
  • WS: wss://rpc.preprod.midnight.network

Alternatives:

  • Blockfrost (needs API key): https://rpc.midnight-preprod.blockfrost.io?project_id=YOUR_KEY
  • Self-host midnight-node via Docker (github.com/midnightntwrk/midnight-node).
  • Preview WS wss://rpc.preview.midnight.network — only if you can switch networks (not Preprod-compatible).

Smoke check:

curl -X POST https://rpc.preprod.midnight.network \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"chain_getHeader","params":[],"id":1}'

currentBlockTime() / in-circuit block metadata

Symptom

No public in-circuit readable block time or height API today — only `blockTime*` comparators.

Cause

Not yet exposed by Compact.

Workaround

File product feedback via the Midnight Service Desk if settlement flows need it.

Source: Midnight team (Nasihudeen Jimoh) responses on the Midnight Discord, June–July 2026, paraphrased for reference. When escalating, open a Service Desk ticket with pinned versions, indexer URLs, and whether you're seeing unshielded/DUST sync errors or hangs.