Credentials prove who. Midnight proves nothing changed.
Hyperledger Identus is self-sovereign identity infrastructure: DIDs, verifiable credentials and presentations, served by a Cloud Agent. Midnight is where you anchor a commitment to a document so a verifier can confirm it existed and is unchanged — without ever seeing its contents. Used together you get an attestation with an author and a tamper-evident record with no data leak. Everything below is from working builds, not the marketing page.
One agent, three services, two SDK edges.
The REST service you actually call
A Scala/JVM service exposing DID registrar, connections, issuance and presentation endpoints. It needs a Postgres and a PRISM node beside it; first boot migrates four separate databases, which is why it looks slow and why it needs 4 GB.
DID method backing
Publishes did:prism documents. Without Cardano ledger backing it resolves only inside your own stack — fine for a demo, but say so: a third party cannot resolve those DIDs.
TypeScript / Kotlin / Swift
Wallet-side libraries for holding credentials and answering presentation requests. In a Lovable app the browser talks to your own server functions, and only the server holds the agent's admin key.
Skip it with connectionless issuance
A Mediator lets remote wallets exchange DIDComm messages. If you only need a credential over a digest, connectionless issuance avoids the whole invitation dance — one REST call, an invitation URL, no established connection.
Prove a predicate, not the credential
A credential carrying a date of birth can back a browser-side age proof: hash the signed JWT into two 128-bit field limbs as a binding, prove over18 with Noir + UltraHonk in the page, and store only the commitment and public inputs. The JWT never leaves the browser, and a credential with no birth claim must be shown as unprovable rather than silently offered.
Digest → credential → commitment.
The shape below is the one verified in IPS Compass (clinical summaries), and it transfers unchanged to a dance licence, a ticket entitlement or an agent mandate. The document never leaves your database; the digest is the only thing that travels, and the ledger only ever sees a salted commitment.
- 01
Serialise and digest
Canonicalise the document (stable key order, normalised whitespace) and hash it with SHA-256. Non-deterministic serialisation is the number-one reason a verifier recomputes a different digest later.
- 02
Issue a credential over the digest
Claims carry the digest, the credential type and at most a derived boolean (e.g.
over18). Never a name, a date of birth, or any body content — a credential is not a place to park data you refused to put on a ledger. - 03
Anchor a salted commitment
commitment = H(domain ‖ digest ‖ salt), inserted into an append-only CompactSet. Persist the salt with the anchor — an anchor whose salt is missing is unverifiable and must fail closed, not read as confirmed. - 04
Verify each link independently
Structural validation, digest match, a real credential (a pending offer with no JWT is not one), and a membership read against the ledger. Report each pass separately so a partial failure names the broken link.
commitment = persistentHash("ips:anchor:v1" || sha256(bundle) || salt)
// contracts/IpsAnchorRegistry.compact — append-only, duplicates rejected
export ledger commitments: Set<Bytes<32>>;
export circuit anchor(commitment: Bytes<32>): [] {
assert(!commitments.member(disclose(commitment)), "already anchored");
commitments.insert(disclose(commitment));
}Insert-only is not a style choice — overwriting an existing key in a public ledger map makes the dust fee balancer panic on the next call. See Known issues.
Three claims almost every demo overstates.
Decoding a JWT
Reading a credential payload is not JWS verification: no signature check, no issuer DID resolution, no status-list lookup. Label it “issuer signature: not verified” until you do all three.
A transaction hash
A tx hash proves you submitted something. On-chain verification is a read: load public state from the indexer and ask whether commitments.member(commitment) holds.
Simulated mode
An in-app mock issuer is invaluable for UI work (alg: none, stub signature, always healthy) — and proves nothing. Mark simulated credentials as such in the UI, every time.
Simulated → Docker → Fly Machines.
docker compose stack reached at http://localhost:8085/cloud-agent. Localhost only — external DIDComm peers need a tunnel./cloud-agent from the stored base URL for this mode — there is no APISIX gateway in front of a direct Fly deploy.Pin every image
- docker.io/identus/identus-cloud-agent:1.40.0
- docker.io/identus/prism-node:2.5.0
- docker.io/postgres:13-alpine
Docker Hub, not GHCR (not anonymously pullable). Never :latest — a re-pushed tag shipped incompatible proving keys mid-demo. Pin digests if you can and record the resolved digest with the deployment.
Four databases, four roles
Create pollux, connect, agent and node as separate databases to avoid migration collisions — and one <db>-application-user LOGIN role inside each. The agent does not connect as the superuser.
-- Identus 1.40's FIRST Flyway statement is
-- ALTER DEFAULT PRIVILEGES ... TO "<db>-application-user"
-- so that LOGIN role must exist per database, or the agent exits 1.
CREATE DATABASE pollux; CREATE DATABASE connect;
CREATE DATABASE agent; CREATE DATABASE node;
\connect pollux
CREATE ROLE "pollux-application-user" WITH LOGIN PASSWORD '<pw>';
GRANT USAGE, CREATE ON SCHEMA public TO "pollux-application-user";
ALTER DEFAULT PRIVILEGES IN SCHEMA public
GRANT ALL PRIVILEGES ON TABLES TO "pollux-application-user";
-- repeat the \connect + CREATE ROLE + GRANT block for connect, agent, node# Fly private DNS (<group>.process.<app>.internal) resolves ONLY when the
# machine declares its process group. 6PN is IPv6-only.
config.metadata.fly_process_group = "identus-postgres" | "prism-node" | "cloud-agent"
JAVA_TOOL_OPTIONS="-Djava.net.preferIPv6Addresses=true \
-Djava.net.preferIPv4Stack=false -XX:MaxRAMPercentage=70"
# Midnight side of the same app
node RPC bind: [::]:9944
indexer: APP__INFRA__API__ADDRESS: "::"
# checks: grace_period 300s · agent memory >= 4096 MB · poll timeout <= 60s
DIDCOMM_SERVICE_URL=https://<app>.fly.dev:8090 # publish internal port 8090# The Machines API has no per-container log endpoint, and a crash-looping
# JVM is drowned out by healthy sibling output. Tee it, then read it back.
sh -c 'identus-cloud-agent > /tmp/agent-boot.log 2>&1; c=$?; exit $c' &
tail -F /tmp/agent-boot.log
# read from outside: POST /v1/apps/<app>/machines/<id>/exec
# { "cmd": ["sh","-c","tail -n 400 /tmp/agent-boot.log"] }Non-negotiables.
- ▸Only a PUBLISHED did:prism carrying an assertionMethod key can sign a credential offer. Filter the issuer picker by resolving each DID and show why the excluded ones are excluded.
- ▸Connectionless issuance omits connectionId; passing one without an established connection returns a 400 that reads like an agent fault.
- ▸DIDComm invitations must advertise a reachable host: publish internal port 8090 and set DIDCOMM_SERVICE_URL=https://<app>.fly.dev:8090. Repair the endpoint on existing apps instead of redeploying.
- ▸grace_period 300s on the health check — first boot migrates four databases and a shorter window restarts the machine mid-migration.
- ▸Agent memory 4 GB or more; anything less gets OOM-killed during that first migration.
- ▸Cap a single readiness poll at 60s — the Machines API rejects longer timeouts with a 400.
- ▸Every provisioning function must degrade gracefully when the Fly token is absent: return an "unconfigured" state the UI can render, never throw inside a loader.
- ▸Treat 404 from a destroy/read as "already gone" and mark the record orphaned — Fly resources vanish outside your app.
- ▸Scope unique indexes on stack tables by (user_id, kind), or provisioning an Identus stack silently overwrites the Midnight one.
- ▸Provision / check / repair / repair-agent-DB / destroy are separate idempotent operations. Repairing a broken agent must never restart a healthy ledger.
- ▸Secrets stay server-side: read the Fly token and admin key inside .handler() bodies, keep agent clients in *.server.ts, and let routes import only *.functions.ts.
- ▸No env var or flag ships without an upstream source that says it is required. Two speculative fixes (a derived wallet seed, a duplicate POSTGRES_* group) cost a debugging cycle and hid the real error.
Symptom, root cause, fix.
| Symptom | Root cause | Fix |
|---|---|---|
| Agent: UnknownHostException resolving identus-postgres.process.<app>.internal | Fly private DNS keys off process-group metadata, not the machine name | Set config.metadata.fly_process_group on every machine; add a repair action that back-fills it on existing stacks |
| zio.FiberFailure: ERROR: role "pollux-application-user" does not exist | Identus 1.40 connects as per-database application roles created by upstream's compose init scripts | Init SQL creates one <db>-application-user per database with GRANT USAGE, CREATE + ALTER DEFAULT PRIVILEGES, inside each DB via \connect hops |
| The role fix has no effect on an existing stack | Postgres init scripts run only against an EMPTY data directory | Destroy and recreate just the Identus Postgres machine (a "Fix agent DB" action) — never a full-stack redeploy that also restarts a healthy ledger |
| Flyway migration fails with a syntax error near FORMAT | Postgres drifted off 13 — FORMAT is reserved from 14 onward | Pin postgres:13-alpine and recreate the machine on a fresh volume |
| Midnight node crash-loops, exit code 1, seconds after start | The 0.22.x node image rejected hand-rolled dev-network CLI flags | midnight-node driven by CFG_PRESET=dev (auto-authoring) instead of flags |
| Node and indexer running but unreachable from sibling machines | Services bound to IPv4 loopback; Fly's private network is IPv6-only | Bind node RPC to [::]:9944, set APP__INFRA__API__ADDRESS: "::", give the JVM preferIPv6Addresses |
| Chain state lost on every machine replacement | The node wrote to the container filesystem | Attach a Fly volume (10 GB) mounted at the node's chain directory |
| Log tail returns failed_precondition: machine not running | A crash-looping machine is down for part of each cycle, so exec has no target | File-based fallbacks and a short post-crash window — but the real fix is stopping the crash, not reading it faster |
| Health probes spin forever while the machine is already dead | Step state derived from probe results alone | Derive state from machine state first, short-circuit downstream probes on a boot failure, and report restart counts + OOM kills |
| GHCR image pull unauthorized | GHCR is not anonymously pullable | Use the Docker Hub tags with explicit versions; never :latest (a re-pushed proof-server:latest shipped incompatible proving keys mid-demo) |
| Agent wallet resource acquisition fails after any restart, deep in a ZIO trace | DEFAULT_WALLET_SEED regenerated randomly per boot | Derive the seed deterministically (hash of app name + a stored salt) so it survives restarts — and never invent env vars upstream does not document |
| Provisioning an Identus stack overwrites the Midnight one for the same user | unique (user_id) on the stack/connection table | Scope the unique index (user_id, kind) so records for different stacks coexist |
| A credential shows a green "verified" badge that no reviewer accepts | Status read from your own database, or a JWT merely decoded rather than signature-checked | Verify against the source of truth in-request — contract membership for the anchor, a real signature check for the credential — and render simulated or unresolvable artefacts as not checked |
The meta-lesson: porting a compose stack to Fly Machines is a translation job. Read the upstream compose and init scripts before writing a single machine spec — every failure above was already answered there.
Where credentials plus private anchoring beat either alone.
Clinical and personal records
An International Patient Summary is authored, digested and attested, and only a salted commitment reaches the ledger. A verifier confirms the summary is the one that was issued and unchanged, without a byte of clinical data leaving the source.
Credential-gated agentic commerce
Gate an A2A / AP2 / x402 flow on a credential before the payment circuit runs. Watch the subjects: the human principal holds the credential, the AI agent holds the mandate — cross-comparing those two DIDs is the classic false “credential mismatch” rejection.
Eligibility and ticketing
Prove over-18, membership or residency from a derived boolean claim, then anchor the entitlement commitment so a door scan is a membership read rather than a database lookup against a list of names.
Licence and provenance for creative work
Issue a credential over a choreography, score or master file digest, anchor it privately, and settle usage on a mimic-token rail. The credential names the author; the ledger proves the file predates the dispute.
Read these.
- Hyperledger Identus documentation↗
- IPS Compass — live app (Identus × Midnight)↗
- ipsmidnight — issues encountered and how they were solved↗
- ipsmidnight — what we would do differently next time↗
- Credential issuance (incl. connectionless flow)↗
- HL7 International Patient Summary IG↗
Sources: the Identus Catalyst console and the Identus NHS console (agent provisioning, diagnostics and the ZK presentation layer) plus the IPS Compass build, which is the one that joins Identus credentials to a Midnight Compact anchor end to end. Dev-network only, unaudited, and not for real patient data.