Incident 35 decision steps

FTX, Alameda, and the segregation break

Customer funds and a sister hedge fund's balance sheet were never supposed to share a wallet. They did. You're the one who finds it.

By Solomon Ajayi · Free to read, no signup

It's Wednesday November 2 2022. CoinDesk has just published Alameda Research's leaked balance sheet, showing that the majority of Alameda's claimed assets are FTT, the token issued by FTX, an affiliated entity. You're a senior engineer at FTX. Until this morning you assumed that customer crypto held at FTX was strictly segregated from anything Alameda touched. Within 72 hours your read-only query against the customer-deposit aggregates and the on-chain wallet balances is going to tell you a different story. The simulation walks you through the week, not as the CEO or the lawyers, but as the engineer with database access who has to decide what to report, when, and to whom. Every choice here actually happened to someone at FTX between November 2 and November 11 2022.

The decisions, beat by beat

Wednesday 2 November 2022, 11:00 ET

The CoinDesk piece is everywhere on crypto Twitter. Slack is moving fast. Trading desk asks: are we OK? Compliance asks: what's our actual reconciliation between customer balances and on-chain holdings right now? You have read access to the wallet aggregator and to the customer-ledger sums. You can run that query in under a minute.

What's the first move?

  • Run the live reconciliation: SUM(customer balances) vs SUM(on-chain wallet assets per asset class)

    Right. This is the single query that separates rumor from fact. You run it before you tell anyone anything, internally or externally. It takes a minute. The answer determines whether the next 72 hours is a comms exercise or an existential crisis.

  • Tweet that all customer funds are safe

    You don't know that yet. Tweeting assurance without the reconciliation in front of you is the move that retroactively constitutes fraud if the reconciliation comes back broken. Verify first.

  • Ask the CEO what to say before checking anything

    You have read access to the ground truth. Asking leadership for a position before pulling the data inverts the chain of evidence, you should be telling them what's true, not the other way around.

  • Start pulling customer withdrawals offline as a precaution

    A withdrawal halt during a rumor cycle confirms the rumor. You only halt when you KNOW there's a problem and the halt is the responsible move. Right now you don't know.

Reconciliation query submitted: customer balances vs on-chain wallet aggregates

The query is read-only and runs in about 40 seconds against the customer-ledger replica and the wallet aggregator. You haven't seen results yet. You've committed only to gathering ground truth, not to any conclusion. The clock is the relevant variable now.

Wednesday 2 November 2022, 11:14 ET

The query comes back. Customer Balances Ledger shows $14.2 billion of USDC obligations across all customers. The on-chain customer hot wallet shows $6.1 billion of USDC. There's an $8.1 billion gap. Tracing the gap: a balance-sheet line for 'Alameda Receivable' sits at $8.1 billion, exactly the missing amount. The shortfall is not theoretical, it's loans extended to Alameda Research, against customer assets, that were never repaid in liquid form.

What's the next move?

  • Escalate immediately to the CCO and General Counsel in writing, with the query output attached

    Right. Once you've confirmed the segregation break, the obligation is to escalate in writing to compliance and legal so the record exists. Verbal-only reporting on this scale doesn't protect customers or you.

  • Delete the query from your history and keep quiet

    That is obstruction. Beyond the criminal exposure, it makes you part of the harm, customers continue depositing into a structurally insolvent exchange while you know it's insolvent.

  • Patch the reconciliation script to net out the Alameda receivable

    That's the move that turned a 2019 problem into a 2022 catastrophe. Hiding a reconciliation break inside the reconciliation IS the failure mode you're trying to stop. Surface it, don't bury it.

  • Run the same query against every other asset class to scope the breach

    Useful eventually, but not first. The bigger urgency is establishing a paper trail with compliance / legal BEFORE you expand the investigation. Otherwise you risk being the only person who knew, for weeks, before reporting.

Escalation sent: written notice to CCO + General Counsel with reconciliation output attached

Email + Slack DM, both with the query output as a CSV. The paper trail now exists. Compliance acknowledges receipt at 11:47. The clock for an internal investigation is now running, and your role in the chain of evidence is documented.

Sunday 6 November 2022, 23:00 ET

Binance's CEO has tweeted he intends to liquidate Binance's FTT holdings. Withdrawal volume on FTX has jumped 8x. The on-chain customer wallet drained another $400M overnight. The team needs to record, at minimum, an honest snapshot of the Alameda receivable against the customer-balance liability so internal records reflect the real position. Post the snapshot entry.

Post the journal entry surfacing the receivable.

  • Alameda Receivable UP $8.1B; Customer Balances Liability UP $8.1B (book the unfunded obligation)

    Correct. The customer-balance liability was already understated; the offsetting asset (a soft receivable from Alameda) belongs on the books explicitly. Both sides UP by $8.1B makes the hole visible in any subsequent report.

  • Customer Balances Liability DOWN $8.1B (write down the customer obligation)

    You can't unilaterally write down what you owe customers. Customers still claim the full $14.2B; pretending otherwise on the books is straight-up fraud and makes the eventual disclosure worse.

  • On-Chain Wallet UP $8.1B (book a phantom asset to balance)

    You'd be claiming wallet assets you cannot point to on chain. That is the textbook definition of phantom assets, and it's what auditors look for first when fraud is suspected.

Book the $8.1B Alameda receivable against customer liability
AccountDebitCredit
Alameda Receivable (Intercompany) (1550)$8,100,000,000.00
Customer Balances Ledger (Outstanding Liability) (2500)$8,100,000,000.00

Intercompany receivable UP $8.1B (asset); customer-balance liability UP $8.1B. The shortfall is now visible in every internal balance sheet and cannot be hidden by future reconciliation passes.

Tuesday 8 November 2022, 09:00 ET

The on-chain customer wallet now holds $1.8B against $14.2B in customer obligations. Withdrawal queue is at 2 hours and growing. The Alameda receivable is uncollectable, Alameda's balance sheet is publicly known to be mostly FTT, which has collapsed 80% in 48 hours. The exchange cannot honor pending withdrawals beyond today's flow. Customer Service is asking what to tell users in the queue.

How do you handle the withdrawal queue?

  • Halt customer withdrawals; issue a public statement naming the liquidity issue

    Right. Continuing to process withdrawals for the lucky few while the exchange is structurally insolvent unfairly favors fast users over slow users with the same legal claim. The halt + clear comms is the least-bad path once insolvency is confirmed.

  • Keep processing first-come-first-served and hope outflows slow

    First-come-first-served during insolvency is the exact wrong distributional outcome, customers with bots and fast UIs get paid 100 cents, customers using mobile get paid 0. Bankruptcy law expects equal treatment of equal claims; honoring some claims now violates that.

  • Process only large customers (institutional accounts)

    Worse than first-come-first-served. Explicitly favoring institutions over retail in an insolvency event compounds the legal liability and is morally indefensible.

  • Tweet that withdrawals are paused for 'routine maintenance'

    Lying about the reason for a withdrawal halt during a bank run is a category-mistake response. Customers know it's a halt; pretending otherwise destroys what credibility remains and triggers regulatory action immediately.

Withdrawals halted; public statement names the liquidity issue

The withdrawal-queue flag flips to halt at 09:14 ET. Public tweet at 09:21 names the liquidity issue without yet using the word 'insolvent'. Customer Twitter erupts; the engineering team locks in the queue snapshot for the bankruptcy estate. From this moment forward, no withdrawal goes out without trustee oversight.

Friday 11 November 2022, 03:00 ET

FTX, Alameda, and 130+ affiliated entities have filed for Chapter 11 protection. The new CEO arriving to oversee the bankruptcy will inherit your ledger. The most useful thing you can hand him is a structural recommendation about why this was possible at all.

Which structural recommendation goes at the top of the handoff doc?

  • Customer assets must live in wallets the exchange operator cannot unilaterally move; reconciliation must run every minute and alert on any non-zero gap

    Right. The root cause was operator-controllable customer wallets plus a reconciliation that wasn't wired to alert. The fix is structural: customer wallets behind multisig the operator can't bypass, and a continuous reconciliation alert that pages the on-call the first time the gap is non-zero.

  • Hire more compliance staff

    More staff in front of a broken architecture doesn't fix the architecture. The breach happened because the reconciliation gap was tolerated, not because compliance was understaffed.

  • Add a public attestation report once per quarter

    Quarterly is too slow. The gap was non-zero for years before it surfaced. The structural fix is minute-by-minute reconciliation with paging, not a quarterly PDF.

  • Move all customer fiat deposits onshore to reduce banking friction

    Misses the root cause. The banking friction was the original justification for routing fiat through Alameda, but the failure was the unchecked commingling that followed. Moving the banking doesn't address the missing reconciliation alert.

Handoff recommendation: operator-non-bypassable wallets + continuous reconciliation alerts

The handoff doc lands on the new CEO's desk at 03:47 ET. Top-of-doc structural recommendation: customer wallets behind multisig the operator cannot bypass, and a continuous reconciliation alert that pages the on-call the first time the gap is non-zero. The bankruptcy estate is now several years' work; the engineering lesson is one paragraph.

What actually happened

FTX filed for Chapter 11 on Friday November 11 2022. The shortfall between customer balances on the exchange and assets actually held in the wallets was eventually reported at roughly $8 billion. The mechanism was approximately what you simulated: customer fiat deposits flowed through Alameda-controlled bank accounts because FTX had banking-relationship limitations, and over time the segregation between customer assets and Alameda's trading book eroded into commingling. The on-chain customer-fund wallet was running structurally short against the customer-balance ledger; FTX could only honor net withdrawals as long as inflows roughly matched outflows. Once the CoinDesk story triggered a bank-run pattern, the shortfall surfaced within four days. Sam Bankman-Fried was convicted on seven counts of fraud and conspiracy in November 2023. The cleanest engineering lesson is that segregation is not a policy you assert on a whitepaper, it's a reconciliation that runs every minute and screams when it breaks. If FTX's reconciliation had been wired to a real alerting pipeline the way you'll wire one in lesson 9, the breach surfaces in 2019, not 2022.

Play it from the engineer's seat

Reading the replay is one thing. Sit in the chair, make the calls live, and watch the consequences land in a real ledger. Free.

More incident replays

Search lessons

Type to find any of the 85 lessons. Press Enter to open.