Markets from First Principles

Every trade needs a buyer and a seller whose prices overlap. Each has a limit: the most the buyer will pay, the least the seller will take. When the buyer's limit is at or above the seller's, some price suits them both and they can trade. When it is below, no price does.

Now it's market day. Through the day, 22 people come into the square one at a time, buyers through one gate and sellers through the other, each wanting to buy or sell a sack of grain, each with their limit.

Nobody haggles. In the middle of the square, people who are waiting stand in line with a note saying their price. Whoever arrives reads the notes. If a waiting note suits their limit, they trade with its owner at once, at the price on the note. If none does, they join the line at their own price and wait.

The first seller found nobody and waited at 47. The next person, a buyer who would have paid 60, bought at 47: the price was set by whoever was already waiting. By evening six sacks have changed hands, at prices from 47 to 52, and ten people are still in line.

This is how the markets in this guide trade, around the clock. Orders arrive one at a time, and each one either trades with an order that is already waiting or waits itself. There is no single price for the day: each trade has its own, set by whoever was waiting.

Here are the notes the day left on the board: five to sell and five to buy. Arrive in a hurry and you don't need to wait. Read the board and take the best note: the cheapest sell note if you want to buy, the highest buy note if you want to sell.

Whoever takes a note is a taker: they pay the price on the board for not having to wait. Everyone who traded on market day was a taker, the moment they arrived. You can also be on the other side, and wait, with a note of your own.

Whoever leaves a note is a maker: they make the board, and they wait for a taker. Notes at the same price are taken in the order they were pinned, so being early counts. Many exchanges charge makers a lower fee than takers (Hyperliquid's fee schedule is one example), because the notes on the board are what let anyone trade at all.

Pin up thousands of notes and the board is too long to read one by one. Add up the notes at each price and you get the order book: the buy notes on one side, called bids, and the sell notes on the other, called asks.

The highest bid is the best bid, the lowest ask is the best ask, and the gap between them is the spread. The point halfway between is the mid price: one number for where the market stands between trades.

Suppose you run a stall that always quotes both sides: you'll buy a sack a little under the going price of 50 and sell one a little over. People who don't want to wait for their note to be taken come to you instead, and the difference between your two prices is how you get paid. This job is called market making.

Most of your customers just need grain, or need to sell some, and each one earns you your gap. But some have read tomorrow's newspaper. They know the price is about to move by 5 coins, and they only come to you when your price is about to be wrong. Each of them costs you the move minus your gap.

With nobody informed, every customer earns you your gap. Let one customer in five know tomorrow's price, and a gap of 1 only breaks even: you earn 1 from four of them and lose 4 to the fifth. The more customers know something, the wider the gap you need.

Set the gap above break-even and you make money, until another stall across the square quotes a narrower one and takes your customers. With enough competition the gap ends up where market makers just break even: wide where many customers know something, narrow where few do. Lawrence Glosten and Paul Milgrom worked this out in 1985, and it's still where economists start when they explain the spread.

Market makers also charge for holding stock they didn't want and for their own running costs, but the informed customers are why the gap widens when news is coming: everyone knows some of the people trading in the next minute know more than they do.

The mid price treats both sides of the book as equal. They often aren't. If ten people wait to buy at 49 and two wait to sell at 51, the next trade is far more likely to clear out the two sellers than the ten buyers, and once the sellers are gone the best ask moves up.

So a better guess at the fair price leans toward the thin side. Weight each side's price by the size waiting on the other side and you get the weighted mid. Drag the queues and watch it move.

The difference between the sizes on the two sides is the book's imbalance. Martin Gould and Julius Bonart found that the imbalance at the best prices predicts the direction of the next price move.

Go back to the board lesson 1 left behind, with the best bid at 51 and the best ask at 53. A maker pinned a note and waited. A taker arrived, read the board and took that note at its price. The trade needed both of them, but only the taker decided it should happen now, and crossed the spread to make it happen.

Exchanges record this. Next to each trade's time, price and size, the tape says which side the taker was on. In a taker buy, the buyer crossed and took a note at the best ask; in a taker sell, the seller crossed and took one at the best bid. Add them up over a minute and you get taker buy volume and taker sell volume. Buys minus sells is the ƒvolume delta.

Below, forty takers arrive at that board, each wanting one sack straight away, while makers keep two sacks waiting at every price. You choose how many of the takers are buying and how many selling, and watch the price.

Forty sacks change hands whatever you choose, so the volume is the same every time. With five buyers in every ten, takers bought 20 and sold 20, and the price ended at 52, where it started. With seven in ten, they bought 28 and sold 12, a volume delta of +16, and the price ended 10 higher, at 62.

Each taker buy removes a sack from the ask side. When buys outrun sells, the asks run out faster than makers replace them, and the best ask has to move up, taking the price with it. The volume says how much traded. The delta says which way the people in a hurry were leaning, and the price follows the lean.

The same delta can come from very different people. Twenty-six sellers might each sell one sack, a few at a time, with buyers cutting in between them. Or one seller might want 12 sacks gone at once.

An order to sell 12 sacks takes the first note at the best bid, then the next, and keeps going down the board until it has all of them. The exchange prints one trade for every note it fills, so the tape shows 12 trades of one sack, all sells, all at the same moment, between the buys of everyone else.

Both tapes hold 14 taker buys and 26 taker sells, a volume delta of −12, and both leave the price at 45. Counted trade by trade, the sizes match too: every trade is one sack, because each fill is the size of the note it took. What gives the big seller away is the order of the trades.

A run is a streak of trades in a row whose takers were on the same side. In the crowd, the longest run of sells was 4 trades. The big seller's order printed 12 in a row. The ƒtrade run imbalance puts the longest run on each side against each other: the longest buy run minus the longest sell run, divided by the two added together. It goes from −1, when only sellers ever kept at it, to 1. The crowd's tape scores −0.33, the big seller's −0.71.

Lesson 2's stall lost money to customers who knew where the price was going. Those customers trade in a recognisable way: they all want the same side, and they come just before the price moves. So flow that piles up on one side while the price jumps around suggests that the people taking a market maker's notes know something. Market makers call such flow toxic, the word David Easley, Marcos López de Prado and Maureen O'Hara used for it in 2012.

The ƒflow toxicity score puts a number on it, a minute at a time. It takes two counts for the minute: the volume delta, and how much the price moved around inside the minute. It measures each against every minute of the day so far as a z-score, where 0 is an ordinary minute and 2 or −2 is far from ordinary. Then it multiplies the two.

Pick one of four minutes below and see where it lands on each count, against a toy morning of 30 ordinary ones.

Take a minute when takers sold 12 sacks more than they bought: far from usual, a z-score of −2.8. If its price moved around an ordinary 3 coins, a z-score of 0.0, the score comes to 0. If its price moved around 6 coins, a z-score of 2.5, the score drops to −7. A large score needs both: flow leaning hard to one side, in a minute when the price is busier than usual. Then the sign says who was pushing, below zero for sellers and above zero for buyers.

A candle keeps five numbers from a stretch of trading: the first price, called the open; the highest and the lowest, the high and the low; the last, the close; and how much traded, the volume. The box runs from the open to the close, and the line through it reaches up to the high and down to the low.

Five numbers can't hold everything that happened. Here are three toy tapes from the board you know, each with 60 sacks traded. Play them, and watch the price, who was buying and selling, and the candle each one leaves.

All three leave the same candle: open 50, high 51, low 44, close 45, 60 sacks. Yet in the slide, sellers kept coming, mostly one sack at a time, and the price stepped down a coin at a time. The jump had ten quiet trades, then one seller who took 24 sacks at 44 in a single trade. And in the tug of war, buyers and sellers took turns, three sacks at a time, while the price kept turning on its way down. Someone reading only the candle can't tell which of these happened.

Measures built from the trades themselves keep some of what the candle throws away. An ƒuptick is a trade at a higher price than the trade before it, and a downtick one at a lower price; the rest print at an unchanged price. Counting them says how the price got from the open to the close: in a few steps or many, and how often it turned back.

An uptick isn't the same as a taker buy. A taker buy that fills at the same price as the trade before is a buy and an unchanged tick. The side says who crossed the spread; the tick says whether the price moved.

ƒTrade sizes sort trades by the money in each: here, small under 100 coins, medium up to 1,000, and large from 1,000. The catalog uses the same cuts in dollars. And ƒflow entropy says how mixed the taking was. Take the share of the money that went through taker buys, and the share through taker sells: the entropy is 0 when one side did all the taking and 0.69 when they split it evenly.

Read each tape three ways: by its ticks, by the size of its trades, and by taker buys against taker sells.

The slide had 7 downticks and only 2 upticks. The tug of war had 9 downticks and 4 upticks: the same distance travelled, with more turning back. In the jump, one large trade carried 38% of the money. And sellers did 88% of the taking in the slide, so its flow entropy is 0.36, while the jump and the tug of war, with both sides active, come to 0.63 and 0.67.

The candle can't say whether the fall from 50 to 44 came in one trade or in many. The ƒjump ratio can. It compares two ways of adding up how much the price moved from trade to trade.

The first is realised variance: square every move and add them up. Squaring makes big moves count much more than small ones. The second is bipower variation, which multiplies each move by the move before it instead, and scales the sum by π/2 so the two agree when the price moves in steady small steps. A big move next to small ones, or next to a trade at an unchanged price, adds little or nothing to it. The jump ratio is the share of realised variance that bipower variation leaves unexplained, the jump part.

With the fall spread over six one-coin moves, the jump ratio is 0.21. Put four coins of it into one trade and it is 0.92; all six at once and it is 1. The three tapes from before score 0.14 for the slide, 0.93 for the jump and 0.13 for the tug of war.

Even six small moves score above zero. Wherever a trade at an unchanged price sits next to a move, that move loses a partner in bipower variation, so part of it counts toward the jump part. On a real tape, where most trades print at the price before them, that happens all the time, so read the jump ratio against other bars of the same market rather than against zero.

In lesson 1 you bought one sack from the cheapest note to sell, and the next buyer paid a little more. Now buy more than the best ask holds. The rest of your order doesn't wait: it takes the next cheapest price, then the next, until it is filled. Traders call this walking the book.

Here is a deeper book, with toy numbers: a unit costs about 100 coins, and prices move one tick, 0.01, at a time. Each column is one level, as tall as the units waiting there: bids on the left, asks on the right, 25 levels of asks in all. Pick a size and send your order.

An order of 40 takes all 10 units at the best ask, then 6 at 100.01, 8 at 100.02, 6 at 100.03, 8 at 100.04 and the last 2 at 100.05. Each fill is a trade of its own, at its level's price. They average 100.0205, which is 2.05 basis points over the best ask. That difference is the order's ƒslippage.

Slippage is measured from the best price you could see when you sent the order: the best ask for a buy, the best bid for a sell. An order the best level can hold has none. Double the order to 80 units and the slippage more than doubles, to 4.48 basis points. Paid on twice as many units, the extra over the best ask comes to more than four times as much: 3.58 coins against 0.82.

How far an order walks depends on how much waits at each level. A book with plenty at every price is called deep, or thick, and one with little is thin. The figure sets the book from above next to one with half as much at every level. Move your order and compare.

Halve the depth and the slippage roughly doubles. At 40 units the thin book charges 4.48 basis points against the thick book's 2.05, and the order walks 10 levels instead of 6. The thick book's line also bends: past 118 units the order reaches the wall of 40 units at 100.13, and the line flattens, because every unit in the wall costs the same.

So a trader who needs to buy a lot can split the order and send the pieces minutes apart, and each piece meets a book that has had time to refill. Whether it refills, and at what price, is the next lesson.

Lesson 2 compared the queues at the best bid and the best ask. With 25 levels a side you can ask the same question at any depth: add up the first 5 levels of each side, or 10, 20 or 25, and compare. The ƒbook imbalance is the difference over the total. It runs from −1 when only asks wait to +1 when only bids do.

In this book the answer depends on how far you look. On the first five levels the bids are heavier, 60 units against 38, an imbalance of +0.22. Across all 25 levels the asks hold twice as much, 312 against 156, an imbalance of −0.33. Both readings are right, and they answer different questions. The first levels are what the next small trades will meet. The deeper ones are what a large order would meet, and they are further from trading: whoever placed them can move or cancel them before the price gets there.

Send 60 units into lesson 5's thick book and the order walks eight levels. Afterwards the best ask is 100.07 and the best bid still 99.99, so the mid has jumped from 99.995 to 100.03. That jump is the peak of the order's impact.

Then the makers come back. New asks appear below 100.07, new bids above 99.99, and the spread narrows again. Where the mid settles depends on what the makers conclude from the order. Someone just bought 60 units in a hurry. Maybe they knew something. Makers can't tell, so they shade their prices up a little: in this toy, a quarter of a tick for every 10 units bought.

Send the order and watch. The book is on the left, asks above and bids below, with outlines where your order took. The chart follows the mid for 30 seconds.

With the figure's starting settings, 60 units lift the mid 3.5 basis points. Makers refill, and it slides back to 1.5 above where it started, and stays there. The 2 basis points that came back are the temporary impact: the cost of emptying the book, paid by the buyer and gone once the book refills. The 1.5 that stayed is the permanent impact, a change in what everyone thinks the price is.

Slow the makers down and something else shows. If they close half the gap every 20 seconds, the same order still shows 2.2 basis points after 30 seconds, of which only 1.5 will stay. Measured then, more of the move looks permanent than really is. How much of a move stays depends on when you look, and every measure of it has to pick a moment.

You rarely see one big order on its own. You see thousands of trades of every size, buys and sells mixed, each moving the price a little. Albert Kyle's 1985 model of trading gives a way to read them together. In it, makers move the price by a fixed amount for every unit bought or sold in a hurry. Plot each trade's size against the move that came with it, and the slope of the line through the dots is that amount. It is called ƒKyle's lambda.

The figure plots 120 toy trades. Each dot is one: its size across, counted plus for a buy and minus for a sell, and the move that came with it up the side. Make the book thinner or thicker.

With 10 units at every level, the slope is 0.053 basis points per unit: a 10-unit buy comes with about half a basis point. Halve the depth to 5 and the slope doubles, to 0.106. Quadruple it to 40 and the slope falls to 0.013. A steep line means a thin market, where every unit traded pushes the price further.

Yakov Amihud proposed a coarser measure in 2002, for stocks, a day at a time: take how far the price moved over the period, whichever way, and divide by how much traded. ƒAmihud illiquidity needs nothing but prices and volume. On the toy it reads the same way as the slope: 0.91 basis points for every 10,000 coins traded with 10 units at each level, twice that with 5. Its blind spot is a period where buyers and sellers cancel out. The price barely moves over it, so it looks liquid however thin the book is.

Lambda and Amihud say how hard trades push. Neither says how much of the push comes back. For that, watch the price after the biggest trades. The ƒlarge-trade reversal takes the biggest tenth of trades by value, looks five trades ahead of each, and averages the return from the big trade's price to the later one.

Here is a toy tape of 50 trades on lesson 5's book. Five are big buys of 60 units, ringed. Choose how many trades ahead to look.

One trade on, the price sits about 1 basis point below where the big buys printed. Five trades on, it is 1.67 below. The big buys printed 3.8 basis points above the mid before them, half the spread plus the walk, so by five trades on about 44% of that had come back.

A negative number after big buys, or a positive one after big sells, means part of the move came back. The measure doesn't flip sells around, so a minute with big trades on both sides can average out to nothing. Read it next to which side the big trades were on.

Every contract has two sides. Whoever bought it is long and gains when the price rises; whoever sold it is short and gains when the price falls, without ever having owned a coin.

The perpetual is measured against an index, a price for the coin built from spot exchanges. At set times the two sides settle up with a payment called funding: every eight hours by default on Binance, every hour on Hyperliquid. When the perpetual trades above the index, longs pay shorts. When it trades below, shorts pay longs.

The payment is the position times the ƒfunding rate. In this toy the rate is simply the premium, the gap as a share of the index: at 0.10% above it, a long holding $10,000 pays $10 at each funding time. Binance and Hyperliquid average the premium over the period and add a small interest term.

While longs pay shorts, a trader can buy a coin on a spot exchange and sell the same amount of the perpetual. One side gains what the other loses, so the trader carries no price risk and collects funding for as long as the premium lasts. This is an arbitrage, and its selling pushes the perpetual's price down toward the index. The next figure starts a week with the perpetual 0.30% above its index.

With nobody arbitraging, the premium sits at 0.30% all week and a long holding $10,000 pays $630 in funding. With three arbitrageurs it's down to 0.03% by the end of the week, and the long pays $246. It never quite reaches zero, because arbitrageurs stop once funding no longer covers their fees and the money tied up in the coin.

The gap between a perpetual's price and its index is its ƒbasis. Funding charges whichever side holds the price away from the index, and pays whoever helps close the gap.

Every contract has one long and one short, so counting either side counts the contracts still open: the ƒopen interest. Each trade has a buyer and a seller, and each of them is either opening a position or closing one.

When both sides open, a contract is created. When both close, one ends. When one opens and the other closes, a contract changes hands and the count stays put. Volume counts every contract traded, so a busy day can leave open interest exactly where it started.

A perpetual asks only for a deposit, the margin, and the venue lets you hold a position several times bigger. Hold $10,000 of bitcoin on $1,000 of margin and your leverage is 10×: every 1% the price moves is 10% of your money.

If the losses use up the whole margin, the venue closes the position for you before anyone else has to cover them, selling at whatever the book will pay. That is a liquidation. In this lesson's toy it happens when the loss reaches the margin: after a 10% fall at 10×, or 5% at 20×. Binance, for one, steps in earlier, at a smaller amount it calls the maintenance margin (its liquidation rules), and it judges the loss by a mark price rather than the last trade, so one odd trade can't liquidate anyone (how Binance builds it).

A liquidation is a forced sale, and a forced sale walks the price down the book (lesson 5), where it can reach the next trader's liquidation price. Below, 20 traders are all long from 100, each with their own leverage.

At 2× and 5× nobody's liquidation line is within reach of 96. At 10× the highest line sits just above 96, and each forced sale pushes the price into the next line until half the crowd, 10 of 20, is gone and the price is at 91. At 20× the chain runs through everyone and ends at 86. The big seller moved the price by 4 every time; what changed was how much everyone had borrowed.

Each liquidation closes a long. When the buyer on the other side is a short closing out, a contract disappears, so a cascade can show up as open interest falling sharply while the price falls.

So over a weekend the perpetual has nothing live to follow. If news about the company breaks on Saturday, the shares can't react until Monday, but the perpetual can. The people trading it have to work out the new price among themselves.

Here is a toy weekend for a stock that closed at 200 on Friday. Pick what happens, then run it from Friday's close to Monday's open, and watch the gap between the perpetual and the stock's last price.

On a quiet weekend the perpetual drifts and the gap stays under half a percent. With good news on Saturday it jumps about 6% and stays up; on Monday the stock opens at 211.20, close to where the perpetual spent Sunday, and the gap closes. With a rumour that is denied on Sunday, the perpetual drops 6.8% and climbs most of the way back, so anyone who sold in a hurry on Saturday sold at a price the stock never traded at.

In this toy the weekend's gap was news the stock hadn't been able to price yet, and the perpetual got there first. It doesn't have to be that way. With fewer people trading, a handful of orders can push the price around (lesson 6), and Monday's open is the first time the weekend's price is checked against the shares.

That leaves the venue with a problem. Funding pulls a perpetual toward its index, and an index built from the stock's price sits still all weekend. Each venue settles this its own way, and a contract's specification says what its index does while the stock's market is closed. The figure tries two simple choices on the good-news weekend: watch the perpetual, the index, and the funding a long pays.

If the index holds Friday's close, the basis is the whole weekend move, and in this toy a long holding $10,000 pays $300 in funding for a price everyone knows is out of date. If the index follows the venue's own trading, the basis stays small and so does funding, but nothing outside the venue checks the price until Monday. Either way, a stock perpetual's basis on a Saturday means something different from its basis on a Tuesday.

In this toy, fewer market makers quote at the weekend, and the ones who do have a harder job. A market maker who sells you the perpetual on a Tuesday afternoon can buy the shares on Nasdaq a moment later and be back to holding nothing. On a Saturday they can't, so they carry the risk until Monday. They quote less, and further from the mid (lesson 2).

Below is the same toy perpetual's book on a Wednesday afternoon and on a Saturday. Choose how much you want to buy straight away, and see what it costs over the mid.

On Wednesday, $25,000 fits inside the best ask and costs 0.5 basis points over the mid. On Saturday the spread is ten times wider, and the same order walks four levels up the book (lesson 5) for 11.6 basis points. Nothing about the contract changed, only who was around to trade it.

Here is one coin on three venues. It is worth 100 dollars on all three, so a basis point is one cent, but their books differ. Venue A has the narrowest spread between its best bid and best ask, and little waiting at each price. B's spread is wider, with one big queue at its best ask and little behind it. C's spread is the widest, and the book behind its best ask is the thickest.

You want to buy. Set the size of your order and see what it pays on each.

Small orders go to A. They fit in its first few prices, so they pay little more than half its narrow spread: $50,000 costs 1 basis point above the mid on A, against 3 on B and 5 on C. At $500,000, A's thin book has run out of cheap prices, and B's big queue fills the whole order at 3. At $5 million every first queue is gone, and C, the venue with the widest spread, is cheapest by far: 9.9 basis points, against 22.8 on B and 50.5 on A. Which venue is cheapest depends on how much you trade.

You can't walk every venue's book before every order, and you rarely see the whole book anyway. What you want instead is a few numbers that sum up each book, each one a question you could ask of any market. Measure three of them on the same three books; the third buys $1M and watches where the mid ends up.

Cost is the spread in basis points of the mid: what you lose if you buy and sell again at once. A market order pays half of it, so lower is better, and A leads with 2.

Depth is the dollars waiting at the best bid and the best ask. It tells you roughly how big an order can be before it pays more than half the spread. More is better, and B leads with $1.2 million.

Impact is how far $1M of net buying moves the price, in basis points, and lower is better. It sees the book behind the best prices, which Depth can't. Behind B's big first queue the book is thin, so $1M moves its mid 2.5 basis points, while C's thick book holds it to 1.

No venue leads on all three, so which one matters depends on your order. The next figure shows where each takes over.

These are the three Aperiodic indices: the Cost Index, the Depth Index and the Impact Index. Each is worked out the same way for every perpetual it covers, so bitcoin, a small coin and a perpetual on Tesla or gold sit on one board. On real data the Impact Index doesn't push $1M into a book. It estimates Kyle's lambda from the trades themselves (lesson 6) and scales it to $1M of net taker flow.

To see where each number takes over, try every order size from $10,000 to $10 million and note the cheapest venue at each. Then make the queue at B's best ask bigger or smaller, and search again.

The cheapest venue changes twice. Up to $250,000 the order fits in A's first prices and the spread is nearly all of what it pays, so the Cost decides. From there to about $1.6 million B wins, because its big first queue fills the order, or a large part of it, at 3 basis points: the Depth decides. Past that every first queue is used up, and what counts is how slowly the price climbs as you keep buying. The Impact decides, and C wins.

Give B a bigger queue and its stretch grows: with $1.5 million at its best ask, B stays cheapest up to $3.3 million. Past that, C still wins. Depth counts only the first queue on each side, and Impact is the number that tells you about the rest.

Here is a toy strategy on six toy coins over four months. Every midnight it buys the two coins that rose most the day before, and sells them at the next midnight. If one falls 8% below what it paid, it sells that one at once, with a stop.

The grey line is its money on point-in-time data. Put a fault into the data and run it again: the orange line is what the backtest reports.

On point-in-time data the strategy loses 11% over the four months. Each fault on its own turns that loss into a gain, and with all three the backtest reports a gain of 53%. The strategy is the same in every run. Only what the backtest knew has changed.

A close that is an hour late gives the signal the first hour of the day it trades. The coins that rose most “yesterday” are partly the coins rising right now, and the backtest buys them at the midnight price, from before that hour. It even sees coin 6 open 45% down after its delisting news, in time to skip it. This is look-ahead bias, and any input stamped earlier than you could have had it does the same.

A recorder that falls behind when trading is busiest drops trades, and the busiest minutes are often the sharpest moves. In the toy, five of them set off the stop on a coin the strategy held, and each time the coin ended the day above its stop. Without those minutes in the data, the backtest never sells, and books the recovery. In real life the stop sold, 8% down.

A list of coins made today holds only the coins that survived. Coin 6 jumped, then collapsed when its delisting was announced in the third month, and it is not on today's list. So the backtest never bought it the night before the crash, and never took the loss. This is survivorship bias.

This lesson is a short version of a course of its own. Building market data, which comes later, is about how market data is built and kept right.