Posts by Yicheng Yang

Streaming variational inference on high-frequency tick data

Exchanges publish per-trade archives; Binance, for instance, distributes aggregate trades as daily and monthly files at data.binance.vision. Widen a pull across symbols and months and the feature matrix can outgrow the RAM of an ordinary workstation; at that point in-memory minibatching stops being an option. PyMC’s Minibatch() randomly slices tensor inputs; it is not itself a disk-backed reader, so it cannot help once the array no longer fits. This notebook fits a hierarchical hurdle–Student-t model of next-event price moves by streaming minibatches from disk with pymc-extras’ DataLoader, on 300,000 synthetic rows that are generated inside the notebook. It tries to teach three things:

Read more ...