Prediction Market Data Downloads: CSV, Parquet and API Exports
A downloadable dataset is useful only if its format preserves the evidence the analysis needs. Flattening a ladder into one price can make a small file and a dishonest backtest.
Use CSV for inspection and small flat extracts, Parquet for large columnar research and REST for bounded or incremental queries. Whichever delivery method you choose, keep market identifiers, source and receive timestamps, bid/ask price-size arrays, settlement fields and explicit gaps. The format should change how bytes are delivered, not the meaning of the recorded market.
Match the format to the workload
| Format | Best for | Watch for |
|---|---|---|
| CSV | Manual inspection and small interoperable samples | Nested ladders require a documented encoding or child table |
| Parquet | Large scans, column pruning and research warehouses | Schema evolution and partition keys must be stable |
| REST JSON | Bounded queries, applications and incremental pulls | Pagination, limits and retry semantics |
| WebSocket | Current updates and live systems | Reconnects, sequence gaps and complete-book seeding |
Fields not to lose
- Venue, event, market and outcome identifiers in their native and normalized forms.
- Exchange/source time and receive/observation time as separate fields.
- Bid prices, bid sizes, ask prices and ask sizes at matching array positions.
- Capture method, interval or event sequence needed to interpret cadence.
- Settlement outcome and reference values stored without leaking into earlier rows.
- A stable null or missing convention that never silently becomes zero.
Partition for the questions you actually ask
Large depth archives should be partitioned by high-selectivity fields such as venue and date, then by a stable market or asset key where it matches the workload. A million tiny files can be as costly as one unpartitioned export, so measure the query engine rather than copying a generic lake layout.
Keep a manifest beside bulk exports. It should record schema version, generation time, requested filters, row or observation counts and any incomplete partitions. That manifest turns an opaque download into a reproducible research input.
DepthFeed delivery choices
DepthFeed's regular research path is REST for filtered historical windows and WebSocket for current normalized frames. Desk workflows can use the full archive and columnar delivery patterns for large studies. The browser tools use the same underlying schema, so an idea can move from a sample to a reproducible backtest without redefining the book.
Start with the smallest representative slice: one venue, one market window and enough observations to test parsing, ordering and a sized fill. Scale only after those invariants pass.
Key takeaways
- 01CSV, Parquet, REST and WebSocket are delivery choices, not interchangeable data meanings.
- 02Preserve full price-size ladders and both source and receive timestamps.
- 03A bulk export needs a schema version and manifest to remain reproducible.
- 04Explicit nulls and gaps are safer than synthetic zeroes or fills.
- 05Validate one representative market before scaling a download.
Validate identifiers, timestamps and ladder shape on a small historical slice. Free Explorer tier, no card.
Get a data sampleView pricing