The Benchmark

6.1 Why realized volatility is the sharpest possible test

Most "do these features help?" questions are vague, because the feature family and the target have no particular relationship and you are really just asking whether a large pile of numbers happens to correlate with something. Realized volatility is different, and it is the reason this benchmark was chosen as the headline study rather than a daily-bar one.

§1.5 established that the lead-lag transform makes quadratic variation visible at level 2. Stated precisely: for the lead-lag embedding of a scalar path, the antisymmetric part of the second level — the Lévy area, doubled — equals the path's discrete quadratic variation exactly.

Not approximately. On a 200-point random walk, computed through rollsig's numpy backend:

QV               184.8037480539841
S12 - S21        184.8037480539842
ratio            1.0000000000000007

This makes the benchmark's question unusually clean. Realized volatility is not something bolted onto signature features from outside; it is a coordinate of the feature vector. The classical estimator is contained in the family by construction. So the question is not the mushy "are signatures useful for volatility" but the sharp one:

The Optiver Realized Volatility Prediction dataset supplies the rest of what a real test needs: genuine order-book paths for 112 stocks, roughly 3,830 ten-minute segments each, a fixed public metric, and a body of published top solutions to reproduce as baselines rather than invent.

6.2 Two backends, two bases

The first thing this benchmark did was make the v0.1 backend choice untenable. Feature building needs one log-signature per segment per window: 20 stocks × ~3,830 segments × 3 windows is roughly 230,000 calls, before the depth study multiplies it again. Timed on a 600-point lead-lag, time-augmented path (1,199 points, dimension 4):

At RoughPy's depth-3 rate the feature build alone is 47 minutes single-threaded; at iisignature's it is 25 seconds. That is the difference between a study you can iterate on and one you run overnight and stop questioning.

So iisignature became a second log_signature backend. It did not go in quietly, because the two backends do not return the same numbers — and this is the interesting part.

iisignature  ... -3.27487 -1.32135 -2.54993 -0.65885  1.14639 -0.39506
roughpy      ...  3.27487  1.22858  1.14639  1.32135  0.65885  0.39506

Silently accepting that would be a real hazard — a model fitted on one backend's columns and scored on the other's would produce plausible-looking garbage. Two things were done instead.

First, a basis-independent oracle. _log_signature_expanded maps either backend's output back into canonical tensor-word coordinates — via iisignature's expanded method for one, ctx.lie_to_tensor for the other — where the redundant-but-canonical representation makes them directly comparable. There, both must equal _log_signature_full_numpy, which computes log(S) as a power series and involves no basis at all. Both do, to 1e-8. That is what establishes that each backend really is returning a log-signature and not something merely log-signature-shaped.

Second, a test that asserts the disagreement itself: level 1 and 2 must match, level 3 must not. If a future release of either library quietly changed convention, that test fails loudly rather than the benchmark drifting.

RoughPy stays the default for log_signature, matching signature; the benchmark asks for iisignature explicitly. The column names come from whichever backend produced them, so get_feature_names_out can never mislabel.

6.3 An irregular stream, and why forward-filling is free

The raw book data is an event stream: a row appears only when the book changed, so seconds_in_bucket is sparse and irregular. Everything downstream wants a regular one-second grid, which means forward-filling — and forward-filling data before feeding it to a model is normally the sort of thing that deserves suspicion.

Here it is provably free, and the reason is §5.1. The signature is invariant under reparametrization: subdividing a segment, or repeating a point, does not change it. Forward-filling inserts no new geometry — it repeats the standing price until the next update, tracing the identical trajectory. The price path the signature sees is literally unchanged.

What does change is the time channel added by time_augment, and it changes for the better. On the raw event stream, a uniformly spaced time channel measures event count — it would say two quiet minutes and two frantic ones took the same amount of time. On the filled grid it measures real elapsed seconds. So the resampling is not a compromise made for convenience; it is what makes the time augmentation mean what it is supposed to mean.

7.1 The arms, and the one thing allowed to vary

A feature-comparison benchmark is worth exactly as much as its control. If the signature arm gets a different learner, different hyperparameters, different folds or even a different row ordering than the baseline, then a favourable number says nothing about signatures.

So: one learner (HistGradientBoostingRegressor, fixed hyperparameters, no tuning), one set of folds, one feature table built in a single pass so every arm sees byte-identical rows in identical order. The only thing that varies between arms is which columns are selected.

Those last three matter more than the headline. A general feature family beating a specialised baseline outright is rare and would be a surprising claim; adding information at the margin is the realistic and still-useful outcome, and it is the one the combined arms measure. Note also that sig is given the same multi-horizon window structure as har, so the comparison tests the feature family rather than the horizon choice.

stock_id goes to every arm equally — each stock has its own baseline volatility level, and withholding that would handicap all arms identically but pointlessly.

7.2 RMSPE, and training on the metric rather than near it

The competition scores root mean squared percentage error. Dividing by the truth means a miss on a quiet segment counts as heavily as the same relative miss on a violent one — which is the right call for volatility, where the quantity ranges over orders of magnitude and a proportional error is what a desk actually cares about.

Training on raw squared error would therefore optimise something the model is not scored on: it would lavish attention on the high-volatility segments where absolute errors are large and neglect the quiet ones entirely. The fix is a one-liner and worth spelling out because it is exact rather than a heuristic:

Weighting each sample by the reciprocal of its squared target turns ordinary squared error into precisely the competition metric. The learner optimises what it is judged on, in every arm equally.

Early stopping is switched off deliberately. scikit-learn's internal validation split is random, and a random split would cut straight across the time_id grouping that the next section exists to protect — a leak smuggled in through a convenience flag.

7.3 The leak that matters — and the split that isn't possible

The plan for this benchmark asked for strict walk-forward cross-validation. On this dataset that is not achievable, and saying so is more useful than pretending otherwise.

The organisers deliberately shuffled and anonymised time_id, so its numeric order is not chronological. There is no timestamp anywhere in the shipped data. A genuine walk-forward split — train on the past, test on the future — cannot be reconstructed from what the competition provides. (Some competitors did reconstruct an ordering from cross-stock volatility correlations; that is a competition-specific exploit, not a foundation for an honest benchmark, and it is out of scope here.)

What is both reconstructable and essential is grouping. A time_id is one instant of market time observed simultaneously across all 112 stocks. Realized volatility is strongly correlated across stocks at the same instant — when the market is turbulent, it is turbulent for everyone. So if one stock's row from a given time_id trains a model that is then scored on a different stock's row from the same time_id, the model has effectively been shown the answer.

GroupKFold on time_id closes that, and a test asserts it directly rather than trusting the class name:

for train_idx, test_idx in evaluate.folds(groups, n_splits=5).split(X, y, groups):
    assert set(groups[train_idx]).isdisjoint(groups[test_idx])

7.4 Causality as a test, not a claim

"No lookahead" is the easiest claim in quantitative research to make and the easiest to be quietly wrong about. Part I tested it for the rolling transformer; the benchmark needs its own version, because the failure mode here is different — an off-by-one in a window slice, or a groupby that silently spans segments.

The test is adversarial rather than assertive. Build one stock's features. Then rebuild them from a dataset in which every later segment's book prices have been overwritten with garbage, and demand that the earlier segments' features are unchanged:

book.loc[book["time_id"] >= N // 2, price_columns] = 999.0
...
pd.testing.assert_frame_equal(original.features.iloc[early], corrupted.features.iloc[early])

If any feature reaches across segment boundaries, corrupting the future moves the past and the test fails. On real data that reach would be a reach into the future.

Two smaller traps get their own tests for the same reason. A suffix-window test confirms the 300-second window genuinely ignores everything in the first 300 seconds — the kind of thing an off-by-one turns into a silent full-window duplicate. And a test asserts the 600-second and 150-second signature blocks actually differ, which would catch a slicing bug that turned the multi-horizon arm into three copies of the same numbers wearing different column names.

8.1 The table

20 stocks, 76,599 ten-minute segments, depth-3 log-signatures over 600/300/150-second causal suffix windows, five grouped folds, one shared learner. The whole feature table — every arm's columns — builds in 21 seconds.

First, the sanity check: the reproduced baselines work. book+har beats the naive persistence predictor by 31.0% (bootstrap CI on the absolute delta [+0.0978, +0.1098], p = 0.000). Whatever the signature arm is being compared against, it is not a strawman.

Now the actual answer, in three parts.

Signatures alone nearly match hand-designed volatility features. 91 log-signature coordinates reach 0.24362 against the 27-feature HAR-RV set's 0.24075 — 1.19% worse. What makes that interesting is not the sign but the size: the signature arm was never told what realized volatility is. It received the geometry of a price path and a truncation depth. A generic construction landing within about one percent of features designed specifically for this target is the genuinely notable number in the table.

They add a little to a weak baseline. sig+har improves on har by 0.40%. So the signature coordinates are not pure redundancy with respect to multi-horizon RV; there is something in them that HAR-RV does not have.

They add nothing to a strong one. sig+book+har scores 0.23133 against book+har's 0.23093 — 0.17% worse, with the group bootstrap putting the whole 95% interval on the wrong side of zero (p(no improvement) = 0.992). Against the order-book baseline, 91 extra columns are cost without benefit.

8.2 Depth and window length: where it turns over

Signature feature count grows fast in the truncation depth — the free Lie algebra on d generators has roughly dⁿ/n independent brackets at level n. The question is where the extra coordinates stop describing anything a ten-minute volatility forecast can use. Same folds, same learner, signature columns only.

Depth turns over at 3. Going from depth 2 to depth 3 buys a real improvement (0.24510 → 0.24362 on the three-window set). Going from 3 to 4 triples the feature count — 91 to 271 — and moves RMSPE by 0.00008, which is a thirtieth of the fold standard deviation. Level-4 brackets are describing path structure that this target cannot use.

The most useful line in the table is a coincidence of feature counts. Depth 4 over a single 600-second window and depth 3 over three nested windows both produce exactly 91 features. They do not perform alike:

Windows show diminishing returns of their own. Adding the 300- and 150-second suffixes to the 600-second window is worth 1.4% at depth 3; adding a 60-second window on top of those is worth a further 0.06%, comfortably inside fold noise. Three horizons is where this task saturates.

8.3 What this licenses claiming, and what it doesn't

The rule was fixed before any number existed, which is the only time such a rule is worth anything. The private leaderboard was rescored on market data from after the competition closed, so no result obtainable from the training data can be turned into a leaderboard position, and none is. Cross-validation here is grouped, not walk-forward (§7.3), so nothing here simulates deployment across time.

There is a second limit, and it is the more important one because it cuts against the conclusion rather than for it.

So the defensible claim is narrower than "signatures don't help here": signatures of the WAP path alone do not beat order-book features. The experiment that would settle the wider question is a multi-channel signature arm carrying spread and imbalance as additional path channels — at which point both arms see the same information and the comparison is clean. That is the obvious next thing to run, and it is deliberately not being run retroactively to rescue the number.

Worth noting what §6.1 bought by framing the question sharply in advance. Because the lead-lag identity puts quadratic variation inside the feature family at level 2, a null result is not an absence of evidence. It says something specific: on ten-minute volatility forecasting, the path detail living above level 2 is either not predictive or is already captured by cheaper order-book statistics. That is a finding about how much path geometry this task can use, and it would have been the write-up either way.

9.1 The redundancy this benchmark pays for three times over

Building the signature arm made the case for a streaming engine concrete in a way the abstract argument for one did not.

Each segment's features come from three nested suffix windows: the last 600 seconds, the last 300, the last 150. The 150-second path is a suffix of the 300-second path, which is a suffix of the 600-second path. Every one is currently computed from scratch, so the final 150 seconds of price data get traversed three separate times, and the total work is 1050 points of path per segment where 600 would do.

Chen's identity (§1.6) says that is unnecessary. The signature of a concatenation is the tensor product of the signatures, so the signature over a suffix can be obtained from the full window's signature by left-multiplying by the inverse of the departing prefix's signature — the group structure of the tensor algebra, doing exactly the work it exists to do. That is the v0.3 streaming engine, and this benchmark is what quantifies its value: not a speedup measured on synthetic paths, but a measured fraction of a real feature pipeline.

The other thing v0.2 hands forward is a caution. The nested-window redundancy is arithmetically removable; whether it is removable at acceptable numerical cost is a separate question. Repeated multiplication by group inverses at depth 4 compounds floating-point error in a way that recomputation does not, and the higher levels are precisely where the coefficients are smallest. That was flagged from the start as something to investigate and document rather than assume away. The oracles from Part I are what will settle it: whatever the streaming engine produces has to match a from-scratch recomputation to a stated tolerance, or the speedup is not real.