Jane Street
Machine Learning

Jane Street is known for a rigorous, data-driven approach to machine learning in finance. Unlike tech companies where ML drives product recommendations, in quant trading ML is used to predict price movements, identify mispricings, and optimize execution.

Key Principles

• Overfitting is the enemy: Financial data is noisy and non-stationary.
• Regime awareness: Models must adapt to changing market conditions.
• Interpretability matters: Black-box models are harder to risk-manage.
• Validation must be realistic: In-sample performance is meaningless; live trading is the only test.

Jane Street Kaggle Competition: https://www.kaggle.com/competitions/jane-street-market-prediction

Feature engineering is arguably more important than model architecture in financial ML.

Common Financial Features

• Price Features: Returns, volatility, momentum, mean reversion signals.
• Volume Features: Volume imbalance, VWAP deviation, trade count anomalies.
• Microstructure Features: Bid-ask spread, order book depth, trade flow imbalance.
• Cross-Asset Features: Correlations, beta to factors, sector momentum.

Feature Validation

• Information coefficient (IC): Correlation between signal and future returns.
• IC decay: How quickly predictive power fades over time.
• Turnover: How frequently the signal changes (affects transaction costs).

Resource: https://www.quantconnect.com/docs/v2/writing-algorithms/machine-learning/key-concepts

Linear Models

• Ridge/Lasso regression: Interpretable, robust to overfitting.
• Factor models: Decompose returns into known risk factors.

Gradient Boosting

• XGBoost/LightGBM: Standard for tabular financial data.
• Handles non-linearity without overfitting as easily as deep learning.

Deep Learning (Cautiously)

• LSTMs/Transformers: For sequence modeling of price data.
• CNNs: Pattern recognition in order book data.
• Limitation: Much more data required; harder to validate.

Jane Street's Kaggle competition famously used complex tabular models with careful feature engineering rather than deep neural networks.

XGBoost: https://xgboost.readthedocs.io/

Financial ML Validation Best Practices

• Walk-Forward Validation: Train on past, test on subsequent period.
• Purged Cross-Validation: Remove samples near the split to avoid look-ahead bias.
• Embargo: Skip samples after the split to handle autocorrelation.
• Multiple testing: Adjust significance levels for the number of models tested.

Overfitting Red Flags

• In-sample Sharpe >> Out-of-sample Sharpe.
• Performance degrades immediately after deployment.
• Feature importance concentrated on one or two features.

Advanced Technique: Combinatorial Purged Cross-Validation (CPCV) for robust backtest evaluation.

ML for Asset Managers: https://www.cfainstitute.org/en/research/foundation/2020/machine-learning-and-asset-management

Deploying ML Models in Production

• Shadow Mode: Run model alongside existing system, compare outputs without risk.
• A/B Testing: Allocate a portion of capital to the new model.
• Gradual Scaling: Increase allocation as model proves itself live.

Monitoring in Production

• Prediction drift: Alert when model outputs change significantly.
• Feature drift: Alert when input features fall out of distribution.
• Performance attribution: Understand where alpha is coming from.

Key Risk Controls

• Position limits: Maximum exposure per model.
• Loss limits: Automatic shutdown on drawdown breach.
• Correlated exposure: Monitor total ML exposure across models.

Resource: https://christophm.github.io/interpretable-ml-book/