Industry Expert Versus Generalist Financial AIs

Should those aiming to exploit machine learning for portfolio construction focus model training on the broad market or specific industries? In their April 2025 paper entitled “Do Machine Learning Models Need to Be Sector Experts?”, Matthias Hanauer, Amar Soebhag, Marc Stam and Tobias Hoogteijling examine return predictability using several machine learning (ML) models trained on a comprehensive set of firm/stock characteristics in three ways:

1. Generalist – trained on all stocks in the sample.
2. Specialist – trained on stocks only within one of 12 industry classifications.
3. Hybrid – integrates overall sample and industry information via industry-neutral mappings from stock characteristics to expected returns.

They employ four ML models, including elastic nets, gradient boosted regression trees, 3-layer neural networks and an equal-weighted ensemble of the three. They train and tune these models with an expanding window with an initial 18-year training set, 12-year validation set and 1-year test set, shifted forward each year but retaining the initial training start point. Input data consists of monthly stock returns and monthly values of 153 firm-level characteristics for U.S. stocks each month at or above the 20th percentile of NYSE market capitalizations . They assign stocks to the 12 industries (including Other), with average weights ranging from 22.5% for Tech to 1.4% for Durables. They then each month sort stocks into tenths (deciles) by machine learnings ensemble-predicted next-month return and reform a volatility-scaled, value-weighted hedge portfolio that is long the decile with the highest expected returns and short the decile with the lowest. Using the specified inputs during January 1957 (January 1986 for a non-U.S. sample) through December 2023, they find that:

(more…)