Incorporating Audio/Video of Stock Analysis

Do AIs that incorporate audio and video (multimodal) aspects of firm/stock analysis (e.g. from YouTube), including tone, delivery style and facial expressions, distill better buy and sell recommendations from financial influencers (finfluencers) than text-only large language models (LLM)? In their May 2025 paper entitled “VideoConviction: A Multimodal Benchmark for Human Conviction and Stock Market Recommendations”, Michael Galarnyk, Veer Kejriwal, Agam Shah, Yash Bhardwaj, Nicholas Meyer, Anand Krishnan and Sudheer Chava test abilities of 16 text-only LLMs and six multimodal LLMs (MLLM) and to extract stock recommendations and associated levels of conviction from a benchmark dataset. This dataset consists of 288 finfluencer videos of 12 minutes or less from 22 YouTube channels during January 2018 through December 2024, each transcribed and annotated by five human experts. These experts identify 687 unique stock recommendation segments in these videos and assign a level of conviction to each. The authors then perform two sets of tests on this benchmark dataset:

• The ability of each LLM/MLLM to mimic human expert analyses with respect to identifying recommended stock tickers, recommended actions (buy or sell) and the level of finfluencer conviction for each recommendation.
• The inherent value of finfluencer recommendations by comparing performances of the following three active portfolios to those of Invesco QQQ Trust (QQQ) and SPDR S&P 500 ETF (SPY):
  1. Buy & Hold: hold each Buy recommendation for six months with equal weights.
  2. Buy & Hold (Weighted by Conviction): hold each Buy recommendation for six months, with positions weighted by relative conviction (position conviction divided by the sum of all position convictions).
  3. Inverse YouTuber: do the opposite of recommendations by selling Buys and buying Sells with equal weights and holding each position for six months.

Using the specified benchmark dataset, they find that:

(more…)