Objective research to aid investing decisions
Menu
Value Allocations for November 2019 (Final)
Cash TLT LQD SPY
Momentum Allocations for November 2019 (Final)
1st ETF 2nd ETF 3rd ETF

Stopping Tests after Lucky Streaks?

Posted in Big Ideas, Investing Expertise

Might purveyors of trading strategies be presenting performance results biased by stopping them when falsely successful? In other words, might they be choosing lucky closing conditions for reported positions? In the December 2018 revision of their paper entitled "p-Hacking and False Discovery in A/B Testing", Ron Berman, Leonid Pekelis, Aisling Scott and Christophe Van den Bulte investigate whether online A/B experimenters bias results by stopping monitored commercial (marketing) experiments based on latest p-value. They hypothesize that such a practice may exist due to: (1) poor training in statistics; (2) self-deception motivated by desire for success; or, (3) deliberate deception for selling purposes. They employ regression discontinuity analysis to estimate whether reaching a particular p-value causes experimenters to end their tests. Using data from 2,101 online A/B experiments with daily tracking of results during 2014, they find that:

Please or subscribe to continue reading...
Gain access to hundreds of premium articles, our momentum strategy, full RSS feeds, and more!  Learn more

Daily Email Updates
Login
Research Categories
Recent Research
Popular Posts