Rayvt: “Any backtest which doesn't include at least the 2008/2009 bear market will be telling you sweet nothings.”

I wouldn’t say sweet nothings, it wouldn’t protect you from another 2008 but it would have still helped you outperform at later dates. I just looked at how my ML/AI models trained from 2004 till end of 2007 did in 2008. You are right they didn’t outperform but neither did they underperform. Likewise when the covid crash happened my ML models went down just as fast as the rest of the market but the interesting observation it one of the best performance periods happened in the following recovery. It appears the ML models found mispriced equities when they were all down.

Ravet: “This is a prime example of survivorship bias. You never hear of the very intelligent individuals who used P123 and got only so-so returns.”

Good point. P123 has what they call designer models developed by individuals who believe they have found an outstanding system which you can subscribe to. The vast majority of these did not perform better than the SP500 post discovery, especially in the early period. So, I believe your observation appears to be correct. Developing a ranking system that outperforms is not an easy couple of hours a week project.

My point is P123 along with their community have developed tools to analyze how hundreds of individual factors have performed over the last 20 years, measure how much information they give you and how they correlate to other factors. Professional investment managers don’t pay out $6K a year for multiple years if they weren’t satisfied. Perhaps that isn’t the best logic as many more pay > $30K a year for a Bloomberg terminal.