Jim:
That's not to say it's impossible to find a new and useful insight this way, but it's a very thorny patch in which to be hunting.

I understand what you are saying and agree it is a complex problem to design models that don’t memorize the past but generalize on the methods that work post discovery. But ML has been working on techniques to minimize this for a long time. That’s not to say it still can’t be a problem but much less likely than just a few years ago.

In my models I trained on different 5 year periods of data and tested on following 2 years after a gap to prevent data leakage from data they had not seen. over 100’s of times they have in over 85% of the time performed well on the unseen data and never worse than average. And no I have never had to retrain them.

Marcos Lopez De Prado’s “Advances in Financial Machine Learning” has the best overall coverage of the problems making ML work post learning.

But from where the rubber hits the road I’ll have to acknowledge that ML doesn’t have a stellar history with capital management firms.

I’m still in the experimental stage, before I throw real money at this game I’ll have some better evidence it’s working post prediction.

Rob