anchak;

Thanks for the suggestions.
I too have been using ML for a while. Way back in 1986 the company I was working for became aware that IBM was in competition with us using ML. A group of us were sent off to learn the capabilities of ML. We ended up developing our own array processors but, in the end, IBM outperformed us. So, way back in the late 80’s and early 90’s a significant percentage of the current ML algorithms were known but limited in capability by the processing capabilities and supporting software.
I have been using NN’s for my investing since 1999 so I am well aware of data mining over fit.
However I haven’t been working in the field for over 20 years so have only kept up by playing at home.

I am using time sample cross validation with a gap and holdout. But I have found that rolling-cross/validation results are not a good as simple time series CV, it appears that the longer past history gives the latter a slight advantage.
I previously thought that as the market is changing over time more recent data would be more valuable. But so far, the longer the training historical data the better results. Training from 2000 for 5 years and testing for 2 years after a gap isn’t as good as training for 17 years and testing for 2 after a gap. Almost always better with a longer training set.

Additionally I’m trying to get an Ensemble of models to build a portfolio.
Aurelien Geron “Hands on Machine Learning with Scikit-Learn, Keras & Tensor Flow” points out in chapter 7 voting classifiers that Ensemble methods work best when the predictors are as independent as possible.
Voting doesn’t seem appropriate to regrssors.

I’ll have to rethink treating where I train/test bull and bear markets, up to now I have been letting them fall into the arbitrary fixed time folds. It was surprising to me to see bear ML test worked on models trained in bull periods without any special considerations.

Thanks again for the advice, you defiantly are more up to date than I on the subject. As I said I’m going to reevaluate some of the methods I’m using. I’m still in the learning mode.

Rob