Love to hear about people's experiences with ML, some going back further than me :)

My first NN projects were in the 90s. I remember Brainmaker, but I think the best package was SNNS:
Wikipedia: SNNS - Wikipedia

Development on SNNS stopped by the late 90s, but it was the best thing available until 2010 or so.

From 2001-2010 I used NNs I developed for all my retirement account investments, similar to what others here were doing with MI. I also had similar results - about 24-29% CAGR. Until 2008, when things went south, including for my NN models. I gave up on those in 2011.

I've been working in Python for about 8 years. I've developed some XGBoost models on time series data that are better than a coin toss, which doesn't necessarily make them actionable. I tried RNNs on the same data, thinking they would do better with the time series. But the so far the XGBoost models have performed better. That seems to be consistent with this persons experience:
towardsdatascience.com - Why you currently do not need deep learning for time series forecasting.

I'm not sure a backtest on data more than a couple years back has much value. If it hasn't worked in the last two years I'd assume the gig is up on that model. If I found something that worked, I'd expect it would be arbitraged away in a year or two, and consider myself lucky if it continued to work after that.

John