No. of Recommendations: 10
At what point can one be reasonably sure that good performance wasn't just the luck of the draw?
I'm disappointed that no one really is keen on digging deeper into this question.
The answer, to me, seems fairly obvious. Since we're dealing with probabilities, the answer is we can never be sure good performance wasn't just luck of the draw. All we can do is draw an arbitrary line in the dry, fine, wind-blown sand and say,"Beyond this point, I'm reasonably confident this algorithm's performance (or mine) is truly good and not merely lucky." And that line gets drawn not merely statistically but also where we're comfortable with the level of risk we're accepting.
Informing the objective part of that line-drawing is history, but there are too few cycles too assess, and too many macro factors to include, or arbitrarily exclude, or not recognize altogether, for those cycles, and--sometimes--"this time it really is different." With no way to tell in real time whether this is the time it's different.
And one more contaminant: since TMF ruined its site, a potful of folks who used to frequent, and contribute to, a Mechanical Investing board have simply left, rather than follow over to this site and this Mechanical Investing board.
Eric Hines
