KILLED · overfit stack
Hypothesis
Stacking many weak predictors under a meta-learner yields a strong combined return forecast.
Math
$$ \hat y = g\big(f_1(\mathbf{x}),f_2(\mathbf{x}),\dots,f_m(\mathbf{x})\big) $$
Method
Stack LSTM, XGBoost, logistic, GP base learners under a meta-regressor with nested cross-validation.
Results
| Cross-val improvement | marginal |
| Out-of-sample (true holdout) | worse than best single |
| Leakage via the stack | hard to fully avoid |
Stacking amplifies overfitting and leakage when the base learners share near-zero-signal targets — the meta-learner fits the validation noise. On a true holdout it underperformed the single best model. Complexity tax with negative return. Killed.
Ensembling weak-or-zero-signal models mostly ensembles their overfitting. Stacking is not a substitute for a target that has signal.