KILLED · no improvement
Hypothesis
Self-attention captures long-range dependencies a recurrent net misses; a Transformer should beat the LSTM (N-038) on directional forecasting.
Math — scaled dot-product attention
Attention re-weights value vectors by query–key similarity:
$$ \text{Attn}(Q,K,V) = \mathrm{softmax}\!\Big(\frac{QK^\top}{\sqrt{d_k}}\Big)V $$
Method
Encoder-only Transformer, same feature set and walk-forward protocol as N-038, tuned over depth/heads.
Results
| Direction accuracy | 50.9% |
| vs LSTM baseline | ≈ identical |
| Overfitting on small crypto history | severe |
A better function approximator on a target with no signal gives a better fit to noise, not better forecasts. No improvement over the LSTM, and worse overfitting given limited history. Killed.
Model capacity helps only when there is structure to capture. On near-random targets, more capacity buys more overfitting, not more edge.