PARKED · sim-to-real
Hypothesis
A PPO agent learns to slice a parent order to minimize implementation shortfall versus TWAP/VWAP, adapting to live book conditions.
Math — clipped surrogate objective
PPO maximizes a clipped policy ratio against the advantage estimate:
$$ L^{CLIP}(\theta)=\mathbb{E}_t\big[\min(\rho_t A_t,\ \mathrm{clip}(\rho_t,1-\epsilon,1+\epsilon)A_t)\big] $$
Method
Train in a simulated LOB with a market-impact model; reward = negative implementation shortfall. Evaluated in-sim only.
Execution (not alpha) is the right place for RL — the reward is well-defined and dense. Parked on the sim-to-real gap: the agent overfits the simulator’s impact model, and we lack a faithful live LOB simulator to trust it with real orders. Promising, not deployable yet.
RL belongs to execution, not direction. But an RL agent is only as honest as its simulator — a wrong impact model trains a confidently wrong agent.