Computer science undergraduate at UMass Amherst. I work on reinforcement learning that stays reliable when the environment changes underneath it.

Most RL results assume the world stands still. Deployed systems don't get that assumption, and the gap between the two is where I spend my time. In practice that splits into proving what a method can and cannot do — regret bounds, sample complexity, and the occasional impossibility result — and building the benchmarks and instrumentation that test whether the theory survives contact with a real training run.

aarav-shah.com · aarnshah@umass.edu · LinkedIn

Quantifying detection bias and recovering habitable zone occurrence rates in the NASA exoplanet archive Aarav Shah, Saummya Salve, Shreyansh Joshi. Astronomy and Computing 57, 101170 (2026). doi.org/10.1016/j.ascom.2026.101170

Audit or Early Warning? A Benchmark for Online Detectors of Reward Hacking

Practitioners hope to catch reward hacking mid-training from signals the trainer already emits — evaluator disagreement, entropy collapse, KL drift. I built a benchmark that induces hacking on demand and measures true quality with a held-out programmatic oracle, which splits the problem cleanly. Hacking is easy to identify after the fact: disagreement separates hacked from benign runs at AUROC 0.98. But nothing gives early warning — every detector's true-positive rate is 0 before quality has already degraded. Disagreement is a consequence of hacking, not a predictor of it.

Draft available on request.

rhob — Reward Hacking Onset Benchmark. Induces hacking on demand, measures true quality with a held-out oracle, and scores any detector per step for lead time and specificity. The result holds across two failure families, two optimizers (GRPO, RLOO), two model families (Qwen, Gemma), and two scales, and regenerates from frozen, hashed data. This is the benchmark behind the paper above.

flight-recorder — Onset-detection diagnostics for GRPO post-training. Hooks into an existing trainer and streams rollout geometry — KL acceleration against the frozen reference, entropy, advantage statistics — as the run proceeds.

inventory-quantum — Offline-RL inventory system. A Temporal Fusion Transformer produces demand distributions, QAOA handles shelf allocation as a QUBO, and Conservative Q-Learning keeps the replenishment policy inside the support of the logged data.

retail-forecasting-benchmark — Regime-dependent benchmark of ARIMA against modern forecasting methods on small-scale retail demand. Which method wins turns out to depend on the demand regime rather than being universal.

AI/ML engineering intern at SAP, building production ML for enterprise workforce systems. Previously at UC Riverside, where I worked with Prof. Saragadam on hyperspectral reconstruction and founded BRIDGE, an interdisciplinary undergraduate research initiative. Silver medal at the 5th International STEM Olympiad, Frankfurt.

What I keep going back to, if you want the shape of the problem:

• Reinforcement Learning for Non-Stationary MDPs: The Blessing of (More) Optimism — Cheung, Simchi-Levi & Zhu
• Non-stationary RL without Prior Knowledge: An Optimal Black-box Approach — Wei & Luo
• Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning — Muppidi, Zhang & Yang