Optimization Dynamics
Comparing optimizers on VQA cost landscapes
Three optimizers — vanilla gradient descent, Adam (adaptive moments), and SPSA (simultaneous perturbation stochastic approximation) — race on the same landscape from the same starting point. Observe how Adam's per-coordinate adaptation navigates narrow valleys, how SPSA's stochastic gradients explore broadly, and how vanilla GD can get trapped.
Two symmetric minima separated by a saddle — tests whether optimizers escape local traps
iteration: 0
Click the landscape to set a starting point, then run all three optimizers simultaneously. Gradient Descent follows the raw gradient; Adam adapts per-coordinate learning rates; SPSA estimates gradients via random perturbations.