A stochastic approximation method that functions by placing conditions on iterative step sizes and whose convergence is
guaranteed under mild conditions. However, the method requires knowledge of the analytical
gradient of the function under consideration.
Kiefer and Wolfowitz (1952) developed a finite difference version of the Robbins-Monro method which maintains the nice convergence properties, while obviating the need for knowledge of the analytic form of the gradient.
Kiefer, J. and Wolfowitz, J. "Stochastic Estimation of the Maximum of a Regression Function." Ann. Math. Stat.23,
462-466, 1952.Robbins, H. and Munro, S. "A Stochastic Approximation
Method." Ann. Math. Stat.22, 400-407, 1951.
