Online Learning : Stochastic Approximation

1139 WordsNov 1, 20165 Pages
4 Online learning: Stochastic Approximation Estimating the mixing density of a mixture distribution remains an interesting problem in the statistics literature. Stochastic approximation (SA) provides a fast recursive way for numerically maximizing a function under measurement error. Using suitably chosen weight/step-size the stochastic approximation algorithm converges to the true solution, which can be adapted to estimate the components of the mixing distribution from a mixture, in the form of recursively learning, predictive recursion method. The convergence depends on a martingale construction and convergence of related series and heavily depends on the independence of the data. The general algorithm may not hold if dependence is present. We have proposed a novel martingale decomposition to address the case of dependent data. 5 Measurement error model: small area estimation We proposed [4] a novel shrinkage type estimator and derived the optimum value of the shrinkage pa- rameter. The asymptotic value of the shrinkage coefficient depends on the Wasserstein metric between standardized distribution of the observed variable and the variable of interest. In the process, we also estab- lished the necessary and sufficient conditions for a recent conjecture about the shrinkage coefficient to hold. The biggest advantage of the proposed approach is that it is completely distribution free. This makes the estimators extremely robust and I also showed that the estimator continues to
Open Document