We found a match
Your institution may have rights to this item. Sign in to continue.
- Title
Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise.
- Authors
Yaji, Vinayaka G.; Bhatnagar, Shalabh
- Abstract
In this paper, we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of nonadditive iterate-dependent Markov noise. We show that the recursion on each timescale tracks the flow of a differential inclusion obtained by averaging the set-valued drift function in the recursion with respect to a set of measures accounting for both averaging with respect to the stationary distributions of the Markov noise terms and the interdependence between the two recursions on different timescales. The framework studied in this paper builds on a recent work by Ramaswamy and Bhatnagar, by allowing for the presence of nonadditive iterate-dependent Markov noise. As an application, we consider the problem of computing the optimum in a constrained convex optimization problem, where the objective function and the constraints are averaged with respect to the stationary distribution of an underlying Markov chain. Further, the proposed scheme neither requires the differentiability of the objective function nor the knowledge of the averaging measure.
- Subjects
DIFFERENTIAL inclusions; STOCHASTIC approximation; CONSTRAINED optimization; NOISE; MARKOV processes
- Publication
Mathematics of Operations Research, 2020, Vol 45, Issue 4, p1405
- ISSN
0364-765X
- Publication type
Article
- DOI
10.1287/moor.2019.1037