We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation.
- Authors
Ahamed, T. P. I.; Borkar, V. S.; Juneja, S.
- Abstract
For a discrete-time finite-state Markov chain, we develop an adaptive importance sampling scheme to estimate the expected total cost before hitting a set of terminal states. This scheme updates the change of measure at every transition using constant or decreasing step-size stochastic approximation. The updates are shown to concentrate asymptotically in a neighborhood of the desired zero-variance estimator. Through simulation experiments on simple Markovian queues, we observe that the proposed technique performs very well in estimating performance measures related to rare events associated with queue lengths exceeding prescribed thresholds. We include performance comparisons of the proposed algorithm with existing adaptive importance sampling algorithms on some examples. We also discuss the extension of the technique to estimate the infinite horizon expected discounted cost and the expected average cost.
- Subjects
COST; STOCHASTIC processes; ESTIMATION theory; MARKOV processes; QUEUING theory; PERFORMANCE; ALGORITHMS
- Publication
Operations Research, 2006, Vol 54, Issue 3, p489
- ISSN
0030-364X
- Publication type
Article
- DOI
10.1287/opre.1060.0291