We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
A Reinforcement Learning Approach for Solving the Mean Variance Customer Portfolio in Partially Observable Models.
- Authors
Asiain, Erick; Clempner, Julio B.; Poznyak, Alexander S.
- Abstract
In problems involving control of financial processes, it is usually complicated to quantify exactly the state variables. It could be expensive to acquire the exact value of a given state, even if it may be physically possible to do so. In such cases it may be interesting to support the decision-making process on inaccurate information pertaining to the system state. In addition, for modeling real-world application, it is necessary to compute the values of the parameters of the environment (transition probabilities and observation probabilities) and the reward functions, which are typically, hand-tuned by experts in the field until it has acquired a satisfactory value. This results in an undesired process. To address these shortcomings, this paper provides a new Reinforcement Learning (RL) framework for computing the mean-variance customer portfolio with transaction costs in controllable Partially Observable Markov Decision Processes (POMDPs). The solution is restricted to finite state, action, observation sets and average reward problems. For solving this problem, a controller/actor-critic architecture is proposed, which balance the difficult tasks of exploitation and exploration of the environment. The architecture consists of three modules: controller, fast-tracked portfolio learning and an actor-critic module. Each module involves the design of a convergent Temporal Difference (TD) learning algorithm. We employ three different learning rules to estimate the real values of: (a) the transition matrices π ^ (j | i k) (t) , (b) the rewards J ^ (i j k) (t) and (c) the resources destined for carrying out a promotion ζ ^ (i ^ | k ^) (t). We present a proof for the estimated transition matrix rule π ^ (j | i k) (t) and showing that it converges when t → ∞. For solving the optimization programming problem we extend the c-variable method for partially observable Markov chains. The c-variable is conceptualized as joint strategy given by the product of the control policy, the observation kernel Q(y|s) and the stationary distribution vector. A major advantage of this procedure is that it can be implemented efficiently for real settings in controllable POMDP. A numerical example illustrates the results of the proposed method.
- Subjects
REINFORCEMENT learning; PARTIALLY observable Markov decision processes; MACHINE learning; OPL (Computer program language); INFORMATION retrieval
- Publication
International Journal on Artificial Intelligence Tools, 2018, Vol 27, Issue 8, pN.PAG
- ISSN
0218-2130
- Publication type
Article
- DOI
10.1142/S0218213018500343