Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework

39 Pages Posted: 29 May 2019

See all articles by Haoran Wang

Haoran Wang

Columbia University - Department of Industrial Engineering and Operations Research (IEOR)

Xun Yu Zhou

Columbia University - Department of Industrial Engineering and Operations Research (IEOR)

Date Written: May 5, 2019

Abstract

We approach the continuous-time mean-variance (MV) portfolio selection with reinforcement learning (RL). The problem is to achieve the best tradeoff between exploration and exploitation, and is formulated as an entropy-regularized, relaxed stochastic control problem. We prove that the optimal feedback policy for this problem must be Gaussian, with time-decaying variance. We then establish connections between the entropy-regularized MV and the classical MV, including the solvability equivalence and the convergence as exploration weighting parameter decays to zero. Finally, we prove a policy improvement theorem, based on which we devise an implementable RL algorithm. We find that our algorithm outperforms both an adaptive control based method and a deep neural networks based algorithm by a large margin in our simulations.

Keywords: Reinforcement learning, mean-variance portfolio selection, entropy regularization, stochastic control, value function, Gaussian distribution, policy improvement theorem

JEL Classification: C02, C61, C63, G11,

Suggested Citation

Wang, Haoran and Zhou, Xunyu, Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework (May 5, 2019). Available at SSRN: https://ssrn.com/abstract=3382932 or http://dx.doi.org/10.2139/ssrn.3382932

Haoran Wang (Contact Author)

Columbia University - Department of Industrial Engineering and Operations Research (IEOR) ( email )

331 S.W. Mudd Building
500 West 120th Street
New York, NY 10027
United States

Xunyu Zhou

Columbia University - Department of Industrial Engineering and Operations Research (IEOR) ( email )

331 S.W. Mudd Building
500 West 120th Street
New York, NY 10027
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
194
Abstract Views
883
rank
187,522
PlumX Metrics