Reputation: 229
Why do we need MDP setting in reinforcement learning
In many reinforcement learning (RL) papers, Markov Decision Process (MDP) is a typical problem setting for RL problem. What is the real benefit of this setting? Some papers use LSTM as their policy network structure which obviously violate the MDP assumption and make more sense.
Upvotes: 0
Views: 376
Answers (1)
Reputation: 6689
Basically, Markov Decision Processes provide a theoretical framework that allows to analyze the convergence guarantees of the algorithms as well as other theoretical properties. Although LSTM and other deep learning approaches combined with RL have reached impressive results, they lack from a solid theoretical background that allow understand or ensure when the algorithm is going to learn something useful, or how far the learned policy will be from the optimal one.
Upvotes: 2
Related Questions
- Why introduce Markov property to reinforcement learning?
- Stochastic state transitions in MDP: How does Q-learning estimate that?
- Why the bandit problem is also called a one-step/state MDP in Reinforcement learning?
- Setting up target values for Deep Q-Learning
- Why and when is deep reinforcement learning needed instead of q-learning?
- What is Optimality in Reinforcement Learning?
- MDP calculation
- MDP & Reinforcement Learning - Convergence Comparison of VI, PI and QLearning Algorithms
- What is the advantage of Deterministic Policy Gradient over Stochastic Policy Gradient?
- When to use Policy Iteration instead of Value Iteration