Understanding state-value and action-value Bellman equations
0
$begingroup$
In Reinforcement Learning , the Bellman Optimality equations are important for defining optimal policies to be taken by a learning algorithm. The following two equations are commonly cited... ... and ... From a high level I understand how each work, I get that the state-value function returns the optimal policy from going from one state to another and I get that the action-value returns the optimal policy of taking an action from a particular state. What I don't understand is why these equations work out mathematically. Why is the max function outside of the equation for the state-action function and why is it placed inside for the action-value function? I must be missing some fundamental information about how each equation works. Can someone explain the difference to me?
...