Finite Horizon

In lecture number 3 we introduced the optimality equations:

$$U_t(h_t) = \max_{a\in A} \{r_t(s_t,a) + \sum_{j\in S} P_t(j\vert s_t,a)U_{t+1}(h_t,a,j)\},$$

Where U_N(h_N)=r_N(s_N) for h_N=(h_N-1,a_N-1,s_N).
We showed:

1.

An optimal policy

2.

A deterministic optimal policy