Probabilities Transition Matrix

Usually the topic of discussion will be a series of t steps, and the object of inquiry will be the probability of making a transition from state s to state j after t steps.
The transition matrix after t steps, starting from state s, using policy $\pi$ is:

$P_{\pi}^{t}(j\vert s)=[P_{d_{t}}\ldots P_{d_{2}}\cdot P_{d_{1}}](j\vert s)=Prob_{\pi}(X_{t+1}=j\vert x_{1}=s)$