Putting It All Together

We've received:

$$\left\Vert \widehat{Q}_l - Q^*\right\Vert \leq \left\Vert \widehat{Q}_l - Q_l \right\Vert + \left\Vert Q_l - Q^* \right\Vert$$ (13)

where

• For any w>0: $\Pr( \left\Vert \widehat{Q}_l - Q_l \right\Vert \geq w \cdot \sum_{i=1}^l{\gamma^i} ) \leq l \cdot \vert S\vert \cdot \vert A\vert \cdot 2e^{-2{m_D}{w^2}}$
• $\left\VertQ_l - Q^*\right\Vert \leq \frac{\gamma^{l-1}}{1-\gamma} \left\Vertv_1 - v_0\right\Vert$

Hence we can sum up all of the above by:

$$\begin{eqnarray*}\lefteqn{ \Pr( \left\Vert\widehat{Q}_l - Q^*\right\Vert \geq ... ...cdot \vert S\vert \cdot \vert A\vert \cdot 2e^{-2{m_D}{w^2}}\\ \end{eqnarray*}$$