derivation of stochastic approximation and its use

[mzaffalon]

How is the stochastic iteration $q_{k+1} = q_k - \alpha_k e(q_k)$ derived?

How is it applied to TD-learning? How are the validity conditions on $e(q)$ satisfied?

Ref: questions on Moodle 1, 2