Description
Motivation and description
Currently the scan
method is used to mark the nodes before applying the actual backpropagation in the graph.
https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19
We can achieve the same results without employing the this pre-marking of nodes. In this proposal the back
function shall not use the scan
method, but rather take care of the nodes itself, without a pre-marking.
https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L74
The functions and files involved are only where the ref
field is used: https://github.com/FluxML/Tracker.jl/blob/master/src/Tracker.jl#L41.
The following structures/functions/algos have to be understood in order to remove scan
:
- struct Tracked{T} and it's members
- the scan, back and back methods in the file
back.jl
.
Note the function back(g::Grads, x::Tracked, Δ) is used for higher-gradients and we should not focus on this use-case in this proposal.