Description
I started feeling that we've been too optimistic about use of specialized pipeTo()
algorithm given an identify transform between the pair of rs/ws.
In the discussion about the Request
object, I've been saying that
- we should be able to
.write()
even before theRequest
gets attached to the final sink (Uploading: write to a writable stream, or pass in a readable stream? yutakahirano/fetch-with-streams#30 (comment)). - we can run specialized
.pipeTo()
internally even with existence of such a buffer.
The following are reduced example of what I said:
Example: id
is writable even before id.readable.pipeTo()
happens.
id.writable.write("hello");
id.writable.write("world");
id.readable.pipeTo(ws);
Example: In the following two examples, rs.pipeTo(ws)
should happen eventually and internally after draining chunks queued in id
.
rs.pipeTo(id.writable);
id.readable.pipeTo(ws); // Eventually rs.pipeTo(ws)
rs.pipeTo(id.writable);
id.readable.pipeTo(ws); // Eventually rs.pipeTo(ws)
Note that not whole of the rs.pipeTo(ws)
algorithm should always happen. Closure of rs
should result in making only id
closed if id.readable.pipeTo(ws)
is changed to id.readable.pipeTo(ws, { preventClose: true })
.
I'd call the optimized main data transfer part of .pipeTo()
as .specialDataTransfer()
which excludes close/error signal handling.
So, let's think of what we should do for the following example:
id.writable.write("hello");
id.writable.write("world");
rs.pipeTo(id.writable);
id.readable.pipeTo(ws);
Here are two issues:
id
already has two chunks queued in it. Ifid.readable.pipeTo(ws)
algorithm is the one ofReadableStream.pipeTo()
as-is, we cannot drain all the data fromid
untilws
stop exerting back-pressure.- Without an ability to stop
rs.pipeTo(id.writable)
, we never be able to switch tors.specialDataTransfer(ws)
.
I guess, the solution is:
- Identity transform flushes all the data queued in it when
id.readable.pipeTo(ws)
is called regardless of back-pressure ofws
. - Provide an (possibly private) API, say
.redirectTo
, on theWritableStream
for.pipeTo()
algorithm to redirect.specialDataTransfer()
tows
.ReadableStream.pipeTo()
is required to check.redirectTo
before everydest.write()
and if it's set, it must switch to users.specialDataTransfer(dest.redirectTo)
.
Maybe this could be evolved to address #307, too.