Validate queueing strategies are sufficiently powerful

[domenic]

I'm going to consolidate a few threads into this one.

Current Spec

The current spec is largely based around Node's high water marks, providing I believe the exact same amount of power, although in a different API package.

ReadableStream

• When enqueuing a chunk, its size, determined by strategy.size, is added to the total queue size.
• After the chunk has been enqueued, if the stream was currently "readable", consult strategy.needsMore(currentTotalSize), to determine the return value of enqueue(chunk).
• Chunks can be dequeued by the user at any time by calling rs.read().

WritableStream

• When enqueuing or dequeing a chunk, the total size is adjusted, and the call to [[syncStateWithQueue]]() adjusts ws.state to be either "waiting" or "writable", depending:
  • If the queue is empty, it always becomes writable.
  • Otherwise, it consults strategy.needsMore(totalCurrentSize); if the return value is truthy, the stream becomes "writable"; if falsy, the stream becomes "waiting".

Ideas and Help Wanted

@tyoshino previously mentioned, in #76 and summarized in #24 (comment) (with some substitutions for the new terminology):

The base ideas behind my proposals are:

• Allow user defined code to hook any action that may result in [enqueue/dequeue] on the internal [queue]
• Allow the code to investigate the contents of the [queue]. Either of:
  • whole contents every time
  • just see newly [enqueued chunk] and return the integer cost of it

@tyoshino, I am curious whether you think the current proposal accomplishes this or not? I suspect not. In which case, what would help? I think I got confused in the various previous threads.

In #76 the thread ended with:

So, I think

• we should be passing along kind of information say, space available (HWM - filled amount) or speed in a RS on pull. It doesn't need to be limiting push, just work as a hint is OK.
• and also there should be some interface for a consumer to tell its consuming speed to the RS (or strategy object?) in addition to the signal of read() calling speed.

This sounds more different than the current proposal, and more along the lines of things that came up while discussing with @slightlyoff and @willchan. It might tie into #111, which will probably be in a separate BinaryStream subclass, but we should be sure the base interface exposes enough information or hooks too.

[domenic]

Oops. I realized there's a bug in the current spec's strategy usage: #121. Will update the "current spec" section once it is fixed.

[domenic]

Bug fixed; let the discussion resume!

[tyoshino]

Great progress, Domenic! Sorry for delay but I'm taking a look at the new revision. Let me reply to each sub-issue separately.

Allow user defined code to hook any action that may result in [enqueue/dequeue] on the internal [queue]

[[callPull]] is still invoked only when [[queue]] becomes empty for the read() method. So, this comment of mine is not yet addressed. That said, by calling wait(), the user can explicitly invoke [[callPull]]. This might be sufficient and may provide more flexibility. I continue to review.

[tyoshino]

[[enqueue]] always returns true when [[state]] was "waiting". This is remainder of 6e2a21d, maybe. Now after merger, can we make this also consult strategy?