Getting the total number of items in a stream

[arvidj]

It is often necessary to get the total number of items in a given stream (e.g. the number of open merge requests). If I understand correctly, this is not yet possible in ocaml-gitlab without requesting all items in the stream and counting them. However, GitLab's API returns the total number of items in the x-total header of responses.

I want to expose this value somehow but I have a hard time wrapping my head around Streams. I've made a proof-of-concept here: arvidj@0dc9f7e but there are some concerns:

• For field total of a Stream that belongs to a gitlab request to be filled, you first have to call next on the stream. This is necessary since it is only at the first call to next that an API call is made.
• I get the impression the whole thing is a bit brittle, I'm not sure restarts, refills and failures are handled correctly.

Another option is making the proposed total field of Stream have the type unit -> int option Monad.t. For Gitlab streams, the total function can capture the endpoint in a closure and query it (possible using a http HEAD request to only retrieve headers) at all calls to total. The value could be memoized. A disadvantage of this approach is that if you want to get the total and also retrieve the items, an additional API request has to be made. A small optimizations would be to retrieve the x-total for during calls to next and fill in the total if it is not yet available.