Skip to content

TaskEx: parallelLimit #143

Open
Open
@bartelink

Description

@bartelink

Replaces #129. TaskEx top level issue: #139

Async.Parallel's optional degree of parallelism parameter was added late in the game, but is critical - dumping an arbitrary unbounded number of work items onto the threadpool is not something that should be easy and/or the default thing to do without due consideration for how that will work under stess.

There are some other shortcomings, which frequently lead to various bespoke helpers proliferating:

  • pipelining is painful, necessitating an explicit argument name (e.g. fun computations -> Async.Parallel(computations, maxDegreeOfParallelism=dop) etc) (note this is not the case for Async.Sequential)
  • before v FSharp.Core v 6.0.6, [there was a stack overflow bug that can tear down the process](// Async.Parallel stack overflow on cancellation of ~2000 uncompleted computations dotnet/fsharp#13165) if >1200 items are started with a throttle and cancellation is triggered quickly (so having a layer between Async.Parallel and direct consumption within an app might be useful)

Current proposed APIs (will be updated inline based on any discussion below):

module Async =
    let parallelLimit maxDegreeOfParallelism computations =
        Async.Parallel(computations, maxDegreeOfParallelism = maxDegreeOfParallelism)

NOTES:

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestNew feature or enhancement requesttopic: task-exRelated to the proposed new TaskEx library, which should get its own oss haven

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions