Skip to content

Conversation

MahdiBM
Copy link
Contributor

@MahdiBM MahdiBM commented Oct 13, 2025

Add an API on top of BufferedReader which splits the file based on its content.

Motivation:

Provides a nice API, instead of users having to go though manually handling BufferedReader.read(while:).
I struggled with this, as documented in https://swift-open-source.slack.com/archives/C9MMT6VGB/p1760115481607159

Modifications:

Add BufferedReader.SplitSequence + stdlib-like functions on BufferedReader to create such a sequence.

Result:

Users can easily split files based on their content.

Copy link
Contributor

@Lukasa Lukasa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial notes, @glbrntt will have more.

extension BufferedReader.SplitSequence: Sendable where BufferedReader: Sendable {}

@available(macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0, *)
extension BufferedReader.SplitSequence.AsyncIterator: Sendable where BufferedReader: Sendable {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These look wrong: I think they need to be when Handle: Sendable.

Copy link
Contributor Author

@MahdiBM MahdiBM Oct 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emmm both are fine. BufferedReader being Sendable implies Handle being Sendable.
This type stores a BufferedReader, so my preference was to not concern it with Handle.
I can use Handle if you prefer that.

@MahdiBM MahdiBM requested a review from Lukasa October 13, 2025 18:33
Copy link
Contributor

@glbrntt glbrntt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely a problem that's worth tackling (you are by no means the first to run into this!) but I don't think this is the right approach.

In my mind the more general problem is having an AsyncSequence<ByteBuffer> from which you want decode a sequence of some other type, e.g. you want a AsyncSequence<Decoded>.

I think the appropriate way to do this would not be to add an extension to NIOFileSystem but instead to extend AsyncSequence where the element type is ByteBuffer in NIOCore. The base API could use a NIOSingleStepByteToMessageDecoder and then we could add helpers on top which setup an appropriate decoder.

This would mean having something like:

extension AsyncSequence where Element == ByteBuffer {
  public func decode<Decoder: NIOSingleStepByteToMessageDecoder, Decoded>(
    using decoder: Decoder
  ) -> SomeNewAsyncSequenceType<Decoder, Decoded> where Decoder.InboundOut == Decoded {
    // ...
  }
}

For reading lines from a file you'd then use the readChunks API rather than the BufferedReader.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants