Skip to content

Add more efficient bucket implementation #5

@helje5

Description

@helje5

Noze.io streams work on batches of items. The items T can be any Swift type due to the use of generics. You can have streams that read and/or write bytes (UInt8) or Characters, or String-lines, or database records, or tweets, etc. In Noze the items are always processed in batches to be efficient. E.g. if a parser reads 5 lines, it won't issue 5 read calls, but just one containing all 5 lines.

The original idea was to make a 'bucket of items' a protocol, like:

protocol Bucket<T> { ... }

And then make existing bucket data structures conform to that, say:

Array<T>: Bucket<T>

and

DispatchData : Bucket<UInt8>

Any stream which would process Bucket<UInt8>'s would have been able to deal with [ UInt8 ] or DispatchData or a memory mapped file handle, etc.

Unfortunately that didn't/doesn't fly (yet?) in Swift 2/3 because you can't use a generic protocol as a type. (essentially this doesn't work: let someBucket : Bucket<UInt8>).

So as a temporary measure we skipped the issue and just used Swift arrays as 'buckets'. That is [T].
Now of course this is pretty slow in a lot of circumstances :-)

Summary: we need more efficient buckets.

One non-ideal solution is to use a base class for buckets and subclasses which wrap the various container classes. Like:

class Bucket<T> {}

class ArrayBucket<T> : Bucket<T> { let items: [ T ] }

class DispatchDataBucket : Bucket<UInt8> { let items : DispatchData }

Presumably this is much better than the [T] and may be the way to go, but obviously having an extra heap allocated wrapper isn't 1337.
Better suggestions are very welcome.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions