Single writer/multiple readers

[danielkornev]

A year ago, I filed a bug describing the subject. OpenMCDF is fantastic, but its inability to support multiple readers make it way, way less useful than it could be.

[ironfede]

Daniel, thank you for your suggestions.
I know the importance of the SWMR theme for OpenMcdf but - at the moment - I have to give priority to bug fixing and ordinary maintenance because of the time I can dedicate to OpenMcdf - too little I know.
Anyway, could you please give me some advice in order to introduce this feature in OpenMcdf if you already have some idea?
Thank you very much,
Best Regards
Federico

[jeremy-visionaid]

@danielkornev It's been several years, but do you still have an interest in this feature? With the rewrite for v3.0 I think it's reasonable to consider adding for v3.1 (at least for the in-process case).

[danielkornev]

Hi Jeremy!

I moved to Redis (out-of-process) in 2016, and to RocksDB (in-process) in 2022.

RocksDB is quite nice though it creates a lot of file on the disk drive.

Wish I could have something like RocksDB but encapsulated into one file per store...

[jeremy-visionaid]

@danielkornev No problem! I think I'll likely keep this ticket open and work on it in the background, as it is still useful to my own work, it's just not high priority right now.

[danielkornev]

Gotcha. Well, if you can outperform RocksDB in terms of multiple readers and writers, that's be awesome. NTFS doesn't like when you write and open many files, which is what RocksDB unfortunately does.

[jeremy-visionaid]

Sadly I doubt CFB as a format is even capable of outperforming a modern design/database, no matter how good the implementation is!

Just curious if you were interested in in-process SWMR or you wanted it to work across processes? I know you've moved on to other things, so mostly just thinking out lound and gathering interests... Neither case is supported today, and as I've been thinking more about it, perhaps v3 isn't so well suited to implementing SWMR... and perhaps not .NET Standard ... and perhaps not even current .NET 8/9... Ideally it would use somthing like RandomAccess from .NET 6+ but coupled with locking like FileStream.Lock. However, those things don't work together, and they also wouldn't work over a MemoryStream. I think it's all possible still (albeit not over streams as a base), but from that point I'm thinking I've probably got other things I should be doing instead (and SWMR would be as personal project feature), so it might be down to someone with a greater need for it to implement it.