Storage Channel Configuration and related error

[DhinchakDhoomDev]

Q1 : I am using a FileNameProvider and Directory to save those entry in specific directories but after restarting the application I am getting error as : "Non-deleted non-empty data file not found: channel 1, file 1"
Source code is as follows :

            StorageFileNameProvider fileNameProvider = StorageFileNameProvider.Builder()
                    .setChannelDirectoryPrefix("channel_")
                    .setDataFilePrefix("data_")
                    .setDataFileSuffix(".databin")
                    .setTransactionsFilePrefix("events_")
                    .setTransactionsFileSuffix(".trbin")
                    .setTypeDictionaryFileName("typeDictionary.txt")
                    .createFileNameProvider();

            NioFileSystem fileSystem = NioFileSystem.New();
            StorageLiveFileProvider fileProvider = Storage
                    .FileProviderBuilder(fileSystem)
                    .setDirectory(fileSystem.ensureDirectoryPath("./saveDir"))
                    .setDeletionDirectory(fileSystem.ensureDirectoryPath("./deleteDir"))
                    .setTruncationDirectory(fileSystem.ensureDirectoryPath("./truncateDir"))
                    .setFileNameProvider(fileNameProvider)
                    .createFileProvider();
            EmbeddedStorageManager storage = EmbeddedStorage.Foundation(
                            Storage.ConfigurationBuilder()
                                    .setChannelCountProvider(Storage.ChannelCountProvider(2))
                                    .setStorageFileProvider(fileProvider)
                                    .createConfiguration()
                    )
                    .start();

I am using example from the following DOCS link

• https://docs.eclipsestore.io/manual/storage/configuration/storage-files-and-directories.html
• https://docs.eclipsestore.io/manual/storage/configuration/using-channels.html

Q2 : There was a feature in Microstream Read-Only Storage Manager

Referenced from the following link :
https://docs.microstream.one/manual/storage/configuration/readonly.html

[hg-ms]

The problem seems to be caused by the dots in setDataFileSuffix and setTransactionsFileSuffix.
It should work if you specify the suffixes without the leading dots e.g.: “databin” and “trbin”.

TODO: check docu and code if there is a bug…

[hg-ms]

Q2 : There was a feature in Microstream Read-Only Storage Manager

The StorageWriteControllerReadOnlyMode is still there. It has been moved to the package org.eclipse.store.storage.types

TODO: Docu seem to be incomplete

[hg-ms]

@DhinchakDhoomDev: Many thanks for the hint to the missing / wrong documentation.

[DhinchakDhoomDev]

One more thing if if you could help me out from the docs shared in MicroStream project. Below is the snippet I would to quote

It is possible to have a Storage Manager which operates in a read-only mode. Since there is a limitation in the Open Source version of MicroStream that you can have only one manager that is connected to one data storage, you can bypass this restriction with this read-only mode.

Q1 Is it possible to write from a thread/micro-service and read from another? (like publish-subscribe pattern)

Because it's been mentioned over here as limitation in the "Open Source" version. That limitation still exist in EclipseStore project.

[hg-ms]

There a two different case to distinguish between:

1. Only one storage manager opens a storage exclusively:
   In that case writing with one thread and reading from another one is not a problem. You just need to take the standard java concurrency topics into account, e.g.: use reentrant read write locks. The storage manager is shared by all treads of your application.
   The read access is done on the already loaded object graph in memory. Write operations require that you ensure that the object graph is not modified while the data to be persisted is collected.
   The io operations (store/load) a done by the single storage manager that syncs io calls internally.

2. More the one storage manager instances access a storage:
   This is the docu part you quoted. It is essential that there is only one storage manager that is allowed to perform modifications to the peristed data. So only one storage manager instance is allowed to have write access to the stored data. Other instances must ensure read only access. One reason for that is that the storage manager instances do not know each other and are not able to detect if another instance modified the persisted data.