Skip to content

Feature Request: Persistency KVS #258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: ulhu_persistency_kvs
Choose a base branch
from
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
@startuml

title Sequence Diagram: KVS Builder

participant "User" as actor
participant "«component» :builder" as builder
participant "«component» :kvs" as kvs

actor -> builder: Create KVS builder
builder --> actor: KVS builder instance

actor -> builder: Set need_defaults flag
builder --> actor: KVS builder instance

actor -> builder: Set need_kvs flag
builder --> actor: KVS builder instance

actor -> builder: Build KVS instance
builder -> kvs: Open KVS with builder config

alt kvs-open
kvs --> builder: KVS instance
builder --> actor: KVS instance
else kvs-open-error
kvs --> builder: KVS ErrorCode
builder --> actor: KVS ErrorCode
end

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
@startuml

title Sequence Diagram: Check if Key contains Default Value

participant "User" as actor
participant "«component» :kvs" as kvs

actor -> kvs: Does key has default value

alt default-key-exists
alt key-exists
alt default-and-key-match
kvs --> actor: Key contains default value
else key-doesnt-exist
kvs --> actor: Key doesn't containt default value
end
else key-doesnt-exist
kvs --> actor: Key contains default value
end
else default-doesnt-exist
kvs --> actor: Key doesn't contain default value
end

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
@startuml

title Sequence Diagram: Delete Key from KVS Instance

participant "User" as actor
participant "«component» :kvs" as kvs

actor -> kvs: Remove key

alt key-exists
kvs --> actor: Successfully deleted key
else key-doesnt-exist
kvs --> actor: Key-Not-Found-Error
end

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
@startuml

title Sequence Diagram: Flush Local Representation to Data File

participant "User" as actor
participant "«component» :kvs" as kvs
participant "«component» :snapshot" as snapshot
participant "«component» :TinyJSON" as json_parser
participant "«component» :fs" as fs
participant "«component» :Adler32" as hasher

actor -> kvs: Flush KVS

kvs -> json_parser: Generate string from local representation

alt json-string
json_parser --> kvs: JSON data string
else json-string-error
json_parser --> kvs: JSON generator error
kvs --> actor: JSON generator error
end

kvs -> snapshot: Rotate snapshots

alt snapshot-rotate
snapshot -> kvs: Snapshots rotated
else snapshot-rotate-error
snapshot --> kvs: Snapshot-Rotate-Error
kvs --> actor: Snapshot-Rotate-Error
end

kvs -> hasher: Create JSON data string hash

alt hash-created
hasher --> kvs: Data hash
else hash-create-error
hasher --> kvs: Hash-Calc-Error
kvs --> actor: Hash-Calc-Error
end

kvs -> fs: Write JSON data string to file

alt file-write
fs --> kvs: File successfully written
else file-write-error
fs --> kvs: File-Write-Error
kvs --> actor: File-Write-Error
end

kvs -> fs: Write JSON data hash to file

alt file-hash-write
fs --> kvs: Hash file successfully written
kvs --> actor: Flush successful
else file-hash-write-error
fs --> kvs: File-Hash-Write-Error
kvs --> actor: File-Hash-Write-Error
end

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
@startuml

title Sequence Diagram: Read Key from KVS Instance

participant "User" as actor
participant "«component» :kvs" as kvs

actor -> kvs: Get value for key

alt key-exists
kvs --> actor: Return value for key
else key-doesnt-exist
alt default-exists
kvs --> actor: Return default value for key
else default-doesnt-exist
kvs --> actor: Key-Not-Found error
end
end

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
@startuml

title Sequence Diagram: Read Data into Local Representation (KvsValue)

participant "Builder" as actor
participant "«component» :kvs" as kvs
participant "«component» :TinyJSON" as json_parser
participant "«component» :fs" as fs
participant "«component» :Adler32" as hasher

actor -> kvs: Open KVS

kvs -> fs: Read defaults file

alt file-exists
fs --> kvs: Defaults file content (JSON)
else file-based-error
fs --> kvs: File-Error
kvs -> actor: File-Error
end

kvs -> fs: Read defaults file hash

alt file-exists
fs --> kvs: Defaults file hash
else file-based-error
fs --> kvs: File-Error
kvs -> actor: File-Error
end

kvs -> hasher: Generate defaults file hash
hasher --> kvs: Defaults file generated hash

alt hash-matches
kvs -> json_parser: Parse JSON data
else hash-match-error
kvs -> actor: Hash-Error
end

alt parsing-success
json_parser --> kvs: Parsed JSON object
else parsing-based-error
json_parser -> kvs: Parser-Error
kvs -> actor: Parser-Error
end

kvs --> actor: KVS instance

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
@startuml

title Sequence Diagram: Restore Snapshot

participant "User" as actor
participant "«component» :kvs" as kvs
participant "«component» :snapshot" as snapshot

actor -> kvs: Restore snapshot

alt snapshot-found
snapshot --> kvs: Snapshot restored into local representation
kvs --> actor: Snapshot restored successfully
else snapshot-error
snapshot --> kvs: Snapshot-Not-Available-Error
kvs --> actor: Snapshot-Not-Available-Error
end

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
@startuml

title Sequence Diagram: Write Key to KVS Instance

participant "User" as actor
participant "«component» :kvs" as kvs

actor -> kvs: Set value for key
kvs --> actor: Value set for key

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
@startuml

interface Kvs {
open
flush_on_exit
reset
get_all_keys
key_exists
get_value
get_default_value
has_default_value
set_default_value
set_value
remove_key
flush
snapshot_count
snapshot_max_count
snapshot_restore
get_kvs_filename
get_hash_filename
}

@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
@startuml
allowmixing

title Static View - key_val_storage

!include kvs_interface.puml


skinparam package {
BackgroundColor #E0E0D0
BorderColor Black
backgroundColor<<feature>> yellow
}

skinparam component {
backgroundColor<<component>> white
}

' Define Features
package "key_val_storage" <<feature>> {
component kvs <<component>>
component fs <<component>>
component tiny_json <<component>>
}

kvs -u- Kvs

kvs ..> tiny_json : use
kvs ..> fs : use


@enduml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
..
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************

Component Architecture
######################

Technical Concept Description
*****************************

The key-value-storage feature (KVS) scope is very small so only the KVS itself
and a another component to handle the persistent storage format are defined.

The API is stable for major version releases. Any changes are discussed in the
CFT. Whenever possible, the existing API is not changed and new functionality
is provided by new function calls.

Key-Value-Storage (KVS)
=======================

The KVS is presented like a dictonary or hashmap implementation to the
application with some extra APIs for flushing data, handling snapshots and
other functionality. All API functions guarantee to not panic or throw
exceptions that can't be handled. Error handling has one of the highest
priorities.

All actions are log- and traceable. The component supports productive and
develop environments.

The KVS is multi-threading safe and allows concurrent access to the same store.
Tracking changes can be done by subscribing to specified keys or the whole
store.

Values that are stored behind the keys can be numbers, text, null, arrays or
objects. Objects and arrays can be used to nest values.

A snapshot logic ensures rollback capabilities if something goes wrong.

TinyJSON
========

TinyJSON is used to serialize and deserialize the data as the well-known format
JSON to and from the permanent storage.
Loading