What is the best way to handle tree load

[Zehir]

Hello,

I have sime issue to load corretly the data.

My game is like a city builder, I have islands, buildings on island, ...
My idea is to have a "file" per save file like Slot 1
Then a partition "common" like with the list of known islands
And a partition "island_x" for the island data.
Then in the island data the list of buildings
And a partition building, ...

First problem, does the ID on the storageAccessor must be unique on the complete save file (with all sub files that could exist) ? It's look like on load the partition property is not used. Is that expected ?
On the dynamic multiplayer example the username is set to id and partition so I suppose yes ?

Then to load data;
So I have a main game StorageAccessor that on _consume_data spawn the islands. But I only want to load the common file at that point because I don't want to load all island data yet.

	LokGlobalStorageManager.current_file = "1"
	lok_storage_accessor.load_data(LokGlobalStorageManager.current_file)
	await lok_storage_accessor.loading_finished

And in that consume function I do;

func _consume_data(res: Dictionary, deps: Dictionary) -> void:
	var game: Game = deps.get(&"game")
	var data: Dictionary = res.get("data")
	var known_island_coords: Array = data.get("known_island_coords", [])
	if known_island_coords.size() == 0:
		await load_island(game, Coords.new(Coords.Precision.ISLAND))
	else:
		for island_coords in known_island_coords:
			await load_island(game, Coords.from_hash(island_coords))

The await here wait for the loading of the island, it's waiting for the node to be instanciated, added to tree, and loading data with a similar script;

static func spawn(_coords: Coords) -> Island:
	var island = prefab.instantiate()
	island.coords = _coords
	island.island_tilemaps_saver = island.get_node("%IslandTilemapsSaver")
	island.island_tilemaps_saver.partition = island.get_save_partition()
	island.island_tilemaps_saver.id = island.get_save_id("tilemap")
	print("load data")
	island.island_tilemaps_saver.load_data(LokGlobalStorageManager.current_file)
	await island.island_tilemaps_saver.loading_finished

But the problem is that the game lok_storage_accessor.loading_finished don't way for the sub StorageAccessor to be loaded too.
So as far I can tell I suppose that the method that call _consume_data don't wait for his completion when they are coroutines.

Or maybe there is better way to do that ?

[Zehir]

I also tried the global LokGlobalStorageManager.load_data(), it's seems that the StorageAccessor instantiated during the consume_data are called to but I am not sure if I was lucky to get them in the correct order or if I would have some problems if the timming is not correct

[nadjiel]

Hello!

If I understand well, you're trying to do something like this, right? Having save slots (in this case, "slot_1") that store different partitions for: the general data ("common"), the islands data ("island_*") and the buildings data ("island_*_building_*).

That looks like a nice approach, I hadn't thought of it this way before.

First problem, does the ID on the storageAccessor must be unique on the complete save file (with all sub files that could exist) ?

So, the LokStorageAccessor.id property should be unique per save file, yes, even if its data is in different partitions. That's because if you have two LokStorageAccessors, for example, with the same id, that will be saved to the same file, the data from one of them will overwrite the data from the other.

It's look like on load the partition property is not used. Is that expected ?

To tell the truth I didn't understand very well this part 🤔

On the dynamic multiplayer example the username is set to id and partition so I suppose yes ?

In that example, setting the partition of the LokStorageAccessors to the players' usernames makes it so the data from the different players stay stored in different partitions identified by their username. If the partition wasn't set, their data would be stored in the same default partition. Both approaches work, they just differ in how the data is organized and accessed, and might differ a little in performance.

static func spawn(_coords: Coords) -> Island:
  var island = prefab.instantiate()
  island.coords = _coords
  island.island_tilemaps_saver = island.get_node("%IslandTilemapsSaver")
  island.island_tilemaps_saver.partition = island.get_save_partition()
  island.island_tilemaps_saver.id = island.get_save_id("tilemap")
  print("load data")
  island.island_tilemaps_saver.load_data(LokGlobalStorageManager.current_file)
  await island.island_tilemaps_saver.loading_finished

About this part, I noticed that you instantiate the Island and try to load its data before adding it to the scene. In theory that shouldn't work since in order to load a LokStorageAccessor's data, that LokStorageAccessor has to be in the scene tree (in order for it to be detected by the LokGlobalStorageManager.

So as far I can tell I suppose that the method that call _consume_data don't wait for his completion when they are coroutines.

That's right, the method that calls it is the LokStorageAccessor.consume_data, which doesn't wait for coroutines.
That's because I hadn't thought about approaches like this when making the plugin, so indeed the LokStorageAccessor.consume_data, which is the method that calls the LokStorageAccessorVersion._consume_data and the other methods involved don't wait for its completion. This looks like a change that should be made, though.

---

I made a test project to try to understand better your issue, and according to what I found out, its possible to realize the tree-like loading, but currently it's not possible to know when that loading as a whole is complete. Because, like you stated there aren't awaits for the data consumption methods.

Here's a zip with this project I made: locker-test.zip

[Zehir]

Thanks again for thie big response. I am out of town today but I will look at your project whan I can.
But yes I think the loaded end signal should wait for coroutines

[nadjiel]

But yes I think the loaded end signal should wait for coroutines

Hello again! I've started working on this issue by doing what you suggested that the data distribution should be awaited by the loading. For that, I've created a new Issue that describes better that particular problem here.

Since I've created that new Issue, I'm going to move this Issue to the new Discussions tab I opened in the repository, so that we can further talk about it there.

[nadjiel]

I also remember getting that kind of error before. That's one of the reasons why initially I didn't make the plugin wait for the user defined _retrive_data and _consume_data methods, because I thought that would raise errors/ warnings if they didn't include the await keyword. But apparently that's okay after all haha

[Zehir]

Well they are there but we don't see them because of this project setting;

[Zehir]

You can ignore the warnings with @warning_ignore("redundant_await");

func consume_data(data: Dictionary) -> void:
	if _version == null:
		return
	if not is_active():
		return
	
	@warning_ignore("redundant_await")
	await _version._consume_data(data, _get_dependencies())

Just but that everywhere you added the await keyword.
You could also check the other warnings and ignore them too

See also @warning_ignore_start("unused_variable")

[nadjiel]

Ohh, there they are! Thanks for pointing that out! It will be nice to add those ignores, so that users don't run the risk of being spammed unnecessarily.

[Zehir]

Yes I had the same problem with my plugin. I just pushed a commit to hide them all Zehir/godot-hexagon-tile-map-layer@b8e2a6b

[Zehir]

Another idea I had before using this plugin was to register the node with the save manager using an ID when adding it. Upon registration, the save manager would call the equivalent of the consume_data function.
The goal was to have an auto load method.

However, it might be better to handle this manually using the load method in the node’s ready function. Additionally, there may not be a valid file to load from at that point, or the node may not have the required dependency.

[nadjiel]

However, it might be better to handle this manually using the load method in the node’s ready function. Additionally, there may not be a valid file to load from at that point, or the node may not have the required dependency.

Well thought! We can't be sure a StorageAccessor's dependencies are initialized just because the StorageAccessor is, so automatically loading data on the StorageAccessor's _ready method would be tricky.

Also, I think not all games would need to load data on start (on ready), so I agree it's better to let this setup up to the users.