Easier heightmap access

[avdstaaij]

There should be Editor.getHeight/Editor.getHeightGlobal methods to get the height at a position, in the former case taking the transform into account.

There should probably be caching functionality similar to Editor.worldSlice/Editor.worldSliceDecay: a system to mark certain heights as "outdated" or update them in a cache (though updating may be costly for the more complex heightmaps). Furthermore, since GDMC HTTP now has a GET /heightmaps endpoint (for a long time already...), it may be possible to avoid getting an entire WorldSlice in some cases.

Some questions on how the height retrieval functions should work still need to be answered:

• What if no WorldSlice is loaded? Should we raise an exception? Make a costly GET /heightmaps call (every time)?
• How do we make it possible to explicitly request only the heightmaps and not the rest of the chunk data? Naturally, we don't want to do this on every getHeight call. Should there be an explicit function for it? Should the getHeight function transparently request the heightmap if it wasn't loaded yet? In the last case, what range should it target?

[Flashing-Blinkenlights]

My suggestion:

1. Valid heightmaps are hardcoded as members of an Enum; ignored blocks are saved for each member
2. For each member in the Enum (i.e. each heightmap type), two global 2D arrays (or equivalent) are created: A bool decay map (akin to the cache) and an int heightmap.
3. When a WorldSlice is made, all decay maps are reset to indicate the WorldSlice is current.
4. When a block is placed, ...

• ...if it's part of the member's ignored blocks and it's equal to the current height: Invalidate decay map and update height downwards (has to check all blocks down until non-ignored block is reached)
• ...if it's not an ignored block and it's greater than the current height: Invalidate decay map and set local height to placed block

The reason for creating a decay map for each heightmap type (as opposed to both combined) is that if a heightmap is fetched and stored somewhere (e.g. by overwriting the local heightmap array), that heightmap type can reference the newest information independently.

Memory

As the max size of the map is typically 1024x1024, and the altitude is a value from range -64 to 320 (comfortably within the range of a numpy short), we can expect at most ~3 MiB per heightmap (numpy bool + short) if all values are saved for the full build area (~750 kiB for 512x512, ~200 kiB for 256x256, ~50 kiB for 128x128).
If all current Minecraft heightmaps (4) were supported in this way, that'd be a maximum memory use of ~12 MiB.

Heightmap Types

Minecraft has four built-in heightmaps which might be of use.

• WORLD_SURFACE: Ignores lookup.AIRS
• OCEAN_FLOOR: Ignores blocks with materials air, bamboo sapling, bubble column, cloth decoration, decoration, fire, frogspawn, lava, plant, portal, powder snow, replaceable fireproof plant, replaceable plant, replaceable water plant, structural air, top snow, water, water plant, and web.
• MOTION_BLOCKING: Same as OCEAN_FLOOR, but lookup.LIQUIDS and waterlogged blocks are not ignored
• MOTION_BLOCKING_NO_LEAVES: Same as MOTION_BLOCKING, but lookup.LEAVES are not ignored

In my opinion, WORLD_SURFACE and MOTION_BLOCKING are the only really useful and optimal heightmaps. Useful custom heightmaps might include:

• No trees
• No replaceables (i.e. everything which would be destroyed by an anvil)
• No structures (artificial blocks)
• Caves (ignores everything except for cave air, not sure if this works)
• Random idea: Tool level heightmaps (top thing that can be mined with wood, stone, iron, diamond, netherite tools)

Custom Heightmaps

In this system, it would also be possible to create custom heightmaps by providing an ignore set, although these heightmaps will always be local and cannot benefit from WorldSlices.

If HTTP GDMC is also updated to permit custom heightmaps, this "ignore-list" approach would also fit in nicely.

[avdstaaij]

Valid heightmaps are hardcoded as members of an Enum; ignored blocks are saved for each member

Putting valid heightmaps in an enum indeed makes sense (but not if we add custom heightmaps).
I would prefer to not store the list of ignored blocks for Minecraft's built-in heightmaps because it's not version-independent (#99), though that might mean that updating the heightmaps based on block placements becomes impossible.

For each member in the Enum (i.e. each heightmap type), two global 2D arrays (or equivalent) are created: A bool decay map (akin to the cache) and an int heightmap.

The heightmaps should not be global, but they could be members of Editor.

When a WorldSlice is made, all decay maps are reset to indicate the WorldSlice is current.

Do note that there's a difference between "a WorldSlice" and Editor.worldSlice. The latter has more integrations with other Editor features, such as caching. Also note that, since GET /heightmaps exists, we do not necessarily have to load an entire WorldSlice if we only need the heightmaps. If we add heightmap caching to Editor, then Editor should indeed have decay maps that get reset when these heightmaps are updated (just like worldSliceDecay for blocks).

When a block is placed, ...

Hmm, now that you've written out what it takes to keep a height cache up-to-date, I'm a bit concerned about the performance hit. It seems like a lot of steps for every block placement. If we implement it, I don't think it should be the default.

Simply marking a height value as outdated is cheaper, but then the performance hit comes when the user requests that height value -- there's no easy to way to get a single height value from GDMC-HTTP. (Maybe this would be a good feature?)

The reason for creating a decay map for each heightmap type (as opposed to both combined) is that if a heightmap is fetched and stored somewhere (e.g. by overwriting the local heightmap array), that heightmap type can reference the newest information independently.

I don't quite understand what you mean in this paragraph. Could you clarify it more? I would expect that we need separate decay maps if we only mark height values as "up-to-date"/"outdated", whereas we could do without decay maps if we fully update height values on every block placement (though see note above).

Memory

I don't think memory will be a concern. Heightmaps should only be loaded on an explicit request anyway, so the memory usage is in the user's hands.

Custom heightmaps

Adding custom heightmaps is an interesting idea. A question is where the logic for them should go. I would prefer to keep Editor as minimal as possible, only containing the core methods needed to interact with GDMC-HTTP (though the transformation system is already an exception to this). However, to support custom heightmaps that invalidate on block placement, they need to be implemented in Editor. Maybe Editor should expose events on block placements, but that would be a significant refactor, and might again be costly...

If GDMC-HTTP implements custom heightmaps internally, this becomes easier: it should then certainly go in Editor.

[Flashing-Blinkenlights]

Putting valid heightmaps in an enum indeed makes sense (but not if we add custom heightmaps).
I would prefer to not store the list of ignored blocks for Minecraft's built-in heightmaps because it's not version-independent (#99), though that might mean that updating the heightmaps based on block placements becomes impossible.

We can always treat built-in and custom heightmaps differently: Built-in gets invalidated, custom gets updated or invalidated based on preference.

The heightmaps should not be global, but they could be members of Editor.

Agreed entirely, that was bad phrasing on my part.

Do note that there's a difference between "a WorldSlice" and Editor.worldSlice. The latter has more integrations with other Editor features, such as caching. Also note that, since GET /heightmaps exists, we do not necessarily have to load an entire WorldSlice if we only need the heightmaps. If we add heightmap caching to Editor, then Editor should indeed have decay maps that get reset when these heightmaps are updated (just like worldSliceDecay for blocks).

You're quite right, I'm being vague, mainly because I can't remember how exactly the WorldSlice and Editor.wordSlice work anymore. I think heightmap caching would be a good idea either way.

Hmm, now that you've written out what it takes to keep a height cache up-to-date, I'm a bit concerned about the performance hit. It seems like a lot of steps for every block placement. If we implement it, I don't think it should be the default.

We can always store the current, outdated value and the last placed per column without updating the original value. That way, if the height is changed repeatedly without requesting the updated height, we can delay the calculation until the last moment possible.

Even if we did have a way of requesting a single height via GDMC-HTTP, I feel like the response time would not be worth it for a single block.

I don't quite understand what you mean in this paragraph. Could you clarify it more? I would expect that we need separate decay maps if we only mark height values as "up-to-date"/"outdated", whereas we could do without decay maps if we fully update height values on every block placement (though see note above).

I'm saying, I'd prefer having a decay map per heightmap type as opposed to a single decay map for any/all heightmaps in the editor. The reason for that preference is because it provides greater flexibility and allows only particular heightmaps to be invalidated, namely only those which are appropriately affected by that Block.

Adding custom heightmaps is an interesting idea. A question is where the logic for them should go. I would prefer to keep Editor as minimal as possible, only containing the core methods needed to interact with GDMC-HTTP (though the transformation system is already an exception to this). [...]

If you want Editor to be as minimal/slim as possible, I think you might have to rethink the Editor's architecture to be more modular.
I'd suggest one of the following design patterns:

1. Mixins and editors with extended functionality based on a base editor class (multiple inheritance)
2. A chain-of-responsibility where you have your basic class functions, but other "plugged-in" strategies can be executed within these functions to achieve more complex functionality. Wrapping these plug-ins into some data structure which can introduce appropriate features into appropriate functions would make sense (e.g. throw in a "caching" feature package and the editor will execute its functions at the appropriate places)
3. Blow the door wide open with strategies, and allow the Editor's functions to be completely replaced

I'd probably go for 1, unless you can think of an elegant interface for option 2.

It doesn't look like GDMC-HTTP will do anything but allow for an ignore-list of blocks to emulate a custom heightmap, storing that information in the client sounds clumsy.