`SemanticLevel.objects` attribute is empty

## Habitat-Sim version
v0.3.3

Habitat is under active development, and we advise users to restrict themselves to stable releases. Are you using the latest [release version](https://github.com/facebookresearch/habitat-sim/releases) of Habitat-Sim?  

**Yes, using v0.3.3 stable release.**

Main branch contains 'bleeding edge' code, but we do appreciate bug reports for it! 

## 🐛 Bug

When loading an MP3D scene—either using `SemanticScene.load_mp3d_house()` directly or through the `Simulator`—the `SemanticLevel.objects` property always returns an empty list, even though the `SemanticLevel` class has an `objects` member variable designed to hold objects associated with that level.  Objects are only accessible through `SemanticLevel.regions` or `SemanticRegion.objects`, but not through `SemanticLevel.objects`.

**Both loading methods exhibit the same bug** since they use the same underlying C++ loading code. 

## Steps to Reproduce

Steps to reproduce the behavior:

1. Create a Python script with the following code:  

```python
from typing import cast

import habitat_sim
from habitat_sim.geo import FRONT, GRAVITY
from habitat_sim.scene import SemanticScene
from magnum import Quaternion, Vector4


def show_objects(scene: SemanticScene):
    # Iterate through levels
    for level in scene.levels:
        print(
            f"Level {level.id}: {len(level.objects)} objects, {len(level.regions)} regions"
        )  # Always prints 0

        # Objects ARE accessible through regions
        for region in level.regions:
            print(f"  Region {region.id}: {len(region.objects)} objects")  # Non-zero


print("====== Via SemanticScene.load_mp3d_house ======")

_HABITAT_MP3D_ROTATION_QUATERNION = Quaternion.rotation(FRONT, GRAVITY)  # type: ignore
HABITAT_MP3D_ROTATION_VECTOR = cast(Vector4, _HABITAT_MP3D_ROTATION_QUATERNION.xyzw)  # type: ignore
scene = SemanticScene()
house_file = "data/scene_datasets/mp3d/v1/mp3d/1LXtFkjw3qL/1LXtFkjw3qL.house"  # path to some house file
SemanticScene.load_mp3d_house(house_file, scene, HABITAT_MP3D_ROTATION_VECTOR)
show_objects(scene)

print("====== Via Simulator ======")

cfg = habitat_sim.simulator.Configuration(
    habitat_sim.simulator.SimulatorConfiguration(),
    [habitat_sim.agent.AgentConfiguration()],
)
cfg.sim_cfg.scene_id = "data/scene_datasets/mp3d/v1/mp3d/1LXtFkjw3qL/1LXtFkjw3qL.glb"
sim = habitat_sim.simulator.Simulator(cfg)

scene = sim.semantic_scene
show_objects(scene)
```

2. Run the script: `python3 habitat-bug-repro.py`

3. Observe that all levels report `0 objects` in both loading methods, while regions contain many objects:

```bash
====== Via SemanticScene.load_mp3d_house ======
Level 0: 0 objects, 16 regions
  Region 0_13: 6 objects
  Region 0_15: 18 objects
  Region 0_16: 16 objects
  Region 0_17: 12 objects
  Region 0_18: 17 objects
  Region 0_19: 14 objects
  Region 0_20: 15 objects
  Region 0_21: 13 objects
  Region 0_22: 6 objects
  Region 0_23: 8 objects
  Region 0_24: 4 objects
  Region 0_25: 22 objects
  Region 0_26: 16 objects
  Region 0_28: 6 objects
  Region 0_29: 19 objects
  Region 0_30: 15 objects
Level 1: 0 objects, 10 regions
  Region 1_5: 30 objects
  Region 1_6: 58 objects
  Region 1_7: 17 objects
  Region 1_8: 25 objects
  Region 1_9: 3 objects
  Region 1_10: 24 objects
  Region 1_11: 40 objects
  Region 1_12: 20 objects
  Region 1_14: 20 objects
  Region 1_27: 9 objects
Level 2: 0 objects, 5 regions
  Region 2_0: 30 objects
  Region 2_1: 12 objects
  Region 2_2: 14 objects
  Region 2_3: 16 objects
  Region 2_4: 25 objects
====== Via Simulator ======
[06:52:47:578779]:[Warning]:[Metadata] SceneDatasetAttributes.cpp(107)::addNewSceneInstanceToDataset : Dataset : 'default' : Lighting Layout Attributes 'no_lights' specified in Scene Attributes but does not exist in dataset, so creating default.
Level 0: 0 objects, 16 regions
  Region 0_13: 6 objects
  Region 0_15: 18 objects
  Region 0_16: 16 objects
  Region 0_17: 12 objects
  Region 0_18: 17 objects
  Region 0_19: 14 objects
  Region 0_20: 15 objects
  Region 0_21: 13 objects
  Region 0_22: 6 objects
  Region 0_23: 8 objects
  Region 0_24: 4 objects
  Region 0_25: 22 objects
  Region 0_26: 16 objects
  Region 0_28: 6 objects
  Region 0_29: 19 objects
  Region 0_30: 15 objects
Level 1: 0 objects, 10 regions
  Region 1_5: 30 objects
  Region 1_6: 58 objects
  Region 1_7: 17 objects
  Region 1_8: 25 objects
  Region 1_9: 3 objects
  Region 1_10: 24 objects
  Region 1_11: 40 objects
  Region 1_12: 20 objects
  Region 1_14: 20 objects
  Region 1_27: 9 objects
Level 2: 0 objects, 5 regions
  Region 2_0: 30 objects
  Region 2_1: 12 objects
  Region 2_2: 14 objects
  Region 2_3: 16 objects
  Region 2_4: 25 objects
```

As shown, all levels report `0 objects` in both loading methods, but each level contains multiple regions with many objects.

## Expected behavior

The `SemanticLevel.objects` property should return all objects within that level, aggregated from all regions in the level, similar to how `SemanticLevel.regions` returns all regions for that level.

## Additional context

**Root Cause:**

Both loading methods use the same underlying C++ code. When you call `sim.semantic_scene`, it returns the semantic scene loaded by the `ResourceManager`, which ultimately calls the same `buildMp3dHouse` function.

Looking at the MP3D loader implementation in [`src/esp/scene/Mp3dSemanticScene.cpp`](https://github.com/facebookresearch/habitat-sim/blob/6c26c4f9ae10ad7f534aa9e59a84dff00782741e/src/esp/scene/Mp3dSemanticScene.cpp#L262-L277), objects are only added to regions, never to levels: 

```cpp
case 'O':  {  // object
  // ...  parsing code ... 
  if (object->parentIndex_ >= 0) {
    object->region_ = scene. regions_[object->parentIndex_];
    object->region_->objects_.push_back(object);  // Added to region
    // MISSING: Never added to level->objects_
  }
  break;
}
```

In contrast, regions ARE properly linked to levels:

```cpp
case 'R':  {  // region
  // ...  parsing code ...
  if (region->parentIndex_ >= 0) {
    region->level_ = scene.levels_[region->parentIndex_];
    region->level_->regions_.push_back(region);  // Regions added to level
  }
  break;
}
```

The `SemanticLevel` class ([`src/esp/scene/SemanticScene.h: 436-463`](https://github.com/facebookresearch/habitat-sim/blob/6c26c4f9ae10ad7f534aa9e59a84dff00782741e/src/esp/scene/SemanticScene.h#L436-L463)) has the data structure in place but the `objects_` vector is never populated by the MP3D loader.

**Workaround:**

Until fixed, manually aggregate objects from regions:

```python
for level in scene.levels:
    level_objects = []
    for region in level.regions:
        level_objects.extend(region.objects)
    print(f"Level {level.id}: {len(level_objects)} objects (manual aggregation)")
```

**Proposed Fix:**

In `src/esp/scene/Mp3dSemanticScene.cpp`, after adding an object to a region, also add it to the region's parent level:

```cpp
if (object->parentIndex_ >= 0) {
  object->region_ = scene.regions_[object->parentIndex_];
  object->region_->objects_.push_back(object);
  
  // Add to level if region has a parent level
  if (object->region_->level_) {
    object->region_->level_->objects_.push_back(object);
  }
}
```

## System Info

Please copy and paste the output from the
[environment collection script](https://github.com/facebookresearch/habitat-sim/tree/main/habitat_sim/utils/collect_env. py)
(or fill out the checklist below manually).

You can run the script with:
```
# For security purposes, please check the contents of collect_env.py before running it.
python src_python/habitat_sim/utils/collect_env.py
```

 - **OS (e.g., Linux)**:  Linux (Ubuntu 20.04 LTS)
   ```
   Linux xxx 5.15.0-139-generic #149~20.04.1-Ubuntu SMP Wed Apr 16 08:29:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   ```
 - **How you installed PyTorch (`conda`, `pip`, source)**: N/A (not using PyTorch directly)
 - **Build command you used (if compiling from source)**: N/A (installed via conda)
 - **Python version**:  3.9.23
 - **GPU models and configuration**: N/A (bug occurs in data loading, not GPU-related)
 - **CUDA version**: N/A
 - **PIP/conda dependencies packages versions**: `habitat-sim=0.3.3` (installed via conda)
 - **Any other relevant information**: 
   - Dataset:  Matterport3D (MP3D)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`SemanticLevel.objects` attribute is empty #2597

Habitat-Sim version

🐛 Bug

Steps to Reproduce

Expected behavior

Additional context

System Info

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

SemanticLevel.objects attribute is empty #2597

Description

Habitat-Sim version

🐛 Bug

Steps to Reproduce

Expected behavior

Additional context

System Info

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`SemanticLevel.objects` attribute is empty #2597