The reproduction results differed significantly from those in the article.

[lixin-zhan]

Thank you for your excellent work contributing to the community!

I am currently reproducing the ScanNet validation results. The results are significantly different from the metrics of 55.0, 76.9, and 88.2 (AP(%), AP@50(%), AP@25(%)) in the paper.

I processed the data according to the readme and performed Discerning Module training (sparseunet) and Completion Module training (AdaPoinTr).

The logs are as follows:
2026-07-01 22:42:02,406 - train - INFO - Epoch 20/20
2026-07-01 22:42:43,327 - train - INFO - Train Loss: 0.1005, Train PointAcc: 0.9723
2026-07-01 22:42:49,427 - train - INFO - ShapeNet Val Loss: 0.0909, Val PointAcc: 0.9759
2026-07-01 22:42:49,427 - train - INFO - ShapeNet validation visualization files
2026-07-01 22:42:50,046 - train - INFO - New best model saved with val_point_acc: 0.9759
2026-07-01 22:42:50,505 - train - INFO - Checkpoint saved for epoch 20
2026-07-01 22:42:50,505 - train - INFO - Training completed! Best validation point accuracy: 0.9759

2026-07-02 15:15:47,582 - AdaPoinTr - INFO - [Epoch 273/300][Batch 1/148] BatchTime = 11.665 (s) DataTime = 10.973 (s) Losses = ['17.9914', '89.0973'] lr = 0.000025
2026-07-02 15:20:34,135 - AdaPoinTr - INFO - [Epoch 273/300][Batch 101/148] BatchTime = 12.006 (s) DataTime = 11.306 (s) Losses = ['18.2599', '88.8213'] lr = 0.000025
2026-07-02 15:22:33,177 - AdaPoinTr - INFO - [Training] EPOCH: 273 EpochTime = 417.257 (s) Losses = ['17.3236', '83.5122']
2026-07-02 15:22:33,180 - AdaPoinTr - INFO - [VALIDATION] Start validating epoch 273
2026-07-02 15:22:38,494 - AdaPoinTr - INFO - Test[67/677] Taxonomy = 2 Sample = tensor(66) Losses = ['34.6152', '3.0232', '28.2338', '1.9605'] Metrics = ['0.0601', '28.2338', '1.9605', '0.0000']
2026-07-02 15:22:40,504 - AdaPoinTr - INFO - Test[134/677] Taxonomy = 2 Sample = tensor(133) Losses = ['75.9641', '16.1089', '63.1367', '11.2670'] Metrics = ['0.0107', '63.1367', '11.2670', '0.0000']
2026-07-02 15:22:44,205 - AdaPoinTr - INFO - Test[201/677] Taxonomy = 2 Sample = tensor(200) Losses = ['53.1951', '7.5695', '41.1235', '4.7586'] Metrics = ['0.0347', '41.1235', '4.7586', '0.0000']
2026-07-02 15:22:57,333 - AdaPoinTr - INFO - Test[268/677] Taxonomy = 2 Sample = tensor(267) Losses = ['42.5562', '4.7803', '31.5042', '2.4846'] Metrics = ['0.0663', '31.5042', '2.4846', '0.0000']
2026-07-02 15:23:00,433 - AdaPoinTr - INFO - Test[335/677] Taxonomy = 2 Sample = tensor(334) Losses = ['42.3119', '4.5287', '36.9089', '3.4035'] Metrics = ['0.0400', '36.9089', '3.4035', '0.0000']
2026-07-02 15:23:13,774 - AdaPoinTr - INFO - Test[402/677] Taxonomy = 2 Sample = tensor(401) Losses = ['56.9935', '8.2331', '46.3031', '5.4537'] Metrics = ['0.0229', '46.3031', '5.4537', '0.0000']
2026-07-02 15:26:08,429 - AdaPoinTr - INFO - Test[469/677] Taxonomy = 2 Sample = tensor(468) Losses = ['33.3820', '3.3375', '26.7495', '1.8445'] Metrics = ['0.0557', '26.7495', '1.8445', '0.0000']
2026-07-02 15:26:12,449 - AdaPoinTr - INFO - Test[536/677] Taxonomy = 2 Sample = tensor(535) Losses = ['40.2985', '4.1853', '33.8746', '2.8723'] Metrics = ['0.0430', '33.8746', '2.8723', '0.0000']
2026-07-02 15:28:05,812 - AdaPoinTr - INFO - Test[603/677] Taxonomy = 2 Sample = tensor(602) Losses = ['41.4820', '5.1728', '32.4629', '2.6728'] Metrics = ['0.0512', '32.4629', '2.6728', '0.0000']
2026-07-02 15:32:58,563 - AdaPoinTr - INFO - Test[670/677] Taxonomy = 2 Sample = tensor(669) Losses = ['42.5027', '4.7288', '33.8338', '2.8408'] Metrics = ['0.0454', '33.8338', '2.8408', '0.0000']
2026-07-02 15:33:54,148 - AdaPoinTr - INFO - [Validation] EPOCH: 273 Metrics = ['0.0436', '37.2733', '4.0746', '0.0000']
2026-07-02 15:33:54,151 - AdaPoinTr - INFO - ============================ TEST RESULTS ============================
2026-07-02 15:33:54,152 - AdaPoinTr - INFO - Taxonomy #Sample F-Score CDL1 CDL2 EMDistance
2026-07-02 15:33:54,152 - AdaPoinTr - INFO - 2 677 0.044 37.273 4.075 0.000
2026-07-02 15:33:54,152 - AdaPoinTr - INFO - Overall 0.044 37.273 4.075 0.000

train_scannet_vae：
2026-07-07 02:09:56 Epoch: 591 [151/151 (100%)]89392, Loss: 45.209, ppo: 1.563, actor: 0.037, critic: 1.636, ent: 1.107, StepR: -0.01, seg: 43.646, mask: 2.087, dice: 15.018, class: 1.589, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.28, infer time: 10.3s, data time: 325.3s, optimize time: 107.2s, Elapsed time: 442.8s (302 iters)
2026-07-07 02:09:56,474:INFO: 50iou percent: 0.251, 25iou percent: 0.364, AVG iou: 0.219, AVG 50iou: 0.674, AVG 25iou: 0.589)
2026-07-07 02:17:33,004:INFO: 2026-07-07 02:17:33 Epoch: 592 [151/151 (100%)]89543, Loss: 45.208, ppo: 1.526, actor: 0.037, critic: 1.598, ent: 1.092, StepR: -0.02, seg: 43.681, mask: 2.087, dice: 15.031, class: 1.592, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.29, infer time: 10.3s, data time: 329.1s, optimize time: 107.3s, Elapsed time: 446.6s (302 iters)
2026-07-07 02:17:33,004:INFO: 50iou percent: 0.261, 25iou percent: 0.360, AVG iou: 0.222, AVG 50iou: 0.678, AVG 25iou: 0.599)
2026-07-07 02:25:08,931:INFO: 2026-07-07 02:25:08 Epoch: 593 [151/151 (100%)]89694, Loss: 45.049, ppo: 1.423, actor: 0.037, critic: 1.498, ent: 1.116, StepR: -0.01, seg: 43.627, mask: 2.087, dice: 14.996, class: 1.601, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.30, infer time: 10.3s, data time: 328.0s, optimize time: 107.4s, Elapsed time: 445.7s (302 iters)
2026-07-07 02:25:08,932:INFO: 50iou percent: 0.252, 25iou percent: 0.354, AVG iou: 0.215, AVG 50iou: 0.673, AVG 25iou: 0.594)
2026-07-07 02:32:45,223:INFO: 2026-07-07 02:32:45 Epoch: 594 [151/151 (100%)]89845, Loss: 45.307, ppo: 1.505, actor: 0.040, critic: 1.577, ent: 1.116, StepR: -0.04, seg: 43.801, mask: 2.094, dice: 15.060, class: 1.605, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.35, infer time: 10.3s, data time: 328.4s, optimize time: 107.6s, Elapsed time: 446.3s (302 iters)
2026-07-07 02:32:45,223:INFO: 50iou percent: 0.257, 25iou percent: 0.359, AVG iou: 0.220, AVG 50iou: 0.679, AVG 25iou: 0.597)
2026-07-07 02:40:20,748:INFO: 2026-07-07 02:40:20 Epoch: 595 [151/151 (100%)]89996, Loss: 45.230, ppo: 1.498, actor: 0.042, critic: 1.567, ent: 1.101, StepR: -0.02, seg: 43.732, mask: 2.091, dice: 15.037, class: 1.601, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.31, infer time: 10.2s, data time: 328.7s, optimize time: 106.7s, Elapsed time: 445.5s (302 iters)
2026-07-07 02:40:20,748:INFO: 50iou percent: 0.256, 25iou percent: 0.348, AVG iou: 0.215, AVG 50iou: 0.680, AVG 25iou: 0.604)
2026-07-07 02:47:57,368:INFO: 2026-07-07 02:47:57 Epoch: 596 [151/151 (100%)]90147, Loss: 45.179, ppo: 1.463, actor: 0.041, critic: 1.531, ent: 1.091, StepR: -0.01, seg: 43.716, mask: 2.089, dice: 15.058, class: 1.577, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.29, infer time: 10.2s, data time: 329.1s, optimize time: 106.8s, Elapsed time: 446.2s (302 iters)
2026-07-07 02:47:57,368:INFO: 50iou percent: 0.252, 25iou percent: 0.362, AVG iou: 0.221, AVG 50iou: 0.684, AVG 25iou: 0.595)
2026-07-07 02:55:34,499:INFO: 2026-07-07 02:55:34 Epoch: 597 [151/151 (100%)]90298, Loss: 45.192, ppo: 1.549, actor: 0.039, critic: 1.622, ent: 1.112, StepR: -0.03, seg: 43.643, mask: 2.085, dice: 15.019, class: 1.589, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.32, infer time: 10.2s, data time: 329.7s, optimize time: 107.1s, Elapsed time: 447.1s (302 iters)
2026-07-07 02:55:34,500:INFO: 50iou percent: 0.253, 25iou percent: 0.358, AVG iou: 0.220, AVG 50iou: 0.685, AVG 25iou: 0.600)
2026-07-07 03:03:10,909:INFO: 2026-07-07 03:03:10 Epoch: 598 [151/151 (100%)]90449, Loss: 45.192, ppo: 1.496, actor: 0.040, critic: 1.567, ent: 1.108, StepR: -0.04, seg: 43.697, mask: 2.086, dice: 15.054, class: 1.579, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.36, infer time: 10.3s, data time: 328.6s, optimize time: 107.1s, Elapsed time: 446.0s (302 iters)
2026-07-07 03:03:10,909:INFO: 50iou percent: 0.249, 25iou percent: 0.357, AVG iou: 0.218, AVG 50iou: 0.682, AVG 25iou: 0.595)
2026-07-07 03:10:46,081:INFO: 2026-07-07 03:10:46 Epoch: 599 [151/151 (100%)]90600, Loss: 45.230, ppo: 1.571, actor: 0.039, critic: 1.644, ent: 1.121, StepR: -0.03, seg: 43.659, mask: 2.088, dice: 15.024, class: 1.586, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.31, infer time: 10.3s, data time: 328.3s, optimize time: 107.0s, Elapsed time: 445.6s (302 iters)
2026-07-07 03:10:46,081:INFO: 50iou percent: 0.257, 25iou percent: 0.363, AVG iou: 0.223, AVG 50iou: 0.682, AVG 25iou: 0.599)
2026-07-08 22:49:29,106:INFO: Transformer with config {'NAME': 'AdaPoinTr', 'num_query': 512, 'num_points': 1024, 'center_num': [512, 256], 'global_feature_dim': 1024, 'encoder_type': 'graph', 'decoder_type': 'fc', 'encoder_config': {'embed_dim': 384, 'depth': 6, 'num_heads': 6, 'k': 8, 'n_group': 2, 'mlp_ratio': 2.0, 'block_style_list': ['attn-graph', 'attn', 'attn', 'attn', 'attn', 'attn'], 'combine_style': 'concat'}, 'decoder_config': {'embed_dim': 384, 'depth': 8, 'num_heads': 6, 'k': 8, 'n_group': 2, 'mlp_ratio': 2.0, 'self_attn_block_style_list': ['attn-graph', 'attn', 'attn', 'attn', 'attn', 'attn', 'attn', 'attn'], 'self_attn_combine_style': 'concat', 'cross_attn_block_style_list': ['attn-graph', 'attn', 'attn', 'attn', 'attn', 'attn', 'attn', 'attn'], 'cross_attn_combine_style': 'concat'}}
2026-07-08 22:57:17,676:INFO: 2026-07-08 22:57:17 Epoch: 590 [151/151 (100%)]89241, Loss: 45.246, ppo: 1.453, actor: 0.029, critic: 1.533, ent: 1.100, StepR: -0.02, seg: 43.794, mask: 2.097, dice: 15.059, class: 1.594, evo_discern: 0.000, lr: 1.000e-04, Traj: 6.29, infer time: 11.8s, data time: 327.3s, optimize time: 117.4s, Elapsed time: 456.4s (302 iters)
2026-07-08 22:57:17,676:INFO: 50iou percent: 0.261, 25iou percent: 0.365, AVG iou: 0.223, AVG 50iou: 0.674, AVG 25iou: 0.597)
2026-07-08 22:57:51,082:INFO:
2026-07-08 22:57:51,082:INFO: ####################################################################################################
2026-07-08 22:57:51,082:INFO: what : AP AP_50% AP_25% | RC RC_50% RC_25% | PR PR_50% PR_25%
2026-07-08 22:57:51,082:INFO: ####################################################################################################
2026-07-08 22:57:51,082:INFO: class_agnostic : 0.149 0.410 0.697 | 0.237 0.515 0.756 | 0.217 0.503 0.732
2026-07-08 22:57:51,082:INFO: ----------------------------------------------------------------------------------------------------
2026-07-08 22:57:51,082:INFO: average : 0.149 0.410 0.697 | 0.237 0.515 0.756 | 0.217 0.503 0.732
2026-07-08 22:57:51,082:INFO: ####################################################################################################

I would greatly appreciate any suggestions you may have! Thank you for your time.

Additionally, I noticed that the Segmentation Network weights in the provided link (https://drive.google.com/file/d/1WoBWTSOvgg4SP_1363y_DjHVrIkBj-Z8/view?usp=sharing) seem to mismatch with the code. Could this be due to the code not being updated?

I look forward to your reply and assistance; I would be extremely grateful.

[D2Simon]

First of all, thank you so much for your interest in our work. However, the results log you obtained appear highly unusual, and I suspect that there may have been an issue with one of the steps in the reproduction process.

Regarding the incorrect link you mentioned, there are two weight links provided in the README. One is the GrabS weight link, which is mainly used to provide pretrained weights for the object-centric network. The other link, [https://drive.google.com/file/d/19pyCfWN7W-vvtqUY5tyyGWBUugf0Gjjw/view], contains the EvObj weights.

I have also sent you an email. I hope I can help you to resolve the issue you encountered during reproduction.

[D2Simon]

@lixin-zhan Hi, I think you could firstly try running the complete module and discerning module using the checkpoints we provided. I rerun the code before uploading it, and the results could be reproduced correctly. I hope this suggestion can help you.