Skip to content

TE2.13: AttributeError: 'types.SimpleNamespace' object has no attribute 'cuda_graph_scope'. #3801

@ko3n1g

Description

@ko3n1g

Describe the bug

2026-03-11T11:18:12.4937444Z 2-task-1-0/0 [default0]:tests/unit_tests/transformer/test_vision_cuda_graphs.py::TestVisionLayerIsGraphable::test_wrong_cuda_graph_impl_returns_false [default0]:FAILED[default0]:
2026-03-11T11:18:12.4941011Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4942176Z 2-task-1-0/0 [default0]:=================================== FAILURES ===================================
2026-03-11T11:18:12.4943134Z 2-task-1-0/0 [default0]:_____ TestVisionLayerIsGraphable.test_wrong_cuda_graph_impl_returns_false ______
2026-03-11T11:18:12.4943637Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4944148Z 2-task-1-0/0 [default0]:self = <tests.unit_tests.transformer.test_vision_cuda_graphs.TestVisionLayerIsGraphable object at 0x7669650ef0e0>
2026-03-11T11:18:12.4944696Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4945000Z 2-task-1-0/0 [default0]:    def test_wrong_cuda_graph_impl_returns_false(self):
2026-03-11T11:18:12.4945519Z 2-task-1-0/0 [default0]:        from megatron.core.transformer.transformer_layer import TransformerLayer
2026-03-11T11:18:12.4945933Z 2-task-1-0/0 [default0]:    
2026-03-11T11:18:12.4946249Z 2-task-1-0/0 [default0]:        config = SimpleNamespace(cuda_graph_impl="local")
2026-03-11T11:18:12.4946669Z 2-task-1-0/0 [default0]:        layer = MagicMock(spec=TransformerLayer)
2026-03-11T11:18:12.4947089Z 2-task-1-0/0 [default0]:        # isinstance check with MagicMock(spec=...) should pass
2026-03-11T11:18:12.4947789Z 2-task-1-0/0 [default0]:>       assert _layer_is_graphable(layer, config) is False
2026-03-11T11:18:12.4948297Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4948600Z 2-task-1-0/0 [default0]:tests/unit_tests/transformer/test_vision_cuda_graphs.py:56: 
2026-03-11T11:18:12.4949053Z 2-task-1-0/0 [default0]:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2026-03-11T11:18:12.4949394Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4949698Z 2-task-1-0/0 [default0]:layer = <MagicMock spec='TransformerLayer' id='130166166340384'>
2026-03-11T11:18:12.4950124Z 2-task-1-0/0 [default0]:config = namespace(cuda_graph_impl='local')
2026-03-11T11:18:12.4950432Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4950692Z 2-task-1-0/0 [default0]:    def _layer_is_graphable(layer, config):
2026-03-11T11:18:12.4950990Z 2-task-1-0/0 [default0]:        """
2026-03-11T11:18:12.4951255Z 2-task-1-0/0 [default0]:        Check if a layer is graphable.
2026-03-11T11:18:12.4951540Z 2-task-1-0/0 [default0]:        """
2026-03-11T11:18:12.4951783Z 2-task-1-0/0 [default0]:    
2026-03-11T11:18:12.4952071Z 2-task-1-0/0 [default0]:        # Only GraphableMegatronModule can be graphed.
2026-03-11T11:18:12.4952543Z 2-task-1-0/0 [default0]:        if not isinstance(layer, GraphableMegatronModule):
2026-03-11T11:18:12.4952911Z 2-task-1-0/0 [default0]:            return False
2026-03-11T11:18:12.4953152Z 2-task-1-0/0 [default0]:    
2026-03-11T11:18:12.4953476Z 2-task-1-0/0 [default0]:        # If cuda_graph_scope is not set, every layer is graphed.
2026-03-11T11:18:12.4953873Z 2-task-1-0/0 [default0]:>       if not config.cuda_graph_scope:
2026-03-11T11:18:12.4954627Z 2-task-1-0/0 [default0]:E       AttributeError: 'types.SimpleNamespace' object has no attribute 'cuda_graph_scope'. Did you mean: 'cuda_graph_impl'?
2026-03-11T11:18:12.4955176Z 2-task-1-0/0 [default0]:
2026-03-11T11:18:12.4955490Z 2-task-1-0/0 [default0]:megatron/core/transformer/cuda_graphs.py:1650: AttributeError
2026-03-11T11:18:12.4955950Z 2-task-1-0/0 [default0]:=============================== warnings summary ===============================

Steps/Code to reproduce bug

https://github.com/NVIDIA/Megatron-LM/actions/runs/22948551554/attempts/1

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions