[Subgraph] Enabling a feature to manage a graph in subgraph level. #2864
Description
Abstract
To accommodate heterogeneous compute engines, it becomes necessary to separately administer distinct graph regions — referred to as subgraphs — each tailored to function according to specific compute engines. This approach differs from our current method of managing only one graph solely at the layer level; therefore, adding a new tier labeled 'subgraph' would allow us to structure our system hierarchically as follows: Graph > Subgraphs > Layers.
Requirements for Subgraphs
- Each subgraph type should be able to define its own operations (e.g., forwarding, backwarding).
- Currently, NNTrainer uses GraphCore to manage layer information within a graph, and the operations (e.g., forwarding, backwarding) in this graph are given from higher levels of the network.
- These functions need to be modified so they can be managed at the subgraph level.
- Subgraphs should be able to define their operations based on the compute engine type (e.g., CPU, NPU, GPU).
- Both topological sorting at the subgraph level and at the layer level are needed.
- Networks should be saved separately for each subgraph.
- This feature should support backward compatibility.
- Various subgraph type should be pluggable
Idea to implement
- In order to fulfill the requirements, I'm planning to implement base class for subgraph that inherits
GraphNode. By doing so, we can manage subgraphs with NetworkCore, which supports topological sort and node management. - As an interface to add a subgraph, I'm planning to add a new property used when
createLayeris called, with a default value.