Replace TVM with Custom Transpiler in Forge Compilation Process

[chandrasekaranpradeep]

Summary

This issue tracks the development and integration of a custom transpiler to replace TVM in the Forge compilation process. The transpiler converts framework-level models (ONNX, and eventually other frameworks) to Forge's Intermediate Representation (TIRGraph), which is then mapped to Forge operations to create Forge modules.

Motivation

Why Replace TVM?

TVM (Apache TVM) is a comprehensive deep learning compiler stack that includes both:

• Runtime components: Execution engine, memory management, device management
• Compilation components: Graph optimization, code generation, operator fusion

Current Problem:

• TVM's runtime components are unnecessary for Forge, as we have our own runtime
• TVM's compilation path adds complexity and intermediate representations (TVM Relay IR → JSON Graphs → Python Forge Module)
• We need more control over the conversion process and framework-specific optimizations
• TVM's dual nature (runtime + compiler) creates unnecessary dependencies

Solution:

• Create a lightweight, purpose-built transpiler that focuses solely on compilation
• Direct conversion: Framework Model → Transpiler → Forge Module
• Full control over model convertion pipeline, optimization passes and code generation

Architecture Overview

Current Compilation Paths

The Forge compiler currently supports two paths for generating Forge modules:

Path 1 (Existing): TVM Relay IR Path
Framework Model → TVM Relay IR → JSON Graphs → Python Forge Module

Path 2 (New): TIR Transpiler Path  
Framework Model → TIRGraph → Python Forge Module

Target Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Framework Models                              │
│  ONNX │ PaddlePaddle │ TensorFlow │ (Other frameworks)          │
└──────────────────────┬──────────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────────┐
│              Transpiler Frontends                                │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │ ONNX Frontend│  │ PaddlePaddle │  │ TensorFlow   │          │
│  │              │  │   Frontend   │  │   Frontend   │          │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘          │
└─────────┼──────────────────┼──────────────────┼─────────────────┘
          │                  │                  │
          └──────────────────┼──────────────────┘
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│         TIRGraph (Transpiler Intermediate Representation)        │
│  • Framework-agnostic computational graph                        │
│  • Similar to TVM Relay IR but Forge-specific                   │
│  • Contains: nodes, inputs, outputs, params, constants           │
└──────────────────────┬──────────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────────┐
│              Code Generation (TranspilerCodeGenerator)           │
│  • TIRGraph → Python Forge Module Code                          │
│  • Maps TIR operations to Forge operations                      │
│  • Generates executable Python code                              │
└──────────────────────┬──────────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Forge Module                                  │
│  • Executable Python module with Forge operations                │
│  • Ready for Forge compilation passes                            │
└──────────────────────┬──────────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────────┐
│            Forge Compilation Pipeline                            │
│  • Graph optimization passes                                     │
│  • MLIR compilation                                              │
│  • Binary generation                                             │
└─────────────────────────────────────────────────────────────────┘