Skip to content

Conversation

@tapspatel
Copy link
Collaborator

@tapspatel tapspatel commented Nov 26, 2025

Compiling a module gives back the mlir file and a golden dictionary

mlir_path, goldens = compile_ttir_module_to_flatbuffer(
            split_module,
            split_builder,
            system_desc_path=system_desc_path,
            test_base=test_base,
            output_root=output_root,
            target="ttnn",
            module_dump=True,
        )

this can be fed into the runtime API for golden evaluation

golden_report = execute_fb(flatbuffer_path, goldens, device=device)

A golden report is returned which has the following format for each op executed on device

{
    'loc("-":9:10)': {
            device_0: {
                'result': 'pass', 
                'expected_pcc': 0.99, 
                'actual_pcc': 1.0, 
                'expected_atol': 1e-08, 
                'actual_atol': 0.0, 
                'expected_rtol': 1e-05, 
                'actual_rtol': 0.0, 
                'allclose': True, 
                'max': 0.0, 
                'mean_absolute_error': 0.0, 
                'root_mean_square_error': 0.0, 
                'cosine_similarity': 1.0000001192092896
            }
        }
    }

@tapspatel tapspatel self-assigned this Nov 26, 2025
@tapspatel tapspatel requested review from a team, ctodTT, dlokeTT and jgrimTT as code owners November 26, 2025 03:45
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.35%. Comparing base (da89b93) to head (063ea62).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6038   +/-   ##
=======================================
  Coverage   69.34%   69.35%           
=======================================
  Files         334      334           
  Lines       51001    51001           
=======================================
+ Hits        35369    35373    +4     
+ Misses      15632    15628    -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants