Skip to content

TorchMeterโฒ๏ธ: Your all-in-one tool for Pytorch model analysis, measuring Params, FLOPs/MACs(aka. MACC or MADD), Memory cost, Inference time and Throughput! ! ๐Ÿš€

License

Notifications You must be signed in to change notification settings

TorchMeter/torchmeter

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

35 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

TorchMeter Banner

๐Ÿš€ ๐’€๐’๐’–๐’“ ๐‘จ๐’๐’-๐’Š๐’-๐‘ถ๐’๐’† ๐‘ป๐’๐’๐’ ๐’‡๐’๐’“ ๐‘ท๐’š๐’•๐’๐’“๐’„๐’‰ ๐‘ด๐’๐’…๐’†๐’ ๐‘จ๐’๐’‚๐’๐’š๐’”๐’Š๐’” ๐Ÿš€

PyPI-Version Python-Badge Pytorch-Badge Coverage-Badge Ruff-Badge License-Badge

  • Docs: https://docs.torchmeter.top (Backup link ๐Ÿ”—)
  • Intro: Provides comprehensive measurement of Pytorch model's Parameters, FLOPs/MACs, Memory-Cost, Inference-Time and Throughput with highly customizable result display โœจ

๐’œ. ๐ป๐’พ๐‘”๐’ฝ๐“๐’พ๐‘”๐’ฝ๐“‰๐“ˆ

โ‘  ๐’๐’†๐’“๐’-๐‘ฐ๐’๐’•๐’“๐’–๐’”๐’Š๐’๐’ ๐‘ท๐’“๐’๐’™๐’š
  • Acts as drop-in decorator without any changes of the underlying model
  • Seamlessly integrates with Pytorch modules while preserving full compatibility (attributes and methods)
โ‘ก ๐‘ญ๐’–๐’๐’-๐‘บ๐’•๐’‚๐’„๐’Œ ๐‘ด๐’๐’…๐’†๐’ ๐‘จ๐’๐’‚๐’๐’š๐’•๐’Š๐’„๐’”

Holistic performance analytics across 5 dimensions:

  1. Parameter Analysis

    • Total/trainable parameter quantification
    • Layer-wise parameter distribution analysis
    • Gradient state tracking (requires_grad flags)
  2. Computational Profiling

    • FLOPs/MACs precision calculation
    • Operation-wise calculation distribution analysis
    • Dynamic input/output detection (number, type, shape, ...)
  3. Memory Diagnostics

    • Input/output tensor memory awareness
    • Hierarchical memory consumption analysis
  4. Inference latency & 5. Throughput benchmarking

    • Auto warm-up phase execution (eliminates cold-start bias)
    • Device-specific high-precision timing
    • Inference latency & Throughput Benchmarking
โ‘ข ๐‘น๐’Š๐’„๐’‰ ๐’—๐’Š๐’”๐’–๐’‚๐’๐’Š๐’›๐’‚๐’•๐’Š๐’๐’
  1. Programmable tabular report

    • Dynamic table structure adjustment
    • Style customization and real-time rendering
    • Real-time data analysis in programmable way
  2. Rich-text hierarchical operation tree

    • Style customization and real-time rendering
    • Smart module folding based on structural equivalence detection for intuitive model structure insights
โ‘ฃ ๐‘ญ๐’Š๐’๐’†-๐‘ฎ๐’“๐’‚๐’Š๐’๐’†๐’… ๐‘ช๐’–๐’”๐’•๐’๐’Ž๐’Š๐’›๐’‚๐’•๐’Š๐’๐’
  • Real-time hot-reload rendering: Dynamic adjustment of rendering configuration for operation trees, report tables and their nested components
  • Progressive update: Namespace assignment + dictionary batch update
โ‘ค ๐‘ช๐’๐’๐’‡๐’Š๐’ˆ-๐‘ซ๐’“๐’Š๐’—๐’†๐’ ๐‘น๐’–๐’๐’•๐’Š๐’Ž๐’† ๐‘ด๐’‚๐’๐’‚๐’ˆ๐’†๐’Ž๐’†๐’๐’•
  • Centralized control: Singleton-managed global configuration for dynamic behavior adjustment
  • Portable presets: Export/import YAML profiles for runtime behaviors, eliminating repetitive setup
โ‘ฅ ๐‘ท๐’๐’“๐’•๐’‚๐’ƒ๐’Š๐’๐’Š๐’•๐’š ๐’‚๐’๐’… ๐‘ท๐’“๐’‚๐’„๐’•๐’Š๐’„๐’‚๐’๐’Š๐’•๐’š
  • Decoupled pipeline: Separation of data collection and visualization
  • Automatic device synchronization: Maintains production-ready status by keeping model and data co-located
  • Dual-mode reporting with export flexibility:
    • Measurement units mode vs. raw data mode
    • Multi-format export (CSV/Excel) for analysis integration

โ„ฌ. ๐ผ๐“ƒ๐“ˆ๐“‰๐’ถ๐“๐“๐’ถ๐“‰๐’พ๐‘œ๐“ƒ

Note

๐‘ช๐’๐’Ž๐’‘๐’‚๐’•๐’Š๐’ƒ๐’Š๐’๐’Š๐’•๐’š

  • OS: windows / linux / macOS
  • Python: >= 3.8
  • Pytorch: >= 1.7.0
โ‘  ๐‘ป๐’‰๐’“๐’๐’–๐’ˆ๐’‰ ๐‘ท๐’š๐’•๐’‰๐’๐’ ๐‘ท๐’‚๐’„๐’Œ๐’‚๐’ˆ๐’† ๐‘ด๐’‚๐’๐’‚๐’ˆ๐’†๐’“

the most convenient way, suitable for installing the released latest stable version

# pip series
pip/pipx/pipenv install torchmeter

# Or via conda
conda install torchmeter

# Or via uv
uv add torchmeter

# Or via poetry
poetry add torchmeter

# Other managers' usage please refer to their own documentation
โ‘ก ๐‘ป๐’‰๐’“๐’๐’–๐’ˆ๐’‰ ๐‘ฉ๐’Š๐’๐’‚๐’“๐’š ๐‘ซ๐’Š๐’”๐’•๐’“๐’Š๐’ƒ๐’–๐’•๐’Š๐’๐’

Suitable for installing released historical versions

  1. Download .whl from PyPI or Github Releases.

  2. Install locally:

    # Replace x.x.x with actual version
    pip install torchmeter-x.x.x.whl  
โ‘ข ๐‘ป๐’‰๐’“๐’๐’–๐’ˆ๐’‰ ๐‘บ๐’๐’–๐’“๐’„๐’† ๐‘ช๐’๐’…๐’†

Suitable for who want to try out the upcoming features (may has unknown bugs).

git clone https://github.com/TorchMeter/torchmeter.git
cd torchmeter

# If you want to install the released stable version, use this: 
# Don't forget to eplace x.x.x with actual version
git checkout vx.x.x  # Stable

# If you want to try the latest development version(alpha/beta), use this:
git checkout master  # Development version

pip install .

๐’ž. ๐’ข๐‘’๐“‰๐“‰๐’พ๐“ƒ๐‘” ๐“ˆ๐“‰๐’ถ๐“‡๐“‰๐‘’๐’น

TorchMeter Demo Refer to tutorials for all scenarios

โ€Œโ‘  ๐‘ซ๐’†๐’๐’†๐’ˆ๐’‚๐’•๐’† ๐’š๐’๐’–๐’“ ๐’Ž๐’๐’…๐’†๐’ ๐’•๐’ ๐’•๐’๐’“๐’„๐’‰๐’Ž๐’†๐’•๐’†๐’“
Implementation of ExampleNet
import torch.nn as nn

class ExampleNet(nn.Module):
    def __init__(self):
        super(ExampleNet, self).__init__()
        
        self.backbone = nn.Sequential(
            self._nested_repeat_block(2),
            self._nested_repeat_block(2)
        )

        self.gap = nn.AdaptiveAvgPool2d(1)

        self.classifier = nn.Linear(3, 2)
    
    def _inner_net(self):
        return nn.Sequential(
            nn.Conv2d(10, 10, 1),
            nn.BatchNorm2d(10),
            nn.ReLU(),
        )

    def _nested_repeat_block(self, repeat:int=1):
        inners = [self._inner_net() for _ in range(repeat)]
        return nn.Sequential(
            nn.Conv2d(3, 10, 3, stride=1, padding=1),
            nn.BatchNorm2d(10),
            nn.ReLU(),
            *inners,
            nn.Conv2d(10, 3, 1),
            nn.BatchNorm2d(3),
            nn.ReLU()
        )

    def forward(self, x):
        x = self.backbone(x)
        x = self.gap(x)
        x = x.squeeze(dim=(2,3))
        return self.classifier(x)
import torch.nn as nn
from torchmeter import Meter
from torch.cuda import is_available as is_cuda

# 1๏ธโƒฃ Prepare your pytorch model, here is a simple examples
underlying_model = ExampleNet() # see above for implementation of `ExampleNet`

# Set an extra attribute to the model to show 
# how torchmeter acts as a zero-intrusion proxy later
underlying_model.example_attr = "ABC"

# 2๏ธโƒฃ Wrap your model with torchmeter
model = Meter(underlying_model)

# 3๏ธโƒฃ Validate the zero-intrusion proxy

# Get the model's attribute
print(model.example_attr)

# Get the model's method
# `_inner_net` is a method defined in the ExampleNet
print(hasattr(model, "_inner_net")) 

# Move the model to other device (now on cpu)
print(model)
if is_cuda():
    model.to("cuda")
    print(model) # now on cuda
โ‘ก ๐‘ฎ๐’†๐’• ๐’Š๐’๐’”๐’Š๐’ˆ๐’‰๐’•๐’” ๐’Š๐’๐’•๐’ ๐’•๐’‰๐’† ๐’Ž๐’๐’…๐’†๐’ ๐’”๐’•๐’“๐’–๐’„๐’•๐’–๐’“๐’†
from rich import print

print(model.structure)
โ‘ข ๐‘ธ๐’–๐’‚๐’๐’•๐’Š๐’‡๐’š ๐’Ž๐’๐’…๐’†๐’ ๐’‘๐’†๐’“๐’‡๐’๐’“๐’Ž๐’‚๐’๐’„๐’† ๐’‡๐’“๐’๐’Ž ๐’—๐’‚๐’“๐’Š๐’๐’–๐’” ๐’…๐’Š๐’Ž๐’†๐’๐’”๐’Š๐’๐’๐’”
# Parameter Analysis
# Suppose that the `backbone` part of ExampleNet is frozen
_ = model.backbone.requires_grad_(False)
print(model.param)
tb, data = model.profile('param', no_tree=True)

# Before measuring calculation you should first execute a feed-forward
# you do **not** need to concern about the device mismatch, 
# just feed the model with the input.
import torch
input = torch.randn(1, 3, 32, 32)
output = model(input) 

# Computational Profiling
print(model.cal) # `cal` for calculation
tb, data = model.profile('cal', no_tree=True)

# Memory Diagnostics
print(model.mem) # `mem` for memory
tb, data = model.profile('mem', no_tree=True)

# Performance Benchmarking
print(model.ittp) # `ittp` for inference time & throughput
tb, data = model.profile('ittp', no_tree=True)

# Overall Analytics
print(model.overview())
โ‘ฃ ๐‘ฌ๐’™๐’‘๐’๐’“๐’• ๐’“๐’†๐’”๐’–๐’๐’•๐’” ๐’‡๐’๐’“ ๐’‡๐’–๐’“๐’•๐’‰๐’†๐’“ ๐’‚๐’๐’‚๐’๐’š๐’”๐’Š๐’”
# export to csv
model.profile('param', show=False, save_to="params.csv")

# export to excel
model.profile('cal', show=False, save_to="../calculation.xlsx")
โ‘ค ๐‘จ๐’…๐’—๐’‚๐’๐’„๐’†๐’… ๐’–๐’”๐’‚๐’ˆ๐’†
  1. Attributes/methods access of underlying model
  2. Automatic device synchronization
  3. Smart module folding
  4. Performance gallery
  5. Customized visulization
  6. Best practice of programmable tabular report
  7. Instant export and postponed export
  8. Centralized configuration management
  9. Submodule exploration

๐’Ÿ. ๐’ž๐‘œ๐“ƒ๐“‰๐“‡๐’พ๐’ท๐“Š๐“‰๐‘’

Thank you for wanting to make TorchMeter even better!

There are several ways to make a contribution:

Before jumping in, let's ensure smooth collaboration by reviewing our ๐Ÿ“‹ contribution guidelines first.

Thanks again !

Note

@Ahzyuan: I'd like to say sorry in advance. Due to my master's studies and job search, I may be too busy in the coming year to address contributions promptly. I'll do my best to handle them as soon as possible. Thanks a lot for your understanding and patience!

โ„ฐ. ๐’ž๐‘œ๐’น๐‘’ ๐‘œ๐’ป ๐’ž๐‘œ๐“ƒ๐’น๐“Š๐’ธ๐“‰

Refer to official code-of-conduct file for more details.

  • TorchMeter is an open-source project built by developers worldwide. We're committed to fostering a friendly, safe, and inclusive environment for all participants.

  • This code applies to all community spaces including but not limited to GitHub repositories, community forums, etc.

โ„ฑ. ๐ฟ๐’พ๐’ธ๐‘’๐“ƒ๐“ˆ๐‘’

TorchMeter is released under the AGPL-3.0 License, see the LICENSE file for the full text. Please carefully review the terms in the LICENSE file before using or distributing TorchMeter. Ensure compliance with the licensing conditions, especially when integrating this project into larger systems or proprietary software.

About

TorchMeterโฒ๏ธ: Your all-in-one tool for Pytorch model analysis, measuring Params, FLOPs/MACs(aka. MACC or MADD), Memory cost, Inference time and Throughput! ! ๐Ÿš€

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks