Is your feature request related to a problem? Please describe.
My team has been using InferenceX in our automated testing workflow to generate workloads and run benchmarks. The benchmarking utilities are proving to be a critical tool for both simulating workloads and tracking performance metrics. Right now, there doesn’t seem to be an official versioning or packaging process for those tools, which makes it trickier to integrate into our testing environment. Currently, we’re handling this by packaging them into in an internal Docker image and including the latest commit hash inside the image.
Describe the solution you'd like
Even though the main focus of the InferenceX project is to create timely and reliable benchmarks of the current state of the inference software ecosystem, Official, versioned releases of InferenceX tools and scripts could be useful for the broader community. Are there plans to publish official versioned releases either via a Docker image, a PyPI package, or both? We’d be happy to help start this work by adapting our current packaging work and creating a PR, but we first wanted to check with maintainers and the community to make sure that is consistent with the future direction of the project.
Describe alternatives you've considered
- Continuing to refine our internal Docker image that vendors InferenceX.
- Creating and maintaining our own Python package that bundles the InferenceX benchmarking utilities, rather than relying on an official package.
Additional context
Not that I can think of at the moment. We mainly wanted to understand the project's long term goals related to versioning and packaging, and express our willingness to contribute to a broad solution.
Is your feature request related to a problem? Please describe.
My team has been using InferenceX in our automated testing workflow to generate workloads and run benchmarks. The benchmarking utilities are proving to be a critical tool for both simulating workloads and tracking performance metrics. Right now, there doesn’t seem to be an official versioning or packaging process for those tools, which makes it trickier to integrate into our testing environment. Currently, we’re handling this by packaging them into in an internal Docker image and including the latest commit hash inside the image.
Describe the solution you'd like
Even though the main focus of the InferenceX project is to create timely and reliable benchmarks of the current state of the inference software ecosystem, Official, versioned releases of InferenceX tools and scripts could be useful for the broader community. Are there plans to publish official versioned releases either via a Docker image, a PyPI package, or both? We’d be happy to help start this work by adapting our current packaging work and creating a PR, but we first wanted to check with maintainers and the community to make sure that is consistent with the future direction of the project.
Describe alternatives you've considered
Additional context
Not that I can think of at the moment. We mainly wanted to understand the project's long term goals related to versioning and packaging, and express our willingness to contribute to a broad solution.