Skip to content

Fix long import times due to processing all pydantic subclasses#689

Open
jaredoconnell wants to merge 3 commits intovllm-project:mainfrom
jaredoconnell:fix/pydantic-imports
Open

Fix long import times due to processing all pydantic subclasses#689
jaredoconnell wants to merge 3 commits intovllm-project:mainfrom
jaredoconnell:fix/pydantic-imports

Conversation

@jaredoconnell
Copy link
Copy Markdown
Collaborator

Summary

Other libraries that use pydantic could end up being processed by GuideLLM, significantly increasing the number of classes processed, noticeably increasing import times for GuideLLM.

Also, in my testing, it makes it so the startup times are less than half of what they were before.

Details

  • Rather than just searching every Pydantic subclass, it only searches sub-classes that we created. So no other libraries will add to the processing burden.
  • On my mac, within the Python virtual environment, this change resulted in a change from around 27.5 seconds before the benchmark starts to 6.5 seconds. This is due to how long it spent processing the classes on my local env. It has been super slow since this was originally added in the refactor.
  • The only classes that use this functionality are BenchmarkConfig, BenchmarkGenerativeTextArgs, and Profile,

Test Plan

  • Run benchmarks as normal, and confirm that the options that depend on registry mix in work.

Related Issues


  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

Other libraries that use pydantic could end up being processed by GuideLLM, significantly increasing the number of classes processed, noticeably increasing import times for GuideLLM.

Assisted-by: Cursor AI
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Copy link
Copy Markdown
Collaborator

@dbutenhof dbutenhof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent! But, as Webb might say, just a few nits ...

def reload_parent_schemas(cls):
"""
Recursively reload schemas for all parent Pydantic models.
Recursively reload schemas for all Pydantic models that reference this
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"for all GuideLLM Pydantic models ..." ?? Yeah, "that reference this" covers that base, but it's a bit less direct.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What type of change would you suggest?

@dbutenhof
Copy link
Copy Markdown
Collaborator

Ha -- you might consider modifying the PR title: it's an incomplete sentence, and it displays as "Fix long import times due to #689", which just looks odd!

@dbutenhof dbutenhof added the bug Represents a user-visible defect label Apr 10, 2026
@jaredoconnell jaredoconnell changed the title Fix long import times due to Fix long import times due to processing all pydantic subclasses Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Represents a user-visible defect

Projects

None yet

Development

Successfully merging this pull request may close these issues.

import guidellm can take a very long time in some environments

2 participants