fix: lazy load transformers in HybridChunker

[ryyhan]

Description

Resolves #459.

Implements lazy loading for transformers and semchunk in HybridChunker. This prevents RuntimeError when using docling-core with only the chunking-openai extra (which does not depend on transformers).

Note: This applies the same lazy-loading pattern accepted in docling PR #2826(docling-project/docling#2826) (fix: transformers models lazy-loaded).

Changes

• Moved semchunk import to _split_using_plain_text method.
• Replaced explicit isinstance check for PreTrainedTokenizerBase with a module path string check (tokenizer.__module__) to avoid importing transformers.

Type of change

• Bug fix (non-breaking change which fixes an issue)

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• New and existing unit tests pass locally with my changes

[github-actions]

✅ DCO Check Passed

Thanks @ryyhan, all your commits are properly signed off. 🎉

[dosubot]

Related Documentation

Checked 6 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[ryyhan]

Hi team,

This PR has passed all CI checks and is ready for review.

Summary:

• Implements lazy loading for transformers in HybridChunker to prevent RuntimeErrors when dependencies are missing.
• Resolves Using docling[chunking-openai] raises import RuntimeError #459.

@dolfim-ibm Could you please take a look? Thanks!