Skip to content

Fix retriever SDG dedup plugin circular import#156

Merged
oliverholworthy merged 2 commits intomainfrom
oholworthy/fix-data-designer-plugin-circular-import
Apr 20, 2026
Merged

Fix retriever SDG dedup plugin circular import#156
oliverholworthy merged 2 commits intomainfrom
oholworthy/fix-data-designer-plugin-circular-import

Conversation

@oliverholworthy
Copy link
Copy Markdown
Contributor

@oliverholworthy oliverholworthy commented Apr 20, 2026

Fixes the error printed in embed receipe sdg stage:

Failed to load plugin from entry point 'data-designer-retrieval-dedup': partially initialized module 'retriever_sdg.deduplication' has no attribute 'dd_retrieval_dedup_plugin' (most likely due to a circular import)

This error didn't prevent the stage from running sucessfully, but is a confusing error to show up in the logs.

  • Avoid a circular import in retriever_sdg.deduplication by publishing the dd_retrieval_dedup_plugin before importing ColumnGeneratorCellByCell, so Data Designer's re-entrant plugin discovery can resolve the plugin while the generator base class import is still in progress.
  • Add __all__ to make the public surface explicit.

Test plan

  • nemotron embedding SDG dedup stage loads without ImportError on plugin discovery
  • Run the retriever SDG dedup step end-to-end against a small sample

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy oliverholworthy self-assigned this Apr 20, 2026
Ensures source edits under vendor/retriever-sdg/ are picked up on the
next run without rebuilding the cached wheel.

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@shan-nvidia shan-nvidia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

@oliverholworthy oliverholworthy merged commit a88a0bb into main Apr 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants