feat: record plugin package provenance on resource creation#1002
Open
michael-johnston wants to merge 3 commits into
Open
feat: record plugin package provenance on resource creation#1002michael-johnston wants to merge 3 commits into
michael-johnston wants to merge 3 commits into
Conversation
Member
|
Wouldn't it be better to have a model called Something like: class ProvenanceInfo:
actuators: dict[str, PackageProvenance] | None
custom_experiments: dict[str, PackageProvenance] | None
operators: dict[str, PackageProvenance] | NoneThis would give us a consistent pattern + extensibility for the future. As a side note, I think we should stop using camelCasing even in Pydantic models, so I would use snake_case everywhere |
Member
Author
Only certain resources have this information though. Its seems strange to have these fields on datacontainer or samplestore - they would also be in full dump of objects and schema. Maybe the object should just be resource subclass can then also subclass this OperationProvenanceInfo etc and put whatever fields are relevant to them. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a DiscoverySpaceResource, OperationResource, or ActuatorConfigurationResource is written to the metastore, the exact PyPI distribution name and version of every plugin involved is now captured and stored alongside it. This makes it straightforward to identify — after the fact — which package versions were used to create a given resource, supporting both replication and debugging.
Previously this was somewhat deducible for some operators as long as the operator name and package name were similar as the identifier contained the operator plugin version. However this breaks down if a package provides multiple operators. For experiments or actuator configurations there was no record.
Changes
Examples:
space - one entry per actuator and custom_experiment used
operation