Skip to content

[FEATURE]: Allow Metrics To Directly Accept Frameworks & Datasets #25

@nmichlo

Description

@nmichlo

Is your feature request related to a problem? Please describe.
Current metrics require that you provide a representation function. This is inconvenient and always repeated. Metrics also always require that the dataset be a DisentDataset.

dataset = DisentDataset(data)
# we always need to produce this same function
get_repr = lambda x: module.encode(x.to(module.device))
# to use with metrics
results = metric_mig(dataset, get_repr)

Describe the solution you'd like
Allow the metrics to directly accept the frameworks instead, and automatically wrap datasets with DisentDataset

# directly use the framework and raw data instead!
results = metric_mig(data, module)

Describe alternatives you've considered

  • Detect if isinstance(obj, DisentFramework) then handle everything automatically!
  • Or add a function to frameworks called get_representation
  • Or detect if an object has the property encode and use that instead!

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentationenhancementNew feature or requestgood first issueGood for newcomersrefactoringInternal changes to the code that do not change the outputs.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions