Skip to content

Conversation

@Bycob
Copy link
Collaborator

@Bycob Bycob commented Mar 5, 2024

@beniz This PR is not ready, I have several questions about the integration of multiple class in joligen:

  • The cls_semantic_nclasses seems to be used only by GAN models, for diffusion, class conditioning uses the f_s_semantic_nclasses parameter. Should we use cls_semantic_nclasses instead? cc @royale
  • Is it fine to have multiple heads to embed the class, like I did? I'm not satisfied of this solution because it adds complexity to the code, I kept single head for backward compatibility with old models
  • Check the backward compatibility with train_config, since *_semantic_nclasses is now an array instead of an int
  • (As @royale suggested, we could sum embeddings of different classes instead of concatening them)

@Bycob Bycob force-pushed the pr_sounds_cls_cond branch from 9c7934e to d91a3c6 Compare March 8, 2024 14:33
@Bycob Bycob requested a review from beniz March 19, 2024 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant