Skip to content

Add v2 RETURNN config serialization - #601

Merged
albertz merged 78 commits into
mainfrom
serialization-v2
Nov 27, 2025
Merged

Add v2 RETURNN config serialization#601
albertz merged 78 commits into
mainfrom
serialization-v2

Conversation

@NeoLegends

@NeoLegends NeoLegends commented May 9, 2025

Copy link
Copy Markdown
Contributor

This PR adds an adapted version of the v2 serialization written by @albertz. I intend for this PR to house discussion on what features to support in v2 serialization and what not to support.

The adaptations I made mainly concern themselves around being compatible with existing RETURNN configs that use the v1 serialization mechanisms like the Import, PartialImport, etc.

Comment thread serialization_v2/serialization.py Outdated
@NeoLegends

This comment was marked as resolved.

Comment thread serialization_v2/serialization.py Outdated
Icemole

This comment was marked as resolved.

Comment thread serialization_v2/serialization.py Outdated
Comment thread serialization_v2/serialization.py Outdated
Comment thread serialization_v2/serialization.py Outdated
albertz

This comment was marked as resolved.

@albertz

albertz commented Nov 20, 2025

Copy link
Copy Markdown
Member

Who else should review this?

@NeoLegends

Copy link
Copy Markdown
Contributor Author

@michelwi Maybe? If you prefer, we can also do a quick pair programming/review session Monday. :)

@michelwi michelwi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the liberty to add some tests. Some of them currently fail.

Comment thread returnn/config.py
Comment on lines +372 to +376
# I'm not really sure about it.
# Our automatic mechanism will find direct imports (e.g. i6_experiments).
# However, it will not find indirect imports (e.g. sisyphus),
# and thus the generated code might fail.
# So add all other paths here which we currently have.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we now sure? -> remove/rewrite comment otherwise, lets try to make sure

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I'm still not sure about it, but that is the solution we are using now. We can strip the "I'm not really sure about it." if you want. But I'm still not sure whether that's the best solution.

But then, when are you really 100% sure about sth? But here, I'm maybe a bit less confident than for most of the other code that I write, so that's why I specifically mentioned it.

Comment thread returnn/config.py Outdated
Comment thread serialization/README.rst Outdated
Comment thread serialization/serialization_v2.py Outdated
Comment thread tests/unit_tests/test_serialization_v2.py Outdated
NeoLegends and others added 4 commits November 26, 2025 15:10
Co-authored-by: michelwi <michelwi@users.noreply.github.com>
I reckon it occurs often enough in RETURNN configs that we can make serialization prettier for it
Co-authored-by: michelwi <michelwi@users.noreply.github.com>
Comment thread tests/unit_tests/test_serialization_v2.py Outdated
@NeoLegends
NeoLegends requested a review from michelwi November 27, 2025 09:34
Comment thread serialization/serialization_v2.py Outdated
Co-authored-by: michelwi <michelwi@users.noreply.github.com>
@albertz
albertz merged commit e21acbd into main Nov 27, 2025
5 checks passed
@albertz
albertz deleted the serialization-v2 branch November 27, 2025 14:25
@NeoLegends

Copy link
Copy Markdown
Contributor Author

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants