Fix[tweakgen]: Make the script work#1411
Draft
pniedzielski wants to merge 7 commits into
Draft
Conversation
pniedzielski
force-pushed
the
fix/tweakgen
branch
2 times, most recently
from
8c594a1 to
7cd5f13
Compare
This patch updates the shebang line in the tweakgen script to use the correct `/usr/bin/env python3` invocation, which is standard on modern Linux and Mac systems. Crucially, there is not a guaranteed `/bin/env` binary, and the Python 3 interpreter is more commonly named `python3` than `python` Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
The string representation of the type object `typing.Union` is not stable across Python versions. In some versions, the representation is `typing.Union`, but in others, it is `<class 'typing.Union'>`. The code we generate is completely incorrect in versions where the latter is printed. This patch uses the string `"Union"` directly, which is actually what we want to output. Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
This patch updates the copyright years both in the tweakgen script itself and in the Python files it generates. Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
pniedzielski
force-pushed
the
fix/tweakgen
branch
from
7cd5f13 to
15a7e27
Compare
This script uses the Python objects that are generated from our XSD files using xsdata in the `schemagen` script. This script appears to have not been run recently, because recent versions of xsdata fix a bug wherein all fields, optional or not, are generated with type `Optional[Dataclass]`, rather than `Dataclass` if the field is required. `tweakgen` assumes that every field has a nested type (i.e., that it is an `Optional`), so it does not perform any generation on non-`Optional` fields. This patch updates `tweakgen` so it first handles the normal `Dataclass` case, and then tries to fallback to the `Optional[Dataclass]` case. Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
With newer versions of xsdata, our generated Python schema files
include the line
from __future__ import annotations
which causes all type annotations in the file to be stored as strings
and evaluated lazily. Because of this, when reflect on these
generated classes in `tweakgen`, the type hints may not yet have been
evaluated, resulting in them sometimes still being strings. In this
case, none of the checks in `generate` will fire, and nothing will be
reified.
This patch adds a call to `typing.get_type_hints`, which evaluates the
type annotation string into a real typing object, which we can then
use as before.
Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
This patch adds a missing arguments when constructing `mqbcfg` and `mqbconf` objects, which were allowed previously by the generated output of old versions of xsdata, which erroneously marked all fields as `Optional`. Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
pniedzielski
force-pushed
the
fix/tweakgen
branch
from
15a7e27 to
590b48f
Compare
This patch updates the copyright years in the generated output of the `schemagen` script and reruns it using a new version of xsdata. This is potentially a breaking change for anyone directly using these generated schema files, who was relying on the previously broken `Optional` behavior. Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
pniedzielski
force-pushed
the
fix/tweakgen
branch
from
590b48f to
aeab754
Compare
Collaborator
Author
|
This is particularly rough. The new xsdata version uses Python 3.10+ features, and we don't have a requirements.txt to pin a particular version (apparently intentionally??). We need Python 3.8 support for the moment, and we run with Python 3.8 in CI. I'll put this work on hold while we figure that out. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request contains a number of changes that enable this script to even be run so we can regenerate
blazingmq/dev/it/tweaks/generated.py.