Skip to content

Commit 4f388bd

Browse files
gpupuckDwarKapex
andauthored
Upgrade seqio to upstream to workaround the MaxText dependency issue for inference (#1225)
Default version built with MaxText image is `seqio==0.0.16` which introduces error ``` 0: Traceback (most recent call last): 0: File "/opt/maxtext/MaxText/inference_microbenchmark.py", line 26, in <module> 0: from jetstream.engine import token_utils 0: File "/opt/jetstream/jetstream/engine/token_utils.py", line 24, in <module> 0: from seqio.vocabularies import SentencePieceVocabulary 0: File "/usr/local/lib/python3.12/dist-packages/seqio/__init__.py", line 18, in <module> 0: from seqio.dataset_providers import * 0: File "/usr/local/lib/python3.12/dist-packages/seqio/dataset_providers.py", line 60, in <module> 0: @dataclasses.dataclass(frozen=True) 0: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0: File "/usr/lib/python3.12/dataclasses.py", line 1258, in wrap 0: return _process_class(cls, init, repr, eq, order, unsafe_hash, 0: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0: File "/usr/lib/python3.12/dataclasses.py", line 994, in _process_class 0: cls_fields.append(_get_field(cls, name, type, kw_only)) 0: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 0: File "/usr/lib/python3.12/dataclasses.py", line 852, in _get_field 0: raise ValueError(f'mutable default {type(f.default)} for field ' 0: ValueError: mutable default <class 'seqio.vocabularies.PassThroughVocabulary'> for field vocabulary is not allowed: use default_factory ``` Upgrade seqio to directly use upstream repo to workaround the issue. (This is not the perfect but workable solution) --------- Co-authored-by: Vladislav Kozlov <[email protected]>
1 parent 92cc80b commit 4f388bd

File tree

1 file changed

+8
-1
lines changed

1 file changed

+8
-1
lines changed

.github/container/Dockerfile.maxtext

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,10 +68,17 @@ for pattern in \
6868
"s|protobuf==3.20.3|protobuf>=3.19.0|g" \
6969
"s|tensorflow-datasets|tensorflow-datasets>=4.8.0|g" \
7070
"s|grain-nightly|grain|g" \
71+
"s|tensorboardx|tensorboardx>=2.6.2|g" \
72+
"s|sentencepiece==0.1.97|sentencepiece>=0.2|g" \
7173
; do
7274
sed -i "${pattern}" ${SRC_PATH_MAXTEXT}/requirements.txt;
7375
done
74-
echo "tensorflow-metadata>=1.15.0" >> ${SRC_PATH_MAXTEXT}/requirements.txt
76+
for requirement in \
77+
"tensorflow-metadata>=1.15.0" \
78+
"seqio@git+https://github.com/google/seqio.git" \
79+
; do
80+
echo "${requirement}" >> ${SRC_PATH_MAXTEXT}/requirements.txt
81+
done
7582
EOF
7683

7784
###############################################################################

0 commit comments

Comments
 (0)