Skip to content

detect() raises ValueError: Unable to avoid copy when deprecated fasttext PyPI package is co-installed with fasttext-predict #31

Description

@asdasde

Summary

When the deprecated fasttext distribution (e.g. fasttext==0.9.2) is installed in the same
Python environment as fast-langdetect, every call to fast_langdetect.detect(...) fails
with:

ValueError: Unable to avoid copy while creating an array as requested.
If using `np.array(obj, copy=False)` replace it with `np.asarray(obj)` to allow a copy when
needed
(no behavior change in NumPy 1.x).

The failure is silent at install time — pip reports both distributions as successfully
installed — and only surfaces on the first detect() call.

Reproduction

python -m venv /tmp/repro && source /tmp/repro/bin/activate
pip install "fast-langdetect==1.0.0" "fasttext==0.9.2"
python -c "import fast_langdetect; print(fast_langdetect.detect('hello world', model='lite'))"

Output:

Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, ...
Traceback (most recent call last):
  ...
  File ".../fast_langdetect/infer.py", line 343, in detect
    labels, scores = ft_model.predict(normalized_text, k=k, threshold=threshold)
  File ".../fasttext/FastText.py", line 232, in predict
    return labels, np.array(probs, copy=False)
ValueError: Unable to avoid copy while creating an array as requested.

The same reproduction succeeds (returns a normal language prediction) when only
fasttext-predict is installed.

Root cause

  1. fast-langdetect>=1.0.0 declares its dependency as fasttext-predict, a maintained,
    NumPy-2-clean fork that exposes only the predict path.
  2. Both the deprecated fasttext distribution and fasttext-predict install files into
    the same top-level fasttext/ package directory (__init__.py, FastText.py,
    fasttext_pybind.*.so). Because the distribution names differ, pip treats them as unrelated
    and silently lets whichever is installed last overwrite the other's files. Verified locally:
    pip uninstall fasttext removes the entire fasttext/ directory even though
    fasttext-predict still claims it.
  3. The deprecated fasttext.FastText._FastText.predict ends with:
    return labels, np.array(probs, copy=False)
    Under NumPy ≥ 2.0, copy=False raises if a copy can't be avoided; from a Python tuple of
    floats, a copy can't be avoided, so it always raises.
  4. fast_langdetect.detect(...) calls ft_model.predict(...)ValueError.

Why this matters

Many environments have the legacy fasttext installed transitively (a common dep through
gensim plugins, langid wrappers, embedding libs, etc.). Adding fast-langdetect to such an
environment produces a hard runtime failure with no actionable hint pointing at the packaging
conflict — the traceback points into fasttext/FastText.py, which most users don't realize is
the wrong distribution.

Environment

  • Python 3.12.x
  • fast-langdetect==1.0.0
  • fasttext==0.9.2 (deprecated)
  • fasttext-predict==0.9.2.4 (also installed; overwritten by fasttext on disk)
  • numpy>=2.0
  • Linux x86_64

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions