Push to webdav ends with git error: 'remote-datalad-annex' is not a git command

[mslw]

This is a user report. Gooey 0.1 (0.1.0+9.g2f5814c) on Windows.

Running push to a webdav sibling (filetree mode) ends with the following error. Files were sent to the remote - we can't know for sure as the remote folder was changed since, but I suspect that git annex export (of file content) worked, and publishing the packaged git repo failed.

Running: Dataset('C:\\projectB04').push(to='sciebo') failed!
(...)
datalad.runner.exception.CommandError: CommandError: '"git" "-c" "diff.ignoreSubmodules=none" "fetch" "sciebo" "git-annex"' failed with exitcode 128 [err: 'git: 'remote-datalad-annex' is not a git command. See 'git --help'.'])

Full traceback

(user name edited out)

Running: Dataset('C:\\projectB04').push(to='sciebo') failed!
Traceback (most recent call last):
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad_gooey\dataladcmd_exec.py", line 158, in _cmdexec_thread
    for res in cmd(**cmdkwargs):
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\interface\utils.py", line 357, in generator_func
    for r in _process_results(
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\interface\utils.py", line 544, in _process_results
    for res in results:
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\core\distributed\push.py", line 262, in __call__
    yield from _push(
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad_next\patches\push_optimize.py", line 175, in _push
    _sync_remote_annex_branch(repo, target, is_annex_repo)
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad_next\patches\push_optimize.py", line 394, in _sync_remote_annex_branch
    repo.call_git(fetch_cmd)
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\dataset\gitrepo.py", line 438, in call_git
    return "".join(
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\dataset\gitrepo.py", line 499, in call_git_items_
    for file_no, line in self._generator_call_git(
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\dataset\gitrepo.py", line 350, in _generator_call_git
    for file_no, content in generator:
  File "_collections_abc.py", line 330, in __next__
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\runner\nonasyncrunner.py", line 108, in send
    self._check_result()
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\runner\nonasyncrunner.py", line 90, in _check_result
    self.runner._check_result()
  File "C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\lib\site-packages\datalad\runner\nonasyncrunner.py", line 272, in _check_result
    raise CommandError(
    datalad.runner.exception.CommandError: CommandError: '"git" "-c" "diff.ignoreSubmodules=none" "fetch" "sciebo" "git-annex"' failed with exitcode 128 [err: 'git: 'remote-datalad-annex' is not a git command. See 'git --help'.'])

Diagnostic infos

(user name edited out)

# WTF
## configuration <SENSITIVE, report disabled by configuration>
## credentials 
  - keyring: 
    - active_backends: 
      - Windows WinVaultKeyring
      - Windows RegistryKeyring
      - EncryptedKeyring with not defined v.1.0 at C:\Users\<username>\AppData\Local\Python Keyring\wincrypto_pass.cfg
      - PlaintextKeyring with no encyption v.1.0 at C:\Users\<username>\AppData\Local\Python Keyring\keyring_pass.cfg
    - config_file: C:\Users\<username>\AppData\Local\Python Keyring\keyringrc.cfg
    - data_root: C:\Users\<username>\AppData\Local\Python Keyring
## datalad 
  - version: 0.17.6
## dataset 
  - branches: 
    - adjusted/master(unlocked)@97439e1
    - git-annex@6312eb4
    - master@41a7d12
  - id: f22c0a10-053b-4890-b4aa-62d769f39a59
  - metadata: <SENSITIVE, report disabled by configuration>
  - path: C:\projectB04
  - repo: AnnexRepo
## dependencies 
  - annexremote: 1.6.0
  - boto: 2.49.0
  - cmd:annex: 10.20220927-ga92546587
  - cmd:bundled-git: UNKNOWN
  - cmd:git: 2.37.3.windows.1
  - cmd:ssh: 8.1p1
  - cmd:system-git: 2.37.3.windows.1
  - cmd:system-ssh: 8.1p1
  - humanize: 4.4.0
  - iso8601: 1.0.2
  - keyring: 23.9.3
  - keyrings.alt: 4.2.0
  - msgpack: 1.0.4
  - platformdirs: 2.5.2
  - requests: 2.28.1
  - tqdm: 4.64.1
## environment 
  - GIT_TERMINAL_PROMPT: 0
  - PATH: C:\Users\<username>\AppData\Local\datalad.org\datalad-gooey\python39\Lib\site-packages\PySide6;C:\Program Files\Intel\iCLS Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\IBM\SPSS\Statistics\25\JRE\bin;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Tobii\Tobii Pro X3-120\;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\PsychoPy3;C:\Program Files\PsychoPy3\DLLs;C:\Program Files\MATLAB\R2021a\bin;C:\Program Files (x86)\SR Research\Eyelink\LIBS;C:\Program Files (x86)\SR Research\Eyelink\bin;C:\Program Files (x86)\SR Research\Eyelink\bin\64;C:\Program Files (x86)\SR Research\Eyelink\LIBS\x64;C:\Program Files\dotnet\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Git\cmd;C:\Users\<username>\AppData\Local\Programs\Python\Python311\Scripts\;C:\Users\<username>\AppData\Local\Programs\Python\Python311\;C:\Users\<username>\AppData\Local\Microsoft\WindowsApps;
## extensions 
  - gooey: 
    - description: Gooey (GUI)
    - entrypoints: 
      - datalad_gooey.askpass.GooeyAskPass: 
        - class: GooeyAskPass
        - load_error: None
        - module: datalad_gooey.askpass
        - names: 
          - gooey-askpass
          - gooey_askpass
      - datalad_gooey.gooey.Gooey: 
        - class: Gooey
        - load_error: None
        - module: datalad_gooey.gooey
        - names: 
      - datalad_gooey.lsdir.GooeyLsDir: 
        - class: GooeyLsDir
        - load_error: None
        - module: datalad_gooey.lsdir
        - names: 
          - gooey-lsdir
          - gooey_lsdir
      - datalad_gooey.status_light.GooeyStatusLight: 
        - class: GooeyStatusLight
        - load_error: None
        - module: datalad_gooey.status_light
        - names: 
          - gooey-status-light
          - gooey_status_light
    - load_error: None
    - module: datalad_gooey
    - version: 0.1.0+9.g2f5814c
  - next: 
    - description: What is next in DataLad
    - entrypoints: 
      - datalad_next.create_sibling_webdav.CreateSiblingWebDAV: 
        - class: CreateSiblingWebDAV
        - load_error: None
        - module: datalad_next.create_sibling_webdav
        - names: 
          - create-sibling-webdav
      - datalad_next.credentials.Credentials: 
        - class: Credentials
        - load_error: None
        - module: datalad_next.credentials
        - names: 
      - datalad_next.tree.TreeCommand: 
        - class: TreeCommand
        - load_error: None
        - module: datalad_next.tree
        - names: 
          - tree
    - load_error: None
    - module: datalad_next
    - version: 0.6.1
## git-annex 
  - build flags: 
    - Assistant
    - Webapp
    - Pairing
    - TorrentParser
    - MagicMime
    - Benchmark
    - Feeds
    - Testsuite
    - S3
    - WebDAV
  - dependency versions: 
    - aws-0.22
    - bloomfilter-2.0.1.0
    - cryptonite-0.29
    - DAV-1.3.4
    - feed-1.3.2.0
    - ghc-8.10.7
    - http-client-0.7.9
    - persistent-sqlite-2.13.0.3
    - torrent-10000.1.1
    - uuid-1.3.15
    - yesod-1.6.1.2
  - key/value backends: 
    - SHA256report E
    - SHA256
    - SHA512E
    - SHA512
    - SHA224E
    - SHA224
    - SHA384E
    - SHA384
    - SHA3_256E
    - SHA3_256
    - SHA3_512E
    - SHA3_512
    - SHA3_224E
    - SHA3_224
    - SHA3_384E
    - SHA3_384
    - SKEIN256E
    - SKEIN256
    - SKEIN512E
    - SKEIN512
    - BLAKE2B256E
    - BLAKE2B256
    - BLAKE2B512E
    - BLAKE2B512
    - BLAKE2B160E
    - BLAKE2B160
    - BLAKE2B224E
    - BLAKE2B224
    - BLAKE2B384E
    - BLAKE2B384
    - BLAKE2BP512E
    - BLAKE2BP512
    - BLAKE2S256E
    - BLAKE2S256
    - BLAKE2S160E
    - BLAKE2S160
    - BLAKE2S224E
    - BLAKE2S224
    - BLAKE2SP256E
    - BLAKE2SP256
    - BLAKE2SP224E
    - BLAKE2SP224
    - SHA1E
    - SHA1
    - MD5E
    - MD5
    - WORM
    - URL
    - X*
  - local repository version: 9
  - operating system: mingw32 x86_64
  - remote types: 
    - git
    - gcrypt
    - p2p
    - S3
    - bup
    - directory
    - rsync
    - web
    - bittorrent
    - webdav
    - adb
    - tahoe
    - glacier
    - ddar
    - git-lfs
    - httpalso
    - borg
    - hook
    - external
  - supported repository versions: 
    - 8
    - 9
    - 10
  - upgrade supported from repository versions: 
    - 2
    - 3
    - 4
    - 5
    - 6
    - 7
    - 8
    - 9
    - 10
  - version: 10.20220927-ga92546587
## location 
  - path: C:\projectB04
  - type: dataset
## metadata_extractors 
  - annex: 
    - distribution: datalad 0.17.6
    - load_error: None
    - module: datalad.metadata.extractors.annex
  - audio: 
    - distribution: datalad 0.17.6
    - load_error: ModuleNotFoundError(No module named 'mutagen')
    - module: datalad.metadata.extractors.audio
  - datacite: 
    - distribution: datalad 0.17.6
    - load_error: None
    - module: datalad.metadata.extractors.datacite
  - datalad_core: 
    - distribution: datalad 0.17.6
    - load_error: None
    - module: datalad.metadata.extractors.datalad_core
  - datalad_rfc822: 
    - distribution: datalad 0.17.6
    - load_error: None
    - module: datalad.metadata.extractors.datalad_rfc822
  - exif: 
    - distribution: datalad 0.17.6
    - load_error: ModuleNotFoundError(No module named 'exifread')
    - module: datalad.metadata.extractors.exif
  - frictionless_datapackage: 
    - distribution: datalad 0.17.6
    - load_error: None
    - module: datalad.metadata.extractors.frictionless_datapackage
  - image: 
    - distribution: datalad 0.17.6
    - load_error: ModuleNotFoundError(No module named 'PIL')
    - module: datalad.metadata.extractors.image
  - xmp: 
    - distribution: datalad 0.17.6
    - load_error: ModuleNotFoundError(No module named 'libxmp')
    - module: datalad.metadata.extractors.xmp
## metadata_indexers 
## python 
  - implementation: CPython
  - version: 3.9.13
## system 
  - distribution:   10/10.0.19044/SP0/Multiprocessor Free
  - encoding: 
    - default: utf-8
    - filesystem: utf-8
    - locale.prefered: cp1252
  - filesystem: 
    - CWD: 
      - path: C:\projectB04
    - HOME: 
      - path: C:\Users\<username>
    - TMP: 
      - path: C:\Users\<USER>~1\AppData\Local\Temp
  - max_path_length: 259
  - name: Windows
  - release: 10
  - type: nt
  - version: 10.0.19044

I am not sure what could cause the unavailability of datalad-annex, since datalad-next is, clearly, installed. The error comes from git itself, and I can replicate it when running the git command in a virtualenv without datalad-next.

If we get a successful report from Gooey 0.2, I think we can safely close this one. But meanwhile - does anybody see something concerning?

[mih]

The reason is likely this: remote-datalad-annex is actually git-remote-datalad-annex called as a subcommand by git. Adn git cannot find it. git-remote-datalad-annex is installed by datalad-next into the environment where itself is installed. This environment's script/bin path does not seem to be known to the PATH setting of the environment that git is running in.

From the diagnostic info you provided it is clear that multiple python versions are in the PATH and also Git. It is unclear to me where datalad-next was installed in relation to that.

Maybe we should add the installation location to the wtf output on extensions.