Skip to content

Make freeze/thaw scripts "smarter" #27

@yarikoptic

Description

@yarikoptic

ATM we just specify them user-wide and then actual git repository which makes use of them does not even know which version of the script it used etc. We already got a problem that we are changing the way we freeze and thus current thaw script is

[f006rq8@discovery-01 ds000003-qc]$ cat /dartfs/rc/lab/D/DBIC/DBIC/archive/bin-annex/thaw-content | grep -v '^#[^!]'
#!/bin/bash

set -eu

{
        # TODO: avoid double invocation for getfacl and ideally may be record
        # somewhere the way how it was all frozen
        # TODO: make scripts smart and "deploy" themselves into a repository
        # TODO: check "better" protections I have tried in
        #  https://git-annex.branchable.com/projects/datalad/bugs-done/can__39__t_make_annex_happy_in_freeze__47__thaw/
        # i.e. D::EVERYONE@:wadTN
        if nfs4_getfacl "$@" | grep -q "D::EVERYONE@:.*w.*D"; then
                # new way which seems to work
                nfs4_setfacl -x D::EVERYONE@:wD -R "$@"
        else
                # it must have been frozen using prior ways
                nfs4_getfacl "$@" | sed -e 's,:rxtn,:rwadxtTnN,g' | nfs4_setfacl -S- -R "$@"
        fi
#
} >&2

so we could account for older way where we explicitly modified permissions to remove "write" ones.

Also these scripts do not care/know if location is with ACL or POSIX, such as e.g. under /scratch (just regular XFS). As a result they fail, and we get git-annex working correctly (just using posix permissions I guess) as a side effect of ignoring them those failing commands: https://git-annex.branchable.com/bugs/git-annex_proceeds_forward_if_freeze_script_fails/?updated .

So, overall the script should somehow annotate any given git repository's config with information on either some "version" of freeze/thaw procedure or just "instantiate" and configure that repository to use specific versions of those scripts, by placing them e.g. under .git/git-annex/freeze-content and .git/git-annex/thaw-content scripts, and configure git-annex to use them:

(datalad) [f006rq8@discovery-01 subdir]$ pwd
/dartfs/rc/lab/D/DBIC/DBIC/CON/asmacdo/tmp/test-local-thaw/subdir

(datalad) [f006rq8@discovery-01 subdir]$ git config -l | grep content
annex.thawcontent-command=/dartfs/rc/lab/D/DBIC/DBIC/archive/bin-annex/thaw-content %path
annex.freezecontent-command=/dartfs/rc/lab/D/DBIC/DBIC/archive/bin-annex/freeze-content %path
annex.thawcontent-command=.git/annex/thaw-content %path
annex.freezecontent-command=.git/annex/freeze-content %path

(datalad) [f006rq8@discovery-01 subdir]$ git config get annex.thawcontent-command
.git/annex/thaw-content %path

(datalad) [f006rq8@discovery-01 subdir]$ git config get annex.freezecontent-command
.git/annex/freeze-content %path

but that doesn't work since git-annex operates from current directory and thus relative path does not work when calling from subfolder:

[2024-10-16 14:47:08.941720897] (Annex.Perms) freezing content ../.git/annex/objects/6k/VJ/MD5E-s115--9a295e3f5f148380d74c3ff3ebdaa173/MD5E-s115--9a295e3f5f148380d74c3ff3ebdaa173            
[2024-10-16 14:47:08.948171243] (Utility.Process) process [2572997] call: sh ["-c",".git/annex/freeze-content '../.git/annex/objects/6k/VJ/MD5E-s115--9a295e3f5f148380d74c3ff3ebdaa173/MD5E-s1
15--9a295e3f5f148380d74c3ff3ebdaa173'"]                                                                                                                                                       
sh: .git/annex/freeze-content: No such file or directory  

filed https://git-annex.branchable.com/todo/specify_freeze__47__thaw_scripts_relative_to_topdir/?updated

So, until we hear back on that, it would only be the case of possibly specifying/storing version of freeze/thaw procedure inside local config, or may be to speed up (since per file) -- just a file, e.g. .git/annex/freezethaw-version with 0 to avoid any freezing in case of POSIX system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions