Skip to content

Fix permissions of step files writen to $TMPDIR #27

@ZJaume

Description

@ZJaume

Right now steps (at least 03.split) that write their output to $TMPDIR. As stated by @jelmervdl, this is probably because LUMI has not $SCRATCH set and files are written to a temporary dir in the node that is owned by the user and not the group. We should have something like this for LUMI that creates the temporary directory and changes ownerships.

Example of giashard doing it well but 03.split not:

$ ll output_wide15_filtered_sample3-shards/es/*/*
output_wide15_filtered_sample3-shards/es/0/1:
total 376M
-rw-r----- 1 zaragoza2 project_462000252 188M May  9 18:04 plain_text.gz
-rw-rw---- 1 zaragoza2 zaragoza2         187M May 10 16:40 sentences.gz
-rw-r----- 1 zaragoza2 project_462000252  12K May  9 18:04 source.gz
-rw-r----- 1 zaragoza2 project_462000252 655K May  9 18:04 url.gz

output_wide15_filtered_sample3-shards/es/1/1:
total 322M
-rw-r----- 1 zaragoza2 project_462000252 162M May  9 18:04 plain_text.gz
-rw-rw---- 1 zaragoza2 zaragoza2         160M May 10 16:39 sentences.gz
-rw-r----- 1 zaragoza2 project_462000252  11K May  9 18:04 source.gz
-rw-r----- 1 zaragoza2 project_462000252 661K May  9 18:04 url.gz

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions