Workers should be able to write to some disk

[ikrommyd]

We've talked about this before @oshadura, I'm just writing this issue for tracking.

It is crucial that workers are able to write somewhere to disk. Now whether that's gonna be over xrootd or whatever, is not my decision. But the workers should have the ability to dump things (arrays in parquet/root format is the common use case) to disk.

[oshadura]

@ikrommyd do you have any example which I can use to test writing? (e.g. your analysis)

[oshadura]

In image with coffea 2025.1.1 we have finally support of writing with x509 proxy. For this you just need to generate x509 and restart Dask Cluster in blue box on the left ("Shut down - > Start").

Elliott Kauffman tried to test this setup and he confirmed it works:

dak_zip = dak.zip(branch_save_dict)
dak_zip.persist().to_parquet(self.write_path + "branches")

where self.write_path is "root://eosuser.cern.ch://eos/user/e/ekauffma/" and he confirmed that can see the file on lxplus.

I will add this information in readthe docs.

[ikrommyd]

I'll try it out!!! So writing over xrootd with a proxy is now availalbe, what's the status of writing on s3 as well?