Operation not permitted

[stevage]

I ran the instructions on an Ubuntu Trusty64 in Vagrant.

$ python get.py
/vagrant/grantnav/.ve/lib/python3.4/site-packages/openpyxl/workbook/names/named_range.py:125: UserWarning: Discarded range with reserved name
  warnings.warn("Discarded range with reserved name")
Traceback (most recent call last):
  File "get.py", line 175, in <module>
    os.link(json_file_name, 'data/json_valid/{}.json'.format(dataset['identifier']))
PermissionError: [Errno 1] Operation not permitted: 'data/json_all/a002400000p1DloAAE.json' -> 'data/json_valid/a002400000p1DloAAE.json'

It looks like the first file did download successfully:

(.ve) vagrant@vagrant-ubuntu-trusty-64:/vagrant/datagetter$ ll data/json_all
total 204
drwxr-xr-x 1 vagrant vagrant    102 Oct  5 23:49 ./
drwxr-xr-x 1 vagrant vagrant    272 Oct  5 23:49 ../
-rw-r--r-- 1 vagrant vagrant 208779 Oct  5 23:49 a002400000p1DloAAE.json
(.ve) vagrant@vagrant-ubuntu-trusty-64:/vagrant/datagetter$ ll data/json_valid
total 0
drwxr-xr-x 1 vagrant vagrant  68 Oct  5 23:45 ./
drwxr-xr-x 1 vagrant vagrant 272 Oct  5 23:49 ../

[Bjwebb]

The script makes use of hardlinks, which it looks like your filesystem doesn't support.

[stevage]

Whoa, two very surprising statements :) Do they need to be hardlinks rather than symlinks? And I wonder why they wouldn't be permitted in this environment (default Trusty Vagrant box).

[Bjwebb]

Yes, symlinks should be fine. Replace https://github.com/ThreeSixtyGiving/datagetter/blob/d95c441405d895c475a82f6c055e46b1b6f806b7/get.py#L142 with os.symlink, or even with shutil.copyfile.

My guess at why this isn't working on your VM, would be that you're using a file-system that's shared with the host machine, so isn't a typical linux file-system.