You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Make this executable and move this script to your path
21
+
chmod +x git-annex-remote-openneuro
22
+
```
17
23
18
24
## Setup
19
25
20
26
The setup step is needed for both uploading _and_ downloading data from OpenNeuro.
21
27
22
-
Run `openneuro login` to configure credentials.
23
-
This prompts you for the required configuration fields and saves this to `.openneuro` in your home directory or profile.
24
-
`openneuro login` will require you to enter an API key.
25
-
You can obtain an API key via a browser at https://openneuro.org/keygen after logging to the OpenNeuro platform via one of the provided authentication services (for example ORCID).
28
+
Run `deno run -A jsr:@openneuro/cli login` to configure credentials.
29
+
This prompts you for the required configuration fields and these are saved in Deno's local storage.
30
+
31
+
`deno run -A jsr:@openneuro/cli login` will require you to enter an API key.
32
+
33
+
You can obtain an API key via a browser on [OpenNeuro](https://openneuro.org/keygen) after logging to the OpenNeuro platform via one of the provided authentication services (for example ORCID).
34
+
35
+
You can also specify this as an option or environment variable.
26
36
27
-
After successfully running `openneuro login`, you can manually configure custom servers by editing the `.openneuro` file.
37
+
```shell
38
+
# For scripts
39
+
export OPENNEURO_API_KEY=<api_key>
40
+
deno run -A jsr:@openneuro/cli login --error-reporting true
41
+
```
28
42
29
43
## Usage
30
44
31
45
### Uploading datasets
32
46
33
47
To upload a new dataset:
34
48
35
-
`openneuro upload <dataset directory>`
49
+
```shell
50
+
# Path to the dataset root (directory containing dataset_description.json)
51
+
deno run -A jsr:@openneuro/cli upload --affirmDefaced path/to/dataset
52
+
```
36
53
37
-
Your dataset must pass validation to upload but warnings can be skipped with `openneuro upload -i <dataset directory>`. A default label is set using the directory name.
54
+
Your dataset must pass validation to upload but warnings can be skipped with `deno run -A jsr:@openneuro/cli upload --ignoreWarnings path/to/dataset`.
38
55
39
56
To resume an interrupted upload or add files to an existing dataset:
deno run -A jsr:@openneuro/cli upload --dataset ds000001 path/to/dataset
61
+
```
42
62
43
63
where <accession_number> is a unique dataset identifier that can be found in the URL. For example accession number for `https://openneuro.org/datasets/ds001555` is `ds001555`.
44
64
45
65
This command will add or replace any files in the dataset but does not delete any files that are only present in the server copy of the dataset.
46
66
47
67
### Downloading datasets
48
68
69
+
Downloads using the CLI will create a DataLad dataset and configure a special remote to retrieve the larger annexed files from OpenNeuro.
deno run -A jsr:@openneuro/cli download --draft <accession number> <destination directory>
81
+
```
82
+
83
+
If the destination directory does not exist, it will be created.
84
+
85
+
To download the annexed objects, use [DataLad](https://datalad.org/) or [git-annex](https://git-annex.branchable.com).
56
86
57
-
If the destination directory does not exist, it will be created. Any files from the dataset that are already present in the directory will be skipped, allowing you to resume an interrupted download.
87
+
```shell
88
+
cd ds000001
89
+
# Make sure you have the `git-annex-remote-openneuro` script available in your path
90
+
git-annex get <path or paths to download>
91
+
# DataLad is supported as well
92
+
datalad get <path or paths to download>
93
+
```
58
94
59
-
## Errors
95
+
### Debugging issues
60
96
61
-
Package: OpenNeuro CLI download option
97
+
```shell
98
+
# To debug issues - enable logging and provide this log to support or open a GitHub issue
99
+
export OPENNEURO_LOG=INFO
100
+
deno run -A jsr:@openneuro/cli upload --affirmDefaced path/to/dataset
101
+
```
62
102
63
-
Issue: `TypeError: path must be a string or Buffer`
103
+
### Implementation Notes
64
104
65
-
Solution: This error is due to a failure to read the `~/.openneuro` configuration file.
66
-
This file can be created with the command `openneuro login`, see the "Setup" step above.
105
+
This tool uses isomorphic git to download, modify, and push datasets using OpenNeuro's [git interface](https://docs.openneuro.org/git.html). Other tools that support git and git-annex repositories such as [DataLad](https://www.datalad.org/) can also be used with the local copy.
Copy file name to clipboardExpand all lines: packages/openneuro-cli.html
+57-22Lines changed: 57 additions & 22 deletions
Original file line number
Diff line number
Diff line change
@@ -269,48 +269,82 @@ <h1>OpenNeuro command line interface<a class="headerlink" href="#openneuro-comma
269
269
<sectionid="install">
270
270
<h2>Install<aclass="headerlink" href="#install" title="Link to this heading">¶</a></h2>
271
271
<olclass="arabic simple">
272
-
<li><p>Install <aclass="reference external" href="https://nodejs.org">Node.js</a> (version 18 or higher)</p></li>
273
-
<li><p>In a terminal type: <codeclass="docutils literal notranslate"><spanclass="pre">npm</span><spanclass="pre">install</span><spanclass="pre">-g</span><spanclass="pre">@openneuro/cli</span></code></p></li>
272
+
<li><p>Install [Deno] (https://deno.land/) via <aclass="reference external" href="https://docs.deno.com/runtime/manual/getting_started/installation">any supported installation method</a>.</p></li>
273
+
<li><p>In a terminal type: <codeclass="docutils literal notranslate"><spanclass="pre">deno</span><spanclass="pre">run</span><spanclass="pre">-A</span><spanclass="pre">jsr:@openneuro/cli</span><spanclass="pre">--help</span></code></p></li>
274
274
</ol>
275
-
<p>If you are using <aclass="reference external" href="https://yarnpkg.com/">yarn</a> you can also perform the installation with <codeclass="docutils literal notranslate"><spanclass="pre">yarn</span><spanclass="pre">global</span><spanclass="pre">add</span><spanclass="pre">@openneuro/cli</span></code>
276
-
(make sure the installation folder is part of your <codeclass="docutils literal notranslate"><spanclass="pre">PATH</span></code> by adding <codeclass="docutils literal notranslate"><spanclass="pre">export</span><spanclass="pre">PATH="$(yarn</span><spanclass="pre">global</span><spanclass="pre">bin):$PATH"</span></code> to <codeclass="docutils literal notranslate"><spanclass="pre">~/.bashrc</span></code>)</p>
275
+
<p>To download annexed files, you will need the git-annex special remote for OpenNeuro.</p>
276
+
<divclass="highlight-shell notranslate"><divclass="highlight"><pre><span></span><spanclass="c1"># A script is provided to wrap the CLI as a special remote</span>
<h2>Setup<aclass="headerlink" href="#setup" title="Link to this heading">¶</a></h2>
280
285
<p>The setup step is needed for both uploading <em>and</em> downloading data from OpenNeuro.</p>
281
-
<p>Run <codeclass="docutils literal notranslate"><spanclass="pre">openneuro</span><spanclass="pre">login</span></code> to configure credentials.
282
-
This prompts you for the required configuration fields and saves this to <codeclass="docutils literal notranslate"><spanclass="pre">.openneuro</span></code> in your home directory or profile.
283
-
<codeclass="docutils literal notranslate"><spanclass="pre">openneuro</span><spanclass="pre">login</span></code> will require you to enter an API key.
284
-
You can obtain an API key via a browser at https://openneuro.org/keygen after logging to the OpenNeuro platform via one of the provided authentication services (for example ORCID).</p>
285
-
<p>After successfully running <codeclass="docutils literal notranslate"><spanclass="pre">openneuro</span><spanclass="pre">login</span></code>, you can manually configure custom servers by editing the <codeclass="docutils literal notranslate"><spanclass="pre">.openneuro</span></code> file.</p>
286
+
<p>Run <codeclass="docutils literal notranslate"><spanclass="pre">deno</span><spanclass="pre">run</span><spanclass="pre">-A</span><spanclass="pre">jsr:@openneuro/cli</span><spanclass="pre">login</span></code> to configure credentials.
287
+
This prompts you for the required configuration fields and these are saved in Deno’s local storage.</p>
288
+
<p><codeclass="docutils literal notranslate"><spanclass="pre">deno</span><spanclass="pre">run</span><spanclass="pre">-A</span><spanclass="pre">jsr:@openneuro/cli</span><spanclass="pre">login</span></code> will require you to enter an API key.</p>
289
+
<p>You can obtain an API key via a browser on <aclass="reference external" href="https://openneuro.org/keygen">OpenNeuro</a> after logging to the OpenNeuro platform via one of the provided authentication services (for example ORCID).</p>
290
+
<p>You can also specify this as an option or environment variable.</p>
291
+
<divclass="highlight-shell notranslate"><divclass="highlight"><pre><span></span><spanclass="c1"># For scripts</span>
<p>Your dataset must pass validation to upload but warnings can be skipped with <codeclass="docutils literal notranslate"><spanclass="pre">openneuro</span><spanclass="pre">upload</span><spanclass="pre">-i</span><spanclass="pre"><dataset</span><spanclass="pre">directory></span></code>. A default label is set using the directory name.</p>
302
+
<divclass="highlight-shell notranslate"><divclass="highlight"><pre><span></span><spanclass="c1"># Path to the dataset root (directory containing dataset_description.json)</span>
<p>Your dataset must pass validation to upload but warnings can be skipped with <codeclass="docutils literal notranslate"><spanclass="pre">deno</span><spanclass="pre">run</span><spanclass="pre">-A</span><spanclass="pre">jsr:@openneuro/cli</span><spanclass="pre">upload</span><spanclass="pre">--ignoreWarnings</span><spanclass="pre">path/to/dataset</span></code>.</p>
294
307
<p>To resume an interrupted upload or add files to an existing dataset:</p>
<p>where <accession_number> is a unique dataset identifier that can be found in the URL. For example accession number for <codeclass="docutils literal notranslate"><spanclass="pre">https://openneuro.org/datasets/ds001555</span></code> is <codeclass="docutils literal notranslate"><spanclass="pre">ds001555</span></code>.</p>
297
313
<p>This command will add or replace any files in the dataset but does not delete any files that are only present in the server copy of the dataset.</p>
298
314
</section>
299
315
<sectionid="downloading-datasets">
300
316
<h3>Downloading datasets<aclass="headerlink" href="#downloading-datasets" title="Link to this heading">¶</a></h3>
317
+
<p>Downloads using the CLI will create a DataLad dataset and configure a special remote to retrieve the larger annexed files from OpenNeuro.</p>
<p>If the destination directory does not exist, it will be created. Any files from the dataset that are already present in the directory will be skipped, allowing you to resume an interrupted download.</p>
<p>If the destination directory does not exist, it will be created.</p>
327
+
<p>To download the annexed objects, use <aclass="reference external" href="https://datalad.org/">DataLad</a> or <aclass="reference external" href="https://git-annex.branchable.com">git-annex</a>.</p>
<h3>Debugging issues<aclass="headerlink" href="#debugging-issues" title="Link to this heading">¶</a></h3>
338
+
<divclass="highlight-shell notranslate"><divclass="highlight"><pre><span></span><spanclass="c1"># To debug issues - enable logging and provide this log to support or open a GitHub issue</span>
<h3>Implementation Notes<aclass="headerlink" href="#implementation-notes" title="Link to this heading">¶</a></h3>
346
+
<p>This tool uses isomorphic git to download, modify, and push datasets using OpenNeuro’s <aclass="reference external" href="https://docs.openneuro.org/git.html">git interface</a>. Other tools that support git and git-annex repositories such as <aclass="reference external" href="https://www.datalad.org/">DataLad</a> can also be used with the local copy.</p>
307
347
</section>
308
-
<sectionid="errors">
309
-
<h2>Errors<aclass="headerlink" href="#errors" title="Link to this heading">¶</a></h2>
<p>Solution: This error is due to a failure to read the <codeclass="docutils literal notranslate"><spanclass="pre">~/.openneuro</span></code> configuration file.
313
-
This file can be created with the command <codeclass="docutils literal notranslate"><spanclass="pre">openneuro</span><spanclass="pre">login</span></code>, see the “Setup” step above.</p>
314
348
</section>
315
349
</section>
316
350
@@ -375,9 +409,10 @@ <h2>Errors<a class="headerlink" href="#errors" title="Link to this heading">¶</
0 commit comments