Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/conf_extlinks.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"guide-issues": ("https://github.com/NeurodataWithoutBorders/nwb-guide/issues/%s", "%s"),
"request-format-support": ("https://github.com/catalystneuro/neuroconv/issues/new?assignees=&labels=enhancement%%2Cdata+interfaces&projects=&template=format_request.yml&title=%%5BNew+Format%%5D%%3A+%s", "%s"),
"path-expansion-guide": ("https://neuroconv.readthedocs.io/en/main/user_guide/expand_path.html%s", "%s"),
"dandi-staging": ("https://gui-staging.dandiarchive.org/%s", "%s"),
"dandi-sandbox": ("https://sandbox.dandiarchive.org/%s", "%s"),
"dandi-archive": ("https://dandiarchive.org/%s", "%s"),
"conda-install": (
"https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation%s",
Expand Down
2 changes: 1 addition & 1 deletion docs/developer_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ Updating Tutorial Screenshots
Before a release, you'll want to update the tutorial screenshots to reflect the latest changes in the application.

#. To regenerate the dataset, you'll need to change ``regenerateTestData`` in the ``tests/e2e/config.ts`` to ``true`` or delete the test dataset directory ``rm -rf ~/NWB_GUIDE/.test``.
#. Create a ``.env`` file with the following content: ``DANDI_STAGING_API_KEY={your_dandi_staging_api_key}`` where ``{your_dandi_staging_api_key}`` is your DANDI staging API key from https://gui-staging.dandiarchive.org.
#. Create a ``.env`` file with the following content: ``DANDI_STAGING_API_KEY={your_dandi_sandbox_api_key}`` where ``{your_dandi_sandbox_api_key}`` is your DANDI sandbox API key from https://sandbox.dandiarchive.org.
#. Run the End-to-End Tests locally using ``npm run test:tutorial``. This will generate new screenshots in the ``docs/assets/tutorials`` directory.
#. Review the new screenshots to ensure they are accurate.
#. If the screenshots are accurate, commit them to the repository. Their paths should be consistent across runs—allowing the new versions to show up on the tutorial.
Expand Down
14 changes: 7 additions & 7 deletions docs/tutorials/dataset_publication.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Dataset Publication
For this tutorial, we'll publish the NWB files created in :doc:`Multi-Session Tutorial </tutorials/multiple_sessions>` as a Dandiset on the DANDI Archive. This workflow works for any collection of NWB files (2.1+), whether they were created by NWB GUIDE or not.

.. note::
Creating an account on DANDI requires approval from the archive administrators. Separate approval is required for both the main archive and the development server.
Creating an account on DANDI requires approval from the archive administrators. Separate approval is required for both the main archive and the sandbox server.

**This tutorial requires an account on the** :dandi-staging:`DANDI Development server <>`. You’ll want to publish your `real` data on the main archive, which will require a separate approval but otherwise follows the same workflow defined in this tutorial.
**This tutorial requires an account on the** :dandi-sandbox:`DANDI Sandbox server <>`. You’ll want to publish your `real` data on the main archive, which will require a separate approval but otherwise follows the same workflow defined in this tutorial.

Once your account is approved, you can move on to the next steps.

Expand All @@ -24,31 +24,31 @@ You'll now notice that the **Exit Pipeline** button has been replaced with **Nex

DANDI Upload
------------
You’ll need to specify your DANDI API keys if you haven’t uploaded from the GUIDE before. These keys are unique between the Main and Development servers.
You’ll need to specify your DANDI API keys if you haven’t uploaded from the GUIDE before. These keys are unique between the Main and Sandbox servers.

.. figure:: ../assets/tutorials/dandi/api-tokens.png
:align: center
:alt: A pop-up asking for DANDI API keys

To get your API key, visit the :dandi-staging:`staging website <>` and click on the profile icon in the top-right corner. This will show a dropdown with a copy button, which will assign your API key to the clipboard.
To get your API key, visit the :dandi-sandbox:`sandbox website <>` and click on the profile icon in the top-right corner. This will show a dropdown with a copy button, which will assign your API key to the clipboard.

.. figure:: ../assets/dandi/api-token-location.png
:align: center
:alt: DANDI Development server API key added
:alt: DANDI Sandbox server API key added

Provide this for the Development API Key value on the GUIDE.

.. figure:: ../assets/tutorials/dandi/api-token-added.png
:align: center
:alt: DANDI staging API key added
:alt: DANDI sandbox API key added

Press the **Submit** button to save your API key. This will populate the **Dandiset** input with a list of Dandisets associated with your account, which you can search by title or ID.

But what if you don't have any Dandisets to upload to? No problem!

Creating a Dandiset
^^^^^^^^^^^^^^^^^^^
If you don't already have a Dandiset on the Development server, you can create one directly from the GUIDE.
If you don't already have a Dandiset on the Sandbox server, you can create one directly from the GUIDE.

Press the **Create New Dandiset** button to open a pop-up that guides you through the required fields for Dandiset creation.

Expand Down
10 changes: 5 additions & 5 deletions docs/tutorials/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,17 @@ In these tutorials, you'll follow along on a :doc:`local installation of the GUI
Watch a video walkthrough of these tutorials `here <https://www.youtube.com/watch?v=EhhdDVuHAZ0>`_.

.. note::
This tutorial focuses on uploading to the DANDI Development server.
This tutorial focuses on uploading to the DANDI Sandbox server.

**When working with real data, you'll want to publish to the Main Archive**. In this case, follow the same steps outlined here—except replace the Development server with the Main Archive.
**When working with real data, you'll want to publish to the Main Archive**. In this case, follow the same steps outlined here—except replace the Sandbox server with the Main Archive.

.. note::

If you intend to complete the Dataset Publication section of this tutorial, you'll need to create an account on the DANDI Archive. This will need to be approved by the archive administrators and will require separate approval for both the main archive and the development server.
If you intend to complete the Dataset Publication section of this tutorial, you'll need to create an account on the DANDI Archive. This will need to be approved by the archive administrators and will require separate approval for both the main archive and the sandbox server.

**This tutorial requires an account on the** :dandi-staging:`DANDI Development server <>`.
**This tutorial requires an account on the** :dandi-sandbox:`DANDI Sandbox server <>`.

We recommend that you create an account on the Development server before you begin the tutorials.
We recommend that you create an account on the Sandbox server before you begin the tutorials.

Before you begin these tutorials, **you'll need to generate the tutorial dataset** using the instructions on the Dataset page.

Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions src/electron/frontend/core/components/DandiResults.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { LitElement, css, html } from "lit";

import { get } from "dandi";
import { isStaging, getAPIKey } from "../../utils/upload";
import { isSandbox, getAPIKey } from "../../utils/upload";

export class DandiResults extends LitElement {
static get styles() {
Expand Down Expand Up @@ -38,9 +38,9 @@ export class DandiResults extends LitElement {

const otherElIds = ["embargo_status"];

const staging = isStaging(this.id);
const type = staging ? "staging" : undefined;
const api_key = await getAPIKey.call(this, staging);
const sandbox = isSandbox(this.id);
const type = sandbox ? "staging" : undefined;
const api_key = await getAPIKey.call(this, sandbox);

const dandiset = await get(this.id, {
type,
Expand Down
4 changes: 2 additions & 2 deletions src/electron/frontend/core/components/Table.js
Original file line number Diff line number Diff line change
Expand Up @@ -670,11 +670,11 @@ export class Table extends LitElement {
let message = "";
let theme = "";
if (warnings.length) {
(theme = "warning"), (message = warnings.map((error) => error.message).join("\n"));
((theme = "warning"), (message = warnings.map((error) => error.message).join("\n")));
} else cell.removeAttribute("warning");

if (errors.length) {
(theme = "error"), (message = errors.map((error) => error.message).join("\n")); // Class switching handled automatically
((theme = "error"), (message = errors.map((error) => error.message).join("\n"))); // Class switching handled automatically
} else cell.removeAttribute("error");

if (theme) cell.setAttribute(theme, "");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ export class GuidedUploadPage extends Page {
formProps: {
validateOnChange: async (name, parent) => {
const value = parent[name];
if (name.includes("api_key")) return await validateDANDIApiKey(value, name.includes("staging"));
if (name.includes("api_key")) return await validateDANDIApiKey(value, name.includes("sandbox"));
},
},
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -330,7 +330,7 @@ export class SettingsPage extends Page {
onUpdate: () => (this.unsavedUpdates = true),
validateOnChange: async (name, parent) => {
const value = parent[name];
if (name.includes("api_key")) return await validateDANDIApiKey(value, name.includes("staging"));
if (name.includes("api_key")) return await validateDANDIApiKey(value, name.includes("sandbox"));
return true;
},
onThrow,
Expand Down
38 changes: 19 additions & 19 deletions src/electron/frontend/core/components/pages/uploads/UploadsPage.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import keyIcon from "../../../../assets/icons/key.svg?raw";
import {
AWARD_VALIDATION_FAIL_MESSAGE,
awardNumberValidator,
isStaging,
isSandbox,
validate,
getAPIKey,
} from "../../../../utils/upload";
Expand Down Expand Up @@ -119,14 +119,14 @@ export function createDandiset(results = {}) {
throw message;
});

const uploadToMain = form.resolved.archive === "main";
const staging = !uploadToMain;
const api_key = await getAPIKey.call(this, staging);
const api = new dandi.API({
token: api_key,
type: staging ? "staging" : undefined,
const uploadToMain = form.resolved.archive === "main";
const sandbox = !uploadToMain;

const api_key = await getAPIKey.call(this, sandbox);

const api = new dandi.API({
token: api_key,
type: sandbox ? "staging" : undefined,
});

await api.authorize();
Expand Down Expand Up @@ -185,15 +185,15 @@ export async function uploadToDandi(info, type = "project" in info ? "project" :

const dandiset_id = dandiset;

const staging = isStaging(dandiset_id); // Automatically detect staging IDs
const api_key = await getAPIKey.call(this, staging);
const payload = {
dandiset_id,
...info.additional_settings,
staging,
api_key,
const sandbox = isSandbox(dandiset_id); // Automatically detect sandbox IDs

const api_key = await getAPIKey.call(this, sandbox);

const payload = {
dandiset_id,
...info.additional_settings,
staging: sandbox,
api_key,
};

if (info.project) payload.project = info.project;
Expand Down Expand Up @@ -269,7 +269,7 @@ export class UploadsPage extends Page {
formProps: {
validateOnChange: async (name, parent) => {
const value = parent[name];
if (name.includes("api_key")) return await validateDANDIApiKey(value, name.includes("staging"));
if (name.includes("api_key")) return await validateDANDIApiKey(value, name.includes("sandbox"));
},
},
}));
Expand Down
6 changes: 3 additions & 3 deletions src/electron/frontend/core/validation/dandi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ const dandiAPITokenRegex = /^[a-f0-9]{40}$/;

import { validateToken } from 'dandi'

export const validateDANDIApiKey = async (apiKey: string, staging = false) => {
export const validateDANDIApiKey = async (apiKey: string, sandbox = false) => {
if (apiKey) {

if (!dandiAPITokenRegex.test(apiKey)) return [{ type: "error", message: `Invalid API key format. Must be a 40 character hexadecimal string` }];

const authFailedError = {type: 'error', message: `Authorization failed. Make sure you're providing an API key for the <a href='https://${staging ? 'gui-staging.' : ''}dandiarchive.org' target='_blank'>${staging ? 'staging' : 'main'} archive</a>.`}
const authFailedError = {type: 'error', message: `Authorization failed. Make sure you're providing an API key for the <a href='https://${sandbox ? 'sandbox.' : ''}dandiarchive.org' target='_blank'>${sandbox ? 'sandbox' : 'main'} archive</a>.`}

const isValid = validateToken({ token: apiKey, type: staging ? 'staging' : undefined }).catch(e => false)
const isValid = validateToken({ token: apiKey, type: sandbox ? 'staging' : undefined }).catch(e => false)
if (!isValid) return [ authFailedError ]
return true
}
Expand Down
16 changes: 8 additions & 8 deletions src/electron/frontend/utils/upload.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import { NotyfNotification } from "notyf";
import dandiGlobalSchema from "../../../schemas/json/dandi/global.json";
import { isNumericString } from "./typecheck";

export const isStaging = (id: string) => parseInt(id) >= 100000;
export const isSandbox = (id: string) => parseInt(id) >= 100000;

type NotificationType = {
type: string;
Expand Down Expand Up @@ -51,9 +51,9 @@ export async function validate(
message: `<b>Invalid ID –</b> Dandiset ID must be 6 digits.`
}]

const staging = isStaging(value)
const type = staging ? "staging" : undefined;
const token = await getAPIKey.call(this, staging);
const sandbox = isSandbox(value)
const type = sandbox ? "staging" : undefined;
const token = await getAPIKey.call(this, sandbox);

const dandiset = await get(value, {
type,
Expand Down Expand Up @@ -106,14 +106,14 @@ export const AWARD_VALIDATION_FAIL_MESSAGE = 'Award number must be properly spac
// this:
export async function getAPIKey(
this: Page,
staging = false
sandbox = false
) {

const whichAPIKey = staging ? "development_api_key" : "main_api_key";
const whichAPIKey = sandbox ? "sandbox_api_key" : "main_api_key";
const DANDI = global.data.DANDI;
let api_key = DANDI?.api_keys?.[whichAPIKey];

const errors = await validateDANDIApiKey(api_key, staging);
const errors = await validateDANDIApiKey(api_key, sandbox);

const isInvalid = Array.isArray(errors) ? errors.length : !errors;

Expand Down Expand Up @@ -154,7 +154,7 @@ export async function getAPIKey(
onClick: async () => {
const value = input.value;
if (value) {
const errors = await validateDANDIApiKey(input.value, staging);
const errors = await validateDANDIApiKey(input.value, sandbox);
if (!errors || !errors.length) {
modal.remove();

Expand Down
16 changes: 8 additions & 8 deletions src/schemas/dandi-upload.schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { Dandiset, getMine } from 'dandi'

import { global } from '../electron/frontend/core/progress'
import upload from './json/dandi/upload.json' assert { type: "json" }
import { isStaging } from '../electron/frontend/utils/upload'
import { isSandbox } from '../electron/frontend/utils/upload'
import { baseUrl, onServerOpen } from '../electron/frontend/core/server/globals'
import { isStorybook } from '../electron/frontend/core/globals'

Expand Down Expand Up @@ -54,16 +54,16 @@ export const regenerateDandisets = async () => {

export const updateDandisets = async (main = true) => {

const staging = !main
const sandbox = !main

// Fetch My Dandisets
const whichAPIKey = staging ? "development_api_key" : "main_api_key";
const whichAPIKey = sandbox ? "sandbox_api_key" : "main_api_key";
const DANDI = global.data.DANDI;
let token = DANDI?.api_keys?.[whichAPIKey];

if (!token) return []

return await getMine({ token, type: staging ? 'staging' : undefined }, { embargoed: true })
return await getMine({ token, type: sandbox ? 'staging' : undefined }, { embargoed: true })
.then((results) => results ? Promise.all(results.map(addDandiset)) : [])
.catch(() => {
return []
Expand All @@ -85,22 +85,22 @@ export const addDandiset = async (info) => {
enumSet.add(id)
idSchema.enum = Array.from(enumSet)

const staging = isStaging(id)
const sandbox = isSandbox(id)

if (!idSchema.enumLabels) idSchema.enumLabels = {}
if (!idSchema.enumKeywords) idSchema.enumKeywords = {}
if (!idSchema.enumCategories) idSchema.enumCategories = {}


const token = global.data.DANDI.api_keys[staging ? "development_api_key" : "main_api_key"];
const token = global.data.DANDI.api_keys[sandbox ? "sandbox_api_key" : "main_api_key"];

info = new Dandiset(info, { type: staging ? "staging" : undefined, token })
info = new Dandiset(info, { type: sandbox ? "staging" : undefined, token })

const latestVersionInfo = (info.most_recent_published_version ?? info.draft_version)!
const enumLabels = `${id} — ${latestVersionInfo.name}`

const isDraft = latestVersionInfo.version === 'draft'
const enumCategories = (isDraft ? 'Unpublished' : '') + (staging ? `${isDraft ? ` - ` : ''}Staging` : '')
const enumCategories = (isDraft ? 'Unpublished' : '') + (sandbox ? `${isDraft ? ` - ` : ''}Sandbox` : '')

const fullInfo = await info.getInfo({ version: latestVersionInfo.version });

Expand Down
6 changes: 3 additions & 3 deletions src/schemas/json/dandi/create.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@
"archive": {
"type": "string",
"enumLabels": {
"staging": "Development Server",
"sandbox": "Sandbox Server",
"main": "Main Archive"
},
"enum": ["main", "staging"],
"description": "Which DANDI server to upload to. <br><small><b>Note:</b> The Development Server is recommended for developers, or users learning to use DANDI</small>",
"enum": ["main", "sandbox"],
"description": "Which DANDI server to upload to. <br><small><b>Note:</b> The Sandbox Server is recommended for developers, or users learning to use DANDI</small>",
"strict": true
},

Expand Down
Loading