Skip to content

Commit 00dade3

Browse files
authored
Tell user to disable multiprocessing if they have more issues (#429)
* tell user to disable multiprocessing if they have more issues * dont need double backtick in .md
1 parent a1a0937 commit 00dade3

2 files changed

Lines changed: 25 additions & 19 deletions

File tree

docs/orchestrators/configuring-anyscale-orchestrator.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -99,8 +99,8 @@ FiftyOne configuration.
9999
When registering your orchestrator with FiftyOne, you will need to supply
100100
credential information, which is stored as a
101101
[FiftyOne Secret](https://docs.voxel51.com/enterprise/secrets.html). The
102-
``secrets`` parameter to
103-
[``fom.register_orchestrator()``](https://docs.voxel51.com/enterprise/management_sdk.html#fiftyone.management.orchestrator.register_orchestrator)
102+
`secrets` parameter to
103+
[`fom.register_orchestrator()`](https://docs.voxel51.com/enterprise/management_sdk.html#fiftyone.management.orchestrator.register_orchestrator)
104104
takes a top level key that must match your orchestrator environment. The
105105
object that follows has key and value pairs that are specific to the
106106
credentials needed to access your orchestrator.
@@ -228,8 +228,11 @@ Additionally:
228228
- Due to a limitation discovered in the connection between Anyscale and
229229
MongoDB Atlas, using more than 4 parallel processes can lead to connection
230230
issues. We recommend setting the environment variable
231-
``FIFTYONE_MAX_PROCESS_POOL_WORKERS`` to ``4`` in your created docker image
231+
`FIFTYONE_MAX_PROCESS_POOL_WORKERS` to `4` in your created docker image
232232
to avoid this issue, if you are using MongoDB Atlas.
233+
- If you still experience connection issues or database-stored cloud
234+
credentials are not being found, you should set
235+
`FIFTYONE_MAX_PROCESS_POOL_WORKERS` to `0` to disable multiprocessing.
233236

234237
## Credential Expiration and Rotation
235238

docs/orchestrators/configuring-databricks-orchestrator.md

Lines changed: 19 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ add them here.
4040
for any zoo model in the [FiftyOne documentation](https://docs.voxel51.com/model_zoo/models.html):
4141
find the model, then look under `Requirements` > `Packages`.
4242

43-
Save your ``DBFS_PATH`` for later as it will be used when creating your
43+
Save your `DBFS_PATH` for later as it will be used when creating your
4444
job configuration. The script will also create the file in your Databricks
4545
account using the Databricks SDK. Alternatively you can create this manually in
4646
the UI.
@@ -90,7 +90,7 @@ correspond to your FiftyOne deployment. These secrets are: Mongo database URI,
9090
FiftyOne encryption key, and FiftyOne pypi url. To follow security best
9191
practices, the code below will create secrets in Databricks. Keep the path of
9292
these secrets with the scope you create, which should look something like:
93-
``secrets/your-scope/FIFTYONE_DATABASE_URI``. We will use these secrets when
93+
`secrets/your-scope/FIFTYONE_DATABASE_URI`. We will use these secrets when
9494
creating the job config environment variables.
9595

9696
```python
@@ -254,7 +254,7 @@ custom plugins. There are many ways to set this up, but here are some examples:
254254
- Give shared volume access to your Databricks
255255

256256
Regardless of your chosen solution, save the absolute file path to be used in
257-
the ``FIFTYONE_PLUGINS_DIR`` environment variable when setting up your job
257+
the `FIFTYONE_PLUGINS_DIR` environment variable when setting up your job
258258
config. Read more about configuring plugins for
259259
[helm](../helm/docs/configuring-plugins.md) and
260260
[docker](../docker/docs/configuring-plugins.md).
@@ -400,7 +400,7 @@ the jobs you created above and keep the following fields for providing to
400400
FiftyOne:
401401

402402
- Host
403-
- The URL you view your account at eg: ``https://1290481.3.gcp.databricks.com/``
403+
- The URL you view your account at eg: `https://1290481.3.gcp.databricks.com/`
404404
- Account Id
405405
- Account drop down (top right)
406406
- Manage account
@@ -420,16 +420,16 @@ FiftyOne:
420420
To register your orchestrator with FiftyOne, you can use the
421421
[FiftyOne Management SDK](https://docs.voxel51.com/enterprise/management_sdk.html#module-fiftyone.management.orchestrator).
422422
You will need to supply the environment you want to run your
423-
orchestrator (``fom.OrchestratorEnvironment.DATABRICKS``), and then the
423+
orchestrator (`fom.OrchestratorEnvironment.DATABRICKS`), and then the
424424
configuration and credential information needed to access that runner. To use
425-
the FiftyOne Management SDK, you will also need an ``API_URI`` set in the
425+
the FiftyOne Management SDK, you will also need an `API_URI` set in the
426426
environment or FiftyOne configuration.
427427

428428
When registering your orchestrator with FiftyOne, you will need to supply
429429
credential information, which is stored as a
430430
[FiftyOne Secret](https://docs.voxel51.com/enterprise/secrets.html). The
431-
``secrets`` parameter to
432-
[``fom.register_orchestrator()``](https://docs.voxel51.com/enterprise/management_sdk.html#fiftyone.management.orchestrator.register_orchestrator)
431+
`secrets` parameter to
432+
[`fom.register_orchestrator()`](https://docs.voxel51.com/enterprise/management_sdk.html#fiftyone.management.orchestrator.register_orchestrator)
433433
takes a top level key that must match your orchestrator environment. The
434434
object that follows has key and value pairs that are specific to the
435435
credentials needed to access your orchestrator.
@@ -469,19 +469,19 @@ fom.register_orchestrator(
469469
```
470470

471471
This will register a new orchestrator with the identifier
472-
``your-orchestrator-name``.
472+
`your-orchestrator-name`.
473473

474474
Additionally, it will save four new Secrets, one each for
475-
``host, accountId, clientId, clientSecret``. Those new secrets will have the
475+
`host, accountId, clientId, clientSecret`. Those new secrets will have the
476476
following names, respectively:
477477

478-
``HOST_YOUR_ORCHESTRATOR_NAME``
479-
``ACCOUNT_ID_YOUR_ORCHESTRATOR_NAME``
480-
``CLIENT_ID_YOUR_ORCHESTRATOR_NAME``
481-
``CLIENT_SECRET_YOUR_ORCHESTRATOR_NAME``
478+
`HOST_YOUR_ORCHESTRATOR_NAME`
479+
`ACCOUNT_ID_YOUR_ORCHESTRATOR_NAME`
480+
`CLIENT_ID_YOUR_ORCHESTRATOR_NAME`
481+
`CLIENT_SECRET_YOUR_ORCHESTRATOR_NAME`
482482

483483
As noted above, if you already had Secrets saved with values you would like to
484-
use, these names could be supplied in place of the values in the ``secrets``
484+
use, these names could be supplied in place of the values in the `secrets`
485485
parameter. Here is an example:
486486

487487
```python
@@ -554,8 +554,11 @@ Additionally:
554554
- Due to a limitation discovered in the connection between Databricks and
555555
MongoDB Atlas, using more than 4 parallel processes can lead to connection
556556
issues. We recommend setting the environment variable
557-
``FIFTYONE_MAX_PROCESS_POOL_WORKERS`` to ``4`` in your job config to avoid
557+
`FIFTYONE_MAX_PROCESS_POOL_WORKERS` to `4` in your job config to avoid
558558
this issue, if you are using MongoDB Atlas.
559+
- If you still experience connection issues or database-stored cloud
560+
credentials are not being found, you should set
561+
`FIFTYONE_MAX_PROCESS_POOL_WORKERS` to `0` to disable multiprocessing.
559562

560563
### Credential Expiration and Rotation
561564

0 commit comments

Comments
 (0)