Skip to content

Commit 3d40460

Browse files
swheatonjleven
swheaton
and
jleven
authored
Documentation: On-demand orc setup instructions (#426)
* docs(do): on-demand orc setup instructions for databricks/anyscale * rabbit * minor rabbit * Point to the model zoo docs for model dependencies * fix trailing whitespace --------- Co-authored-by: Josh Leven <josh@voxel51.com>
1 parent bed4758 commit 3d40460

6 files changed

Lines changed: 846 additions & 0 deletions

File tree

‎docker/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ regarding FiftyOne Enterprise.
6262
- [:page_facing_up: Optional: Upload Run Logs](#page_facing_up-optional-upload-run-logs)
6363
- [:desktop_computer: GPU-Enabled Workloads](#desktop_computer-gpu-enabled-workloads)
6464
- [:bricks: Custom Plugin Images](#bricks-custom-plugin-images)
65+
- [:on: On-Demand Delegated Operator Executors](#on-on-demand-delegated-operator-executors)
6566
- [Step 8: Identity Provider (IdP) and Authentication (CAS)](#step-8-identity-provider-idp-and-authentication-cas)
6667
- [:information_source: IdP configuration](#information_source-idp-configuration)
6768
- [:hammer_and_wrench: Optional: CAS Customization Instructions](#hammer_and_wrench-optional-cas-customization-instructions)
@@ -438,6 +439,12 @@ and deploy **custom plugin images**. You can base them on
438439
- ML libraries (e.g. PyTorch, OpenCV)
439440
- Internal SDKs or models
440441

442+
### :on: On-Demand Delegated Operator Executors
443+
444+
FiftyOne Enterprise v2.11 introduces support for on-demand delegated operator
445+
executors for Databricks and Anyscale. Please refer to the
446+
[configuration documentation](https://github.com/voxel51/fiftyone-teams-app-deploy/blob/main/docs/configuring-on-demand-orchestrator.md).
447+
441448
## Step 8: Identity Provider (IdP) and Authentication (CAS)
442449

443450
### :information_source: IdP configuration

‎docs/configuring-on-demand-orchestrator.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
<!-- markdownlint-disable no-inline-html line-length -->
2+
<!-- markdownlint-disable-next-line first-line-heading -->
3+
<div align="center">
4+
<p align="center">
5+
6+
<img alt="Voxel51 Logo" src="https://user-images.githubusercontent.com/25985824/106288517-2422e000-6216-11eb-871d-26ad2e7b1e59.png" height="55px"> &nbsp;
7+
<img alt="Voxel51 FiftyOne" src="https://user-images.githubusercontent.com/25985824/106288518-24bb7680-6216-11eb-8f10-60052c519586.png" height="50px">
8+
9+
</p>
10+
</div>
11+
<!-- markdownlint-enable no-inline-html line-length -->
12+
13+
---
14+
15+
# Configuring On-Demand Orchestrators
16+
17+
As of FiftyOne Enterprise v2.11.0, delegated operations can be run on-demand
18+
in select external compute platforms (orchestrators).
19+
20+
This is the list of supported orchestrators and their configuration guides:
21+
22+
- [Anyscale](./orchestrators/configuring-anyscale-orchestrator.md)
23+
- [Databricks](./orchestrators/configuring-databricks-orchestrator.md)

‎docs/orchestrators/configuring-anyscale-orchestrator.md‎

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
<!-- markdownlint-disable no-inline-html line-length -->
2+
<!-- markdownlint-disable-next-line first-line-heading -->
3+
<div align="center">
4+
<p align="center">
5+
6+
<img alt="Voxel51 Logo" src="https://user-images.githubusercontent.com/25985824/106288517-2422e000-6216-11eb-871d-26ad2e7b1e59.png" height="55px"> &nbsp;
7+
<img alt="Voxel51 FiftyOne" src="https://user-images.githubusercontent.com/25985824/106288518-24bb7680-6216-11eb-8f10-60052c519586.png" height="50px">
8+
9+
</p>
10+
</div>
11+
<!-- markdownlint-enable no-inline-html line-length -->
12+
13+
---
14+
15+
# Anyscale On-Demand Orchestrator Setup
16+
17+
This document provides a step-by-step guide to configuring FiftyOne Enterprise
18+
to use [Anyscale](https://www.anyscale.com/) as an orchestrator for running
19+
delegated operations on-demand.
20+
21+
## Compute Config
22+
23+
Define and manage the machines, scaling, and other configurations for computer
24+
resources [directly in Anyscale](https://docs.anyscale.com/configuration/compute/overview/).
25+
26+
## Dependency Management
27+
28+
[Dependency management](https://docs.anyscale.com/configuration/dependency-management/dependency-overview)
29+
can be done using Anyscale Container Images. The images are written with
30+
Docker-like specification and must be
31+
[customized](https://docs.anyscale.com/configuration/dependency-management/dependency-container-images#customizing-a-container-image)
32+
to include FiftyOne and, optionally, any custom operators.
33+
34+
Note: Some zoo models require additional packages. You can check the
35+
requirements for any zoo model in the [fiftyone documentation](https://docs.voxel51.com/model_zoo/models.html):
36+
find the model, then look under `Requirements` > `Packages`.
37+
38+
1. Determine base for Container Image using one of the following options:
39+
- An [Anyscale base image](https://docs.anyscale.com/reference/anyscale-base-images)
40+
- A custom image meeting the [requirements](https://docs.anyscale.com/configuration/dependency-management/image-requirement)
41+
1. Add the FiftyOne Python package to the Container Image
42+
(_see following example Dockerfile_)
43+
44+
```dockerfile
45+
FROM anyscale/ray:2.46.0-slim-py312
46+
47+
ARG FIFTYONE_ENTERPRISE_PYPI_TOKEN
48+
ARG FIFTYONE_ENTERPRISE_VERSION
49+
50+
# Install system level packages here
51+
52+
# Install fiftyone
53+
RUN pip install fiftyone==${FIFTYONE_ENTERPRISE_VERSION} \
54+
--index-url https://${FIFTYONE_ENTERPRISE_PYPI_TOKEN}@pypi.dev.fiftyone.ai/simple/ \
55+
--extra-index-url "https://pypi.org/simple"
56+
57+
# Install extra python packages here.
58+
59+
# Install custom operators here.
60+
```
61+
62+
1. Build and tag the Container Image (_see following example command_)
63+
64+
```commandline
65+
docker build . \
66+
-t fiftyone-anyscale-example \
67+
--build-arg FIFTYONE_ENTERPRISE_VERSION=2.11 \
68+
--build-arg FIFTYONE_ENTERPRISE_PYPI_TOKEN=abc123
69+
```
70+
71+
1. Push Container Image to an
72+
[Anyscale supported Docker registry](https://docs.anyscale.com/configuration/dependency-management/dependency-byod#step-2-push-your-image)
73+
74+
```commandline
75+
docker push fiftyone-anyscale-example
76+
```
77+
78+
## Create Service Creds
79+
80+
Create service creds that FiftyOne will use, give the service access to run the
81+
jobs you created above, and keep the following fields for providing to
82+
FiftyOne:
83+
84+
- auth_token
85+
86+
## Register Orchestrator in FiftyOne
87+
88+
To register your orchestrator with FiftyOne, you can use the
89+
[FiftyOne Management SDK](https://docs.voxel51.com/enterprise/management_sdk.html#module-fiftyone.management.orchestrator).
90+
You will need to supply the environment you want to run your orchestrator
91+
(`fom.OrchestratorEnvironment.ANYSCALE`), and then the configuration and
92+
credential information needed to access that runner. To use the FiftyOne
93+
Management SDK, you will also need an `API_URI` set in the environment or
94+
FiftyOne configuration.
95+
96+
When registering your orchestrator with FiftyOne, you will need to supply
97+
credential information, which is stored as a
98+
[FiftyOne Secret](https://docs.voxel51.com/enterprise/secrets.html). The
99+
``secrets`` parameter to
100+
[``fom.register_orchestrator()``](https://docs.voxel51.com/enterprise/management_sdk.html#fiftyone.management.orchestrator.register_orchestrator)
101+
takes a top level key that must match your orchestrator environment. The
102+
object that follows has key and value pairs that are specific to the
103+
credentials needed to access your orchestrator.
104+
105+
When supplying one of the values, a new Secret will be created for you that
106+
securely stores the information provided. These can be managed via the
107+
Secrets manager.
108+
109+
Optionally, if you have an existing Secret that already has the credentials
110+
you’d like to use, you can provide the name of that Secret and it will be used
111+
instead of creating a new one. Examples of both options are included below.
112+
113+
Example snippet using the Management SDK to register an Anyscale orchestrator:
114+
115+
```python
116+
import fiftyone.management as fom
117+
fom.register_orchestrator(
118+
instance_id="your-orchestrator-name",
119+
description="Your orchestrator description",
120+
environment=fom.OrchestratorEnvironment.ANYSCALE,
121+
config={
122+
fom.OrchestratorEnvironment.ANYSCALE: { # config
123+
"jobQueueName": "your-job-queue-name",
124+
"imageUri": "your-image-uri",
125+
"executionComputeConfig": "your-execution-compute-config",
126+
"registrationComputeConfig": "your-registration-compute-config", # optional
127+
"idleTimeoutS": 300, # optional, defaults to 300
128+
"pluginsDir": "your-plugins-dir", # optional
129+
}
130+
},
131+
secrets={
132+
fom.OrchestratorEnvironment.ANYSCALE: { # secrets
133+
"authToken": "your-anyscale-auth-token"
134+
}
135+
}
136+
)
137+
```
138+
139+
This will register a new orchestrator with the identifier
140+
`your-orchestrator-name`.
141+
142+
Additionally, it will save a new Secret for the value supplied in authToken.
143+
That new secret will have the following name:
144+
145+
`AUTH_TOKEN_YOUR_ORCHESTRATOR_NAME`
146+
147+
As noted above, if you already had Secrets saved with values you would like to
148+
use, these names could be supplied in place of the values in the `secrets`
149+
parameter. Here is an example:
150+
151+
```python
152+
import fiftyone.management as fom
153+
fom.register_orchestrator(
154+
instance_id="your-orchestrator-name",
155+
description="Your orchestrator description",
156+
environment=fom.OrchestratorEnvironment.ANYSCALE,
157+
config={
158+
fom.OrchestratorEnvironment.ANYSCALE: { # config
159+
"jobQueueName": "your-job-queue-name",
160+
"imageUri": "your-image-uri",
161+
"executionComputeConfig": "your-execution-compute-config",
162+
"registrationComputeConfig": "your-registration-compute-config", # optional
163+
"idleTimeoutS": 300, # optional, defaults to 300
164+
"pluginsDir": "your-plugins-dir", # optional
165+
}
166+
},
167+
secrets={
168+
fom.OrchestratorEnvironment.ANYSCALE: { # secrets
169+
"authToken": "EXISTING_AUTH_TOKEN_SECRET"
170+
}
171+
}
172+
)
173+
```
174+
175+
In this case, new Secrets will not be created since valid names for existing
176+
secrets have been provided. Those existing Secrets will be associated with the
177+
orchestrator.
178+
179+
## Refresh Orchestrator Operators
180+
181+
Before you can do this step make sure you’ve added the optional dependency
182+
`anyscale` into your FiftyOne API deployment, and set the environment
183+
variable `API_EXTERNAL_URL` (the external Teams API base URL) used by Anyscale
184+
workers to talk back to FiftyOne during registration/refresh. The external URL
185+
can be found under `/settings/api_keys` in the UI. The API must be exposed
186+
outside of the internal network
187+
([helm](../../helm/docs/expose-teams-api.md) / [docker](../../docker/docs/expose-teams-api.md)).
188+
189+
This step is only required if you’ve added a plugin directory with custom
190+
plugins to your Anyscale environment.
191+
192+
Once your orchestrator is registered in FiftyOne you can now refresh the
193+
available operators for that environment. To do so, go to any dataset/runs page
194+
and select your orchestrator on the right hand side.
195+
196+
Select the “refresh” button and click “confirm” when prompted. This will kick
197+
off a job in your Anyscale that will tell FiftyOne what operators are available
198+
in that environment. Once you see the job is complete, reload the page and
199+
verify your “available operators” show the ones that you have configured.
200+
201+
In the future, anytime you add new operators to your Anyscale environment, you
202+
will go through this same workflow or you can run that same task again directly
203+
in Anyscale.
204+
205+
## Additional Considerations
206+
207+
Your Anyscale service account will need the following permissions for your
208+
cloud storage platform of choice:
209+
210+
- Storage Bucket Viewer
211+
- Storage Object Viewer
212+
- Write permissions, If you setup
213+
[cloud storage logging](https://docs.voxel51.com/enterprise/plugins.html#logs)
214+
- Blob sign permission, if the plugin uses signed URLs and your cloud platform
215+
requires additional permissions.
216+
217+
Additionally:
218+
219+
- “Anyscale” is not automatically built into the API image so you’ll need to
220+
add it as an extra dependency
221+
- Make sure your API service has the environment variable `API_EXTERNAL_URL`
222+
set to your API_URI since this will be used to set the API endpoint in your
223+
Anyscale workers
224+
225+
## Credential Expiration and Rotation
226+
227+
In order to rotate your Anyscale credentials in FiftyOne:
228+
229+
1. Regenerate credentials through the Anyscale UI or SDK
230+
1. Update the credentials in FiftyOne using the following FOM commands:
231+
232+
```python
233+
import fiftyone.management as fom
234+
235+
orc = fom.get_orchestrator("<your-orc-instance-id>")
236+
fom.update_secret(
237+
key=orc.secrets['auth_token'],
238+
value="<new_credentials>",
239+
)
240+
```

0 commit comments

Comments
 (0)