Skip to content

Commit de50b1b

Browse files
ramonfigueiredoramonfigueiredo
authored
Improve sandbox install script, pyproject metadata, and update documentation (#1751)
- Updated `sandbox/install-client-sources.sh` to: - Add comments and safety checks - Automatically tag the version based on VERSION.in - Support local macOS builds using grpcio~=1.71.0 and grpcio-tools~=1.71.0 - Updated `cuesubmit/pyproject.toml` to fix script entry point via [project.scripts] - Required to build cuesubmit and launch it via `cuesubmit &` - Revised `requirements.txt` to use grpcio~=1.71.0 and grpcio-tools~=1.71.0 to fix RuntimeError - grpc version 1.69.0 is too old; upgrade to grpcio~=1.71.0 - Fix RuntimeError: The grpc package installed is at version 1.69.0, but the generated code in `comment_pb2_grpc.py` depends on `grpcio>=1.71.0` - Rewrote `sandbox/README.md` with clearer step-by-step setup instructions, including use of `sandbox/install-client-sources.sh` - Updated main `README.md` to reflect improvements in sandbox usage and mention quick install and testing instructions These changes make it easier to locally build and test OpenCue Python clients from source, especially on macOS and with virtual environments. **Link the Issue(s) this Pull Request is related to.** - #1749 - #1750
1 parent e508849 commit de50b1b

File tree

5 files changed

+189
-7
lines changed

5 files changed

+189
-7
lines changed

‎README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,23 @@ For more information on installing, using, and administering OpenCue, visit
4242

4343
Watch YouTube videos on the [OpenCue Playlist](https://www.youtube.com/playlist?list=PL9dZxafYCWmzSBEwVT2AQinmZolYqBzdp) of the Academy Software Foundation (ASWF) to learn more.
4444

45+
# Quick installation and tests
46+
47+
Read the [OpenCue sandbox documentation](https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/sandbox/README.md)
48+
to learn how to set up a local OpenCue environment.
49+
50+
- The sandbox environment offers an easy way to run a test OpenCue deployment locally, with all components running in
51+
separate Docker containers or Python virtual environments.
52+
- It is ideal for small tests, development work, and for those new to OpenCue who want a simple setup for
53+
experimentation and learning.
54+
55+
To learn how to run the sandbox environment, see https://www.opencue.io/docs/quick-starts/.
56+
57+
# OpenCue full installation
58+
59+
Guides for system admins deploying OpenCue components and installing dependencies are available in the
60+
[OpenCue documentation](https://www.opencue.io/docs/getting-started/).
61+
4562
# Meeting notes
4663

4764
Starting from May 2024, all Opencue meeting notes are stored on the [Opencue Confluence page](http://wiki.aswf.io/display/OPENCUE/OpenCue+Home).

‎cuesubmit/pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ default-version = "0.0.0"
2424
[tool.hatch.build.targets.wheel]
2525
packages = ["cuesubmit"]
2626

27-
[console_scripts]
27+
[project.scripts]
2828
cuesubmit = "cuesubmit.__main__:main"
2929

3030
# --- Pytest configuration ---

‎requirements.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
future==1.0.0
22
grpcio==1.48.2;python_version<"3.7"
33
grpcio-tools==1.48.2;python_version<"3.7"
4-
grpcio==1.69.0;python_version>="3.7"
5-
grpcio-tools==1.69.0;python_version>="3.7"
4+
grpcio~=1.71.0;python_version>="3.7"
5+
grpcio-tools~=1.71.0;python_version>="3.7"
66
mock==2.0.0
77
packaging==20.9;python_version<"3.7"
88
packaging==24.1;python_version>="3.7"

‎sandbox/README.md‎

Lines changed: 153 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,165 @@
11
# OpenCue sandbox environment
22

3-
The sandbox environment provides a way to run a test OpenCue deployment. You can use the sandbox
4-
environment to run small tests or development work. The sandbox environment runs OpenCue components
5-
in separate Docker containers on your local machine.
3+
The sandbox environment offers an easy way to run a test OpenCue deployment locally, with all components running in
4+
separate Docker containers or Python virtual environments. It is ideal for small tests, development work, and for
5+
those new to OpenCue who want a simple setup for experimentation and learning.
66

77
To learn how to run the sandbox environment, see
88
https://www.opencue.io/docs/quick-starts/.
99

10+
## Usage example
11+
12+
If you don’t already have a recent local copy of the OpenCue source code, you must do one of the following:
13+
14+
- Download and unzip the [OpenCue source code ZIP file](https://github.com/AcademySoftwareFoundation/OpenCue/archive/master.zip).
15+
- If you have the git command installed on your machine, you can clone the repository:
16+
17+
```bash
18+
git clone https://github.com/AcademySoftwareFoundation/OpenCue.git
19+
```
20+
21+
For developers, you can also use the following commands to set up a local copy of the OpenCue source code:
22+
23+
- Fork the [Opencue repository](https://github.com/AcademySoftwareFoundation/OpenCue) using your GitHub account.
24+
- Clone your forked repository:
25+
```bash
26+
git clone https://github.com/<username>/OpenCue.git
27+
```
28+
29+
### 1. Deploying the OpenCue Sandbox Environment
30+
31+
- Run the services: DB, Flyway, Cuebot, and RQD
32+
- A PostgreSQL database
33+
- A Flyway database migration tool
34+
- A Cuebot server
35+
- An RQD rendering server
36+
37+
In one terminal, run the following commands:
38+
39+
- Create required directories for RQD logs and shots:
40+
41+
```bash
42+
mkdir -p /tmp/rqd/logs /tmp/rqd/shots
43+
```
44+
45+
- Change to the root of the OpenCue source code directory
46+
47+
```bash
48+
cd OpenCue
49+
```
50+
51+
- Build the Cuebot container from source
52+
53+
**Note:** Make sure [Docker](https://www.docker.com/) is installed and running on your machine.
54+
55+
```bash
56+
docker build -t opencue/cuebot -f cuebot/Dockerfile .
57+
```
58+
59+
- Deploy the sandbox environment
60+
- This command will start the services (db, flyway, cuebot, rqd) in the background and create a network for them to
61+
communicate with each other.
62+
63+
```bash
64+
docker compose up
65+
```
66+
67+
**Notes:**
68+
- Use `docker compose up -d` to run the services in detached mode.
69+
- Use `docker compose down` to stop the services and remove the network.
70+
- Use `docker compose logs` to view the logs of the services.
71+
- Use `docker compose ps` to view the status of the services.
72+
- Use `docker compose exec <service> bash` to open a shell in the container of the specified service.
73+
- Use `docker compose exec <service> <command>` to run a command in the container of the specified service.
74+
- Use `docker compose down --volumes` to remove the volumes created by the services.
75+
- Use `docker compose down --rmi all` to remove all images created by the services.
76+
77+
### 2. Installing the OpenCue Client Packages
78+
79+
In a second terminal, run the following commands:
80+
81+
- Make sure you are in the root of the OpenCue source code directory
82+
83+
```bash
84+
cd OpenCue
85+
```
86+
87+
- Create a virtual environment for the Python packages
88+
89+
```bash
90+
python3 -m venv sandbox-venv
91+
```
92+
93+
- Activate the virtual environment
94+
95+
```bash
96+
source sandbox-venv/bin/activate
97+
```
98+
99+
- Upgrade pip to the latest version
100+
101+
```bash
102+
pip install --upgrade pip
103+
```
104+
105+
- Install the required Python packages
106+
107+
```bash
108+
pip install -r requirements.txt
109+
pip install -r requirements_gui.txt
110+
```
111+
112+
- Install the OpenCue Python client libraries from source
113+
- This option is mostly used by developers that contribute to the OpenCue project.
114+
- It is recommended if you want to test the latest changes in the OpenCue source code.
115+
- To install Opencue from source, run:
116+
117+
```bash
118+
./sandbox/install-client-sources.sh
119+
```
120+
121+
**Note:** The latest version of the OpenCue source code might include changes that are incompatible with the prebuilt
122+
OpenCue images of Cuebot and RQD on Docker Hub used in the sandbox environment.
123+
124+
Alternatively, you can use the script `./sandbox/install-client-archive.sh` to download, extract, and install specified
125+
OpenCue client packages for a given release version from GitHub. To learn how to run the sandbox environment, see
126+
https://www.opencue.io/docs/quick-starts/.
127+
- To install the latest versions of the OpenCue client packages, you must configure the installation script with the
128+
version number.
129+
- The script `sandbox/get-latest-release-tag.sh` will automatically fetch this for you, but you can also look up the
130+
version numbers for OpenCue releases on GitHub.
131+
132+
```bash
133+
export VERSION=$(sandbox/get-latest-release-tag.sh)
134+
```
135+
136+
### 3. Testing the Sandbox Environment
137+
138+
To test the sandbox environment, run the following commands inside the Python virtual environment:
139+
140+
- To verify the successful installation and connection between the client packages and sandbox, list the hosts in the
141+
sandbox environment:
142+
143+
```bash
144+
cueadmin -lh
145+
```
146+
147+
- Launch the CueGUI (Cuetopia/CueCommander) app for monitoring and controlling jobs:
148+
149+
```bash
150+
cuegui &
151+
```
152+
153+
- Launch the CueSubmit app for submitting jobs:
154+
155+
```bash
156+
cuesubmit &
157+
```
158+
10159
## Monitoring
11160

12161
To get started with monitoring there is also an additional Docker compose file which sets up
13162
monitoring for key services.
14163

15164
To learn how to run the sandbox environment with monitoring,
16-
see https://www.opencue.io/docs/other-guides/monitoring-with-prometheus-loki-and-grafana/.
165+
see https://www.opencue.io/docs/other-guides/monitoring-with-prometheus-loki-and-grafana/.

‎sandbox/install-client-sources.sh‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,12 @@
11
#!/bin/bash
22

3+
# Installs OpenCue Python client libraries from source.
4+
#
5+
# Read the [OpenCue sandbox documentation](https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/sandbox/README.md)
6+
# to learn how to set up a local OpenCue environment.
7+
8+
# This script should be run from the root of the OpenCue repository.
9+
310
set -e
411

512
# Install all client packages.
@@ -8,6 +15,15 @@ then
815
echo "Installing pre-built cuebot package"
916
pip install ${OPENCUE_PROTO_PACKAGE_PATH}
1017
else
18+
VERSION_TAG="$(cat VERSION.in | tr -d '[:space:]')"
19+
# Check if the tag already exists
20+
if git tag --list "$VERSION_TAG" | grep -q "^$VERSION_TAG$"; then
21+
echo "Tag $VERSION_TAG already exists."
22+
else
23+
echo "Creating tag $VERSION_TAG"
24+
git tag "$VERSION_TAG"
25+
fi
26+
# Build the proto package
1127
pip install proto/
1228
fi
1329
pip install pycue/ pyoutline/ cueadmin/ cuesubmit/ cuegui/

0 commit comments

Comments
 (0)