Skip to content

Commit 2ad98ac

Browse files
MRGuziXMRGuziX
committed
working locally and deployment for crew ai agent
1 parent 8a89678 commit 2ad98ac

25 files changed

Lines changed: 7673 additions & 3 deletions

‎agents/base/crewai-websearch-agent/Dockerfile‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Use Python 3.12 slim image as base
2+
FROM python:3.12-slim
3+
4+
# Set working directory
5+
WORKDIR /app
6+
7+
# Create a non-privileged user for OpenShift
8+
# UID 1001 is commonly used in OpenShift deployments
9+
# Adding to root group (GID 0) is the OpenShift convention for arbitrary UIDs
10+
RUN groupadd -r appuser && useradd -r -u 1001 -g 0 -m -d /home/appuser appuser
11+
12+
# Install uv for fast dependency management
13+
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
14+
15+
# Copy project files for dependency installation
16+
COPY pyproject.toml .
17+
COPY src/ ./src/
18+
19+
# Install the project and its dependencies using uv
20+
RUN uv pip install --system --no-cache .
21+
22+
# Copy the application entrypoint
23+
COPY main.py .
24+
25+
# Pre-create the directory that CrewAI writes to at import time
26+
# (crewai.utilities.paths.db_storage_path -> ~/.local/share/app/)
27+
# Make everything group-writable (GID 0) for OpenShift arbitrary UID support
28+
RUN mkdir -p /home/appuser/.local/share/app \
29+
&& chown -R appuser:0 /app /home/appuser \
30+
&& chmod -R g=u /app /home/appuser
31+
32+
# Switch to non-privileged user
33+
USER appuser
34+
35+
# Expose port 8080 (OpenShift standard)
36+
EXPOSE 8080
37+
38+
# Set environment variables
39+
ENV PORT=8080
40+
ENV HOME=/home/appuser
41+
ENV PYTHONPATH=/app:/app/src
42+
43+
# Run the application
44+
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

‎agents/base/crewai-websearch-agent/LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎agents/base/crewai-websearch-agent/README.md‎

Lines changed: 321 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,321 @@
1+
# A Base CrewAI LLM app template with function calling capabilities 🚀
2+
3+
## 📖 Table of Contents
4+
* [Introduction](#-introduction)
5+
* [Directory structure and file descriptions](#-directory-structure-and-file-descriptions)
6+
* [Prerequisites](#-prerequisites)
7+
* [Installation](#-installation)
8+
* [Configuration](#-configuration)
9+
* [Modifying and configuring the template](#-modifying-and-configuring-the-template)
10+
* [Testing the template](#-testing-the-template)
11+
* [Running the application locally](#-running-the-application-locally)
12+
* [Deploying on IBM Cloud](#-deploying-on-ibm-cloud)
13+
* [Querying the deployment](#-querying-the-deployment)
14+
* [Running the graphical app locally](#-running-the-graphical-app-locally)
15+
* [Evaluating agent](#-evaluating-agent)
16+
* [Cloning template (Optional)](#-cloning-template-optional)
17+
18+
## 🤔 Introduction
19+
20+
This repository provides a basic template for LLM apps built using CrewAI framework. It also makes it easy to deploy them as an AI service as part of IBM watsonx.ai for IBM Cloud[^1].
21+
22+
An AI service is a deployable unit of code that encapsulates the logic of your generative AI use case. For an in-depth description of AI services, please refer to the [IBM watsonx.ai documentation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ai-services-templates.html?context=wx&audience=wdp).
23+
24+
[^1]: _IBM watsonx.ai for IBM Cloud_ is a full and proper name of the component we're using in this template and only a part of the whole suite of products offered in the SaaS model within IBM Cloud environment. Throughout this README, for the sake of simplicity, we'll be calling it just an **IBM Cloud**.
25+
26+
**Highlights:**
27+
28+
* 🚀 Easy-to-extend agent and tool modules
29+
* ⚙️ Configurable via `config.toml`
30+
* 🌐 Step-by-step local and cloud deployment
31+
32+
## 🗂 Directory structure and file descriptions
33+
34+
The high level structure of the repository is as follows:
35+
36+
```
37+
crewai-websearch-agent/
38+
├── src/
39+
│ └── crewai_web_search/
40+
│ ├── crew.py
41+
│ ├── tools/
42+
│ └── config/
43+
├── schema/
44+
├── ai_service.py
45+
├── config.toml.example
46+
├── template.env
47+
└── pyproject.toml
48+
```
49+
50+
* **`crewai_web_search`** folder: Contains CrewAI agents configuration yaml files (`src/crewai_web_search/config`) and auxiliary files used by the deployed function. These files provide various framework specific definitions and extensions. This folder is packaged and sent to IBM Cloud during deployment as a [package extension](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ml-create-custom-software-spec.html?context=wx&audience=wdp#custom-wml).
51+
* **`schema`** folder: Contains request and response schemas for the `/ai_service` endpoint queries.
52+
* **`ai_service.py`** file: Contains the function to be deployed as an AI service defining the application's logic
53+
* **`config.toml.example`** file: A configuration file with placeholders that stores the deployment metadata. After downloading the template repository, copy the contents of the `config.toml.example` file to the `config.toml` file and fill in the required fields. `config.toml` file can also be used to tweak the model for your use case.
54+
* **`template.env`**: A file with placeholders for necessary credentials that are essential to run some of the `ibm-watsonx-ai-cli` commands and to test agent locally. Copy the contents of the `template.env` file to the `.env` file and fill the required fields.
55+
## 🛠 Prerequisites
56+
57+
* **Python 3.11**
58+
* **[Poetry](https://python-poetry.org/)** package manager (install via [pipx](https://github.com/pypa/pipx))
59+
* IBM Cloud access and permissions
60+
61+
## 📥 Installation
62+
63+
To begin working with this template using the Command Line Interface (CLI), please ensure that the IBM watsonx AI CLI tool is installed on your system. You can install or upgrade it using the following command:
64+
65+
1. **Install CLI**:
66+
67+
```sh
68+
pip install -U ibm-watsonx-ai-cli
69+
```
70+
71+
2. **Download template**:
72+
```sh
73+
watsonx-ai template new "base/crewai-websearch-agent"
74+
```
75+
76+
Upon executing the above command, a prompt will appear requesting the user to specify the target directory for downloading the template. Once the template has been successfully downloaded, navigate to the designated template folder to proceed.
77+
78+
> [!NOTE]
79+
> Alternatively, it is possible to set up the template using a different method. For detailed instructions, please refer to the section "[Cloning template (Optional)](#-cloning-template-optional)".
80+
81+
3. **Install Poetry**:
82+
83+
```sh
84+
pipx install --python 3.11 poetry
85+
```
86+
87+
4. **Install the template**:
88+
89+
Running the below commands will install the repository with dev environment in a separate virtual environment
90+
91+
```sh
92+
poetry install --with dev
93+
```
94+
95+
5. **(Optional) Activate the virtual environment**:
96+
97+
```sh
98+
source $(poetry -q env use 3.11 && poetry env info --path)/bin/activate
99+
```
100+
101+
6. **Export PYTHONPATH**:
102+
103+
Adding working directory to PYTHONPATH is necessary for the next steps.
104+
105+
```sh
106+
export PYTHONPATH=$(pwd):${PYTHONPATH}
107+
```
108+
109+
## ⚙️ Configuration
110+
111+
1. Copy `template.env` → `.env`.
112+
2. Copy `config.toml.example` → `config.toml`.
113+
3. Fill in IBM Cloud credentials.
114+
115+
## 🎨 Modifying and configuring the template
116+
117+
[config.toml](config.toml) and [.env](.env) files should be filled in before either deploying the template on IBM Cloud or executing it locally.
118+
Possible config parameters are given in the provided file and explained using comments (when necessary).
119+
120+
121+
The template can also be extended to provide additional key-value data to the application. Create a special asset from within your deployment space called _Parameter Sets_. Use the _watsonx.ai_ library to instantiate it and later reference it from the code.
122+
For detailed description and API please refer to the [IBM watsonx.ai Parameter Set's docs](https://ibm.github.io/watsonx-ai-python-sdk/core_api.html#parameter-sets)
123+
124+
125+
Sensitive data should not be passed unencrypted, e.g. in the configuration file. The recommended way to handle them is to make use of the [IBM Cloud® Secrets Manager](https://cloud.ibm.com/apidocs/secrets-manager/secrets-manager-v2). The approach to integrating the Secrets Manager's API with the app is for the user to decide on.
126+
127+
128+
The [crew.py](src/crewai_web_search/crew.py) creates an AI crew, which consists of agents and their tasks.
129+
For detailed info on how to modify the crew object please refer to [CrewAI's official docs](https://docs.crewai.com/quickstart)
130+
131+
132+
The [ai_service.py](ai_service.py) file encompasses the core logic of the app alongside the way of authenticating the user to the IBM Cloud.
133+
For a detailed breakdown of the ai-service's implementation please refer the [IBM Cloud docs](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/ai-services-create.html?context=wx)
134+
135+
136+
[tools.py](src/crewai_web_search/tools/tools.py) file stores the definition for tools enhancing the chat model's capabilities.
137+
To add a new tool, create a class that extends the `crewai.tools.BaseTool` base class and has a `_run` method defined.
138+
139+
## 🧪 Testing the template
140+
141+
The `tests/` directory's structure resembles the repository. Adding new tests should follow this convention.
142+
For exemplary purposes only the tools and some general utility functions are covered with unit tests.
143+
144+
Running the below command will run the complete tests suite:
145+
```sh
146+
pytest -r 'fEsxX' tests/
147+
```
148+
149+
## 💻 Running the application locally
150+
151+
It is possible to run (or even debug) the ai-service locally, however it still requires creating the connection to the IBM Cloud.
152+
153+
Ensure `config.toml` and `.env` are configured.
154+
155+
You can test and debug your AI service locally via two alternative flows:
156+
157+
### ✅ Recommended flow: CLI
158+
159+
```sh
160+
watsonx-ai template invoke "<PROMPT>"
161+
```
162+
163+
### ⚠️ Alternative flow: Python Script (Deprecated)
164+
165+
1. **Run Python Script**:
166+
167+
```sh
168+
python examples/execute_ai_service_locally.py
169+
```
170+
171+
2. **Ask the model**:
172+
173+
Choose from some pre-defined questions or ask the model your own.
174+
175+
> [!WARNING]
176+
> This flow is deprecated and will be removed in a future release. Please migrate to recommended flow as soon as possible.
177+
178+
## ☁️ Deploying on IBM Cloud
179+
180+
Follow these steps to deploy the model on IBM Cloud.
181+
182+
Ensure `config.toml` and `.env` are configured.
183+
184+
You can deploy your AI service to IBM Cloud via two alternative flows:
185+
186+
### ✅ Recommended flow: CLI
187+
188+
```sh
189+
watsonx-ai service new
190+
```
191+
192+
*Config file updates automatically with `deployment_id`.*
193+
194+
### ⚠️ Alternative flow: Python Script (Deprecated)
195+
196+
```sh
197+
python scripts/deploy.py
198+
```
199+
200+
*Script prints `deployment_id`; update `config.toml`.*
201+
202+
> [!WARNING]
203+
> This flow is deprecated and will be removed in a future release. Please migrate to recommended flow as soon as possible.
204+
205+
## 🔍 Querying the deployment
206+
207+
You can send inference requests to your deployed AI service via two alternative flows:
208+
209+
### ✅ Recommended flow: CLI
210+
211+
```sh
212+
watsonx-ai service invoke --deployment_id "<DEPLOYMENT_ID>" "<PROMPT>"
213+
```
214+
215+
*If `deployment_id` is set in `.env`, omit the flag.*
216+
217+
```sh
218+
watsonx-ai service invoke "<PROMPT>"
219+
```
220+
221+
### ⚠️ Alternative flow: Python Script (Deprecated)
222+
223+
Follow these steps to inference your deployment. The [query_existing_deployment.py](examples/query_existing_deployment.py) file shows how to test the existing deployment using `watsonx.ai` library.
224+
225+
1. **Initialize the deployment ID**:
226+
227+
Initialize the `deployment_id` variable in the [query_existing_deployment.py](examples/query_existing_deployment.py) file.
228+
The _deployment_id_ of your deployment can be obtained from [the previous section](#%EF%B8%8F-deploying-on-ibm-cloud) by running [scripts/deploy.sh](scripts/deploy.py)
229+
230+
2. **Run the script for querying the deployment**:
231+
232+
```sh
233+
python examples/query_existing_deployment.py
234+
```
235+
236+
> [!WARNING]
237+
> This flow is deprecated and will be removed in a future release. Please migrate to recommended flow as soon as possible.
238+
239+
## 🖥️ Running the graphical app locally
240+
241+
You can also run the graphical application locally using the deployed model. All you need to do is deploy the model and follow the steps below. Detailed information for each app is available in its README file.
242+
243+
1. **Download the app**:
244+
245+
```bash
246+
watsonx-ai app new
247+
```
248+
249+
2. **Configure the app**:
250+
251+
All required variables are defined in the `.env` file.
252+
Here is an example of how to create the **WATSONX_BASE_DEPLOYMENT_URL**:
253+
`https://{REGION}.ml.cloud.ibm.com/ml/v4/deployments/{deployment_id}`
254+
255+
256+
```bash
257+
cd <app_name>
258+
cp template.env .env
259+
```
260+
261+
3. **Start the app**:
262+
263+
```bash
264+
watsonx-ai app run
265+
```
266+
267+
3. **Start the app in development mode**:
268+
269+
```bash
270+
watsonx-ai app run --dev
271+
```
272+
273+
This soultion allows user to make changes to the source code while the app is running. Each time changes are saved the app reloads and is working with provided changes.
274+
275+
## 📊 Evaluating agent
276+
If you want to evaluate your agent, you can do so using the following command.
277+
278+
```bash
279+
$ watsonx-ai template eval --tests test.jsonl --metrics answer_similarity,answer_relevance --evaluator llm_as_judge
280+
```
281+
282+
The `eval` command supports several options
283+
284+
__Options:__
285+
- `--tests`: [Required] one or more input data files (in jsonl format) for evaluation
286+
- `--metrics`: [Optional] one or more evaluation metric. If multiple metrics are specified, they must be separated by a comma. If not specified all possible metrics will be used
287+
- `--evaluator`: [Optional] a model name for evaluation, or `llm_as_judge` can be used for a predefined choice (`meta-llama/llama-3-3-70b-instruct`, or `mistralai/mistral-small-3-1-24b-instruct-2503` if former is not available). If not provided, metrics are computed using the method `token_recall`.
288+
289+
__Supported Evaluation Metrics__:
290+
- `answer_similarity` _(can be evaluated with `--evaluator`)_
291+
- `answer_relevance` _(can be evaluated with `--evaluator`)_
292+
- `text_reading_ease`
293+
- `unsuccessful_request_metric`
294+
- `text_grade_level`
295+
296+
The metrics are calculated using the **IBM watsonx.governance SDK** library. You can find more details about these metrics in the official documentation [here](https://ibm.github.io/ibm-watsonx-gov/).
297+
298+
> [!WARNING]
299+
> The `eval` command requires Python version >=3.10,<=3.12
300+
301+
---
302+
303+
**Enjoy your coding! 🚀**
304+
305+
---
306+
307+
## 💾 Cloning template (Optional)
308+
309+
1. **Clone the repo** (sparse checkout):
310+
311+
In order not to clone the whole `IBM/watsonx-developer-hub` repository we'll use git's shallow and sparse cloning feature to checkout only the template's directory:
312+
313+
```sh
314+
git clone --no-tags --depth 1 --single-branch --filter=tree:0 --sparse https://github.com/IBM/watsonx-developer-hub.git
315+
cd watsonx-developer-hub
316+
git sparse-checkout add agents/base/crewai-websearch-agent
317+
cd agents/base/crewai-websearch-agent/
318+
```
319+
320+
> [!NOTE]
321+
> From now on it'll be considered that the working directory is `watsonx-developer-hub/agents/base/crewai-websearch-agent`

0 commit comments

Comments
 (0)