From 2b25de364c65c446147745c5ecac77598fbf6680 Mon Sep 17 00:00:00 2001 From: Mark Andrew Miller <25185273+turbomam@users.noreply.github.com> Date: Tue, 7 Jul 2026 09:52:34 -0400 Subject: [PATCH 1/5] docs: sync env var docs with what the code reads Make README, .env.example, and the code agree: every variable the code reads via os.environ is documented and has a placeholder, and nothing unread appears in either. - fix cp .env-example -> .env.example (broke the first quickstart step) - drop DEFAULT_MODEL/MAX_TOKENS/TEMPERATURE from README (no code reads them) - drop OPENALEX_API_KEY/UNPAYWALL_EMAIL from .env.example (no code reads them) - document CONTACT_EMAIL and CLOUD_ML_REGION - add the NMDC_* ingestion-tuning vars to README and .env.example - fix a non-existent test path in the README Closes #95 Co-Authored-By: Claude Opus 4.8 (1M context) --- .env.example | 17 ++++++++++------- README.md | 23 ++++++++++++++++------- 2 files changed, 26 insertions(+), 14 deletions(-) diff --git a/.env.example b/.env.example index 540c712..67db73d 100644 --- a/.env.example +++ b/.env.example @@ -23,10 +23,13 @@ GEMINI_REGION=region # OpenAlex: required since Feb 2026 for >100k requests/day CONTACT_EMAIL=support@microbiomedata.org -# OpenAlex API key (optional — email-based auth is sufficient for moderate usage). -# Request at https://docs.openalex.org/how-to-use-the-api/api-key -# OPENALEX_API_KEY= - -# Unpaywall email (required — used as the email param in API calls). -# See https://unpaywall.org/products/api -# UNPAYWALL_EMAIL= \ No newline at end of file +# Ingestion tuning (all optional; defaults shown) +# Safety limits on characters read from untrusted XML payloads in DOI resolvers +NMDC_EDI_MAX_XML_CHARS=2000000 +NMDC_DATAONE_SOLR_MAX_XML_CHARS=2000000 +# HTTP retry and connection-pool settings for publication/DOI requests +NMDC_HTTP_RETRY_ATTEMPTS=3 +NMDC_HTTP_RETRY_BACKOFF_SECONDS=0 +NMDC_HTTP_MAX_RETRY_DELAY_SECONDS=30 +NMDC_HTTP_POOL_CONNECTIONS=20 +NMDC_HTTP_POOL_MAXSIZE=100 \ No newline at end of file diff --git a/README.md b/README.md index 664a164..f815167 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ A Python application for the NMDC Submission portal metadata suggestor tool, pow You will need to set up a `.env` file. Copy the example first: ```bash -cp .env-example .env +cp .env.example .env ``` Environment variables used by `LLMClient` and `ConversationManager`: @@ -23,7 +23,8 @@ Environment variables used by `LLMClient` and `ConversationManager`: - `AI_INCUBATOR_BASE_URL`: Base URL for the PNNL AI Incubator API. - `GOOGLE_APPLICATION_CREDENTIALS`: Path to a GCP service account JSON file (for Vertex AI). - `VERTEX_PROJECT_ID`: (Optional) GCP project id for Vertex. If not provided, the SDK will attempt to infer it from credentials. -- `GEMINI_REGION`: (Optional) GCP region for Gemini/Vertex (defaults to `us-east5` or `CLOUD_ML_REGION`). +- `GEMINI_REGION`: (Optional) Vertex region override for Gemini calls (falls back to `CLOUD_ML_REGION`, then `us-east5`). +- `CLOUD_ML_REGION`: (Optional) Default Vertex region for non-Gemini model calls (defaults to `us-east5`). - `CBORG_KEY`: API key for CBORG (when using `access_provider=cborg`). - `CBORG_BASE_URL`: Base URL for the CBORG API. @@ -161,7 +162,7 @@ uv run pytest uv run pytest --cov=src/nmdc_metadata_suggestor_ai_tool # Run specific test file -uv run pytest tests/test_example.py +uv run pytest tests/test_recommendation_pipeline.py ``` ### Code Quality @@ -192,11 +193,19 @@ uv sync ## Configuration -Configuration is managed through environment variables or a `.env` file. See `.env.example` for available options: +Configuration is managed through environment variables or a `.env` file. See `.env.example` for all options. Beyond the LLM access variables above: -- `DEFAULT_MODEL`: Default LLM model to use -- `MAX_TOKENS`: Maximum tokens for LLM responses -- `TEMPERATURE`: Temperature for LLM responses (0.0-1.0) +- `CONTACT_EMAIL`: Contact email sent in User-Agent headers for the Crossref polite pool and OpenAlex (defaults to `support@microbiomedata.org`). + +Advanced ingestion tuning (all optional; defaults shown): + +- `NMDC_EDI_MAX_XML_CHARS`: Max characters read from an untrusted EDI metadata XML payload (default `2000000`). +- `NMDC_DATAONE_SOLR_MAX_XML_CHARS`: Max characters read from an untrusted DataONE Solr XML payload (default `2000000`). +- `NMDC_HTTP_RETRY_ATTEMPTS`: Retry attempts for publication/DOI HTTP requests (default `3`). +- `NMDC_HTTP_RETRY_BACKOFF_SECONDS`: Base backoff in seconds between retries (default `0`). +- `NMDC_HTTP_MAX_RETRY_DELAY_SECONDS`: Cap in seconds on retry delay (default `30`). +- `NMDC_HTTP_POOL_CONNECTIONS`: HTTP connection pool size (default `20`). +- `NMDC_HTTP_POOL_MAXSIZE`: HTTP connection pool max size (default `100`). ## Docker Development Workflow From 601c8c36281672bcb07b35e9ed67d48f14eab552 Mon Sep 17 00:00:00 2001 From: Mark Andrew Miller <25185273+turbomam@users.noreply.github.com> Date: Tue, 7 Jul 2026 14:37:26 -0400 Subject: [PATCH 2/5] docs: fix .env.example region placeholder and misleading comment CLOUD_ML_REGION and GEMINI_REGION shipped as the literal placeholder 'region'. GEMINI_REGION is read first and only falls back to CLOUD_ML_REGION when unset, so a copied .env with GEMINI_REGION=region sends 'region' to Vertex and gets a confusing 404. Ship the real default (us-east5) and correct the comment (it takes precedence when set, it does not default to CLOUD_ML_REGION unless unset). Co-Authored-By: Claude Opus 4.8 (1M context) --- .env.example | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/.env.example b/.env.example index 67db73d..f9a9b3a 100644 --- a/.env.example +++ b/.env.example @@ -12,10 +12,11 @@ CBORG_BASE_URL=base_url GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json # GCP project ID used by Vertex AI VERTEX_PROJECT_ID=your_gcp_project_id -# Default Vertex region for non-Gemini model calls -CLOUD_ML_REGION=region -# Optional Gemini region override (defaults to CLOUD_ML_REGION) -GEMINI_REGION=region +# Default Vertex region (the code default is us-east5 if unset) +CLOUD_ML_REGION=us-east5 +# Optional override of the region for Gemini calls. If set to any value it takes +# precedence over CLOUD_ML_REGION, so do not leave a placeholder here. +GEMINI_REGION=us-east5 # DOI / Publication APIs # Contact email used in User-Agent headers for Crossref polite pool and OpenAlex. From 667f022e463b4077b26e957b0491b3ff0c3ae73a Mon Sep 17 00:00:00 2001 From: Olivia Hess Date: Fri, 10 Jul 2026 11:06:40 -0400 Subject: [PATCH 3/5] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f815167..7cbbf58 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ Environment variables used by `LLMClient` and `ConversationManager`: - `GOOGLE_APPLICATION_CREDENTIALS`: Path to a GCP service account JSON file (for Vertex AI). - `VERTEX_PROJECT_ID`: (Optional) GCP project id for Vertex. If not provided, the SDK will attempt to infer it from credentials. - `GEMINI_REGION`: (Optional) Vertex region override for Gemini calls (falls back to `CLOUD_ML_REGION`, then `us-east5`). -- `CLOUD_ML_REGION`: (Optional) Default Vertex region for non-Gemini model calls (defaults to `us-east5`). +- `CLOUD_ML_REGION`: (Optional) Default Vertex region for `access_provider=gcp` (used as the fallback when `GEMINI_REGION` is unset; defaults to `us-east5`). - `CBORG_KEY`: API key for CBORG (when using `access_provider=cborg`). - `CBORG_BASE_URL`: Base URL for the CBORG API. From 4a341578f283f912c62b5dd2703cddfa897b9324 Mon Sep 17 00:00:00 2001 From: "olivia.hess" Date: Fri, 10 Jul 2026 11:10:49 -0400 Subject: [PATCH 4/5] remove docker documentation --- README.md | 58 +------------------------------------------------------ 1 file changed, 1 insertion(+), 57 deletions(-) diff --git a/README.md b/README.md index 7cbbf58..e2c9ab7 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ Environment variables are loaded from a `.env` file in the project root via set in your shell take precedence over `.env` values (`override=False` is the default). -### Option 1: Using uv (Local Development) +### Using uv (Local Development) 1. **Install uv** (if not already installed): ```bash @@ -94,31 +94,6 @@ response = conversation.generate(model="gemini-2.5-flash", max_tokens=1024, gemi print(response) ``` -### Option 2: Using Docker - -1. **Clone the repository**: - ```bash - git clone https://github.com/microbiomedata/nmdc-metadata-suggestor-ai-tool.git - cd nmdc-metadata-suggestor-ai-tool - ``` - -2. **Configure environment**: - ```bash - cp .env.example .env - # Edit .env and add your API keys - ``` - -3. **Run with Docker Compose** (development): - ```bash - docker-compose up - ``` - -4. **Or build and run production image**: - ```bash - docker build -t nmdc-suggestor . - docker run --env-file .env nmdc-suggestor - ``` - ## Development ### Project Structure @@ -207,37 +182,6 @@ Advanced ingestion tuning (all optional; defaults shown): - `NMDC_HTTP_POOL_CONNECTIONS`: HTTP connection pool size (default `20`). - `NMDC_HTTP_POOL_MAXSIZE`: HTTP connection pool max size (default `100`). -## Docker Development Workflow - -### Interactive Development - -For interactive development with hot-reload: - -```bash -# Start container in background -docker-compose up -d - -# Execute commands in the container -docker-compose exec app uv run pytest -docker-compose exec app uv run ruff format - -# Access shell -docker-compose exec app bash - -# Stop container -docker-compose down -``` - -### Production Build - -```bash -# Build production image -docker build -t nmdc-suggestor:latest . - -# Run production container -docker run --env-file .env nmdc-suggestor:latest -``` - ## Contributing 1. Fork the repository From 0f86748317c16b3f673545414426ecc4ceb2ac26 Mon Sep 17 00:00:00 2001 From: "olivia.hess" Date: Fri, 10 Jul 2026 11:12:21 -0400 Subject: [PATCH 5/5] remove project structure graph - hard to maintain --- README.md | 31 ------------------------------- 1 file changed, 31 deletions(-) diff --git a/README.md b/README.md index e2c9ab7..aee786d 100644 --- a/README.md +++ b/README.md @@ -96,37 +96,6 @@ print(response) ## Development -### Project Structure - -``` -nmdc-metadata-suggestor-ai-tool/ -├── src/ -│ └── nmdc_metadata_suggestor_ai_tool/ -│ ├── __init__.py -│ ├── recommendation_pipeline.py # Pipeline orchestration -│ ├── llm_client.py # LLM client for AI interactions -│ ├── cli/ -│ │ ├── __init__.py -│ │ └── doi_cli.py # DOI operations CLI -│ ├── models/ -│ │ ├── __init__.py -│ │ ├── doi.py # DOI data models -│ │ └── llm_output.py # LLM output model -│ └── publication_ingestion/ -│ ├── __init__.py -│ ├── download_pdf.py # PDF retrieval logic -│ └── retreive_pdf_link.py # PDF link discovery -├── tests/ # Test files -├── scripts/ # Vertex AI test scripts -├── docs/ # Documentation -├── pyproject.toml # Project dependencies and metadata -├── Dockerfile # Production Docker image -├── Dockerfile.dev # Development Docker image -├── docker-compose.yml # Docker Compose configuration -├── .env.example # Example environment variables -└── README.md # This file -``` - ### Running Tests ```bash