Force upgrade latest apex

Author: Zikkying

apex/skills/apex-flow/SKILL.md

@@ -107,6 +107,7 @@ Options to offer via AskQuestion:
   - If it exists, use `create` for a new job or `refresh-global` for an existing job; both convert the key to a fresh ticket and write it to `global.json`.
   - Verify that `global.json` contains a non-empty `bohrium_config.ticket` before submission.
   - `run.sh` must only install/verify APEX and call `apex submit`. Do not add ticket API calls or depend on `BOHRIUM_ACCESS_KEY` inside the APEX container.
+  - Install with `python3 -m pip install --upgrade --no-cache-dir apex-flow`.
    See `reference/submission.md`.
 7. **Project ID from environment only.** `generate_config.py` reads `BOHRIUM_PROJECT_ID` (or `--project-id`). Never hardcode a project ID (including old examples like `13529`) into `global.json`, docs, or prompts.
 8. **Hard-validate inside the task directory before every upload.** Run:

apex/skills/apex-flow/reference/submission.md

@@ -17,7 +17,7 @@ APEX uses a **two-layer submission architecture**:
                            ▼
 ┌─────────────────────────────────────────────────────────────────┐
 │  Outer Bohrium Job (APEX image, c1_m2_cpu)                     │
-│  1. pip install apex-flow (latest, no version pin)             │
+│  1. pip install --upgrade --no-cache-dir apex-flow             │
 │  2. apex submit param.json -c global.json -n "<wf-name>"      │
 │  → Connects to workflows.deepmodeling.com                      │
 │  → Waits for completion and retrieves results automatically    │
@@ -69,14 +69,11 @@ the job directory is packaged:
 
 ## Outer Job run.sh Template
 
-The outer Bohrium job always uses this pattern. **Do NOT pin apex-flow version** — this ensures the latest code (with bug fixes) is used and propagated to inner steps via `upload_packages`.
-
 ```bash
 #!/bin/bash
 set -eo pipefail
 
-# Always install latest apex-flow (DO NOT pin version)
-pip install apex-flow 2>&1 | tail -3
+python3 -m pip install --upgrade --no-cache-dir apex-flow 2>&1 | tail -5
 python3 -c "import apex; print(f'APEX version: {apex.__version__}')"
 
 # Authentication is already stored in global.json by generate_config.py.
@@ -98,7 +95,7 @@ fi
 ```
 
 **Key points:**
-- `pip install apex-flow` (no `==` pin) → always gets latest from PyPI
+- Must use `pip install --upgrade --no-cache-dir apex-flow` (no `==` pin).
 - The installed APEX code is automatically sent to inner dflow steps via `upload_packages`
 - `run.sh` does not access `BOHRIUM_ACCESS_KEY` or modify `global.json`
 - **单次执行，不重试** — 失败即退出，由用户决定是否重新提交

apex/skills/apex-flow/scripts/generate_config.py

@@ -860,14 +860,54 @@ def main():
         json.dump(param_config, f, indent=4)
     print(f"Written: {param_path}")
 
-    # Write a submission helper script
-    submit_script = output_dir / "submit.sh"
+    # Write outer-job run.sh (force-upgrade APEX; bare pip install keeps the
+    # image-bundled older version and uploads that stale code to inner steps).
+    run_script = output_dir / "run.sh"
     cmd = f'apex submit param.json -c global.json -f {args.flow_type} -n "{workflow_name}"'
+    with open(run_script, "w") as f:
+        f.write("#!/bin/bash\n")
+        f.write("set -eo pipefail\n\n")
+        f.write("# Upgrade to latest apex-flow.\n")
+        f.write(
+            "python3 -m pip install --upgrade --no-cache-dir apex-flow "
+            "2>&1 | tail -5\n"
+        )
+        f.write(
+            'python3 -c "import apex; '
+            'print(f\'APEX version: {apex.__version__}\')"\n\n'
+        )
+        f.write(
+            "# Authentication is already stored in global.json by "
+            "generate_config.py.\n"
+        )
+        f.write("set +eo pipefail\n")
+        f.write(f"{cmd} 2>&1 | tee apex_submit.log\n")
+        f.write("EXIT_CODE=${PIPESTATUS[0]}\n")
+        f.write("set -eo pipefail\n\n")
+        f.write("if [ $EXIT_CODE -eq 0 ]; then\n")
+        f.write(
+            '    echo "=== APEX workflow completed and results retrieved ==="\n'
+        )
+        f.write('    echo "Retain apex_submit.log and the workflow ID."\n')
+        f.write("    exit 0\n")
+        f.write("else\n")
+        f.write('    echo "APEX failed (exit $EXIT_CODE)"\n')
+        f.write("    tail -50 apex_submit.log 2>/dev/null || true\n")
+        f.write("    exit 1\n")
+        f.write("fi\n")
+    os.chmod(run_script, 0o755)
+    print(f"Written: {run_script}")
+
+    # Keep a minimal submit.sh for local/debug use inside an already-upgraded env.
+    submit_script = output_dir / "submit.sh"
     with open(submit_script, "w") as f:
         f.write("#!/bin/bash\n")
-        f.write(f"# APEX submission command (run inside APEX container)\n")
+        f.write("# APEX submission command (run inside APEX container)\n")
         f.write(f"# Workflow name: {workflow_name}\n")
-        f.write(f"# Flow type: {args.flow_type}\n\n")
+        f.write(f"# Flow type: {args.flow_type}\n")
+        f.write(
+            "# Prefer run.sh for Bohrium outer jobs (it upgrades apex-flow).\n\n"
+        )
         f.write(f"{cmd}\n")
     os.chmod(submit_script, 0o755)
     print(f"Written: {submit_script}")