ci: separate hermetic unit tests from online E2E tests

[XuanYang-cn]

Problem

The PR workflow runs make unittest, but the target executes only:

pytest tests/test_dataset.py::TestDataSet::test_download_small

This test downloads data from S3 and Aliyun OSS, so it is an online E2E/integration test—not a unit test.

As a result:

• PR CI does not run the pure local unit-test suite.
• CI depends on external network and storage availability.
• New local tests can pass manually but remain unprotected by CI.
• Failures do not clearly distinguish code regressions from external-service failures.

The repository already defines an integration pytest marker, but classification is incomplete. Running:

pytest --collect-only -m "not integration" tests

still collects service-dependent tests and fails on missing optional SDKs such as Chroma, Pinecone, Turbopuffer, and MySQL.

Some files also mix both test types, such as test_milvus.py, test_pgvector.py, and test_dataset.py.

Proposed approach

Separate tests into two explicit categories:

1. Pure local unit tests

   • No network access.
   • No live database or external service.
   • No credentials or large dataset downloads.
   • Deterministic and fast.
   • Run on every pull request.

2. Online E2E tests

   • Require dataset downloads, a running database, cloud credentials, containers, or optional provider SDKs.
   • Run in dedicated jobs or workflows with their prerequisites documented.
   • Can be scheduled, manually triggered, or enabled for selected PRs.

Prefer separate directories so unit-test collection does not import E2E-only dependencies:

tests/
├── unit/
└── e2e/

Add explicit Make targets:

unit-test:
	PYTHONPATH=`pwd` python3 -m pytest tests/unit

e2e-test:
	PYTHONPATH=`pwd` python3 -m pytest tests/e2e -svv

The existing unittest target can temporarily alias unit-test for compatibility.

Acceptance criteria

• make unit-test passes with only pip install -e ".[test]".
• Every PR runs the complete local unit-test suite.
• Unit tests require no network, credentials, external services, or downloaded datasets.
• Online E2E tests have a separate command and CI job/workflow.
• test_download_small is moved to the E2E category.
• Mixed test modules are split or clearly reorganized.
• Optional E2E dependencies and service prerequisites are documented.
• CI job names clearly distinguish unit tests from online E2E tests.

[linhongyu510]

I would like to contribute this in a focused first PR. I propose to: (1) create a hermetic tests/unit suite that avoids importing optional provider SDKs, (2) move test_download_small to tests/e2e, (3) add unit-test and e2e-test Make targets while keeping unittest as a compatibility alias, and (4) update PR CI to run the full local unit suite. I would leave broader provider-by-provider E2E reorganization for follow-up PRs. Would this first phase be acceptable?