Replace Pydantic with custom validation logic, restore v0.1.5 config and registry algorithm (#73)

* Replace Pydantic with lightweight custom validation

Remove pydantic and pydantic-core dependencies entirely. Add
confection/_validation.py with Schema base class, dynamic schema
creation, and type validation supporting unions, generics, forward
references, Literal, generators, and constrained types (StrictBool,
PositiveInt, StrictFloat).

Key changes:
- New _validation.py module replaces all Pydantic functionality
- _registry.py uses custom Schema/create_schema/Field/ValidationError
- util.py Generator class uses metaclass for isinstance checks
- Removed test_pydantic_generators.py (Pydantic-specific tests)
- Updated all test imports from pydantic to confection._validation
- Removed pydantic from setup.cfg install_requires

* Add ensure_schema shim for pydantic BaseModel backward compatibility

Downstream libraries (spaCy, thinc) pass pydantic BaseModel subclasses
to registry.resolve(schema=...) and registry.fill(schema=...). The shim
transparently converts these to our Schema interface:

- Extracts fields from pydantic v1 (__fields__) or v2 (model_fields)
- Extracts config from v1 (inner class Config) or v2 (model_config dict)
- Handles v1 Extra enum for extra='forbid'/'allow'
- Delegates model_validate to the original pydantic class so that
  pydantic validators, strict types, and constraints keep working
- Recursively converts nested BaseModel field annotations
- Caches conversions for performance

Also fixes validate_type to accept constrained subtypes (e.g. pydantic's
StrictInt which inherits from int) by checking issubclass(annotation,
type(value)) as a fallback.

* Tighten exception handling in _validation.py

- resolve_type_hints: catch (NameError, AttributeError, TypeError)
  instead of bare Exception
- _is_pydantic_model: catch (ImportError, ModuleNotFoundError) on
  import, use else clause for issubclass so import errors and logic
  errors are never conflated
- _pydantic_model_validate: resolve the concrete pydantic
  ValidationError class(es) up front via _get_pydantic_validation_error()
  so the except clause catches only pydantic validation errors, not
  arbitrary exceptions

* Port validation module onto stock v0.1.5 registry architecture

Start from confection v0.1.5 (last pydantic v1 release) and replace
pydantic with our validation module, keeping the registry code (_fill,
_make, etc.) intact:

- Replace pydantic imports with validation.py (Schema, Field, etc.)
- Replace srsly with stdlib json
- BaseModel -> Schema, __fields__ -> model_fields, parse_obj -> model_validate
- copy_model_field -> _override_field_to_any
- create_model -> create_schema
- Add ensure_schema at resolve/fill/_fill entry points
- Drop pydantic and srsly from install_requires

102/108 stock tests passing. Remaining 5 failures need test updates
for __fields__ -> model_fields and pydantic-specific assertions.

* Fix remaining test failures

- test_make_promise_schema: __fields__ -> model_fields
- test_resolve_schema_coerced: expect raw values (no pydantic coercion)
- test_validation_custom_types: fix pydantic v1 validator calling —
  skip 3-arg validators needing config objects, use field shim for 2-arg
- test_config_reserved_aliases: reorder validator hooks before
  constrained-subtype fallback so StrictBool/PositiveInt validators run
- test_config_fill_without_resolve: reset model_validate to Schema's
  own after _override_field_to_any mutates fields
- test_frozen_struct_deepcopy: unfreeze frozen dict/list defaults so
  _update_from_parsed can write to them

111/111 stock v0.1.5 tests passing.

* Fix pydantic-delegating validation for promise-containing fields

- _override_field_to_any: create a new Schema subclass instead of
  mutating the cached wrapper, so ensure_schema cache isn't corrupted
- Add _contains_promise: detect nested promises in dict values
- Override fields to Any when value contains nested promises and
  resolve=False (matches stock confection's behavior)
- Fix pydantic v1 validator calling: skip 3-arg validators needing
  config objects, use field shim for 2-arg constraint validators
- Reorder validator hooks before constrained-subtype fallback

111/111 confection tests, 21/21 spaCy serialize_config tests passing.

* Drop pydantic for custom validation logic

* Port test_pydantic_shim and test_config_values from main branch

- test_pydantic_shim: 16/16 pass — tests ensure_schema backward compat
- test_config_values: 89/102 pass — Hypothesis property tests for config
  parsing. 13 failures are json serialization edge cases from srsly→json
  migration (whitespace handling, negative numeric strings), not pydantic
  related.

Total: 202 passed across all test files.

* Port double-parsing fix from main and remove isdigit hack

Cherry-pick the config parsing fixes from dbc8484 and a4e2d58:

- Simplify before_read: just warn about single quotes, no unquoting
  (the unquoting caused double-parsing of string values like '0')
- Add _coerce_for_string_context: coerce values in compound string
  interpolation (bare refs keep JSON type, compounds join as strings)
- Rewrite before_get: detect bare refs (len(L)==1) vs compound
- Fix _get_section_ref: guard against value == SECTION_PREFIX,
  use replace(prefix, '', 1) to avoid stripping too much
- Remove isdigit() hack in try_dump_json (no longer needed without
  the unquoting in before_read)
- Remove JSON_EXCEPTIONS constant (no longer used)

215/215 tests passing, 0 failures.

* Fix requirements.txt: drop srsly/mypy, keep pydantic as test-only dep

* Format

* Migrate tooling from black/isort/flake8/mypy to ruff

- Replace black, isort, flake8 with ruff in CI, requirements, config
- Remove mypy config from setup.cfg
- Add ruff config to pyproject.toml
- Fix lint issues: unused imports, unused vars, trailing whitespace,
  type comparison (== -> is), import sorting
- CI: single 'ruff check' + 'ruff format --check' step

* Add pyright to CI lint step

* Skip pydantic v1 tests on Python 3.14+

pydantic v1 compat layer is broken on Python 3.14. Skip test_config.py
and test_pydantic_shim.py entirely on 3.14+. Fall back to our own
StrictBool in tests/util.py when pydantic is unavailable.

* Format

* Validate TypeVar bounds and constraints

When validate_type encounters a TypeVar with a bound, validate against
the bound. For constrained TypeVars, try each constraint. This catches
type mismatches like passing ndarray where SeqT (bound to
Union[Ragged, Padded, List[...]]) is expected.

* Fix pyright errors in source files

- FieldInfo.annotation: type as Any instead of None
- alias_gen: add callable() check for type narrowing
- sys.modules.get: guard against None module name
- NewType __supertype__: suppress reportFunctionMemberAccess
- Extra.value: suppress reportAttributeAccessIssue (guarded by hasattr)
- pydantic ValidationError.errors(): suppress (typed as Exception)
- SimpleFrozenDict.update: suppress override warning

7 remaining errors are pre-existing ConfigParser internals from stock.

* Move pydantic compat imports to module scope

Replace in-function try/except pydantic imports with module-level
imports that set _PydanticV1BaseModel, _PydanticV2BaseModel, etc.
to None when pydantic is not installed. Cleaner and avoids repeated
import attempts on every call.

* Configure pyright to exclude tests, use pyproject.toml settings

3 remaining errors are pre-existing stock ConfigParser internals.

* Fix type errors

* Format

* Pyright

* isort

* Fix ruff I001: format import block in confection/__init__.py

* Add comprehensive tests for validate_type

44 tests covering all type validation branches:
- None, Annotated, Union, Optional, pipe syntax
- Literal, NewType, TypeVar (unbound, bound, constrained)
- Plain types: bool, int, float, str, Path
- Generic types: Callable, List, Dict, Tuple (bare, fixed, variable),
  Set, FrozenSet, Sequence, Iterable, Mapping, Iterator, Type[X]
- Schema-as-dict validation
- Pydantic hooks (__get_pydantic_core_schema__, __get_validators__)
- Generator passthrough
- _validate_schema: extra forbid/allow, missing required, aliases
- ensure_schema passthrough
- model_dump, Schema.from_function

validation.py coverage: 66% -> 87%

* Format

* Fix pyright ignore comments to survive ruff formatting

* Fix pyright ignore on all pydantic import lines

Ruff splits combined imports into separate statements. Each from line
needs its own pyright: ignore[reportMissingImports] comment since
pydantic is not installed during CI lint.

Author: honnibal

.github/workflows/tests.yml

@@ -5,39 +5,26 @@ on:
     branches: [main]
   pull_request:
     branches: ["*"]
-  workflow_dispatch: # allows you to trigger manually
+  workflow_dispatch:
 
-# When this workflow is queued, automatically cancel any previous running
-# or pending jobs from the same branch
 concurrency:
   group: tests-${{ github.ref }}
   cancel-in-progress: true
 
 jobs:
-  validate:
-    name: Validate
+  lint:
+    name: Lint
     runs-on: ubuntu-latest
     steps:
-      - name: Check out repo
-        uses: actions/checkout@v6
-
-      - name: Configure Python version
-        uses: actions/setup-python@v6
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
         with:
           python-version: "3.13"
+      - run: pip install ruff pyright
+      - run: ruff check confection
+      - run: ruff format confection --check
+      - run: pyright confection
 
-      - name: black
-        run: |
-          python -m pip install black -c requirements.txt
-          python -m black confection --check
-      - name: isort
-        run: |
-          python -m pip install isort -c requirements.txt
-          python -m isort confection --check
-      - name: flake8
-        run: |
-          python -m pip install flake8 -c requirements.txt
-          python -m flake8 confection --show-source --statistics
   tests:
     name: Test
     strategy:
@@ -81,17 +68,10 @@ jobs:
       - name: Test import
         run: python -c "import confection" -Werror
 
-      - name: Install test requirements with Pydantic v2
+      - name: Install test requirements
         run: |
           python -m pip install -U -r requirements.txt
-          python -m pip install -U pydantic
-
-      - name: Run tests for Pydantic v2
-        run: |
-          python -c "import pydantic; print(pydantic.VERSION)"
-          python -m pytest --pyargs confection -Werror
 
-      - name: Test for import conflicts with hypothesis
+      - name: Run tests
         run: |
-          python -m pip install hypothesis
-          python -c "import confection; import hypothesis"
+          python -m pytest --pyargs confection