Distributed REFRESH INDEX for incremental scalar index maintenance

[ivscheianu]

Design proposal: #788

Incremental maintenance for scalar indexes (ALTER TABLE ... REFRESH INDEX), plus three pre-existing bugs the work surfaced.

Items 1-3 are independent of the design outcome and can land first: they are bugs on main today that affect CREATE INDEX, OPTIMIZE and VACUUM.

• 1. fix: normalize WITH-clause option names at the parser boundary #791 — Normalize WITH clause option names at the parser boundary. Option names are compared case-sensitively, while the parser preserves the spelling as written, so CREATE INDEX ... WITH (TRAIN = false) is silently ignored and builds a trained index. OPTIMIZE and VACUUM have the same gap.
• 2. fix: keep index segment fragment coverage contiguous #792 — Keep index segment coverage contiguous. feat: balance scalar index fragment batches #758 replaced contiguous fragment batching with row-balanced assignment. Lance only compacts fragments covered by an identical set of index segments, so interleaved coverage leaves OPTIMIZE with no two fragments it can group, and an indexed table stops coalescing entirely. Row balance and contiguity are compatible.
• 3. fix: name the index on uncommitted segment builds #793 — Name the index on uncommitted segment builds and set replace. The builds let Lance derive their own name, which is {column}_idx, so its collision pre-check fires against an existing index of that name on the same column. Re-running CREATE INDEX id_idx on column id fails on every task.
• 4. feat: report index coverage, segment count and size in SHOW INDEXES #794 — Index coverage in SHOW INDEXES: indexed_percent, num_segments, size_bytes, derived from metadata the command already fetches.
• 5. fix: make distributed index builds version-consistent and report real coverage #795 — Make distributed index builds version-consistent and report real coverage. Tasks
  each resolved the latest version independently instead of the one the driver planned over, and
  CREATE INDEX reported the fragment count it planned rather than the one the commit achieved.
• 6. ALTER TABLE ... REFRESH INDEX — feat: add distributed REFRESH INDEX for incremental index maintenance #784.

Known gaps recorded in the proposal rather than fixed here: a concurrent DROP INDEX can be undone by an in-flight refresh (needs lance-format/lance#6806), and a partially covered zonemap index prunes uncovered fragments so a predicate on the indexed column can return fewer rows than the table holds (reproducible with CREATE INDEX alone).

Edit: the zonemap gap is fixed by #781.