v1.0.0b64

Fixed

• _process_sort now emits idx->'{key}' (JSONB operator) instead of idx->>'{key}' (text operator) for the FieldIndex fallback. Text-cast sorting compared everything lexicographically, so a FieldIndex over a numeric attribute ranked "10" before "2". JSONB comparison is type-aware: numbers sort numerically, strings lexically, so a homogeneous FieldIndex now always sorts correctly regardless of value type. Affects any FieldIndex with numeric source data (counters, priorities, prices, weights).

See #158, PR #159.

v1.0.0b63

Changed

• PGCatalogBrain is now catalog-independent. _catalog dropped from __slots__; getURL uses zope.globalrequest.getRequest() only; getObject / _unrestrictedGetObject resolve the traversal root lazily via a new _traversal_root() helper (getSite().getPhysicalRoot() first, getRequest().PARENTS[-1] as fallback). The catalog reference that ZODB prefetch needs has moved onto the transient CatalogSearchResults container. This keeps brains cacheable / pickleable / re-queueable without dragging the Acquisition chain along.

• The catalog= keyword on PGCatalogBrain.__init__ is accepted but ignored — kept until the next major version for signature compatibility.

See #156.

v1.0.0b62

Fixed

• _process_index now falls back to the correct handler for built-in
  Plone indexes when they are missing from the IndexRegistry —
  previously the miss fell through to _handle_field which emits
  idx->>'name', bypassing the dedicated typed columns and their
  indexes. On aaf-6 prod this produced 4-9 second seq-scans over a
  450k-row object_state table for every folder-listing query,
  saturating the worker pool and causing intermittent Varnish
  backend-fetch errors. Resolution happens via a new
  _builtin_index_type(name) helper that combines two sources: the
  three Plone-native specials (path, effectiveRange,
  SearchableText) hardcoded because their SQL lives inside the
  handler, and every TEXT[]-typed ExtraIdxColumn (derived at
  dispatch time — currently allowedRolesAndUsers and
  object_provides, extensible via
  register_extra_idx_column). The correct handler then uses
  path / allowed_roles / object_provides /
  searchable_text columns and the DateRangeIndex composite
  clause. Explicit registry entries still win so addons can override
  behavior. Closes #154.

• _handle_keyword no longer crashes with TypeError: 'DateTime' object is not iterable when a caller passes a non-str, non-iterable
  value (e.g. a Zope DateTime or a Python datetime) as the
  query for a KeywordIndex. The old coercion assumed the value was
  either a str or iterable; anything else (DateTime, int,
  …) hit list(value) and raised. Values are now coerced via
  str() into a single-element list, matching the JSONB storage
  shape (keyword arrays always contain strings). Closes #152.

v1.0.0b61

Fixed

• _field_range on FieldIndex no longer silently returns zero rows
  for numeric range queries. Two stacked bugs: [max, min] order
  from the caller was not normalized (produced always-false SQL), and
  idx->>'key' comparisons ran lexicographically on text — so
  '46.1' <= '5.0' <= '49.0' included values outside the range.
  _field_range now sorts min/max and casts idx->>'key' to
  ::numeric when the range values are int / float. String
  values (ISO dates etc.) keep text comparison. Affected aaf-6 prod
  map-widget bbox filter via collective.collectionfilter — closes
  #150.

• _CatalogCompat.getIndex and _CatalogIndexesView.__getitem__ no
  longer silently fall back to the raw ZCatalog index when they
  cannot find the catalog tool — that fallback returned empty BTrees
  and masked #143 / #146 for weeks. A new private helper
  _resolve_catalog tries three paths in order (__parent__ →
  Acquisition chain → zope.component.hooks.getSite().portal_catalog)
  and raises RuntimeError if all three fail.

• _CatalogCompat.indexes property now self-heals a missing
  __parent__ on first access via getSite().portal_catalog.
  First page render after deploy persists __parent__; no second
  upgrade-step click required on sites where the #139 upgrade ran
  before that fix landed. Zero-touch prod recovery.

Added

• PGIndex._apply_index(request, resultset=None) — ZCatalog-
  compatible low-level query entry point. Returns
  (IITreeSet(zoids), (index_name,)). Reuses
  _QueryBuilder._process_index so every registered IndexType
  (FIELD, KEYWORD, PATH, DATE, DATE_RANGE, UUID, TEXT, BOOLEAN, GOPIP)
  plus every IPGIndexTranslator utility works for free. No
  implicit security filtering — matches ZCatalog semantics; use
  catalog(**query) for secured results. Emits a
  DeprecationWarning once per caller site.

• _PGIndexMapping.__getitem__ / __len__ — round out the
  PG-backed mapping so Plone core callers
  (plone.app.uuid.utils, plone.app.vocabularies.Keywords)
  work against catalog._catalog.getIndex(name)._index without
  needing catalog.Indexes[name] acquisition.

• _PGIndexMapping.items() / values() raise
  NotImplementedError with guidance pointing at uniqueValues,
  _apply_index, and catalog(**query) as alternatives. No
  Plone-core caller uses them on a wrapped index; a concrete usecase
  can land in a future issue with server-side-cursor streaming.

• PGIndex._index property emits a DeprecationWarning on
  access — signals callers that the BTree-shaped API is an
  emulation and suggests the preferred pgcatalog-native
  alternatives.

Closes #146.

v1.0.0b60

Fixed

• _PGIndexMapping (backing PGIndex._index) is now iterable and
  branches its SQL on the index type. plone.app.vocabularies.Keywords
  iterates index._index directly to populate the tag-autocomplete
  widget in the Plone edit form — the previous mapping had no
  __iter__ and its keys() coerced JSONB arrays to their text
  representation. Net effect on b59: typing into the Schlagwort
  field offered zero suggestions even when matching keywords existed.
  keys() / __iter__ now use the same UNION ALL expansion as
  uniqueValues for KEYWORD, and get() uses idx->key @> to_jsonb(value::text) so membership checks against a keyword
  actually match. Follow-up to #143.

v1.0.0b59

Fixed

• PGIndex.uniqueValues() now branches on the wrapped index type.
  For IndexType.KEYWORD the JSONB value is a list of tags, so the
  SQL uses jsonb_array_elements_text to expand it into individual
  entries. Previously all index types went through idx->>key,
  which coerces a JSONB array to its JSON text representation —
  producing entries like '["Werkvortrag", "Tirol"]' instead of
  'Werkvortrag' / 'Tirol'. Callers (the querystring
  composer vocabulary, plone.app.vocabularies.Keywords,
  collective.collectionfilter tag clouds, etc.) now see the
  distinct set of elements as expected.

  A defensive UNION ALL branch treats a scalar row under the
  same keyword key (corrupt/legacy data) as a single-value keyword
  so the query does not raise cannot extract elements from a scalar. _maybe_wrap_index passes the registered
  IndexType through to PGIndex so callers that build the
  wrapper directly keep getting the (correct) scalar path by
  default.

  Closes #143.

v1.0.0b58

Fixed

• Register SanitizeRowsModifier as an IQueryModifier utility so
  that malformed Collection-query rows (missing or non-string i
  field) are dropped before plone.app.querystring.parseFormquery
  processes them. Upstream queryparser.py:73 otherwise sets
  query[None] = ... and the subsequent catalog(**parsedquery)
  in querybuilder._makequery fails at the Python level with
  TypeError: keywords must be strings — which the Collection edit
  widget cannot recover from. With the sanitizer in place the
  preview renders again, so editors can open their Collections and
  repair corrupted Subject / tag data through the UI.

  This is a defensive workaround for an upstream gotcha, not a fix for
  the underlying plone.app.querystring behavior. Closes #142.

v1.0.0b57

Fixed

• PGCatalogBrain.getObject() now mirrors upstream
  Products.ZCatalog.CatalogBrains.AbstractCatalogBrain.getObject:
  the parent path is traversed unrestricted and only the final
  object is restrictedTraverse-checked. Previously the full
  path went through restrictedTraverse, so any intermediate
  container with stricter permissions than the leaf raised
  AccessControl.unauthorized.Unauthorized — even though the
  catalog filter had already authorized access to the target.
  Sites with a private parent folder publishing individual public
  items (the common "kalender/event-xyz" pattern) hit this on every
  anonymous render. Closes #141.

v1.0.0b56

Fixed

• The v1->v2 upgrade step was silently no-op on production sites. GenericSetup
  invokes upgrade handlers with the portal_setup tool as the context, but
  _resolve_compat only understood the ImportContext shape (getSite()),
  so the setup-tool call path hit the return None, None branch and logged
  migrate_catalog_indexes: no _CatalogCompat found; skipping — while
  GenericSetup happily bumped the profile version to 2. Net effect: the
  persisted _CatalogCompat kept its legacy indexes attribute and the
  new indexes property then raised AttributeError for _raw_indexes,
  which Acquisition swallowed and replaced with the tool's indexes()
  method — surfacing as 'function' object has no attribute 'keys' from
  catalog.indexes.keys() and 'method' object is not subscriptable from
  catalog.indexes[name].

  _resolve_compat now also walks aq_parent(context) to reach the Plone
  site when the context is the setup tool, so the normal ZMI
  manage_upgrades path migrates the state as intended.

• Made _CatalogCompat.indexes self-healing: if the legacy indexes
  attribute is still in __dict__ (unmigrated or fresh-install site that
  skipped v1), the property moves it to _raw_indexes on first access and
  marks the instance dirty. This avoids the Acquisition-swallowed
  AttributeError failure mode even when the upgrade step never ran.

  Closes #139.

v1.0.0b55

Fixed

• Wrap portal_catalog._catalog.indexes[name] access (#137, PR #138). Plone and addon code commonly reaches into the catalog via the non-API-conform pattern cat._catalog.indexes[name] / .get(name) / .items(). Previously this returned raw ZCatalog index objects with empty BTrees, so queries silently returned no results. _CatalogCompat.indexes is now a property returning a transient view that wraps each index with PGIndex (same behavior as catalog.Indexes[name]). Custom PATH-type indexes and other special indexes (idx_key=None) continue to be returned raw.

  Based on prior prototyping by @thet on the thet/indexes-wrapper branch.

  Migration: GenericSetup profile bumped from v1 to v2. The upgrade step renames the persisted indexes attribute to _raw_indexes and sets __parent__ on the compat so aq_parent can reach the catalog tool. Run Plone Site Setup → Add-ons → plone.pgcatalog → Upgrade on existing sites, or let the next runAllImportSteps on the default profile pick it up.

  Likely-affected callers include plone.base.utils.check_id (reserved-name check), plone.restapi.search.query.Query.get_index, plone.app.discussion, plone.app.referenceablebehavior, plone.volto, collective.collectionfilter, and collective.exportimport — per-package verification is recommended after upgrade.

Full changelog: v1.0.0b54...v1.0.0b55