Skip to content

@@contenttree-fetch slows down fetching indexes on brains for objects with large catalog indexes #32

Open
@fredvd

Description

@fredvd

We ran into this issue on a site with a folder with 260 items that had a lot of words indexed in the SearchableText catalog. (Custom Dexterity contenttype with a file field containing a PDF that's indexed).

The related items behavior in plone.app.relationfield is using @@contenttree-fetch as well, default setup on Plone 4 with plone.app.contenttypes and trying to browse into the folder mentionned above takes between 30 and 40 seconds to render the listing.

Trying to profile and looking at a thread dump while the request is in progress there was always access to the method isBrainSelectable on PathSource at https://github.com/plone/plone.formwidget.contenttree/blob/master/plone/formwidget/contenttree/source.py#L161-L165 .

   def isBrainSelectable(self, brain):
        if brain is None:
            return False
        index_data = self.catalog.getIndexDataForRID(brain.getRID())
        return self.selectable_filter(brain, index_data)

These last two lines pull in a lot of index_data from the catalog, which takes a l a lot of time, especially when each content item has a lot of words.

Pull request #31 fixes the case when there are no criteria at all in CustomFilter to operate on, which makes fetching the index_data unnecessary in the first place. This fixes the related items behavior in plone.app.relationfield which doesn't use any filtering on the content type.

Plone 5 uses a different querying mechanism for plone.app.widgets related items, so it's not much use fixing this, just documented the behavior if someone else still runs into this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions