Filter on whether a path is an event in the dashboard.

[GeoWill]

Thanks for this project, it's very nice to use 🙂 .

I recently added two events to a page. At a high level it is to let us check whether we had any information to show the user.

Actual code

    <script>
        const postElectionsCount = JSON.parse(document.getElementById('postElectionsCount').textContent);

        function trackElectionsShownEvent(postElectionsCount) {
            const hasElections = postElectionsCount > 0;
            const eventData = hasElections ? {
                path: 'POSTCODE_SEARCH_ELECTION_SHOWN',
                title: 'Postcode search user had an election',
                event: true,
            } : {
                path: 'POSTCODE_SEARCH_NO_ELECTION_SHOWN',
                title: 'Postcode search user had no election',
                event: true,
            };

            window.goatcounter.count(eventData);
        }

        document.addEventListener('DOMContentLoaded', function() {
            trackElectionsShownEvent(postElectionsCount);
        });
    </script>

These events are fired on pages with different paths. So, if I understand correctly, a visit to a given page will fire off two counts to goat counter, one where the path corresponds to the page url, and another for the event triggered by the js. In the dashboard I can see counts for the pages visited and the event path. I think this means on the dashboard our totals now include a double count in terms of actual visitors.

Is there a way for us to filter out events for when we're just looking at figures on the dashboard? I didn't see anything in the UI. Ideally I want to be able to easily check the number of visitors to a page, or set of pages, without including the events that they also triggered.

[arp242]

I added support for is:event and is:pageview in the filter. This works as you'd expect: is:event foo shows all events containing foo. Intentionally not documented for now until I completely figure out more advanced search syntax and build some reasonably nice UI or it (that's #194). But this is pretty easy to add so why not.

The filter gets saved with the "default view", so you can save this as the default if you want.

I think this should solve your use case?

[GeoWill]

Thanks! This is exactly the sort of thing I was after.

I am a bit confused by what I'm seeing, but this might just be me 'holding it wrong'.

The totals for is:pageview and is:event are much less (combined) than the total with no filter applied. My mental model is that they should add up to whatever the total was. I guess this might be how we've configured things, so I will dig into this more myself. My hunch is that the filtered totals look more like 'human traffic' and the totals without filters are counting something extra. I thought I'd post it here, in case it was something you had an idea about, but no pressure to investigate.

Screenshots:

• No filter (36714 visits)

• Pageviews only (2199 visits)

• Events 197 visits

[arp242]

Yes, the numbers should add up exactly; all it does is add where event = [true/false] in the SQL query (it's all super simple; see: 906f3a0). Not sure why it doesn't for you. Can you try doing a hard refresh just to be sure it's not some frontend issue?

[GeoWill]

I've checked in another browser, and done hard refresh and that doesn't change the issue. The issue is only on one of the three sites we have goatcounter on. However only one of them is using events, whereas the other two just track pageviews (at the moment).

[arp242]

It gets the list of path IDs to filter in one query, and then passes the result of that to further queries to fetch the list of pages, browser overview, etc. This avoids having to run the same search 6+ times.

The query it runs is:

select path_id from paths
where
	site_id = :site
	{{:only_event    and event=1}}
	{{:only_pageview and event=0}}
	and (
		lower(path) like lower(:filter)
		{{:match_title or lower(title) like lower(:filter)}}
	)
-- The limit is here because that's the limit in SQL parameters; the returned
-- []int64 is passed as parameters later on.
--
-- Having (and scrolling!) more than 65k pages is a rather curious usage
-- pattern, but it has happened. This was just because they were sending data
-- with many unique IDs in the URL which really ought to be removed.
limit 65500

That limit 65500 is probably your problem, as your site has 1,545,556 unique paths (more than 65k). The events one is correct as you only have 4 events. This only gets run if you fill in anything in the filter, so the normal (non-filtered) view is correct if you have >65k pages, but any filter is not.

I think that number can be removed at least for PostgreSQL as it uses arrays now. Need to verify and think about SQLite.

[GeoWill]

Right. That makes sense. I guess our site might be a bit of an outlier in that regard 😄. Understanding usage across the different urls is useful for us though.

[arp242]

I think that number can be removed at least for PostgreSQL as it uses arrays now.

So turns out PostgreSQL doesn't really like it if you use an array of 1.5M items; that will add about 30 seconds in query planning time alone while the CPU is at 100%, times 6 or 7 queries. I spent some time looking at where PostgreSQL is spending all of this time, and almost all of it is in the array deconstruction. So meh – that won't really work.

Performance when using a join seems fine in some quick tests; need to do some testing to see what the best option is at different scales; maybe always using a join is fine? Or maybe need to switch if the number of pages gets too large.