Decompose topic filters into ORs of ANDs

[Shaptic]

What

Decomposes the existing, flattened NestedTopicArray which is simply a list of topic filters that are OR'd together into a disjunction of conjunctions, meaning each topic filter is kept together as an AND clause and only the total set of filters is ORd together.

You can see the differences in SQL as follows:

Details

-- before 
SELECT id, event_data, transaction_hash, ledger_close_time
FROM events 
WHERE 
    id >= ? AND 
    id < ? AND
    contract_id IN (?,?,?,?,?) AND (
        topic1 IN (?,?,?,?,?) OR 
        topic2 IN (?,?,?,?,?)
    )
ORDER BY id ASC

-- after
SELECT id, event_data, transaction_hash, ledger_close_time
FROM events 
WHERE
    id >= ? AND 
    id < ? AND 
    contract_id IN (?,?,?,?,?) AND (
        (topic1 = ? AND topic2 = ?) OR 
        (topic1 = ? AND topic2 = ?) OR 
        (topic1 = ? AND topic2 = ?) OR 
        (topic1 = ? AND topic2 = ?) OR 
        (topic1 = ? AND topic2 = ?)
    )
ORDER BY id ASC

which demonstrates that the topic filter lists themselves are preserved at the SQL level rather than filtered out at the service level after the fact.

Why

By OR'ing everything together, we unnecessarily include extra rows in complex filtering cases.

The included benchmark proves the efficacy of this method, run on this branch vs. main:

Details

$ go test ./cmd/stellar-rpc/internal/methods -run ^$ -bench BenchmarkGetEvents -benchtime=1x 
goos: linux
goarch: amd64
cpu: 13th Gen Intel(R) Core(TM) i7-1360P
BenchmarkGetEventsTopicFilters-16    	       1	  34372128 ns/op	20023952 B/op	  275417 allocs/op
BenchmarkGetEventsTopicFilters-16    	       1	  15437162 ns/op	11233016 B/op	  126252 allocs/op

The top run is on main while the second one is on this branch.

You can see there is around a 50% reduction in both CPU and memory usage across the board due to most of the filtering happening in the database rather than client-side.

Known limitations

n/a

[Copilot]

Pull request overview

This PR refactors topic filtering in event queries by decomposing the flattened topic filter structure into a disjunction of conjunctions (OR of ANDs). Previously, all topic filters were OR'd together at each topic position, which unnecessarily included extra rows that required client-side filtering. The new approach preserves the complete filter structure at the SQL level, resulting in approximately 50% reduction in CPU and memory usage as demonstrated by the included benchmark.

Changes:

• Introduced new type definitions (TopicCondition, TopicFilter, TopicFilters) to represent topic filters as OR of ANDs
• Refactored combineTopics function to build the new structured filter format
• Updated database query builder to construct SQL with nested AND/OR conditions
• Added benchmark test to demonstrate performance improvements

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
cmd/stellar-rpc/internal/db/event.go	Introduced new type definitions for topic filtering and updated GetEvents query builder to support OR of ANDs structure
cmd/stellar-rpc/internal/methods/get_events.go	Refactored combineTopics to decompose topic filters into conjunctive clauses within a disjunction
cmd/stellar-rpc/internal/methods/get_events_test.go	Added BenchmarkGetEventsTopicFilters to measure performance improvements

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The comment 'Each topic is an OR...' is misleading. According to the protocol structure, each element in filter.Topics (which is a TopicFilter) represents a single topic pattern that should match all its segments (AND), and multiple TopicFilters in the array are OR'd together. The comment should clarify this more accurately, for example: 'Each TopicFilter is OR'd together...'

Suggested change

	// Each topic is an OR...
	for _, topicFilter := range filter.Topics {
	conditions := make(db.TopicFilter, 0, len(topicFilter))
	// ...but each segment within a topic is an AND.
	// Each TopicFilter (topic pattern) in filter.Topics is OR'd together...
	for _, topicFilter := range filter.Topics {
	conditions := make(db.TopicFilter, 0, len(topicFilter))
	// ...but each segment within a TopicFilter is ANDed.

[sreuland]

lgtm. could update the title of the pr to mention this is for db specifics such as Decompose db topic filter query into ... just to discern it's an internal sql model thing and not related to external rpc request model.