log reader without running full lp service #172
Conversation
|
|
||
| if topic != eventSig { | ||
| // eventSig == 0 bypasses filtering | ||
| if eventSig != 0 && topic != eventSig { |
There was a problem hiding this comment.
I think I scoped this utility incorrectly when I first wrote it. This parsing utility should only handle parsing and return the sig, body, and error. The filtering and skipping should really be handled in each domain’s logic instead.
Could you update it accordingly? Otherwise, I can push the changes to this branch myself
| } | ||
|
|
||
| if opcode != eventSig { | ||
| // eventSig == 0 bypasses filtering |
There was a problem hiding this comment.
👀 here too. Added to be consistent despite I'm not using internal messages right now
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a standalone log reader feature that allows retrieval of external message output logs from TON blockchain addresses within a specified block range without running the full LogPoller service.
- Adds a new
LogReaderinterface and implementation for direct log retrieval - Extends existing interfaces to support single-address transaction fetching
- Creates a message parser that bypasses filtering for standalone use
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/logpoller/log_reader.go | Implements the core LogReader functionality with GetLogs method |
| pkg/logpoller/interfaces.go | Adds LogReader and MessageParser interfaces, updates TxLoader interface |
| pkg/logpoller/backend/txparser/utils.go | Modifies parsing functions to support bypass filtering when eventSig is 0 |
| pkg/logpoller/backend/txparser/parser.go | Adds MessageParser implementation and improves import organization |
| pkg/logpoller/backend/loader/account/loader.go | Exposes FetchTxsForAddress method for single-address transaction loading |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
pkg/logpoller/log_reader.go
Outdated
| } | ||
|
|
||
| // NewLogReader creates a new LogReader instance. | ||
| func NewLogReader(client ton.APIClientWrapped, lggr logger.Logger, loader TxLoader, msgParser MessageParser) LogReader { |
There was a problem hiding this comment.
It looks like the new LogReader is essentially taking on the responsibilties of both the Loader and Parser from the existing LogPoller. I’m not entirely convinced that introducing a separate subcomponent is justified just for the sake of merging those functions — is there a specific motivation for creating this abstraction?
Another point I’d like to bring up is how we’re currently structuring O11Y ingestion. Right now, O11Y will consume types.Log into its ingestion pipeline based on the TON LogPoller. But we should think carefully about whether we want to pass along the refined Log type (which is domain-specific to log polling) or instead rely on each chain’s native raw transaction type.
In either case, O11Y ultimately performs the same step of converting input into its own log type. The question is whether it’s better to give O11Y a domain-specific log that might need constant updates (e.g., adding fee information or other chain-specific fields in the future), or just pass the raw transactions directly and let O11Y handle the conversion in a consistent way.
My concern is that if we continue extending the LogPoller type every time O11Y needs more fieds, we’ll end up overloading that type with responsibilities that don’t really belong to it. That’s why I’m leaning toward passing raw transactions instead — it feels more future-proof and keeps responsibilities clearer.
And if O11Y only consumes raw transactions, then there’s no need to introduce a separate structured type at all. In that case, a single component would be sufficient to handle ingestion, which would further simplify the design and reduce unnecessary abstractions. What do you think?
|
|
||
| if topic != eventSig { | ||
| // eventSig == 0 bypasses filtering | ||
| if eventSig != 0 && topic != eventSig { |
There was a problem hiding this comment.
I think I scoped this utility incorrectly when I first wrote it. This parsing utility should only handle parsing and return the sig, body, and error. The filtering and skipping should really be handled in each domain’s logic instead.
Could you update it accordingly? Otherwise, I can push the changes to this branch myself
| } | ||
|
|
||
| if opcode != eventSig { | ||
| // eventSig == 0 bypasses filtering |
| } | ||
|
|
||
| // messageParserWithoutFilters provides a way to parse messages without using log poller filters | ||
| type messageParserWithoutFilters struct{} |
There was a problem hiding this comment.
I don't think we'll need to add this if we remove filtering from parsers
2 participants
Adding feature to retrieve external messages out (logs) from an address on a given blockrange (fromBlockSeqNo, toBlockSeqNo] without running the full LogPoller service