Description
The epoch parsing code is in need of significant refactoring and bug fixes. It has led to the following issues:
- delays in rendering content: 10x content not visible until pop-out #264
- lagging event history: Events that return app-db unchanged seem to cause the event history to lag #268
dispatch-sync
invisible: Handle dispatch-sync better #154nil
app-db state: Have Appdb state at startup #263, Template Issues #151- inconsistent use of event filters: Events filters in settings don't filter out events from the trace tab #175
- duplication of subs between the current epoch and intra-epoch subs
- invalid timing information
The relevant code is in ::receive-new-traces
, parse-traces
, surrounding code in the metamorphic
ns and in subs that use the app-db state derived from the epoch parsing.
Step 1: Move ignored-events
filtering from event to subs
In ::recieve-new-traces
there is a remove
to filter ignored events (via settings) before epochs are stored in app-db.
This causes some issues:
- Partial epoch states in app-db leading to odd workarounds in subscriptions which are not all handled
- If the user removes or adds an epoch filter with existing traces in app-db, 10x is in an invalid state as it does not have a correct 'view' of epochs according to the settings.
The remove
filter needs to be deleted from ::recieve-new-traces
and added to the appropriate subs instead.
Step 2: Change metamorphic/parse-traces
so that dispatch-sync
is its own epoch
The delimitation of dispatch-sync
epochs is not detected in metamorphic/parse-traces
. This needs to be fixed so a dispatch-sync
is returned as a distinct epoch.
Step 3: Move all parsing from ::recieve-new-traces
event to metamorphic
ns
::receive-new-traces
contains too much data transformation/parsing related code. If this is moved to metamorphic
ns it would be easier to write unit tests to debug/check correct behavior and so forth.
New APIs/functions should be considered as part of this move, as should renaming. The current names are certainly not great.
Tests should be added for all the significant epoch parsing use cases/situations.
At least the following should be moved:
parse-traces
parse-state
changessubscription-info
subscription-match-state
subscription-matches
new-sub-state
timing
mapvnew-matches
mapquiescent?
all-matches
retained-matches
first-id-to-retain
retained-traces
:matches-by-id
into...map:selected-epoch-id
:selected-epoch-index
Step 4: Performance
Epoch parsing and associated code is currently the biggest hit on 10x performance.
Once more of the code is moved into functions in metamorphic
with unit tests various optimisations could be explored such as storing the internal parse state as transient vs persistent collections, doing less string parsing etc.