Open
Conversation
sk1p
force-pushed
the
k2is
branch
2 times, most recently
from
7472ec9 to
366d2a4
Compare
sk1p
force-pushed
the
k2is
branch
2 times, most recently
from
559f933 to
2c3d13c
Compare
- timestamp_recorder: set local_addr depending on sector - ipc_test: pre-fault the whole shm area - warmup: increase number of blocks sent through - add a "circuit breaker" if the `assembly_channel` fills up too fast - `k2is_live.py`: allow setting a filename pattern and controlling the synchronization mode
Still slightly broken, but useful still for starting to receive multicast traffic (for example for capturing), or for testing the receive code in isolation.
* Split frame module into base, is, and summit modules * main: allocate shm with correct slot size * Fix `decode_u16_vec` to correctly take header into account * Fully implement `K2Frame` trait for `K2SummitFrame` * Slightly improve error handling; shut down assembly threads when one of the workers has died * Lots of clippy fixes
Meaning, work towards multiple acquisitions after another with minimal delay inbetween.
Otherwise, the recv on the second channel will, in the worst case, be limited to the timeout of the first recv. Also work on fixing the missing first frame issue.
Also some cleanup..
If a frame is 'dropped multiple times', that means the acquisition index is counted up too often, which can then result in the re-ordering buffer being non-empty at the end of an acquisition, which is not expected.
It's a lot faster to just use `getattr`/`call0` to call Python functions instead of compiling a Python module each time the function is called.
Otherwise, there is a high chance of having multiple dropped frames at the beginning of an acquisition, because receiving from multicast sockets does not initialize instantaneously.
Extract some configuration into `AcquisitionRuntimeConfig`
Instead, writing will be implemented as part of the common infrastructure.
Previously it didn't properly set the local IP address, so it only worked in the replayed-data scenario.
Only iterate over the frames received, and decode them in-line into a numpy array.
Especially handle I/O errors during initialization, when the event pump mechanism isn't running yet.
Error handling TBD, IS mode TBD, ...
Also don't build in container env for now - we can't easily install third-party libs there.
Get rid of system dependencies; re-enable building of the wheel of `libertem_k2is` in CI.
maturin complained: Caused by: The module name must not contain a minus `-` (Make sure you have set an appropriate [lib] name or [tool.maturin] module-name in your pyproject.toml)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opened to keep track of TODOs.
TODO
ipc-testversionK2Frameinto a thread-local variant with aSlotForWriting(K2FrameForWriting?), and an immutable variant to pass around for further processingPython parts
multiversionusage for especially hot functionsparallel-udp-replay-Ctarget-cpu=nativefor optimal performanceparallel-udp-replay