chore: sync upstream#2
Open
BRAVO68WEB wants to merge 140 commits into
Open
Conversation
Instead of having postprocessing stages built directly into rpicam_app.so, allow them to be loaded dynamically at runtime. These .so files will be installed to the /<system lib>/rpicam-apps-postproc/ directory. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Split the existing postprocessing stages into 3 .so files: - Core stages - OpenCV stages - TFLite stages These are installed into the /<system lib>/rpicam-apps-postproc/ directory. Additionally, switch enable_tflite and enable_opencv meson option types to "feature". Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Ensure we alway have a valid factory map object, even if no stages have been registered. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Allow the user to set a custom path for the postprocessing library location. This is useful for local builds. Update regression tests to use the build location of the postprocessing libs. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Allow the caller to allocate buffers use use for the conversion externally. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
For consistency, use 'feature' types for enable_libav, enable_drm, enable_egl and enable_qt meson options. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Introduce a new set of postprocessing stages using the Hailo 8L AI
accellerator. The following new stages are included:
Resnet Classifier
MobileNet SSD
Yolov5/Yolov6/Yolov8/YoloX detection
Yolov5 segmentation
Yolov8 pose estimation
A new script runs at build time to download the relevant Hailo device
models for all the above stages. These model files get installed into
the /usr/{local/}share/hailo-models directory in the "meson install"
stage.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reduce the minimum runtime for some tests as they can fail regression runs. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
These files were pulled from libcamera with the LGPL-2.1-or-later license. Since Raspberry Pi is the sole author of these files, update the license in this repo to BSD-2-Clause, matching the other source files. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Install the postprocessing JSON config assets as part of the meson install stage. They are put into <prefix>/share/rpi-camera-assets/ Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Allow the low resolution stream width, height and pixel fromat to be set through the JSON postprocessing file. The parameters are stored in the "rpicam-apps" key. Currently only "RGB", "BGR" and "YUV420" formats are supported. Configuration in the JSON file will override any user supplied values through the command line. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
If the ISP is configured for RGB output, bypass the YUV -> RGB software conversion in the Hailo postprocessing stages. Note that we still need to make a copy of the low res image buffer to pass into the Hailo HW, otherwise we get some intermittent low memory mapping error from the Hailo device driver. This needs further investigation. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
This avoids the need to add these parameters to the command line. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Remove all hard-coded tensor size consts from the Hailo postprocessing stages. Add a HailoPostProcessingStage::InputTensorSize() helper to fetch the input tensor size supplied in the JSON file instead. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
This is a more robust way of creating the latest file symlink, and allows the symlink to be created in a directory different to the output files. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Do not set HDR control switch on all IMX708 subdev nodes available. Rather, traverse through sysfs and find the correct subdev node to toggle the HDR control on and off. We aim to match the selected camera device id when deciding which subdev node to use. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The change for rpi::ScalerCrops changes the behaviour of how crops are advertised - we now need a non-RAW output stream to be configured to return the available crop control. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add a HailoPostProcessingStage::ConvertInferenceCoordinates() helper that converts from the inference (low res) image co-ordinate space to the main image space, accounting for different scaler crops. Fixup the Yolo inference and pose estimation stages to use this new helper. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add a new command line argument "--lores-par" that when set, preserves the aspect ratio of the low res stream. This is only possible on the PiSP platform. The default behaviour is to have this switch disabled, allowing for identical behaviour betwen VC4 and PiSP platforms. This requires the use of the rpi::ScalerCrops vendor control. This switch can also be triggered via the "rpicam-apps.lores.par" key in the postprocessing JSON file. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
This has changed between Tappas versions, so test for the correct lib dir to maintain backward compatibility. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
…tworks The v4.18 Hailo release combines the cases for both types of postprocessing. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
This uses NMS processing similar to the other Yolo networks. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add a 14-bit entry to the supported format table. Signed-off-by: Csaba Nagy
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Switch to using vectors for the list of points/confidences so that multiple poses can be drawn. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add postprocessing stages using the IMX500 inference output. MobileNet SSD object inference and PoseNet pose estimation software stages have been implemented in this commit. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The --metadata option was documented but not functional in rpicam-jpeg. While rpicam-still correctly saves metadata files, rpicam-jpeg never called the save_metadata() function. This commit adds: - Include for output/output.hpp - save_metadata() function implementation - Call to save_metadata() after jpeg_save() Tested with JSON, TXT, and stdout formats - all working correctly. Fixes: #885 Signed-off-by: Kletternaut <tomge68@gmail.com>
Add support for mono formats in the dng writer. Authored by Will Whang. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Download scrips for the IMX500 and Hailo device models are now removed since they are not directly packaged with rpicam-apps. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Allow callers to specify custom flages to dlopen in the DlLib class. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add support for the Hailo H10 accelerator in the hailo-postprocess sw stage template. This also requires a change to remove linkage to libhailort.so as they are different for each device. Instead, we do a dlopen on the library so file at runtime to avoid any version issues with linkage. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Because of an API mismatch in versions of libhailort for H8/H10, we cannot use the exisitng mechanism of getting the hardware architecture. Replace this with a system call to: hailortcli fw-control identify and extract the architecture string from stdout which should be correct. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Some protocols open their own file handes, so we mustn't try to do this for them as it will fail. The correct way to do this is to check the context's oformat->flags, not the flags field directly. The reason is that the flags report the current state of the context, whereas oformat->flags reports what this particular output format requires, which is therefore correct. The consequence is that we can output RTSP streams directly, which is useful for publishing to media servers like MediaMTX using a runOnDemand command. Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Make the application remember that sync has been achieved, so that subsequent frames will not be skipped even if they are missing the usual sync metadata. This can happen at fast framerates, when the algorithm does not run every frame. Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
This adds a new post-processing stage that automatically blurs detected objects
in the video stream based on their COCO class names. Useful for privacy
applications and real-time object obscuring.
Key features:
- Blur objects by COCO class name (e.g., "person", "cup", "wine glass")
- Multiple blur types: pixelate (fast), gaussian (smooth), median (noise reduction)
- Configurable blur strength and bounding box expansion
- Live processing - no post-production editing needed
- Works seamlessly with hailo_yolo_inference and other detection stages
- Clean recordings possible by omitting object_detect_draw_cv (no bounding boxes)
The stage depends on object detection results from stages like hailo_yolo_inference
and applies blur effects directly to the main video stream.
Example configuration:
{
"hailo_yolo_inference": { ... },
"object_blur": {
"overlay_blur": ["person", "cell phone"],
"blur_type": "pixelate",
"blur_strength": 16
}
}
Includes:
- object_blur_stage.cpp: Implementation with OpenCV
- object_blur.json: Example configuration
- object_blur_stage.md: Comprehensive bilingual documentation (EN/DE)
- meson.build: Build configuration (added to opencv-postproc)
Signed-off-by: Kletternaut <tomge68@gmail.com>
This is caused by OpenCV headers using the C11 Atomic keyword in C++ source files. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Instead of overwriting the Request::controls() list, merge it with the local list as per the 0.7 libcamera API changes. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Ensure the "packed" flag is set so these will be unpacked to u16. Set the "white level" for monochrome as well as for Bayer formats. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
Call the internal test system for PRs to replaces the existing GitHub actions. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Replace for loop with while(true) Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add some basic regression tests for hailo and imx500 postprocessing stages. The tests will attempt to detect if the devices are attached to the platform before running. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Check if we have 2 cameras, and run a simple dual viewfinder test in parallel processes. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The Hilao post-processing stage segfaults at process exit due to a static destruction order bug in libhailort. The VDevice singleton is held in a static unique_ptr which gets destroyed during exit() teardown, but by that point other statics inside libhailort that the VDevice destructor depends on have already been destroyed, causing a use-after-free crash in Buffer::~Buffer(). The fix is to intentionally leak the VDevice by storing it as a raw pointer instead of a unique_ptr. The OS reclaims the memory at process exit anyway, and this avoids triggering the buggy libhailort destructor. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Add .pre-commit-config.yaml with clang-format, ruff, shellcheck, trailing-whitespace, end-of-file-fixer, check-json, and gitlint hooks. Add .gitlint config requiring Signed-off-by and 80 char line limits. Update .clang-format PackConstructorInitializers to BinPack. Delete utils/checkstyle.py and add utils/pre-commit-hook template. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Also suppress -Wdeprecated-enum-enum-conversion for the opencv-postproc module, as OpenCV 4 headers perform arithmetic between unrelated enum types which becomes a hard error under C++20 with -Werror. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The post-processing detect test unconditionally hard-fails if object_detect_tf.json is absent. Move the JSON-file check inside the try block so the existing "model unavailable, skipping test" branch also covers missing JSON, matching the comment's stated intent. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
The lores size 640x640 ended up larger than the cropped hires size, so the test would throw an exception. Fix this. Add a tolerance for ISO checks, imx296 reports ISO 199 for a set of 200. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Previously it could crash in the (rare) case that Qt preview was requested but could not be found; there was no path to fall back to the null preview. Now it will continue to search for EGL, DRM or null preview. Tidied code a little. Signed-off-by: Nick Hollinghurst <nick.hollinghurst@raspberrypi.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.