Add Kanzi codec support#506
Open
BICHENG wants to merge 1 commit into
Open
Conversation
The earlier Kanzi integration discussion in mcmilk#503 still had Kanzi C++ codec issues that could affect real 7z archive creation and extraction. This import uses BICHENG/kanzi-cpp 3420e938874b5f3a4058e76ce6926968c10402e1, where the affected 7z-facing cases are fixed and covered by the new CI check. This adds KANZI as a 7z archive compression method. The method name is KANZI, the method id is 0x4F71107, and 7z can create, list, test, and extract .7z archives using it. Included: - Kanzi C++ v2.5.3 source based on flanglet/kanzi-cpp de6860a6794d6432ae2f252fda381ed30445f60e, with the 7z integration fixes from BICHENG/kanzi-cpp 3420e938874b5f3a4058e76ce6926968c10402e1; - Apache-2.0 license text in C/kanzi/LICENSE; - the 7z encoder/decoder adapter under CPP/7zip/Compress/Kanzi*; - method registration for 0x4F71107 / KANZI; - Windows, Linux, and 7zFM builds that compile the Kanzi encoder, decoder, and vendored source into 7z; - 7z list/property output for the KANZI method name, level, block, and checksum; - 7zFM compression dialog support for Kanzi, mx0..mx9, and up to 64 threads; - README and external method documentation for the Kanzi source, method id, and method header: 7z stores version, level, checksum, and block size, while the Kanzi stream records transform and entropy. Archive tests cover the operations users run through 7z: create, list, test, extract, stdin, update/delete, method properties, and SHA-256 comparison. They run with the existing Linux, Windows, ARM, and darkmode builds, and include mx8/mx9 default decode, -m0=kanzi:b64m -mx8 -mmt=1 for TPAQ large blocks, and -m0=kanzi:b1k -mx5..7 -mmt=64 for small-block high-thread archives. The new kanzi-codec-quality CI check runs the full option matrix on Linux x64: Kanzi level, block, checksum, thread count, and archive operation coverage for 4,327 cases. Platform coverage remains in the existing platform jobs with the representative Kanzi cases above. Resolved in the shipped source: - flanglet/kanzi-cpp#36 for TPAQ large-block handling; - flanglet/kanzi-cpp#38 for ROLZ inverse bounds and transform test crashes; - MSVC C++20 ThreadPool result type handling; - small-block high-thread output-block assignment. The MSVC C++20 build issue is reached by the Windows darkmode build. MSVC can use /std:c++20 without updating __cplusplus, so the old concurrent.hpp check could take the pre-C++17 branch and refer to std::result_of, which C++20 has removed. The fix also checks _MSVC_LANG so C++17/C++20 builds use std::invoke_result_t. The small-block high-thread issue is reached by -m0=kanzi:b1k -mx5..7 -mmt=64. With a known input size, Kanzi assigns jobs-per-task from the input block count. The old output stream capped the known block count at MAX_CONCURRENCY - 1. With 64 jobs and enough small blocks to use all task slots, the last slot kept a zero jobs-per-task value and archive creation could fail. The fix caps the known block count at MAX_CONCURRENCY, matching Kanzi's documented 1..64 job range. The platform test now covers this 7z command shape directly. Known issues: - flanglet/kanzi-cpp#39 remains an upstream Java/C++ TPAQX and CM compatibility issue. This 7z method creates and extracts archives with the C++ implementation on both sides. - This change integrates Kanzi as a 7z archive codec. Standalone .knz handling and custom transform/entropy UI are outside this change. - Kanzi streams currently support up to 64 jobs, and 7zFM shows the same thread limit. Larger-machine tuning remains a performance follow-up; it does not affect 7z archive creation, extraction, or option correctness here. This supersedes the earlier Kanzi integration discussion in mcmilk#503 and closes mcmilk#486. CI cost: This adds one visible kanzi-codec-quality check. In GitHub Actions 27525025443, that check took 5m46s and the full workflow took 10m38s; the longest job was linux x64 at 10m28s.
|
Author
|
@sebres @mcmilk https://sonarcloud.io/project/issues?pullRequest=506&open=AZ7JzYLa-uowLXpUNiO7&id=mcmilk_7-Zip-zstd Added an empty alphabet guard: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Kanzi codec support
The earlier Kanzi integration discussion in #503 still had Kanzi C++ codec issues that could affect real 7z archive creation and extraction. This import uses BICHENG/kanzi-cpp 3420e938874b5f3a4058e76ce6926968c10402e1, where the affected 7z-facing cases are fixed and covered by the new CI check.
This adds KANZI as a 7z archive compression method. The method name is KANZI, the method id is 0x4F71107, and 7z can create, list, test, and extract .7z archives using it.
Included:
Archive tests cover the operations users run through 7z: create, list, test, extract, stdin, update/delete, method properties, and SHA-256 comparison. They run with the existing Linux, Windows, ARM, and darkmode builds, and include mx8/mx9 default decode, -m0=kanzi:b64m -mx8 -mmt=1 for TPAQ large blocks, and -m0=kanzi:b1k -mx5..7 -mmt=64 for small-block high-thread archives.
The new kanzi-codec-quality CI check runs the full option matrix on Linux x64: Kanzi level, block, checksum, thread count, and archive operation coverage for 4,327 cases. Platform coverage remains in the existing platform jobs with the representative Kanzi cases above.
Resolved in the shipped source:
The “MSVC C++20 build” issue is reached by the Windows darkmode build. MSVC can use /std:c++20 without updating __cplusplus, so the old concurrent.hpp check could take the pre-C++17 branch and refer to std::result_of, which C++20 has removed. The fix also checks _MSVC_LANG so C++17/C++20 builds use std::invoke_result_t.
The “small-block high-thread issue” is reached by -m0=kanzi:b1k -mx5..7 -mmt=64. With a known input size, Kanzi assigns jobs-per-task from the input block count. The old output stream capped the known block count at MAX_CONCURRENCY - 1. With 64 jobs and enough small blocks to use all task slots, the last slot kept a zero jobs-per-task value and archive creation could fail. The fix caps the known block count at MAX_CONCURRENCY, matching Kanzi's documented 1..64 job range. The platform test now covers this 7z command shape directly.
Known limits:
MAX_CONCURRENCYhard-codedto 64, and 7zFM shows the same thread limit. Larger CPU (e.g EPYC 9654) tuning remains a performance follow-up; it does not affect 7z archive creation, extraction, or option correctness here.This supersedes the earlier Kanzi integration discussion in #503 and closes #486.
Extra CI:
This adds kanzi-codec-quality check. about 6min.