Merge pull request #206 from indunet/develop

Develop

Author: Deng-Ran

CHANGELOG-zh.md

@@ -1,5 +1,7 @@
 # 更新日志
 
+本文件用于记录本项目的所有重要变更。
+
 ## [3.12.3] - 2025-11-23
 ### 新增
 - 新增 `@BcdType` 注解与 `BcdCodec`，支持定长 packed BCD 整数（支持大小端），映射到 `int` / `Integer` 类型。

README-zh.md

@@ -15,21 +15,16 @@ FastProto 是一个面向 **二进制通信协议** 的高性能序列化 / 反
 ## *设计理念*
 
 * **声明式优于命令式：** 协议字段用注解定义，结构一目了然，代码即文档。
-* **性能与可维护并重：** 在架构层做反射缓存与零拷贝等优化，在保持高吞吐的同时兼顾可维护性。
+* **性能与可维护并重：** 在架构层做反射缓存等优化，在保持高吞吐的同时兼顾可维护性。
 * **兼容工程现实：** 支持大小端混排、位域、BCD、CRC 等传统协议特征，方便对接存量系统。
 
 ## *核心功能*
 
-* **注解驱动：** 字段用注解标记，解析和封装一目了然。
-* **类型丰富：** 支持 Java 原始类型、无符号类型、字符串、时间以及集合。
-* **灵活地址：** 提供反向地址，适配变长协议。
-* **可变长度：** 通过 `lengthRef` 引用计数字段，支持变长字符串/数组与结构体数组。详见[可变长度与结构体数组](docs/variable-length.md)。
-* **动态偏移：** 使用 `offsetRef` 引用偏移字段，简化“头部存绝对位置、正文在后”的格式。详见[动态偏移](docs/dynamic-offset.md)。
-* **字节顺序可选：** 大端或小端随心切换。
-* **公式支持：** Lambda 或自定义类均可实现编解码公式。
-* **校验和/CRC：** 使用 `@Checksum` 注解一次性定义起始地址、长度与存放地址，内置 CRC8（SMBus、MAXIM）、CRC16（MODBUS、CCITT）、CRC32/CRC32C、CRC64（ECMA/ISO）、LRC、XOR 等。
-* **生态集成：** 提供 Netty 编/解码器与 Kafka Serializer/Deserializer/Serde，可即插即用。详见[Netty 集成](docs/help.html#netty-integration)与[Kafka 集成](docs/help.html#kafka-integration)。
-* **多种API：** 兼顾效率与易用性。
+* **注解驱动协议映射：** 用少量注解描述二进制布局，替代手写偏移和位运算。
+* **类型与布局支持丰富：** 覆盖基础类型（含无符号）、字符串、时间、数组/集合，并支持可变长度、动态偏移和反向地址。
+* **表示精确可控：** 可配置字节序/位序，支持 BCD、位域以及自定义编解码公式。
+* **内置校验和/CRC：** 通过 `@Checksum` 或工具方法直接使用常见 CRC / 校验和算法。
+* **生态与 API 友好：** 提供 Netty 编解码器、Kafka 序列化组件，以及无需注解的链式 API。
 
 查看[更新日志](CHANGELOG-zh.md)获取版本历史，英文版请查阅[CHANGELOG](CHANGELOG.md)。
 
@@ -190,6 +185,7 @@ FastProto支持Java基础数据类型，考虑到跨语言跨平台的数据交
 | @Int16Type  |      Short/short/Integer/int       |     short      | 2 字节 |  
 | @Int32Type  |            Integer/int             |      int       | 4 字节 | 
 | @Int64Type  |             Long/long              |      long      | 8 字节 |
+| @BitFieldType |         Integer/int             |      --        | 1..31 位 |
 | @UInt8Type  |            Integer/int             | unsigned char  | 1 字节 |   
 | @UInt16Type |            Integer/int             | unsigned short | 2 字节 |   
 | @UInt32Type |             Long/long              |  unsigned int  | 4 字节 |   

README.md

@@ -15,21 +15,16 @@ FastProto is a high-performance serialization/deserialization **library** design
 ## *Design Philosophy*
 
 - **Declarative over imperative:** Protocol fields are defined via annotations, making the structure self‑evident so that code doubles as documentation.
-- **Performance with maintainability:** Architectural optimizations such as reflection caching and zero‑copy are built in to keep high throughput without sacrificing readability and maintainability.
+- **Performance with maintainability:** Architectural optimizations such as reflection caching are built in to keep high throughput without sacrificing readability and maintainability.
 - **Grounded in engineering reality:** Supports mixed endianness, bit fields, BCD, CRC and other traditional protocol features to integrate smoothly with existing systems.
 
 ## *Key Features*
 
-- **Annotation-Driven:** Quickly map binary data to Java fields.
-- **Broad Type Support:** Works with primitives, unsigned numbers, strings, time types, arrays and collections.
-- **Flexible Addressing:** Reverse addressing for variable-length packets.
-- **Variable-Length Fields:** Use `lengthRef` to reference a count field; supports variable-length strings/arrays and struct arrays. See [Variable Length and Struct Arrays](docs/variable-length.md).
-- **Dynamic Offset:** Use `offsetRef` to bind a field's offset to a previously decoded numeric field (useful for header-with-pointer formats). See [Dynamic Offset](docs/dynamic-offset.md).
-- **Configurable Byte Order:** Choose big-endian or little-endian to match your protocol.
-- **Custom Formulas:** Use lambdas or classes to transform values during encode/decode.
-- **Checksum/CRC:** Single-annotation `@Checksum` to define start, length and storage offset; built-ins include CRC8 (SMBus, MAXIM), CRC16 (MODBUS, CCITT), CRC32/CRC32C, CRC64 (ECMA/ISO), plus LRC and XOR.
-- **Integrations:** Netty codecs and Kafka Serializer/Deserializer/Serde for drop‑in use. See [Netty Integration](docs/help.html#netty-integration) and [Kafka Integration](docs/help.html#kafka-integration).
-- **Easy APIs:** Multiple APIs tuned for efficiency and reliability.
+- **Annotation‑driven protocol mapping:** Describe binary layouts with simple field annotations instead of manual offset and bit operations.
+- **Rich type & layout support:** Primitives (including unsigned), strings, time types, arrays/collections, variable‑length fields, dynamic offsets and reverse addressing.
+- **Precise control of representation:** Configurable byte/bit order, BCD, bit fields and custom encode/decode formulas.
+- **Built‑in checksum/CRC:** Single `@Checksum` annotation or utility methods covering common CRC and checksum algorithms.
+- **Ecosystem integrations:** Ready‑to‑use Netty codecs and Kafka Serializer/Deserializer/Serde, plus fluent APIs for use without annotations.
 
 See the [CHANGELOG](CHANGELOG.md) for recent updates.
 
@@ -187,9 +182,10 @@ FastProto supports Java primitive data types, taking into account cross-language
 | @AsciiType  |          Character/char           |      char      | 1 bytes |   
 |  @CharType  |          Character/char           |       --       | 2 bytes |
 |  @Int8Type  |       Byte/byte/Integer/int       |      char      | 1 byte  |  
-| @Int16Type  |     Short/short/Integer/int     |     short      | 2 bytes |  
+| @Int16Type  |     Short/short/Integer/int       |     short      | 2 bytes |  
 | @Int32Type  |            Integer/int            |      int       | 4 bytes | 
 | @Int64Type  |             Long/long             |      long      | 8 bytes |
+| @BitFieldType |         Integer/int             |      --        | 1..31 bits |
 | @UInt8Type  |            Integer/int            | unsigned char  | 1 byte  |   
 | @UInt16Type |            Integer/int            | unsigned short | 2 bytes |   
 | @UInt32Type |             Long/long             |  unsigned int  | 4 bytes |   

docs/annotation-mapping.md

@@ -33,6 +33,7 @@ FastProto uses annotations placed on fields to describe how bytes map to values.
 | @Int16Type  |     Short/short / Integer/int     |     short      | 2 bytes |
 | @Int32Type  |            Integer/int            |      int       | 4 bytes |
 | @Int64Type  |             Long/long             |      long      | 8 bytes |
+| @BitFieldType |         Integer/int             |      --        | 1..31 bits |
 | @UInt8Type  |            Integer/int            | unsigned char  | 1 byte  |
 | @UInt16Type |            Integer/int            | unsigned short | 2 bytes |
 | @UInt32Type |             Long/long             |  unsigned int  | 4 bytes |

docs/byte-and-bit-order.md

@@ -61,6 +61,23 @@ public class Flags {
 }
 ```
 
+### Bit field (1..31 bits, supports bitOrder + byteOrder)
+```java
+import org.indunet.fastproto.*;
+import org.indunet.fastproto.annotation.*;
+
+@DefaultByteOrder(ByteOrder.LITTLE)
+@DefaultBitOrder(BitOrder.LSB_0)
+public class BitFields {
+  // Use lengthRef to determine bit width dynamically; logical start bitOffset=6, spans bytes
+  @BitFieldType(offset = 0, bitOffset = 6, lengthRef = "$width", byteOrder = ByteOrder.LITTLE, bitOrder = BitOrder.LSB_0)
+  int value;
+
+  @UInt8Type(offset = 2)
+  int width; // Provides the length; must be declared before use
+}
+```
+
 ### Array with per‑element endianness
 ```java
 import org.indunet.fastproto.*;

docs/dynamic-offset.md

@@ -1,7 +1,7 @@
 ## Dynamic Offset with offsetRef
 
 ### Overview
-Some formats store an absolute position in a header and place the actual data later in the file. Use `offsetRef` to point a field’s `offset()` to a previously decoded numeric field in the same class. The source field name can be prefixed with `$`.
+Some formats store an absolute position in a header and place the actual data later in the file. Use `offsetRef` to point a field’s `offset()` to a previously decoded numeric field in the same class. The source field name must be prefixed with `$`.
 
 ### Rules
 - The referenced field must appear earlier in the class and be decoded before the target.

docs/variable-length.md

@@ -8,7 +8,7 @@ Variable-length fields are common in binary protocols. FastProto supports them v
 
 ### Basics: length vs lengthRef
 - `length`: a fixed number of elements/bytes.
-- `lengthRef`: the name of a field in the same class that holds the length/count; you may optionally prefix it with `$` (e.g. `$count`).
+- `lengthRef`: the name of a field in the same class that holds the length/count; it **must** be prefixed with `$` (e.g. `$count`).
 - If both are set, `lengthRef` takes precedence for the effective length.
 
 ### Strings with lengthRef
@@ -87,7 +87,7 @@ public class PacketVar {
 
 ### Tips and caveats
 - Ensure the referenced length field is decoded before the variable-length field (i.e., it appears earlier in the class with a lower offset).
-- `lengthRef` accepts optional leading `$` and is case-sensitive with respect to the Java field name.
+- `lengthRef` is case-sensitive and must start with `$` followed by the field name.
 - For strings, you can also set `charset`.
 - For boolean bit arrays, use `BoolArrayType` (byte-aligned). Per-bit packed variable-length booleans are not supported.
 - With `@AutoType`, you must still supply `offset` and `lengthRef` for arrays/strings; FastProto infers only the data type, not the layout.

pom.xml

@@ -226,6 +226,13 @@
             <version>3.9.1</version>
             <scope>provided</scope>
         </dependency>
+        <!-- SLF4J binding for tests to avoid NOP logger warning -->
+        <dependency>
+            <groupId>org.slf4j</groupId>
+            <artifactId>slf4j-simple</artifactId>
+            <version>1.7.36</version>
+            <scope>test</scope>
+        </dependency>
     </dependencies>
 
     <build>

src/main/java/org/indunet/fastproto/annotation/BitFieldType.java

@@ -0,0 +1,75 @@
+/*
+ * Copyright 2019-2025 indunet.org
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.indunet.fastproto.annotation;
+
+import org.indunet.fastproto.BitOrder;
+import org.indunet.fastproto.ByteOrder;
+import org.indunet.fastproto.graph.resolve.validate.DecodingFormulaValidator;
+import org.indunet.fastproto.graph.resolve.validate.EncodingFormulaValidator;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Bit field type, supports unsigned value extraction from 1..31 bits.
+ *
+ * @author Deng Ran
+ * @since 3.13.0
+ */
+@DataType
+@Validator({DecodingFormulaValidator.class, EncodingFormulaValidator.class})
+@Target(ElementType.FIELD)
+@Retention(RetentionPolicy.RUNTIME)
+public @interface BitFieldType {
+    /*
+     * The byte offset of the field in the binary data.
+     */
+    int offset() default Integer.MIN_VALUE;
+
+    /*
+     * Reference to an offset field name in the same class. Accepts optional leading '$'.
+     */
+    String offsetRef() default "";
+
+    /*
+     * The bit offset (logical index 0..7) within the starting byte.
+     */
+    int bitOffset();
+
+    /*
+     * The bit width of the field (1..31). If 0, a valid lengthRef must be provided.
+     */
+    int length() default 0;
+
+    /*
+     * Reference to a length field name in the same class. Accepts optional leading '$'.
+     */
+    String lengthRef() default "";
+
+    /*
+     * The byte order for multi-byte bit ranges, priority higher than @DefaultByteOrder.
+     */
+    ByteOrder[] byteOrder() default {};
+
+    /*
+     * The bit order within each byte, priority higher than @DefaultBitOrder.
+     */
+    BitOrder[] bitOrder() default {};
+}
+

src/main/java/org/indunet/fastproto/codec/BitFieldCodec.java

@@ -0,0 +1,58 @@
+/*
+ * Copyright 2019-2025 indunet.org
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.indunet.fastproto.codec;
+
+import lombok.val;
+import org.indunet.fastproto.annotation.BitFieldType;
+import org.indunet.fastproto.exception.DecodingException;
+import org.indunet.fastproto.exception.EncodingException;
+import org.indunet.fastproto.io.ByteBufferInputStream;
+import org.indunet.fastproto.io.ByteBufferOutputStream;
+
+/**
+ * Codec for BitFieldType, unsigned width 1..31 bits mapped to Integer.
+ */
+public class BitFieldCodec implements Codec<Integer> {
+    @Override
+    public Integer decode(CodecContext context, ByteBufferInputStream inputStream) {
+        try {
+            val type = context.getDataTypeAnnotation(BitFieldType.class);
+            val bitOrder = context.getBitOrder(type::bitOrder);
+            val byteOrder = context.getByteOrder(type::byteOrder);
+            int length = type.length();
+
+            return inputStream.readBits(type.offset(), type.bitOffset(), length, bitOrder, byteOrder);
+        } catch (IndexOutOfBoundsException | IllegalArgumentException e) {
+            throw new DecodingException("Fail decoding bit field type.", e);
+        }
+    }
+
+    @Override
+    public void encode(CodecContext context, ByteBufferOutputStream outputStream, Integer value) {
+        try {
+            val type = context.getDataTypeAnnotation(BitFieldType.class);
+            val bitOrder = context.getBitOrder(type::bitOrder);
+            val byteOrder = context.getByteOrder(type::byteOrder);
+            int length = type.length();
+
+            outputStream.writeBits(type.offset(), type.bitOffset(), length, bitOrder, byteOrder, value);
+        } catch (IndexOutOfBoundsException | IllegalArgumentException e) {
+            throw new EncodingException("Fail encoding bit field type.", e);
+        }
+    }
+}
+