feat: add cdylib target, harden C FFI, fix capture offsets, and add timestamp event boundaries

[jackluo923]

Description

Add several improvements to the Rust lexer's build targets, FFI safety, correctness,
and functionality:

• cdylib target: Add cdylib to crate-type so the library can be loaded via dlopen
  (e.g. Python cffi).

• Runtime-configurable debug output: Replace unconditional println! in DFA/NFA
  construction with a debug_println! macro gated behind an AtomicBool, controllable at
  runtime via debug::set_debug(). This lets upper-layer consumers (e.g. Python bindings)
  suppress noisy internal output by default and enable it only when needed.

• Fix capture boundary offsets: The DFA used inclusive byte offsets (input[i..=j],
  input[..=consumed]), causing incorrect capture slices. For example, matching
  (?<key>[a-z]+)=(?<val>[0-9]+) against foo=123 would return 12 instead of 123 for
  the val capture because the end offset pointed to the start of the last character rather
  than past it. Fix by tracking consumed_end as an exclusive byte offset and using
  input[i..j] / input[..consumed_end].

• Fix delimiter hex encoding: pattern_for_delimiters formatted codepoints with {:x},
  producing \u{a} for newline, which the regex parser rejected. Fix by zero-padding with
  {:02x} to produce \u{0a}.

• Support underscores in capture group names: Named capture groups like (?<my_var>...)
  are common convention but were rejected by the regex parser. Add support by switching from
  alphanumeric1 to take_while1(|c| c.is_alphanumeric() || c == '_').

• Fix FFI panic on invalid input: FFI functions like schema_add_rule and lexer_new
  would panic (abort the calling process) on invalid UTF-8 or bad regex patterns. Now they
  return bool / null pointer so callers can handle errors gracefully.

• Harden FFI lifecycle: lexer_new returns *mut Lexer (null on failure) instead of
  Box<Lexer>. lexer_delete takes *mut Lexer with explicit null check. Add set_debug,
  schema_rule_count, and schema_rule_name introspection functions.

• Timestamp rules + event boundary detection: Add is_timestamp field to Schema::Rule
  and add_timestamp_rule() method. Add is_event_start field to Fragment, set when a
  timestamp rule matches at byte offset 0 or immediately after a newline, enabling downstream
  consumers to split multi-line log events.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• cargo test — all unit tests pass, including new tests for capture boundaries and event
  boundary detection
• End-to-end validation via
  log-surgeon-ffi 0.1.0b10 — the
  published Python package uses the Rust cdylib from this branch as its cffi backend,
  exercising the FFI, capture offsets, and timestamp event boundary features against real
  log data across Python 3.9–3.13 on Linux and macOS

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

• 🔍 Trigger a full review

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}