feat(log): structured logging

[GrapeBaBa]

Summary

Add a full-featured structured logging module for lodestar-z

Key features

• Comptime static dispatch pipeline: Filter → Diagnostic → Append, all resolved at compile time with zero runtime overhead for disabled log levels
• Three layout formats: TextLayout (human-readable with ANSI color), JsonLayout, LogfmtLayout
• Multiple appender types: WriterAppend (sync), AsyncAppend (lock-free ring buffer + background drain), RollingFileWriter (size/date rotation), OpenTelemetryAppend (OTLP JSON)
• Composable filters: LevelFilter, ScopeFilter, EnvFilter (parses RUST_LOG-style directives like warn,fork_choice=debug)
• Diagnostics: StaticDiagnostic (pre-bound attrs), ThreadLocalDiagnostic (per-thread context)
• std.log bridge: Drop-in std_options so existing std.log.scoped() calls route through the pipeline
• Global dispatcher builders: initConsoleDispatcher, initFileDispatcher, initCombinedDispatcher for common setups
• Microsecond timestamps: ISO 8601 UTC format (2024-08-11T22:44:57.172105Z)
• stderr by default: Following Unix convention (stdout = data, stderr = diagnostics)
• Extracted ring_buffer.zig: Generic lock-free SPSC ring buffer, reusable outside logging

Architecture

Logger(N) → AnyDispatcher → Dispatcher(Dispatches)
                               ├─ Dispatch(Filters, Diagnostics, Appends)
                               │    ├─ LevelFilter / ScopeFilter / EnvFilter
                               │    ├─ StaticDiagnostic / ThreadLocalDiagnostic
                               │    └─ WriterAppend(Layout) / AsyncAppend(Layout)
                               └─ Dispatch(...)

Module structure

Consolidated from 20+ separate files into 6 focused modules:

• record.zig — Scope, Level, Attr, Record, comptime interface checks
• filter.zig — LevelFilter, ScopeFilter, EnvFilter
• diagnostic.zig — StaticDiagnostic, ThreadLocalDiagnostic
• layout.zig — TextLayout, JsonLayout, LogfmtLayout, ISO 8601 formatting
• append.zig — WriterAppend, AsyncAppend, RollingFileWriter, OpenTelemetryAppend
• dispatch.zig — Dispatch, Dispatcher, AnyDispatcher, Logger

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new, highly performant, and flexible structured logging module for lodestar-z, drawing inspiration from the logforth architecture. The module focuses on compile-time optimizations, offering a variety of output formats and destinations, and advanced filtering mechanisms to provide a robust and configurable logging solution. While the core logging implementation files are not included in the provided diff, the changes primarily involve setting up development environment configurations and tooling related to the project.

Highlights

• Comptime Static Dispatch Pipeline: Implemented a logging pipeline (Filter → Diagnostic → Append) with compile-time static dispatch, ensuring zero runtime overhead for disabled log levels.
• Flexible Layout Formats: Introduced three distinct layout formats: TextLayout (human-readable with ANSI color), JsonLayout, and LogfmtLayout for diverse output needs.
• Multiple Appender Types: Added various appender types, including synchronous WriterAppend, asynchronous AsyncAppend (using a lock-free ring buffer), RollingFileWriter for log rotation, and OpenTelemetryAppend for OTLP JSON export.
• Composable Filters and Diagnostics: Provided composable filters (LevelFilter, ScopeFilter, EnvFilter supporting RUST_LOG-style directives) and diagnostics (StaticDiagnostic, ThreadLocalDiagnostic) for fine-grained control.
• std.log Bridge and Global Dispatchers: Included a bridge to std.log to route existing std.log.scoped() calls and global dispatcher builders (initConsoleDispatcher, initFileDispatcher, initCombinedDispatcher) for easy setup.
• Microsecond Timestamps and stderr Default: Configured microsecond timestamps in ISO 8601 UTC format and set stderr as the default output, adhering to Unix conventions.
• Generic Lock-Free Ring Buffer: Extracted ring_buffer.zig as a generic lock-free SPSC ring buffer, making it reusable beyond the logging module.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a cross-implementation fork choice test comparator agent and updates subproject references. Feedback was provided regarding the inclusion of absolute local paths in the agent configuration, which hinders portability. Additionally, several local IDE state files and log directories were committed; these should be removed and ignored to adhere to the project's zero technical debt policy and prevent environment-specific noise.

[gemini-code-assist]

The agent configuration contains absolute local paths (e.g., /Users/grapebaba/...). These paths are specific to your local machine and will not work for other contributors or in CI environments. Please use relative paths or placeholders to ensure the agent is portable.

Suggested change

	| lodestar-z | Zig | `/Users/grapebaba/Documents/projects/lodestar-z/src/fork_choice/` |
	| Lodestar | TypeScript | `/Users/grapebaba/Documents/projects/lodestar/packages/fork-choice/` |
	| Prysm | Go | `/Users/grapebaba/Documents/projects/prysm/beacon-chain/forkchoice/` |
	| lodestar-z | Zig | `./src/fork_choice/` |
	| Lodestar | TypeScript | `../lodestar/packages/fork-choice/` |
	| Prysm | Go | `../prysm/beacon-chain/forkchoice/` |

[gemini-code-assist]

The workspace.xml file contains local IDE state, including absolute paths and window configurations. This file should not be committed to the repository as it is user-specific and can lead to merge conflicts. It should be added to the project's .gitignore file.

References

1. TigerBeetle has a 'zero technical debt' policy. Committing local IDE configuration files adds unnecessary noise and maintenance burden to the repository. ^(link)

[gemini-code-assist]

This log file (and the .remember/ directory) appears to be local noise from a script execution. It contains shell errors and environment-specific paths that are not relevant to the codebase. It should be removed from the pull request.

References

1. TigerBeetle has a 'zero technical debt' policy. Avoid committing temporary files or logs that are not part of the application's functional code or documentation. ^(link)