Skip to content

Conversation

@0xOmarA
Copy link
Contributor

@0xOmarA 0xOmarA commented Jul 9, 2025

Summary

This PR improves how logging is done in the differential testing framework adding more structured logs which allows us to quickly filter through logs and associate logs together.

Description

This PR replaces the log crate with the tracing crate and replaces the env_logger with the tracing-subscriber crate. This is done to allow us to get log messages with a lot more structured context that we can filter on.

Why?

Tl;Dr: By replacing log with tracing and adding some spans throughout the code we get significantly better logs with a lot more context with little maintenance overhead.

Reading the logs of any multi-threaded program is difficult as the logs from all threads all appear together in the same stream. In the case of this framework, this makes it hard to tell which metadata file is causing errors, and which specific test cases are causing errors.

Take the following as an example:

[2025-07-08T14:29:11Z ERROR revive_dt_core::driver] Failed to construct legacy transaction: instance TickMathLoop not deployed

The above log message leaves us with many questions:

  • Which metadata file caused this error?
  • Which specific test case caused this error?
  • What happened during the execution of this metadata file and are there any other logs associated with it that can point to more information?

If we wanted to answer all of these questions with the log crate we would need to pass the entire execution context up the stack with each call that we make and we would need to keep re-logging this information in a consistent manner, which adds a lot of development and maintenance overhead.

Using tracing and and some spans (which I added in this PR) allows for more information to be added to logs which allows us to filter logs for specific metadata files and test cases a lot easier.

The above log becomes the following:

2025-07-09T17:22:54.209866Z ERROR Running driver{metadata_file_path="era-compiler-tests/solidity/complex/defi/UniswapV4/test.json"}:Executing case{case="default" case_idx=0}: revive_dt_core::driver: Failed to construct legacy transaction: instance TickMathLoop not deployed
2025-07-09T17:22:54.209878Z  WARN Running driver{metadata_file_path="era-compiler-tests/solidity/complex/defi/UniswapV4/test.json"}: retester: metadata era-compiler-tests/solidity/complex/defi/UniswapV4/test.json failure: instance TickMathLoop not deployed

We can see in this case that it's quite clear to us which metadata file encountered this error, it's the era-compiler-tests/solidity/complex/defi/UniswapV4/test.json file and the associated test cases.

This allows for us to more easily filter for logs for a specific metadata file or for a specific case which we need at this stage in development to be able to quickly iterate, fix bugs, and get this framework to a production-level.

From a maintenance point of view, there's little maintenance overhead needed when using tracing due to their use of spans and the span relationships which means that we do not need to pass the entire execution context up the stack just for logging, we get that for free (from a maintenance point of view) with tracing and a few well positioned spans.

Note

This PR is not meant to fix all of the issues that we have around logging or improve all areas of the logging. Rather, it's meant to lay the foundation for everything else that's to come.

This commit updates how logging is done in the differential testing
harness to use `tracing` instead of using the `log` crate. This allows
us to be able to better associate logs with the cases being executed
which makes it easier to debug and understand what the harness is doing.
@0xOmarA 0xOmarA requested review from activecoder10 and xermicus July 9, 2025 17:44
Copy link
Member

@xermicus xermicus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@xermicus xermicus added this pull request to the merge queue Jul 10, 2025
Merged via the queue into main with commit 0513a4b Jul 10, 2025
4 of 5 checks passed
@xermicus xermicus deleted the feature/better-logging branch July 14, 2025 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants