Fix: Compiler directives being merged onto single line (#2073)

[kbrunham-intel]

Fix: Compiler directives being merged onto single line (#2073)

Problem

The formatter was incorrectly merging consecutive compiler directives onto a single line. For example:

`timescale 1 ps / 1 ps
`default_nettype none

Would be formatted as:

`timescale 1 ps / 1 ps `default_nettype none

This affected all SystemVerilog compiler directives including `timescale, `default_nettype, `resetall, `celldefine, and others.

Root Cause

The issue had two parts:

1. Token Classification: The IsPreprocessorKeyword() function only recognized preprocessor control flow directives (PP_* tokens like `ifdef, `define) but did not include compiler directives (DR_* tokens like `timescale, `default_nettype).

2. Tree Unwrapping: The tree unwrapper was not properly creating separate partitions for:

   • Compiler directive nodes (kTimescaleDirective, kTopLevelDirective)
   • Bare compiler directive leaf tokens (like `resetall, `celldefine) that don't have parent nodes

Changes

1. verible/verilog/parser/verilog-token-classifications.cc

Extended IsPreprocessorKeyword() to include all DR_* compiler directive tokens:

• DR_timescale, DR_default_nettype, DR_resetall
• DR_celldefine, DR_endcelldefine
• DR_unconnected_drive, DR_nounconnected_drive
• DR_suppress_faults, DR_nosuppress_faults
• DR_enable_portfaults, DR_disable_portfaults
• DR_delay_mode_*, DR_default_decay_time, DR_default_trireg_strength
• DR_pragma, DR_uselib, DR_begin_keywords, DR_end_keywords
• DR_protect, DR_endprotect

This ensures these tokens receive kMustWrap spacing decisions in the token annotator.

2. verible/verilog/formatting/tree-unwrapper.cc (Node handling)

Added NodeEnum::kTimescaleDirective and NodeEnum::kTopLevelDirective to the list of constructs that should be formatted as unindented unwrapped lines, alongside preprocessor control flow clauses.

3. verible/verilog/formatting/tree-unwrapper.cc (Leaf handling)

Added handling in the Visit(SyntaxTreeLeaf) function for preprocessor keyword leaves (bare DR_* tokens without parent nodes). These directives now:

• Start new unwrapped lines
• Are unindented when at top level or inside preprocessor clauses
• Respect context indentation when inside module ports or other structures

4. verible/verilog/formatting/formatter_test.cc

Added comprehensive regression tests covering:

• Multiple consecutive compiler directives ( `timescale + `default_nettype)
• Various directive combinations ( `resetall + `celldefine + `timescale)
• Directives before module declarations
• Different directive types ( `suppress_faults, `enable_portfaults, `delay_mode_*)
• Directives with arguments ( `default_decay_time, `begin_keywords)

All test cases verify that consecutive directives remain on separate lines with no indentation when at top level.

Testing

• All existing formatter tests pass
• New test cases specifically validate the fix for this issue
• Verified correct behavior with real-world SystemVerilog files

[kbrunham-intel]

@hzeller : Please share your thoughts on this PR. I believe this fixes the bug.

[hzeller]

Nice!
I suggest to break out the classification of compiler directives into different functions so that it is easier to compose in places where needed.

[hzeller]

I'd probably make a IsCompilerDirective() classification function, in which we put all these tokens then, so that it has a nice name.

Note, IsPreprocessorKeyword() is also used in formatting/align.cc and formatting/token-annotator.cc - it would probably be good to check if the additional tokens in this IsPreprocessorKeyword classification would be a good addition of what these are doing (add unit tests there).

Then, if having the IsCompilerDirective tokens showed as a win in the updated unit tests, we can

IsPreprocessorKeyword(...) {
...
 switch (token_type) {
   ...
  default:
     return IsCompilerDirective(token_type);
 }
}

If not, users of this function can be manually specific if they need both, e.g. by saying IsPreprocessorKeyword(foo) || IsCompilerDirective(foo)

[kbrunham-intel]

Hi @hzeller.

I've made the changes recommended. Please let me know if you have any other feedback or areas of improvement.

[kbrunham-intel]

Hi @hzeller,

Please let me know if there is any additional feedback I could follow.

Thanks.

[hzeller]

Thanks!