[Bug-Candidate]: Fail to scan in a project that supports both Hardhat and Foundry, keeps deleting the `build-info` folder when `slither .` is used

[SquilliamX]

Describe the issue:

I have a project that uses both foundry and hardhat. Slither throws an error whenever I run it slither .

I have already upgraded slither with using python3 -m pip install slither-analyzer -U

When I run slither . it throws this error:

'forge clean' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
'forge config --json' running
'forge build --build-info --skip */test/** */scripts/** --force' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
Traceback (most recent call last):
  File "/home/smartchain/.local/bin/slither", line 8, in <module>
    sys.exit(main())
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 753, in main
    main_impl(all_detector_classes=detectors, all_printer_classes=printers)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 859, in main_impl
    ) = process_all(filename, args, detector_classes, printer_classes)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 96, in process_all
    compilations = compile_all(target, **vars(args))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 722, in compile_all
    compilations.append(CryticCompile(target, **kwargs))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 211, in __init__
    self._compile(**kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 633, in _compile
    self._platform.compile(self, **kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/foundry.py", line 86, in compile
    hardhat_like_parsing(crytic_compile, self._target, build_directory, self._target)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 52, in hardhat_like_parsing
    raise InvalidCompilation(txt)
crytic_compile.platform.exceptions.InvalidCompilation: Compilation failed. Can you run build command?
out/build-info is not a directory.

Now i thought this was because of my foundry.toml which is here:

[profile.default]
solc_version="0.8.25"
evm_version = "cancun"
optimizer = true
optimizer_runs = 200
revert_strings = "strip"
isolate = true

ffi = true
src = "test"
out = "artifacts/foundry"
script = "scripts"
cache_path = "cache/foundry"
fs_permissions = [{ access = "read", path = "./deployments" }]
verbosity = 3

[fmt]
tab_width = 2
bracket_spacing = true
number_underscore = "thousands"

[profile.production.fuzz]
runs = 66_666

[profile.overkill.fuzz]
runs = 6_666_666

and I thought the out = "artifacts/foundry" line was causing the error, but this is not the case. I changed the out = "artifacts/foundry" to out = "out" and then ran forge build and then slither . it still throws an error. Now this error is very weird because when I compile the project with forge build, it creates the out folder with all the data inside of it. But when i run slither . right after, it deletes everything in the out folder and says it is empty! The error this is:

$ slither .
'forge clean' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
'forge config --json' running
'forge build --build-info --skip */test/** */scripts/** --force' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
Traceback (most recent call last):
  File "/home/smartchain/.local/bin/slither", line 8, in <module>
    sys.exit(main())
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 753, in main
    main_impl(all_detector_classes=detectors, all_printer_classes=printers)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 859, in main_impl
    ) = process_all(filename, args, detector_classes, printer_classes)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 96, in process_all
    compilations = compile_all(target, **vars(args))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 722, in compile_all
    compilations.append(CryticCompile(target, **kwargs))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 211, in __init__
    self._compile(**kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 633, in _compile
    self._platform.compile(self, **kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/foundry.py", line 86, in compile
    hardhat_like_parsing(crytic_compile, self._target, build_directory, self._target)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 52, in hardhat_like_parsing
    raise InvalidCompilation(txt)
crytic_compile.platform.exceptions.InvalidCompilation: Compilation failed. Can you run build command?
out/build-info is not a directory.

I searched the issues in the slither repo and I found that i could run this command in my terminal: slither . --compile-force-framework hardhat --ignore-compile

However this throws another error. the error this time is

slither . --compile-force-framework hardhat --ignore-compile
Problem deserializing hardhat configuration, using defaults: Expecting value: line 2 column 1 (char 1)
Traceback (most recent call last):
  File "/home/smartchain/.local/bin/slither", line 8, in <module>
    sys.exit(main())
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 753, in main
    main_impl(all_detector_classes=detectors, all_printer_classes=printers)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 859, in main_impl
    ) = process_all(filename, args, detector_classes, printer_classes)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 96, in process_all
    compilations = compile_all(target, **vars(args))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 722, in compile_all
    compilations.append(CryticCompile(target, **kwargs))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 211, in __init__
    self._compile(**kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 633, in _compile
    self._platform.compile(self, **kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 183, in compile
    hardhat_like_parsing(crytic_compile, self._target, build_directory, hardhat_working_dir)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 52, in hardhat_like_parsing
    raise InvalidCompilation(txt)
crytic_compile.platform.exceptions.InvalidCompilation: Compilation failed. Can you run build command?
/home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol/artifacts/build-info is not a directory.

The last line in the error says artifacts/build-info is not a directory, which is true because in my artifacts folder there is two sub folders, foundry and hardhat with data inside of each sub-folder. However when I change my foundry.toml to out = "artifacts" , I run forge build and then run slither . --compile-force-framework hardhat --ignore-compile , it throws another error:

$ slither . --compile-force-framework hardhat --ignore-compile
Problem deserializing hardhat configuration, using defaults: Expecting value: line 2 column 1 (char 1)
Traceback (most recent call last):
  File "/home/smartchain/.local/bin/slither", line 8, in <module>
    sys.exit(main())
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 753, in main
    main_impl(all_detector_classes=detectors, all_printer_classes=printers)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 859, in main_impl
    ) = process_all(filename, args, detector_classes, printer_classes)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 96, in process_all
    compilations = compile_all(target, **vars(args))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 722, in compile_all
    compilations.append(CryticCompile(target, **kwargs))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 211, in __init__
    self._compile(**kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 633, in _compile
    self._platform.compile(self, **kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 183, in compile
    hardhat_like_parsing(crytic_compile, self._target, build_directory, hardhat_working_dir)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 52, in hardhat_like_parsing
    raise InvalidCompilation(txt)
crytic_compile.platform.exceptions.InvalidCompilation: Compilation failed. Can you run build command?
/home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol/artifacts/build-info is not a directory.

Now this is very interesting because after I run forge build with my foundry.toml with out = "artifacts" it created the build-info folder, but when i run slither . --compile-force-framework hardhat --ignore-compile , it deletes the entire build-info in my artifacts folder.

Code example to reproduce the issue:

The github repo is here: https://github.com/exactly/protocol

Version:

0.10.2

[0xalpharush]

What version of foundry are you using? Alos, does slither . --compile-force-framework hardhat without --ignore compile work?

[SquilliamX]

What version of foundry are you using? Alos, does slither . --compile-force-framework hardhat without --ignore compile work?

this is my version of foundry: forge 0.2.0 (d431f74 2024-04-27T00:17:17.280985407Z)

and no slither . --compile-force-framework hardhat does not work, when i run this command, below is my output:

'npx hardhat clean' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
'npx hardhat clean --global' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
Problem deserializing hardhat configuration, using defaults: Expecting value: line 2 column 1 (char 1)
'npx hardhat compile --force' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
Traceback (most recent call last):
  File "/home/smartchain/.local/bin/slither", line 8, in <module>
    sys.exit(main())
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 753, in main
    main_impl(all_detector_classes=detectors, all_printer_classes=printers)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 859, in main_impl
    ) = process_all(filename, args, detector_classes, printer_classes)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 96, in process_all
    compilations = compile_all(target, **vars(args))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 722, in compile_all
    compilations.append(CryticCompile(target, **kwargs))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 211, in __init__
    self._compile(**kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 633, in _compile
    self._platform.compile(self, **kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 183, in compile
    hardhat_like_parsing(crytic_compile, self._target, build_directory, hardhat_working_dir)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 52, in hardhat_like_parsing
    raise InvalidCompilation(txt)
crytic_compile.platform.exceptions.InvalidCompilation: Compilation failed. Can you run build command?
/home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol/artifacts/build-info is not a directory.

[elopez]

I am able to successfully build and analyze this repo (https://github.com/sherlock-audit/2024-04-interest-rate-model) as follows, using forge:

1. install npm dependencies (otherwise forge build fails)
2. run slither with foundry-out-directory pointing to the correct artifacts, and foundry-compile-all (see note 1 below)

npm i
slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all

It also works if you use hardhat as follows:

1. install npm dependencies (otherwise hardhat build fails)
2. run slither with hardhat-artifacts-directory pointing to the correct artifacts (see note 2 below)

npm i
slither . --compile-force-framework hardhat --hardhat-artifacts-directory artifacts/hardhat/

Note 1: if you use the default arguments that skip tests and scripts, foundry for some reason won't build the repo, it will produce zero artifacts, and yet claim all is good and up to date. It can be worked around by using foundry-compile-all -- maybe this is something worth reporting to the Foundry team? Interestingly, it only happens if you have two skip paths, but not with just one.

% forge clean; forge build --build-info --skip '*/test/**' '*/scripts/**' 
[⠊] Compiling...
No files changed, compilation skipped

% forge clean; forge build --build-info --skip '*/test/**'                
[⠊] Compiling...
[⠃] Compiling 49 files with 0.8.25
[⠢] Solc 0.8.25 finished in 2.93s
Compiler run successful!

% forge clean; forge build --build-info --skip '*/scripts/**'  
[⠊] Compiling...
[⠔] Compiling 114 files with 0.8.25
[⠢] Solc 0.8.25 finished in 24.51s
Compiler run successful with warnings:
...

% forge clean; forge build --build-info                                   
[⠊] Compiling...
[⠆] Compiling 125 files with 0.8.25
[⠊] Solc 0.8.25 finished in 25.32s
Compiler run successful with warnings:
...

% forge clean; forge build --build-info --skip '*/test/**' '*/scripts/**' 
[⠊] Compiling...
No files changed, compilation skipped

% forge clean; forge build --build-info --skip '*/test/**' --skip '*/scripts/**' 
[⠊] Compiling...
No files changed, compilation skipped

Note 2: normally, the Hardhat artifacts directory is auto-detected by extracting the config via the hardhat console, but in this case the console unconditionally prints a table every time it's opened, so the autodetection breaks (Problem deserializing hardhat configuration...). However, it can still be manually configured with the hardhat-artifacts-directory flag.

[SquilliamX]

slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all

yeah it doesnt work for me. I build the project and create the build-info files but when i run slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all it deletes all the build-info files and tells me to run the build command again. It's a loop.

[elopez]

@SquilliamX can you post the log of running the following commands?

slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all

forge clean
forge build --build-info 

[SquilliamX]

@SquilliamX can you post the log of running the following commands?

slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all

forge clean
forge build --build-info 

when i run these commands, these are my outputs:

slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all:

 slither . --foundry-out-directory artifacts/foundry/ --foundry-compile-all
'forge clean' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
'forge build --build-info' running (wd: /home/smartchain/code/audits/2024-04-interest-rate-model-SquilliamX/protocol)
Traceback (most recent call last):
  File "/home/smartchain/.local/bin/slither", line 8, in <module>
    sys.exit(main())
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 753, in main
    main_impl(all_detector_classes=detectors, all_printer_classes=printers)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 859, in main_impl
    ) = process_all(filename, args, detector_classes, printer_classes)
  File "/home/smartchain/.local/lib/python3.10/site-packages/slither/__main__.py", line 96, in process_all
    compilations = compile_all(target, **vars(args))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 722, in compile_all
    compilations.append(CryticCompile(target, **kwargs))
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 211, in __init__
    self._compile(**kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/crytic_compile.py", line 633, in _compile
    self._platform.compile(self, **kwargs)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/foundry.py", line 86, in compile
    hardhat_like_parsing(crytic_compile, self._target, build_directory, self._target)
  File "/home/smartchain/.local/lib/python3.10/site-packages/crytic_compile/platform/hardhat.py", line 52, in hardhat_like_parsing
    raise InvalidCompilation(txt)
crytic_compile.platform.exceptions.InvalidCompilation: Compilation failed. Can you run build command?
artifacts/foundry/build-info is not a directory.

forge clean: does not return anything

forge build --build-info:

forge build --build-info
[⠢] Compiling...
[⠒] Compiling 129 files with 0.8.25
[⠘] Solc 0.8.25 finished in 80.98s
Compiler run successful with warnings:
Warning (2018): Function state mutability can be restricted to view
  --> test/DebtPreviewer.t.sol:93:3:
   |
93 |   function testPreviewEmptyLeverage() external {
   |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:181:3:
    |
181 |   function testPreviewLeverageBalancerAvailableLiquidity() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:282:3:
    |
282 |   function testLeverageRatesCrossAsset() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:311:3:
    |
311 |   function testLeverageRatesSameAsset() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:352:3:
    |
352 |   function testLeverageRatesZeroPrincipalCrossAsset() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:373:3:
    |
373 |   function testLeverageRatesZeroPrincipalSameAsset() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:389:3:
    |
389 |   function testLeverageRatesWithNativeBorrow() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/DebtPreviewer.t.sol:409:3:
    |
409 |   function testLeverageRatesWithNegativeNativeResult() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to pure
  --> test/FixedLib.t.sol:70:3:
   |
70 |   function testMaturityRangeLimit() external {
   |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
    --> test/Market.t.sol:2717:3:
     |
2717 |   function testInitiallyUnfrozen() external {
     |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/Previewer.t.sol:192:3:
    |
192 |   function testPreviewDepositAtMaturityWithEmptyMaturity() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/Previewer.t.sol:197:3:
    |
197 |   function testPreviewDepositAtMaturityWithEmptyMaturityAndZeroAmount() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/Previewer.t.sol:202:3:
    |
202 |   function testPreviewDepositAtMaturityWithInvalidMaturity() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
    --> test/Previewer.t.sol:1501:3:
     |
1501 |   function testExactlyReturningInterestRateModelData() external {
     |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
    --> test/Previewer.t.sol:2170:3:
     |
2170 |   function testAccountsWithEmptyAccount() external {
     |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
  --> test/PriceFeedPool.t.sol:23:3:
   |
23 |   function testPriceFeedPoolReturningPrice() external {
   |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
  --> test/PriceFeedWrapper.t.sol:29:3:
   |
29 |   function testPriceFeedWrapperReturningPrice() external {
   |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
   --> test/Protocol.t.sol:631:3:
    |
631 |   function checkInvariants() internal {
    |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to view
  --> test/VotePreviewer.t.sol:37:3:
   |
37 |   function testExternalVotes() external {
   |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to pure
  --> test/exploit.t.sol:92:3:
   |
92 |   function testForcedTypeCastingOverflow() external {
   |   ^ (Relevant source part starts here and spans across multiple lines).

Warning (2018): Function state mutability can be restricted to pure
   --> test/exploit.t.sol:109:3:
    |
109 |   function testTimeUnitsOverflow() external {
    |   ^ (Relevant source part starts here and spans across multiple lines).

[elopez]

@SquilliamX do you still have foundry configured to write artifacts to artifacts/foundry/? are there any files on artifacts/foundry/build-info/ after running forge clean; forge build --build-info ? can you post the output of forge config?