Nested Contracts cannot import from `source-dir` only from their directory

[charlescrain]

Version

$ yarn chanterelle --version
yarn run v1.19.2
warning ../package.json: No license field
$ /Users/charlescrain/dev/pixura-contracts/node_modules/.bin/chanterelle --version
Chanterelle 4.0.0
Default Solc 0.5.16+commit.9c3226ce.Emscripten.clang

Example project structure:

chanterelle.json -- assume `source-dir: contracts/`
contracts/ 
   -> Test1.sol
   -> Test/Test2.sol

Consider Test1.sol:

pragma solidity ^0.4.24;

contract Test1 {
    function test1() public view returns (string) {
        return "test1";
    }
}

Both of the following contracts fail with the same error:

Attempt 1

pragma solidity ^0.4.24;

import "../Test1.sol";

contract Test2 {
    function test2() public view returns (string) {
        return "test2";
    }
}

Error for first try:

Test2.sol:3:1: ParserError: Source "Test1.sol" not found: Error: ENOENT: no such file or directory, open '/Users/charlescrain/dev/pixura-contracts/contracts/Test/Test1.sol'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at /Users/charlescrain/dev/pixura-contracts/output/Node.FS.Sync/index.js:43:23
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Effect.Exception/foreign.js:37:16
    at __do (/Users/charlescrain/dev/pixura-contracts/output/Chanterelle.Internal.Compile/index.js:190:155)
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Language.Solidity.Compiler/foreign.js:113:41
import "../Test1.sol";
^--------------------^

Attempt 2

pragma solidity ^0.4.24;

import "Test1.sol";

contract Test2 {
    function test2() public view returns (string) {
        return "test2";
    }
}

Error for attempt 2:

Test2.sol:3:1: ParserError: Source "Test1.sol" not found: Error: ENOENT: no such file or directory, open '/Users/charlescrain/dev/pixura-contracts/contracts/Test/Test1.sol'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at /Users/charlescrain/dev/pixura-contracts/output/Node.FS.Sync/index.js:43:23
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Effect.Exception/foreign.js:37:16
    at __do (/Users/charlescrain/dev/pixura-contracts/output/Chanterelle.Internal.Compile/index.js:190:155)
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Language.Solidity.Compiler/foreign.js:113:41
import "Test1.sol";
^-----------------^

[iostat]

Interesting... attempt 1 should've worked. Just out of curiousity, can you try import ../../Test.sol? I wonder if ../ is being expanded relative to the filename somewhere (i.e., contracts/Test2/Test.sol/../Test1.sol, leading to contracts/Test2/Test.sol), instead of the directory the file is in contracts/Test2/../Test1.sol which would resolve correctly to contracts/Test1.sol.

Attempt 2's failure makes perfect sense.

[iostat]

Also chanterelle version please.

[charlescrain]

@iostat updated with version it's Chanterelle 4.0.0

here's output using import ../../Test1.sol

Test2.sol:3:1: ParserError: Source "Test1.sol" not found: Error: ENOENT: no such file or directory, open '/Users/charlescrain/dev/pixura-contracts/contracts/Test/Test1.sol'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at /Users/charlescrain/dev/pixura-contracts/output/Node.FS.Sync/index.js:43:23
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Effect.Exception/foreign.js:37:16
    at __do (/Users/charlescrain/dev/pixura-contracts/output/Chanterelle.Internal.Compile/index.js:190:155)
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Language.Solidity.Compiler/foreign.js:113:41
import "../../Test1.sol";
^-----------------------^

[charlescrain]

@iostat It appears to ignore all ../. I moved them to the same directory was able to make it compile even though I left the import "../../Test1.sol";

[iostat]

Do you have a solc-version specified in chanterelle.json? Try overriding it to 0.4.24, cause it almost seems to me like something that solc is doing. We just use the read file callback that's in the solc API, where it requests a path and iirc the only substitution we do is make it relative to the Chanterelle project directory/source-dir.

It's also possible that the logic in

chanterelle/src/Chanterelle/Internal/Compile.purs

Lines 121 to 139 in f315d39

	-- | load a file when solc requests it
	-- | TODO: secure it so that it doesnt try loading crap like /etc/passwd, etc. :P
	-- | TODO: be more clever about dependency resolution, that way we don't even have to do
	-- | any remappings!
	loadSolcCallback
	:: ChanterelleModule
	-> FilePath
	-> ChanterelleProjectSpec
	-> String
	-> Effect (Either String String)
	loadSolcCallback (ChanterelleModule mod) root (ChanterelleProjectSpec project) filePath = do
	let modRoot = Path.dirname mod.solPath
	isAbs = Path.isAbsolute filePath
	modRootWithoutRoot = fromMaybe modRoot $ stripPrefix (Pattern root) modRoot
	fullPath = if isAbs
	then filePath
	else Path.normalize (Path.concat [root, modRootWithoutRoot, filePath])
	log Debug ("Root: " <> root <> " :: modRoot: " <> modRoot <> " :: Solc load: " <> filePath <> " -> " <> fullPath)
	catchException (pure <<< Left <<< show) (Right <$> (FSS.readTextFile UTF8 fullPath))

is unsound. Can you rerun with chanterelle -v Debug compile?

[charlescrain]

@iostat Yes it's specified at 0.4.24 in the chanterelle.json. here's the output

020-03-11T18:37:46.923Z [INFO] compiling contract module Test1
2020-03-11T18:37:46.931Z [WARN] Solidity compiler warning in module Test1:
Test1.sol:4:5: Warning: Function state mutability can be restricted to pure
    function test1() public view returns (string) {
    ^ (Relevant source part starts here and spans across multiple lines).

2020-03-11T18:37:46.933Z [DEBUG] path /Users/charlescrain/dev/pixura-contracts/contracts/build/contracts exists and is a directory
2020-03-11T18:37:46.936Z [INFO] compiling contract module Test/Test2
2020-03-11T18:37:46.937Z [DEBUG] Root: /Users/charlescrain/dev/pixura-contracts :: modRoot: /Users/charlescrain/dev/pixura-contracts/contracts/solidity/Test :: Solc load: Test1.sol -> /Users/charlescrain/dev/pixura-contracts/contracts/solidity/Test/Test1.sol
2020-03-11T18:37:46.938Z [ERROR] Solidity compiler error in module Test/Test2:
Test/Test2.sol:3:1: ParserError: Source "Test1.sol" not found: Error: ENOENT: no such file or directory, open '/Users/charlescrain/dev/pixura-contracts/contracts/solidity/Test/Test1.sol'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at /Users/charlescrain/dev/pixura-contracts/output/Node.FS.Sync/index.js:43:23
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Effect.Exception/foreign.js:37:16
    at __do (/Users/charlescrain/dev/pixura-contracts/output/Chanterelle.Internal.Compile/index.js:190:155)
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:16
    at /Users/charlescrain/dev/pixura-contracts/output/Effect/foreign.js:12:20
    at /Users/charlescrain/dev/pixura-contracts/output/Language.Solidity.Compiler/foreign.js:113:41
import "../Test1.sol";
^--------------------^

The line:

2020-03-11T18:37:46.937Z [DEBUG] Root: /Users/charlescrain/dev/pixura-contracts :: modRoot: /Users/charlescrain/dev/pixura-contracts/contracts/solidity/Test :: Solc load: Test1.sol -> /Users/charlescrain/dev/pixura-contracts/contracts/solidity/Test/Test1.sol

Says Test is the modRoot, could that be the issue?

[iostat]

What stands out to me most is that solc is specifically requesting that we give it Test1.sol, instead of ../Test1.sol (Hence the log output Solc load: Test1.sol -> ....). So in principle I’m going to say this is a solc bug — though it can construed as a feature to avoid leaking stuff like requesting ../../../../etc/passwd), but I’m not going to discount trying to add a few options to solc-options in chanterelle.json to try and work around it (especially if it’s actually a feature). Can you try with newer or older versions of solc and see if they do the same truncation behavior?

[iostat]

Test as the modRoot makes sense, as Test/Test2.sol is the module requesting an import

[charlescrain]

Tested with latest version and no longer an issue

[charlescrain]

My bad I didn't fully recreate the test. This is still a problem.