Add support for solcjs (npm-installed Solidity compiler) for cross-architecture compatibility

[dguido]

Support for solcjs (npm-installed Solidity compiler) in crytic-compile

Background

The Solidity compiler ecosystem has two main distribution methods:

1. Native binaries (solc) - Platform-specific executables distributed by the Ethereum Foundation
2. JavaScript/WASM version (solcjs) - Cross-platform compiler distributed via npm

Currently, crytic-compile only supports native solc binaries and fails when encountering solcjs, limiting its use in cross-architecture environments where native binaries are unavailable.

The Problem

Architecture Compatibility Challenge

Native Solidity binaries have limited architecture support:

• x86_64: Full support for all versions
• ARM64 (Apple Silicon, AWS Graviton, etc.): Limited or no support for older versions
• Other architectures: Generally unsupported

This creates problems for:

• Developers on Apple Silicon Macs
• CI/CD pipelines running on ARM64 instances
• Docker containers targeting multiple architectures
• Projects requiring older Solidity versions (e.g., 0.4.x) on non-x86_64 platforms

Current Failure Mode

When crytic-compile encounters solcjs instead of native solc, it fails with:

crytic_compile.platform.exceptions.InvalidCompilation: Invalid solc compilation /usr/lib/node_modules/solc/soljson.js:1
var Module;if(!Module)Module=...

This happens because:

1. solcjs is actually a Node.js script, not a native binary
2. crytic-compile uses subprocess.Popen with executable=shutil.which(cmd[0]) to invoke solc
3. Without a shell, the Node.js shebang (#!/usr/bin/env node) isn't processed
4. The JavaScript content is returned as stdout instead of being executed

Affected Code Locations

The issue manifests in multiple places:

• crytic_compile/platform/solc.py:385: subprocess.Popen(..., executable=shutil.which(cmd[0]))
• crytic_compile/platform/solc.py:556-565: Similar subprocess invocations
• crytic_compile/utils/subprocess.py:35-50: Common subprocess utility

Proposed Solution

Option 1: Add Native solcjs Support (Recommended)

Create a new platform type SolcJS that:

1. Detects when solcjs is available instead of solc
2. Handles the different CLI interface (solcjs uses different flags than solc)
3. Translates between command-line formats

Implementation approach:

class SolcJS(AbstractPlatform):
    NAME = "solcjs"
    TYPE = Type.SOLCJS
    
    @staticmethod
    def is_supported(target: str, **kwargs: str) -> bool:
        # Check if solcjs is available and target is .sol file
        solcjs_path = shutil.which("solcjs")
        if solcjs_path and target.endswith(".sol"):
            # Verify it's actually solcjs not native solc
            return _is_nodejs_script(solcjs_path)
        return False
    
    def compile(self, crytic_compile: "CryticCompile", **kwargs: str) -> None:
        # Translate solc arguments to solcjs format
        # e.g., --combined-json abi,bin → --abi --bin + manual JSON assembly
        # Invoke with proper Node.js execution

Benefits:

• Clean abstraction aligned with crytic-compile's architecture
• Automatic fallback when native solc unavailable
• Transparent to downstream tools (Slither, Manticore, etc.)

Option 2: Modify Existing Solc Platform

Enhance the existing Solc platform to detect and handle both native and JS versions:

def _run_solc(...):
    # Check if solc is actually solcjs
    if _is_solcjs(solc_path):
        return _run_solcjs(...)
    else:
        # Existing native solc logic

Benefits:

• Smaller change footprint
• Single platform for all direct Solidity compilation

Drawbacks:

• Mixes two different compiler interfaces in one class
• More complex error handling

Option 3: Fix subprocess invocation

Modify subprocess calls to use shell=True or explicitly invoke through Node.js:

# Instead of:
subprocess.Popen(cmd, executable=shutil.which(cmd[0]))

# Use:
subprocess.Popen(cmd, shell=True)  # Let shell handle shebang
# OR
subprocess.Popen(["node", solc_path] + cmd[1:])  # Explicit Node.js

Drawbacks:

• Security implications of shell=True
• Doesn't address CLI incompatibilities between solc and solcjs

CLI Differences to Handle

The main differences between solc and solcjs:

Feature	solc	solcjs
Combined JSON	--combined-json abi,bin	Not supported (must use --abi --bin separately)
Output format	Direct JSON output	Separate files or stdout
Remappings	--remap	Different format
Import paths	--allow-paths	--base-path and --include-path

Testing Requirements

1. Unit tests: Mock solcjs responses for different Solidity versions
2. Integration tests: Test with actual npm-installed solcjs
3. Cross-platform CI: Test on x86_64 and ARM64 architectures
4. Version compatibility: Test with solcjs versions matching solc 0.4.x through 0.8.x

Migration Path

1. Phase 1: Implement solcjs support behind a feature flag
2. Phase 2: Auto-detect and use solcjs when native solc unavailable
3. Phase 3: Document usage and limitations
4. Phase 4: Consider deprecating workarounds in downstream projects

Alternative Workarounds (Current State)

Projects currently work around this limitation by:

1. Creating wrapper scripts that translate between solc and solcjs
2. Using Docker images with pre-installed native binaries (x86_64 only)
3. Skipping tests on unsupported architectures
4. Using remote compilation services

None of these are ideal solutions.

Related Issues

• solc-select doesn't support ARM64 Linux binaries
• Foundry increasingly used in projects, which has better cross-platform support
• Growing adoption of ARM64 in cloud (AWS Graviton) and development (Apple Silicon)

Impact

Adding solcjs support would:

• Enable crytic-compile on all architectures
• Remove a significant barrier for Apple Silicon developers
• Reduce Docker image complexity
• Enable older Solidity version support on modern hardware
• Improve CI/CD flexibility with ARM64 instances

Recommended Approach

We recommend Option 1 (Native solcjs Support) because:

1. It aligns with crytic-compile's existing architecture
2. Provides clean separation of concerns
3. Allows for proper testing and maintenance
4. Enables gradual rollout and testing

Next Steps

1. Validate the proposed approach with maintainers
2. Implement prototype of SolcJS platform
3. Test with real-world projects using various Solidity versions
4. Document limitations and usage
5. Coordinate with downstream projects for testing

Code References

Key files that need modification:

• /crytic_compile/platform/solcjs.py (new file)
• /crytic_compile/platform/all_platforms.py:19 (add import)
• /crytic_compile/platform/types.py (add SOLCJS type)
• /crytic_compile/crytic_compile.py:600-618 (platform selection logic)

Use Case Example

This issue was discovered while trying to run Manticore tests in Docker on ARM64. The Dockerfile installs solcjs via npm since native solc 0.4.25 isn't available for ARM64, but crytic-compile fails to recognize and properly execute solcjs, making cross-architecture testing impossible.

Conclusion

Supporting solcjs would significantly improve crytic-compile's usability across different architectures and environments. While there are multiple approaches, adding proper solcjs support as a new platform provides the cleanest and most maintainable solution.