Bug: sam local invoke - FileExistsError

[RB-Lab]

This issue may be related: #3184

Description:

I am running Java 17 Lambda function locally and I am getting the [Errno 17] File exists. As far, as I understand, the problem is caused by license file named LICENSE, and a folder named license in META-INF/ folder inside the .jar file. .jar file system is case-sensitive, and Mac OS FS is case-insensitive, so when SAM tries to unpack files, it cannot create a folder, because file LICENSE already exists.

Steps to reproduce:

Here's my pom.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.big-corp</groupId>
    <artifactId>our_amasing_project</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <maven.shade.plugin.version>3.2.1</maven.shade.plugin.version>
        <maven.compiler.plugin.version>3.6.1</maven.compiler.plugin.version>
        <exec-maven-plugin.version>1.6.0</exec-maven-plugin.version>
        <aws.java.sdk.version>2.20.17</aws.java.sdk.version>
        <aws.lambda.java.version>1.2.0</aws.lambda.java.version>
        <junit5.version>5.8.1</junit5.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>${aws.java.sdk.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <!-- AWS SDK -->
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>software.amazon.awssdk</groupId>
                    <artifactId>netty-nio-client</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>software.amazon.awssdk</groupId>
                    <artifactId>apache-client</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>secretsmanager</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>software.amazon.awssdk</groupId>
                    <artifactId>netty-nio-client</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>software.amazon.awssdk</groupId>
                    <artifactId>apache-client</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>url-connection-client</artifactId>
        </dependency>

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-core</artifactId>
            <version>${aws.lambda.java.version}</version>
        </dependency>


        <!-- Azure SDK -->
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-identity</artifactId>
            <version>1.5.0</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.graph</groupId>
            <artifactId>microsoft-graph</artifactId>
            <version>5.22.0</version>
        </dependency>

        <!-- Utils -->
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.15.2</version>
        </dependency>


        <!-- Test Dependencies -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>${junit5.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${maven.compiler.plugin.version}</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>${maven.shade.plugin.version}</version>
                <configuration>
                    <createDependencyReducedPom>false</createDependencyReducedPom>
                    <finalName>email_to_case_lambdas</finalName>
                    <filters>
                        <filter>
                            <artifact>*:*</artifact>
                            <excludes>
                                <!-- Suppress module-info.class warning-->
                                <exclude>module-info.class</exclude>
                            </excludes>
                        </filter>
                    </filters>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Observed result:

Error: [Errno 17] File exists: '/var/folders/7k/1jpr7ybs15x1ly785348rjgh0000gp/T/tmp48bohh4f/META-INF/license'
Traceback:
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/decorators.py", line 93, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 184, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 149, in wrapped
    return_value = func(*args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/lib/utils/version_checker.py", line 42, in wrapped
    actual_result = func(*args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/cli/main.py", line 95, in wrapper
    return func(*args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/commands/local/invoke/cli.py", line 100, in cli
    do_cli(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/commands/local/invoke/cli.py", line 206, in do_cli
    context.local_lambda_runner.invoke(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/commands/local/lib/local_lambda.py", line 144, in invoke
    self.local_runtime.invoke(
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 324, in wrapped_func
    return_value = func(*args, **kwargs)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/local/lambdafn/runtime.py", line 189, in invoke
    container = self.create(function_config, debug_context, container_host, container_host_interface)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/local/lambdafn/runtime.py", line 72, in create
    code_dir = self._get_code_dir(function_config.code_abs_path)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/local/lambdafn/runtime.py", line 290, in _get_code_dir
    decompressed_dir: str = _unzip_file(code_path)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/local/lambdafn/runtime.py", line 524, in _unzip_file
    unzip(filepath, temp_dir)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/local/lambdafn/zip.py", line 92, in unzip
    extracted_path = _extract(file_info, output_dir, zip_ref)
  File "/opt/homebrew/Cellar/aws-sam-cli/1.90.0/libexec/lib/python3.8/site-packages/samcli/local/lambdafn/zip.py", line 55, in _extract
    return zip_ref.extract(file_info, output_dir)
  File "/opt/homebrew/Cellar/python@3.8/3.8.17_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/zipfile.py", line 1630, in extract
    return self._extract_member(member, path, pwd)
  File "/opt/homebrew/Cellar/python@3.8/3.8.17_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/zipfile.py", line 1697, in _extract_member
    os.mkdir(targetpath)

An unexpected error was encountered while executing "sam local invoke".

Expected result:

SAM RUNS

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Mac OS Ventura 13.4.1
2. sam --version: SAM CLI, version 1.90.0
3. AWS region: any

sam --info
{
  "version": "1.90.0",
  "system": {
    "python": "3.8.17",
    "os": "macOS-13.4.1-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.2",
    "aws_cdk": "2.80.0 (build bbdb16a)",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Here's content of META-INF folder inside the jar:

[RB-Lab]

A workaround is to add a filter to shade plugin:

<filter>
    <artifact>*:*</artifact>
    <excludes>
        <exclude>META-INF/*.txt</exclude>
        <exclude>META-INF/*.MD</exclude>
        <exclude>META-INF/LICENSE*</exclude>
        <exclude>META-INF/NOTICE*</exclude>
    </excludes>
</filter>

[hawflau]

@RB-Lab Thanks for raising the issue and providing a workaround. We will try to reproduce the issue.