Description
Describe the bug
Scanning Excalidraw (https://github.com/excalidraw/excalidraw) results in an error and no ort output.
To Reproduce
Steps to reproduce the behavior:
- Check out repository for excalidraw at following link 'https://github.com/excalidraw/excalidraw/releases/tag/v0.17.3'
- Run Analyzer from dockerfile built from ORT master (no issues), via:
docker run -v ~/gitRepos/ort_output/:/home/ort/.ort/config/ -v ~/gitRepos/excalidraw/:/download -v ~/gitRepos/ort_output/ort_excalidraw/analyze/:/analyze ort:latest analyze -i /download -o/analyze
Config is just
ort:
analyzer:
allowDynamicVersions: true
- Finally do run the scanner:
docker run --network=host -e ORT_CONFIG_DIR=/config -v ~/gitRepos/ort_output/ort_excalidraw/:/config/ -v ~/gitRepos/ort_output/ort_excalidraw/analyze/:/analyze -v ~/gitRepos/ort_output/ort_excalidraw/scan/:/scan ort:latest --info scan -i /analyze/analyzer-result.yml -o/scan
- See error
I also tried adding options:
-e "JAVA_OPTS=-Xms512M -Xmx10g -XX:MaxMetaspaceSize=6g -Dkotlin.daemon.jvm.options='-Xmx16g'"
--memory="32g"
and various combinations of sizes, that seemed to not help.
I also tried using a postgres database which significantly reduced scan time, however the error remains the same.
Expected behavior
No error.
Console / log output
09:19:42.855 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git checkout 0136e8a05ef864666a6e95b0de8b679b0bc93338' in '/tmp/ort-DefaultWorkingTreeCache18354474298420537781'...
Exception in thread "main" java.nio.file.NoSuchFileException: /tmp/ort-DefaultWorkingTreeCache18354474298420537781/.git/gc.log.lock
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:171)
at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.base/java.nio.file.Files.readAttributes(Files.java:1853)
at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:220)
at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:277)
at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:374)
at java.base/java.nio.file.Files.walkFileTree(Files.java:2820)
at kotlin.io.path.PathsKt__PathUtilsKt.visitFileTree(PathUtils.kt:1147)
at kotlin.io.path.PathsKt__PathUtilsKt.visitFileTree(PathUtils.kt:1201)
at kotlin.io.path.PathsKt__PathUtilsKt.visitFileTree$default(PathUtils.kt:1193)
at kotlin.io.path.PathsKt__PathRecursiveFunctionsKt.copyToRecursively(PathRecursiveFunctions.kt:234)
at kotlin.io.path.PathsKt__PathRecursiveFunctionsKt.copyToRecursively(PathRecursiveFunctions.kt:81)
at kotlin.io.path.PathsKt__PathRecursiveFunctionsKt.copyToRecursively$default(PathRecursiveFunctions.kt:72)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.downloadFromVcs$lambda$1(ProvenanceDownloader.kt:117)
at org.ossreviewtoolkit.downloader.DefaultWorkingTreeCache.use(WorkingTreeCache.kt:65)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.downloadFromVcs(ProvenanceDownloader.kt:104)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.access$downloadFromVcs(ProvenanceDownloader.kt:80)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader$download$1.invokeSuspend(ProvenanceDownloader.kt:96)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader$download$1.invoke(ProvenanceDownloader.kt)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader$download$1.invoke(ProvenanceDownloader.kt)
at org.ossreviewtoolkit.utils.ort.UtilsKt$runBlocking$1.invokeSuspend(Utils.kt:231)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at org.ossreviewtoolkit.utils.ort.UtilsKt.runBlocking(Utils.kt:231)
at org.ossreviewtoolkit.utils.ort.UtilsKt.runBlocking$default(Utils.kt:229)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.download(ProvenanceDownloader.kt:96)
at org.ossreviewtoolkit.scanner.provenance.ProvenanceDownloader$DefaultImpls.downloadRecursively(ProvenanceDownloader.kt:64)
at org.ossreviewtoolkit.scanner.provenance.DefaultProvenanceDownloader.downloadRecursively(ProvenanceDownloader.kt:80)
at org.ossreviewtoolkit.scanner.Scanner.createMissingArchives(Scanner.kt:708)
at org.ossreviewtoolkit.scanner.Scanner.scan(Scanner.kt:179)
at org.ossreviewtoolkit.scanner.Scanner$scan$3.invokeSuspend(Scanner.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:100)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:263)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:95)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:69)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:47)
at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.runScanners(ScannerCommand.kt:242)
at org.ossreviewtoolkit.plugins.commands.scanner.ScannerCommand.run(ScannerCommand.kt:148)
at com.github.ajalt.clikt.core.CoreCliktCommandKt.parse(CoreCliktCommand.kt:107)
at com.github.ajalt.clikt.core.CoreCliktCommandKt.main(CoreCliktCommand.kt:78)
at com.github.ajalt.clikt.core.CoreCliktCommandKt.main(CoreCliktCommand.kt:90)
at org.ossreviewtoolkit.cli.OrtMainKt.main(OrtMain.kt:88)
Environment
Output of the ort requirements -l commands
command:
09:42:44.616 [main] INFO org.ossreviewtoolkit.model.config.OrtConfiguration - Using ORT configuration file '/config/config.yml'.
Hoplite is configured to infer which sealed type to choose by inspecting the config values at runtime. This behaviour is now deprecated in favour of explicitly specifying the type through a discriminator field. In 3.0 this new behavior will become the default. To enable this behavior now (and disable this warning), invoke withExplicitSealedTypes() on the ConfigLoaderBuilder.
09:42:44.936 [main] INFO org.ossreviewtoolkit.utils.common.EnvironmentVariableFilter - EnvironmentVariableFilter initialized with denySubstrings = [key, pass, pwd, token, user] and allowNames = [CARGO_HTTP_USER_AGENT, COMPOSER_ALLOW_SUPERUSER, CONAN_LOGIN_ENCRYPTION_KEY, CONAN_LOGIN_USERNAME, CONAN_PASSWORD, CONAN_USERNAME, CONAN_USER_HOME, CONAN_USER_HOME_SHORT, DOTNET_CLI_CONTEXT_ANSI_PASS_THRU, GIT_ASKPASS, GIT_HTTP_USER_AGENT, GRADLE_USER_HOME, HACKAGE_USERNAME, HACKAGE_PASSWORD, HACKAGE_KEY, PWD, USER, USERPROFILE].
______________________________
/ \_______ \__ ___/ The OSS Review Toolkit, version DOCKER-SNAPSHOT
| | | | _/ | | built with JDK 21.0.5+11-LTS, running under Jav
| | | | | \ | | Executing 'requirements' as 'ort' on Linux
\________/ |____|___/ |____| with 16 CPUs and a maximum of 10240 MiB of memo
Environment variables:
ORT_CONFIG_DIR = /config
ORT_DATA_DIR = /home/ort/.ort
HOME = /home/ort
JAVA_HOME = /opt/java/openjdk
ANDROID_HOME = /opt/android-sdk
Looking for ORT configuration in the following file:
/config/config.yml
09:42:45.330 [main] INFO org.reflections.Reflections - Reflections took 270 ms to scan 94 urls, producing 155 keys and 1369 values
Scanners:
- Askalono: Requires 'askalono' in no specific version. Tool not found.
- BoyterLc: Requires 'lc' in no specific version. Tool not found.
- Licensee: Requires 'licensee' in no specific version. Tool not found.
09:42:45.599 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'scancode --version' in '/home/ort'...
* ScanCode: Requires 'scancode' in version >=30.0.0. Found version 32.3.1.
PackageManagers:
09:42:46.760 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'bazel --version' in '/home/ort'...
* BazelCommand: Requires 'bazel' in version >=7.0.0. Found version 7.0.1.
09:42:50.771 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'bower --version' in '/home/ort'...
* BowerCommand: Requires 'bower' in version >=1.8.8. Found version 1.8.14.
09:42:51.138 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'buildozer --version' in '/home/ort'...
+ BuildozerCommand: Requires 'buildozer' in no specific version. Found version redacted.
09:42:51.159 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'cargo --version' in '/home/ort'...
* CargoCommand: Requires 'cargo' in no specific version. Found version 1.84.0.
09:42:51.243 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'pod --version --allow-root' in '/home/ort'...
* CocoaPodsCommand: Requires 'pod' in version >=1.11.0. Found version 1.16.2.
09:42:51.915 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'composer --no-ansi --version' in '/home/ort'...
* ComposerCommand: Requires 'composer' in version >=1.5.0. Found version 2.8.4.
09:42:52.000 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'conan --version' in '/home/ort'...
* ConanCommand: Requires 'conan' in version >=1.44.0 and <2.0.0. Found version 1.64.1.
09:42:52.517 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'go version' in '/home/ort'...
* GoCommand: Requires 'go' in version >=1.21.1. Found version 1.23.0.
09:42:52.553 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'npm --version' in '/home/ort'...
* NpmCommand: Requires 'npm' in version >=6.0.0 and <11.0.0. Found version 10.9.2.
09:42:52.671 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'nuget-inspector --version' in '/home/ort'...
+ NuGetInspector: Requires 'nuget-inspector' in no specific version. Could not determine the version.
09:42:53.047 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'pipenv --version' in '/home/ort'...
* PipenvCommand: Requires 'pipenv' in version >=2018.10.9. Found version 2023.12.1.
09:42:53.679 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'pnpm --version' in '/home/ort'...
* PnpmCommand: Requires 'pnpm' in version >=5.0.0 and <10.0.0. Found version 9.9.0.
09:42:54.072 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'poetry --version' in '/home/ort'...
* PoetryCommand: Requires 'poetry' in no specific version. Found version 1.8.3.
09:42:54.505 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'dart --version' in '/home/ort'...
* Pub: Requires 'dart' in version >=2.10.0. Found version 2.18.4.
09:42:54.532 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'python-inspector --version' in '/home/ort'...
* PythonInspector: Requires 'python-inspector' in version >=0.9.2. Found version 0.10.0.
09:42:54.837 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'sbt --version' in '/home/ort'...
* SbtCommand: Requires 'sbt' in no specific version. Found version copying runtime jar...
sbt version in this project: 1.10.0
sbt script version: 1.10.0.
09:43:13.913 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'stack --version' in '/home/ort'...
* StackCommand: Requires 'stack' in version >=2.1.1. Found version 3.3.1.
09:43:13.953 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'swift --version' in '/home/ort'...
* SwiftCommand: Requires 'swift' in no specific version. Found version 5.10.1.
09:43:14.375 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'yarn --version' in '/home/ort'...
* YarnCommand: Requires 'yarn' in version >=1.3.0 and <1.23.0. Found version 1.22.22.
VersionControlSystems:
09:43:14.581 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git --version' in '/home/ort'...
* Git: Requires 'git' in version >=2.29.0. Found version 2.34.1.
09:43:14.595 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'repo --version' in '/home/ort'...
* GitRepo: Requires 'repo' in no specific version. Found version 2.50 (launcher).
09:43:14.672 [main] INFO org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'hg --version' in '/home/ort'...
* Mercurial: Requires 'hg' in no specific version. Found version 6.9.1.
Prefix legend:
- The tool was not found in the PATH environment.
+ The tool was found in the PATH environment, but not in the required version.
* The tool was found in the PATH environment in the required version.
ScanCode license texts found in '/opt/scancode-license-data'.
Not all tools requirements were satisfied:
! Some tools were not found in their required versions.
Additional context
Running with --debug
it seems that this happens during some cleanup.