Error when running iped-webapi.jar

[rcaforense]

Hello everyone,

I'm trying to run the iped-webapi.jar component as described in the official wiki, using the following command:

#java -jar lib/iped-webapi.jar --host=127.0.0.1 --port=1234 --sources=casosweb.json

However, I get the following error when starting the server:

Exception in thread "main" java.lang.NoSuchMethodError:
'org.reflections.util.ConfigurationBuilder org.reflections.util.ConfigurationBuilder.setExpandSuperTypes(boolean)'
at io.swagger.jaxrs.config.BeanConfig.classes(BeanConfig.java:284)
at io.swagger.jaxrs.config.BeanConfig.scanAndRead(BeanConfig.java:250)
at io.swagger.jaxrs.config.BeanConfig.setScan(BeanConfig.java:231)
at iped.engine.webapi.Main.startServer(Main.java:40)
at iped.engine.webapi.Main.main(Main.java:91)

My environment:

Operating system: Windows 10 x64
Java version: 11.0.26 (BellSoft LibericaJDK)
Maven version: 3.9.9
IPED version: 4.2.2

I’m not sure if this error is due to an internal incompatibility with some library (reflections, swagger, etc.), or if I'm executing the .jar incorrectly. I would really appreciate any guidance or clarification on whether this is a known issue or a mistake on my side.

Thank you very much for your time and for the great work on this tool.

[lfcnassif]

@marcosammoura, could you take a look at this? It was also reported on #2449. This feature has a simple documentation written by its original author @atilaromero on https://github.com/sepinf-inc/IPED/wiki/Web-API, but he moved to another job.

[marcosammoura]

Hi, @lfcnassif. I'll take a look.

[filipesimoes]

I’ve tracked down the issue to a transitive dependency coming from the iped-parsers-impl component.
The dependency com.webcohesion.ofx4j:ofx4j brings in org.reflections:reflections:0.9.10, which is outdated and triggers the error reported here.

The problem can be resolved by overriding the transitive dependency and explicitly forcing the newer version:

org.reflections:reflections:0.10.2

This upgrade resolves the compatibility issue for the affected functionality.

I have tested this fix specifically for the functionality related to this error, and it works as expected.
However, I was not able to run the full build with all tests enabled, so additional verification may be needed to ensure there are no regressions in other areas.

I will prepare and submit a pull request with this change shortly so it can be reviewed and merged.

[marcosammoura]

Hi, @filipesimoes. Many thanks for working on this issue and sending PR #2600.

At first, adding reflections:0.10.2 to IPED's pom.xml file solves the issue.
However, I see some challenges:

• ofx4j requires reflections:0.9.10 (even up to the latest ofx4j:1.38)
• reflections:0.10.x introduces some breaking changes with older versions (see footnote on reflections' Usage)

So, directly switching to reflections:0.10.2 could eventually work, although this couldn't be a permanent solution due to the Reflections library 0.10+ aforementioned breaking changes!

A possible path towards a stable solution might involve coding a dependency shading strategy using Maven Shade plugin (see a brief discussion on this subject).

It is also worth to mention that Reflections library is currently not under active development or maintenance, so the solution to the issue concerning IPED's web API should be in line with IPED's roadmap. What do you think, @lfcnassif?

[lfcnassif]

Thanks @marcosammoura for your analysis. We've already used maven shade plugin in the past, to put incompatible versions of Lucene and ElasticSearch together, fortunately we don't need it anymore for them.

Although reflections-0.10.x has API breaking changes, they don't seem to affect ofx4j-1.36, possibly the changed API is not used. I just run regression tests on my 379 OFX files regression database, no additional exceptions were thrown, the same number of XLS reports were generated, and I got the same number of results when searching for common words, so I think the upgrade to reflections-0.10.2 is fine. Thanks @filipesimoes and @marcosammoura for helping!

[marcosammoura]

Great news, @lfcnassif! Things got really straightforward and the work has already been done by @filipesimoes!

[paulobreim]

I reindexed with this snap and now I got this error.

i:\pcbsnap\iped>java -jar lib/iped-webapi.jar --host=127.0.0.1 --port=1234 --sources=.
Exception in thread "main" java.io.FileNotFoundException: . (Acesso negado)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(FileInputStream.java:213)
at java.base/java.io.FileInputStream.(FileInputStream.java:152)
at java.base/java.io.FileInputStream.(FileInputStream.java:106)
at iped.engine.webapi.Sources.askSources(Sources.java:149)
at iped.engine.webapi.Sources.init(Sources.java:54)
at iped.engine.webapi.Main.startServer(Main.java:45)
at iped.engine.webapi.Main.main(Main.java:91)

tks

[lfcnassif]

Your command is incorrect, --sources option should point to a json file, see https://github.com/sepinf-inc/IPED/wiki/Web-API

[paulobreim]

Thanks Nassif,
Then another problem appeared, which is the Java version on Windows is higher than 11, in my case 21, so I did it with jre\bin\java.exe...

tks