mvn install gives error on tests

[Yicong-Huang]

Hi,

I found this repo through your excellent papers (Scotty, Cutty). I am trying to build the Flink adaptor and use it with the latest Flink distribution (1.14+).

However I met error with mvn install, I've attached the log here:

Results :

Tests in error: 
  twoWindowsTest(de.tub.dima.scotty.slicing.aggregationstore.test.windowTest.PunctuationWindowTupleTest): class java.lang.String cannot be cast to class java.lang.Integer (java.lang.String and java.lang.Integer are in module java.base of loader 'bootstrap')

Tests run: 64, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Scotty Window Processor 0.4:
[INFO] 
[INFO] Scotty Window Processor ............................ SUCCESS [  0.144 s]
[INFO] core ............................................... SUCCESS [  0.752 s]
[INFO] state .............................................. SUCCESS [  0.068 s]
[INFO] slicing ............................................ FAILURE [  0.711 s]
[INFO] flink-connector .................................... SKIPPED
[INFO] benchmark .......................................... SKIPPED
[INFO] samza-connector .................................... SKIPPED
[INFO] spark-connector .................................... SKIPPED
[INFO] kafkaStreams-connector ............................. SKIPPED
[INFO] beam-connector ..................................... SKIPPED
[INFO] storm-connector .................................... SKIPPED
[INFO] demo ............................................... SKIPPED
[INFO] flink-demo ......................................... SKIPPED
[INFO] spark-demo ......................................... SKIPPED
[INFO] storm-demo ......................................... SKIPPED
[INFO] beam-demo .......................................... SKIPPED
[INFO] kafka-demo ......................................... SKIPPED
[INFO] samza-demo ......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.751 s
[INFO] Finished at: 2022-06-27T10:48:58-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project slicing: There are test failures.
[ERROR] 
[ERROR] Please refer to /Users/yicong-huang/IdeaProjects/SWAT-RT/scotty-window-processor/slicing/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :slicing

Could you please help me identify what's going wrong? Also, would it be available to build with sbt in the near future (or simply publish on maven central should help)?

Thanks in advance!

[julianev]

Hi @Yicong-Huang,

We believe that the punctuation window tuple test fails because it is using the tuple implementation of flink.
The problem is that the scotty core is not using the latest flink distribution as you are.
Maybe you can try to build nevertheless by skipping the tests with the -DskipTests command.

Thank you for this suggestion, we plan to publish on maven central soon!

Best,
Juliane

[dchristle]

Hi @julianev,

I'm hitting this same error and my intent is to try to use scotty with Flink 1.15.2. However, even on the latest commit b8f57b00a9303de3dcb9ab89fc23e4fc3738057e, where the flink-connector POM is using version 1.9.2 of flink-streaming-java_2.11, the same test fails with the same error when I run mvn clean package. Can you reproduce this locally? If not, I wonder if something about our testing environments is different.

Best,
David

[jelos98]

This happens for me when doing a git clone of master, then mvn install or mvn test as well.

Tests in error:
  twoWindowsTest(stream.scotty.slicing.aggregationstore.test.windowTest.PunctuationWindowTupleTest): java.lang.String cannot be cast to java.lang.Integer

We believe that the punctuation window tuple test fails because it is using the tuple implementation of flink.

I think it's simpler than that. You're trying to pass a Tuple2<**String**, Integer> to a SlicingWindowOperator<Tuple2<**Integer**, Integer>> with an AggregateFunction<Tuple2<**Integer**, Integer>..... > attached. That doesn't seem like it's valid Java.

https://github.com/TU-Berlin-DIMA/scotty-window-processor/blob/master/slicing/src/test/java/stream/scotty/slicing/aggregationstore/test/windowTest/PunctuationWindowTupleTest.java#L16

https://github.com/TU-Berlin-DIMA/scotty-window-processor/blob/master/slicing/src/test/java/stream/scotty/slicing/aggregationstore/test/windowTest/PunctuationWindowTupleTest.java#L26

https://github.com/TU-Berlin-DIMA/scotty-window-processor/blob/master/slicing/src/test/java/stream/scotty/slicing/aggregationstore/test/windowTest/PunctuationWindowTupleTest.java#L58

The error attached above (which I also get) doesn't seem like it's flink-version related at all - it appears to be a bog standard java compilation error because the code is expecting a different generic-typed tuple, and you're mixing ints and strings in the test.
Perhaps some JDKs are better than others about checking generic types - and you're using one that doesn't bother?

It doesn't compile with any JDK version that I have access to at work (are you by chance using Oracle's JDK 8, as opposed to OpenJDK/Temurin?).

The only way I can think of that this would compile, let alone pass as a test, would be if your JDK's compiler is ignoring the typing entirely. As an outsider, it seems like if you're expecting this test to work, you're at best relying on likely undefined behavior w.r.t. type erasure, abusing the generic type, and possibly making some strong assumptions about serialization as well.

I would humbly request you consider changing the bits of the test that use Tuple2<String, Integer> to be Tuple2<Integer, Integer> as the typing requires. Or perhaps there's a way to limit this particular test to only run if you're running whatever JDK you're using if there's a legitimate reason you'd expect the "wrong" tuple to work here.

Tests fail with:
$ mvn -v
Java version: 11.0.20.1, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64

nor:
$ mvn -v
Java version: 1.8.0_382 , vendor: Temurin, runtime: /jdk8u382-b05/jre

(Ubuntu 20.02 in WSL2)