MOJO: ArrayIndexOutOfBoundsException running prediction on a system with different endianness

[exalate-issue-sync]

A mojo model created on a workstation with LITTLE ENDIAN architecture, gives the following error when used to predict a score on a server with BIG ENDIAN architecture:

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 10496
at hex.genmodel.algos.tree.SharedTreeMojoModel.scoreTree(SharedTreeMojoModel.java:136)
at hex.genmodel.algos.tree.SharedTreeMojoModel.scoreAllTrees(SharedTreeMojoModel.java:453)
at hex.genmodel.algos.gbm.GbmMojoModel.score0(GbmMojoModel.java:26)
at hex.genmodel.algos.gbm.GbmMojoModel.score0(GbmMojoModel.java:50)
at hex.genmodel.easy.EasyPredictModelWrapper.predict(EasyPredictModelWrapper.java:663)
at hex.genmodel.easy.EasyPredictModelWrapper.preamble(EasyPredictModelWrapper.java:535)
at hex.genmodel.easy.EasyPredictModelWrapper.predictBinomial(EasyPredictModelWrapper.java:368)

The problem is related with the 'endianness' used by the ByteBufferWrapper class to parse compressed trees; this class is implemented to use native byte order of the server, causing error when the byte order is different from the byte order used in compressing model's trees.

We suggest to use the 'endianness' saved in the model.ini file of mojo zip to configure the endianness in the ByteBufferWrapper and correctly read model's binary data.

[exalate-issue-sync]

Ruslan Dautkhanov commented: +1

Any updates on this ?

We have to create models on Linux but our scoring servers are in Solaris.. and it's not going to change any time soon.

[exalate-issue-sync]

Matteo Capano commented: @Ruslan:

On our scoring server we load mojos and calculate score through a custom Java program, based on this page's example:

https://github.com/h2oai/h2o-3/blob/master/h2o-docs/src/product/productionizing.rst#mojo-quick-start

The "workaround" we adopted was to copy this Java source and force the same endianness of the models:

https://github.com/h2oai/h2o-3/blob/master/h2o-genmodel/src/main/java/hex/genmodel/utils/ByteBufferWrapper.java

//_bb = ByteBuffer.wrap(buf, 0, buf.length).order(ByteOrder.nativeOrder());
_bb = ByteBuffer.wrap(buf, 0, buf.length).order(ByteOrder.LITTLE_ENDIAN);

Then we configured our classloader to use the modified class instead of the original class (alternatively, you can build H2O from source or replace the class file into the h2o-genmodel.jar).

[exalate-issue-sync]

Ruslan Dautkhanov commented: Just noticed this update.. Thank you for sharing this [~accountid:557058:6cebc8ac-5a13-4f51-a509-0f4c166f92f9]

[DinukaH2O]

JIRA Issue Migration Info

Jira Issue: PUBDEV-5046
Assignee: New H2O Bugs
Reporter: Matteo Capano
State: Open
Fix Version: N/A
Attachments: N/A
Development PRs: N/A

[wendycwong]

https://h2oai.atlassian.net/browse/PUBDEV-5046