Perfect Javacpp-pytorch , let's make

[mullerhai]

Hi @saudet ,

we’ve made huge progress over the last few days.
javacpp-pytorch now finally supports real parameters. This is extremely significant, especially for the global torch.class, which contains thousands of commonly used operators. It’s very beginner-friendly, and it will also greatly help with developing new upstream and downstream components based on javacpp-pytorch in the future.
AOT CPU & CUDA support. This capability far surpasses the inference performance of model formats like Microsoft’s ONNX. Of course, the results will be even better when combined with downstream KV-cache development in the future.
AOT Package Loader support.
Ubuntu CPU: loading and inference work fine.
CUDA: loading works; inference just needs a GPU driver upgrade.
Mac CPU: still missing OpenMP library adaptation.
Windows: since all my AOT models are transpiled from Python, they require the Triton library, but we found that there is no usable Triton on Windows. I will probably need to port and transfer the model files to Windows for testing. I think the issues are minor overall.
At this point, I believe we are very close to perfect.
However, there are still a few things we could try to resolve. These do not involve algorithms or validation—they are mostly about aligning with the standard directory structure and naming conventions of the official C++ libtorch and PyTorch Python libraries. The main goal is to organize the most commonly used modules and classes clearly so people can find them easily.
Compared to the entire project, I think we should prioritize optimizing these commonly used modules first. For model training, the most essential ones are:
layer, loss, dataset, dataloader, sampler, optimizer, activation.
They must be placed in the most convenient directories for quick lookup and search.

1. Unify naming for the Data module
   data, dataset, dataloader are the most frequently used modules for training.
   I strongly prefer keeping the naming consistent with libtorch. The Java prefix confuses users and makes it hard to find the right components.
   For example, it took me four years to realize that JavaDataset is equivalent to PyTorch Python’s Dataset.
   Please remove the Java prefix:
   JavaDataset → Dataset
   JavaTensorDataset → TensorDataset
   JavaRandomDataloader → RandomDataLoader
   JavaSequentialDataLoader → SequentialDataLoader
   JavaDistributedSequentialTensorDataLoader → DistributedSequentialTensorDataLoader
   JavaStatefulTensorBatchDataset → StatefulTensorBatchDataset
   and so on, including their base and parent classes.
   Also, move them into:
   org.bytedeco.pytorch.data
   or
   org.bytedeco.pytorch.utils.data
   Additionally, please put all Sampler classes into a dedicated package, similar to:
   org.bytedeco.pytorch.sampler
2. Organize layers, losses, activations, modules, optimizers
   Please move all of these into the org.bytedeco.pytorch.nn hierarchy:
   LayerImpl
   LossFnImpl
   ActivateFunc*Impl
   Right now they are scattered in the root directory, so users often can’t find them—including me in past issues, where I thought we were missing layers that actually existed.
   There are about:
   90+ layers
   30+ activation functions
   7–8 loss functions
   Please place them under:
   org.bytedeco.pytorch.nn
   org.bytedeco.pytorch.nn.loss
   Also include:
   SequentialImpl, ModuleList, ModuleDict, AnyModule, Module, JitModule
   And all optimizers (around 7–8) under:
   org.bytedeco.pytorch.Optimizer
3. Improve the release process
   Waiting 6 months for a single update is far too slow.
   On the other hand, snapshot versions are not convenient for integration in production environments, and snapshot packages also take a very long time to download.
   I suggest adding RC or Beta versions to the pipeline, similar to how Lance and LangChain4j currently operate.
4. Improve sbt-javacpp platform filtering
   I hope sbt-javacpp can automatically detect and filter the local platform for javacpp-related native libraries.
   Right now, if you don’t filter manually, it downloads libraries for all platforms, which is extremely slow and painful.
   That’s about it.
   In my opinion, javacpp-pytorch is already nearly perfect. There are almost no major usability issues, and all functionality works correctly.

best regards

[mullerhai]

let's see here
https://mvnrepository.com/artifact/org.lance/lance-core, it has. RC version and. beta version @saudet
https://mvnrepository.com/artifact/dev.langchain4j/langchain4j-embeddings-all-minilm-l6-v2-q/1.11.0-beta19

[mullerhai]

Hi, @saudet
I try to make one code dir sample like that ,not real correct , I also think if do like that ,also complex

[mullerhai]

Hi @saudet
Although modifying the directory structure of the code is quite complex, we still hope to complete the creation and application of this directory structure when translating the cpp libtorch code into the java glue layer code. It is true that having most of the code scattered in the root directory all the time is really inconvenient for everyone to use. From my personal experience, there are times when I am not sure whether certain classes have actually been implemented; I end up spending a long time searching for them, and even miss them frequently. If all the code were organized in an orderly manner consistent with their namespaces, it would benefit all users. This is something we have to do—someone will definitely raise an issue to report this difficulty in the future. The predicament with javacpp-pytorch is real, as the codebase is simply too large. Thank you for your hard work.

here you can see the javacpp-pytorch。scala3 binding code dir

https://github.com/bytedeco/pytorch/tree/main/storch_core/src

[mullerhai]

HI @saudet ,

I later discussed with other engineers who are familiar with and use JavaCPP compilation, and they told me that if we want to distribute the compiled Java glue layer code into different packages according to namespaces, we would need to write more mapper files, each of which must explicitly declare the corresponding C++ classes and perform mappings, as well as list the C++ file names in the header. Most importantly, this would generate many more .dll and .so library files, causing platform library bloat. He suggested that it might be better to handle the classification of the glue layer code into different package directories downstream, but I believe that if we could solve this issue—unfriendly to beginners—at its root, you, as the author and repository maintainer, should be an expert with a strong sense of code structure. I’m curious how you view this issue: what is the best way to resolve this unbeginner-friendly problem more elegantly without introducing additional complexity? Or, in your opinion, is this not even an issue—as long as the functionality works? I look forward to discussing this with you when you have time. It would be even better if this workload could be handled by Claude or Gemini.
Since Java does not support package file definitions to declare a series of upstream classes within the same package, while Python and Scala do support it, if downstream Java writes wrapper classes to integrate and unify the namespace, it will lose originality, causing the downstream to constantly perform toNative operations.