[Good First Issue]: [JAVA API] Async demo for Java API 2.0

[likholat]

Context

OpenVINO Runtime supports inference in either synchronous or asynchronous mode. The key advantage of the Async API is that when a device is busy with inference, the application can perform other tasks in parallel (for example, populating inputs or scheduling other requests) rather than wait for the current inference to complete first.

What needs to be done?

We had async inference support and async demo for Java API 1.0 before, now the async mode should be supported for OV Java API 2.0.

It's necessary to:

• Add Face Detection Async Java Sample for OV API 2.0 based on the deprecated sample and Async Mode.
• If needed, implement missing Java bindings of C++ methods required for OV asynchronous demo via the Java Native Interface (JNI). Such methods as start_async() and wait() have already been added for Java API, but you will probably find more unsupported OV Java methods while developing demo.

Here is an example of OV Java method implementation for infer() method of InferRequest class:
Java part of implementation:

// openvino_contrib/blob/master/modules/java_api/src/main/java/org/intel/openvino/InferRequest.java

package org.intel.openvino;

public class InferRequest extends Wrapper {
    ...

    public void infer() {
        Infer(nativeObj);
    }

    private static native void Infer(long addr);
}

CPP part of method implementation:

//openvino_contrib/blob/master/modules/java_api/src/main/cpp/openvino_java.hpp

...
JNIEXPORT void JNICALL Java_org_intel_openvino_InferRequest_Infer(JNIEnv *, jobject, jlong);

// openvino_contrib/blob/master/modules/java_api/src/main/cpp/infer_request.cpp

#include <jni.h>
#include "openvino/openvino.hpp"
#include "openvino_java.hpp"
#include "jni_common.hpp"

using namespace ov;
JNIEXPORT void JNICALL Java_org_intel_openvino_InferRequest_Infer(JNIEnv *env, jobject obj, jlong addr)
{
    JNI_METHOD("Infer",
        InferRequest *infer_request = (InferRequest *)addr;
        infer_request->infer();)
}
...

Example Pull Requests

[JAVA_API][GSOC] Add Java API bindings

Resources

• What is OpenVINO toolkit?
• OpenVINO Contrib repo
• How to build OpenVINO with extra modules
• Java bindings for OpenVINO
• OV Java Samples
• Async Mode
• Java Programming Tutorial - Java Native Interface (JNI)
• Java async demo for depricated OV API 1.0
• OpenVINO™ API 2.0 Transition Guide
• Intel DevHub Discord channel - engage in discussions, ask questions and talk to OpenVINO developers

Contact points

@likholat

Ticket

No response

[AishaniPandey]

.take

[AishaniPandey]

Hello. I wish to take this issue in preparation for GSoc 2024

[AishaniPandey]

.take

[AishaniPandey]

@likholat could you kindly add me as an assignee for this issue.
Thanks :)

[AishaniPandey]

Implementation Plan for Asynchronous Java Sample in OpenVINO Java API 2.0

Subject: Accelerated Implementation of Async Java Sample for OpenVINO Java API 2.0

Introduction
This plan aims to implement a Face Detection Asynchronous Java Sample for OpenVINO Java API 2.0 within a tight timeline. The focus is on integrating necessary Java bindings and developing a robust async demo.

Timezone and Availability
My timezone is CET (GMT+1). I can dedicate 3 hours daily to this project.

Objective
To create an asynchronous Java sample for the OpenVINO Java API 2.0, ensuring compatibility and efficiency, and to add any missing Java bindings for C++ methods via JNI.

Expedited Implementation Steps and Timeline

Initial Setup and Analysis (Day 1)

• Set up the development environment.
• Review existing async demos and understand the upgrades in Java API 2.0.

Development of Java Bindings (Day 2-3)

• Implement Java bindings for essential C++ methods (like start_async, wait).
• Test bindings for functionality and compatibility.

Asynchronous Java Sample Development (Day 4-5)

• Develop the core structure of the Face Detection Async Java Sample.
• Integrate and test the new Java bindings within the sample.

Sample Refinement and Testing (Day 6)

• Optimize the sample for performance and adherence to coding standards.
• Conduct comprehensive testing to ensure proper functionality.

Documentation, Review, and Pull Request (Day 7)

• Update documentation to reflect new features and instructions.
• Finalize the sample and prepare for submission.
• Create and submit a pull request with a detailed implementation summary.

Buffer Period (Day 8-10)

• Use this time to address feedback, make adjustments, or conduct additional testing if needed.

Conclusion
This condensed plan aims for the swift and efficient integration of an asynchronous Java sample in the OpenVINO Java API 2.0. I'm committed to maintaining quality and functionality within this expedited timeframe.

Please feel free to reach out for any clarifications or discussions regarding this plan.

Best regards,
Aishani Pandey
pandeyaishani@gmail.com

[likholat]

Hi @AishaniPandey, thank you for the detailed integration plan, it looks good to me!
Please note that this issue can't be a GSoc project, but can be done as a preparation step for this program.

[likholat]

Hi @AishaniPandey, do you have any update on this task?
Please let me know if you have questions, you can also create a draft PR to discuss.

[AishaniPandey]

Hi likholat,

I was occupied with college midterm assessments from the past few days and I missed your message.

I will be back on track with an updated plan within next few days.

[HarshG1308]

Hello Likholat,

I could resolve this issue.