-
Notifications
You must be signed in to change notification settings - Fork 44
Description
Hello!
I am trying to implement the animation pipeline as described in the NVIDIA ACE documentation.
I have successfully run all containers and can see the video stream in my browser. However, I am encountering issues when trying to verify the audio stream. Here are the details:
Issue 1: Incorrect .proto files in the provided validation script
The documentation suggests using the following script to run the Audio2Face (A2F) pipeline:
audio2face_in_animation_pipeline_validation_app
However, this script contains outdated or incorrect .proto files, at least for A2F version 1.2.
Specifically:
The ACE repository defines AudioStreamHeader without an emotion_params property.
This discrepancy seems to be causing an error: StatusCode.UNIMPLEMENTED.
Issue 2: Missing stream_id in AudioStreamHeader of the A2F 3D Samples repository
With the following script from the Audio2Face 3D Samples repository:
audio2face_3d_microservices_interaction_app
I was able to run it, but the AudioStreamHeader does not include a stream_id field, which, according to the documentation, should be present.
To address this, I modified the code as follows:
# from nvidia_ace.controller.v1_pb2 import AudioStream, AudioStreamHeader
from nvidia_ace.a2f.v1_pb2 import AudioStream, AudioStreamHeader
from nvidia_ace.animation_id.v1_pb2 import AnimationIds
And added the stream_id field:
audio_stream_header = AudioStream(
audio_stream_header=AudioStreamHeader(
animation_ids=AnimationIds(
request_id=str(uuid4()),
target_object_id='avatar',
stream_id='my_stream_id',
),
)
# .... rest of the code remains the same
)
Now, I am receiving the following error message:
Received status message with value: 'Input Error: audio header channel count is 0, expected 1.'
Status code: '3'
I have ensured that I am setting the channel count to 1, yet the error persists. Previously, everything worked when using nvidia_ace.controller.v1_pb2 proto files, and the only change I made was adding audio_stream_header.
I suspect it a mismatch between the .proto files used on the client and server.
Could you clarify the correct .proto files that should be used for A2F version 1.2?
Is there an updated repository or documentation that specifies the correct schema for AudioStreamHeader?
Any guidance on resolving the "channel count is 0" error would be greatly appreciated.
Thank you in advance for any assistance or relevant documentation links!