Identity lock on references image

[samuellucky2424-afk]

Please is there any identity lock when I am using the character swipe or let me just see when I upload my reference image with my prompt the attribute character is not the same as my uploaded image and the character keep on changing in real time well the latency was very very nice it was fast responding good and clean boats the identity preservation was not working I don't know if you guys added it but it will be better if you add it please so we can be able to achieve this decart alternative perfectly

[EasonXiao-888]

Thanks for the awesome feedback! Glad the model feels fast and responsive.

To give you some transparency on the 08/14 checkpoint: while trained on general RV2V, its primary optimization right now is try-on (as shown in readme). General IP replacements absolutely work, but you might see some subject drifting as we are still updating the training data for it. (We'll open-source our training scripts soon so you can fine-tune it yourself!)

We'd love to investigate your specific case. Could you share your reference image, exact prompt, and a short screen recording?
Until our next update, here are two quick tips for better consistency:
Try-on: Stick to the "from Image 1" pattern. The model loves this syntax.
General Edits: Use a VLM to enhance your prompt with explicit static attributes of your reference (color, texture, unique features). Spelling these out acts as a strong anchor.

Hope this helps! Let us know if you have any questions. 🚀

[samuellucky2424-afk]

https://github.com/user-attachments/assets/b8af2835-7608-43bf-ada5-ffc7998e2338 So here is my result I was trying to add identity lock of which it worked for some short. Before I started experiencing a very intensive glowing And I was trying to modify my repo of which i was try to make it capture every single expression to make the output feel realistic as decart but it almost impossible i always go through Ur paper and doc so if there might be a guild, even currently I'm still try to achieve it

[samuellucky2424-afk]

More precisely, I would define the target behavior as:

Reference identity stability: face shape, eyes, nose, lips, hairline, proportions, and recognizable identity remain stable across the stream.
Expression transfer: smiles, blinks, mouth articulation, eyebrow motion, cheek movement, and subtle facial muscle changes follow the source.
Head-motion transfer: yaw, pitch, roll, nodding, turning and natural pose changes follow the source.
Skin fidelity: preserve realistic pores, tone, highlights, shadows and texture instead of becoming plastic, glowing or oversmoothed.
Temporal consistency: no identity flicker, sudden face changes, texture crawling or frame-to-frame instability.
Low temporal lag: expressions should appear as close as possible to when they happen in the source stream.
Long-session stability: the identity should remain stable not just for 5–10 seconds, but across longer live sessions.

[Dai-Wenxun]

@samuellucky2424-afk Your last comment is genuinely useful — those seven bullets read as an eval rubric (identity stability / expression transfer / head-motion transfer / skin fidelity / temporal consistency / lag / long-session stability), and that's how we're treating them internally. Adding a few things to what @EasonXiao-888 said:

On the intensifying "glow": what your video shows is accumulation drift — in long-horizon streaming generation, small per-chunk errors compound, and they surface exactly as you describe: fine at first, then progressive glowing / over-smoothing the longer the session runs. Today the practical mitigation is to restart the stream periodically (a fresh session starts from clean conditioning); long-session stability is squarely on our list, and your rubric makes the target concrete.

On where the model stands: JoyAI-Video-Edit is a general streaming editor. Expression and head-motion transfer come from the v2v conditioning on your source stream — those follow well, as you saw early in your session. A hard identity lock to a reference image is the weakest of your seven axes today: as Eason said, the current release (08/14 — the joyai_video_edit_dit_0811.pth file) optimizes the reference pathway for try-on first, and general IP replacement is still being data-iterated. Once the training scripts are open-sourced, fine-tuning on your specific target identity is likely the strongest route to the Decart-style lock you're after.

One thing that matters for fidelity testing: if any of your judgments came from the hosted demo (https://huggingface.co/spaces/wxDai/joyai-video-edit), note it runs 480p @ 24 fps to stay real-time, while the model is trained at 720p — skin texture, pores and fine identity cues are exactly what 480p hurts most, so it can read as "plastic/glowing" sooner than a local run would. For serious evaluation, deploy locally at 720p (DEPLOYMENT.md, bash deploy/run_server.sh) and keep --num-inference-steps 2.

And to let us actually reproduce the glow: could you attach the reference image + exact prompt from that recording? The video alone doesn't let us re-run your case.

[samuellucky2424-afk]

Prompt: Replace the main subject with the person in the reference image.

And Please I will be happy if you can give us an enlightenment on when the next version of this model will be released and I really appreciate for the responds.

[samuellucky2424-afk]

And secondly I wasn't running the model on the demo platform on huggingface no I Deployed the model on runpod serverless GPU H200 because wanted it full ability so i will be happy to hear from you guys wish you the best

[Dai-Wenxun]

@samuellucky2424-afk Perfect, that's exactly what we needed — and thanks for the clarification: since you're on your own H200 deployment, ignore my demo-resolution note above (do confirm you're running the 720p settings with --num-inference-steps 2, though). Two things stand out immediately in your setup:

1. The prompt is the weakest possible pattern for identity. "Replace the main subject with the person in the reference image" gives the model zero attribute anchors — this is exactly the case @EasonXiao-888's VLM tip addresses. For your specific reference, try something like:

Replace the main subject with the man from Image 1: a young East Asian man with short black slicked-back hair, thick straight eyebrows, dark brown eyes, a defined jawline with light stubble, wearing a white collared pajama shirt. Keep his face shape, hairline and skin tone constant.

Spelled-out static attributes act as a per-frame anchor and measurably reduce drift.

2. The reference image itself is working against you. It's a low-light shot with half the face in shadow — the model can't read the skin texture and fine identity cues it's supposed to lock onto, so it fills them in (which also feeds the "glowing/plastic" look). A well-lit, frontal, evenly-exposed reference photo will noticeably improve both identity stability and skin fidelity. Worth re-testing with both changes before anything else.

On the next release: we don't pre-announce checkpoint dates, but the concrete near-term items are RTX 5090 / low-VRAM support landing this week, training scripts open-sourcing soon (per Eason above — per-identity fine-tuning is the strongest path to a hard lock), and the reference/IP-replacement data iteration is actively ongoing — your case is exactly the kind feeding it. The long-session glow on your H200 is the accumulation-drift point from my previous comment; periodic stream restarts remain the practical mitigation until that lands.

[samuellucky2424-afk]

Alright thank you so I was able to like add the IP identity preservation and here was the result I did test it this evening never mind i'm from Africa i'm a black guy so I did some tests this evening here in Nigeria and this was my result I used this prompt where I tried to enhance my prompt I used this and prompt below and then I will attach the video results

PROMPT: Replace the main subject with the exact same adult person from Image 1.
Keep the same face shape, hairline, eyes, nose, lips, hairstyle, natural
skin tone and visible skin texture on the face, neck, arms and hands in
every frame. Transfer only the source person's pose, eye blinks, eyebrow
motion, jaw and mouth motion, hand motion and body motion. Keep identity,
age, hairstyle and skin tone unchanged.

VIDEO: test video

Apologies that I used link alright I was unable to upload the video it was 133MB file size so please you can watch it through the Google Drive link