[Bug]: v1.8 optical flow crashes #961
Description
Have you read the latest version of the FAQ?
- I have visited the FAQ page right now and my issue is not present there
Is there an existing issue for this?
- I have searched the existing issues and checked the recent builds/commits of both this extension and the webui
Are you using the latest version of the Deforum extension?
- I have Deforum updated to the lastest version and I still have the issue.
What happened?
Running Deforum with these options in Hybrid Video Tab (Optical Flow: Farneback)
User friendly error message: Error: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all(). Please, check your schedules/ init values.
Steps to reproduce the problem
- Start a fresh Automatic1111/Deforum instance
- Leave everything at default, except insert a custom Input video in the init tab
- In hybrid tab enable "Generate Input Frames", Optical Flow and Farneback
- Hit generate and wait until it crashes
What should have happened/how would you fix it?
It used to work fine in v1.6
Torch version
torch: 2.1.2+cu121
On which platform are you launching the webui with the extension?
Local PC setup (Windows)
Deforum settings
https://gist.github.com/chrizzlemadizzle/8c1746ebe00dc14918b1bec9a61e4fcf
Webui core settings
https://gist.github.com/chrizzlemadizzle/579d30c11ca5c05c8b89888934a03445
Console logs
Already up to date.
venv "C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
Launching Web UI with arguments: --xformers --autolaunch --medvram
ControlNet preprocessor location: C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2024-03-16 20:41:34,865 - ControlNet - INFO - ControlNet v1.1.441
2024-03-16 20:41:35,032 - ControlNet - INFO - ControlNet v1.1.441
Loading weights [4199bcdd14] from C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\models\Stable-diffusion\revAnimated_v122EOL.safetensors
2024-03-16 20:41:35,511 - ControlNet - INFO - ControlNet UI callback registered.
Creating model from config: C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\configs\v1-inference.yaml
*Deforum ControlNet support: enabled*
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
Startup time: 23.6s (prepare environment: 6.1s, import torch: 8.5s, import gradio: 1.8s, setup paths: 1.4s, initialize shared: 0.4s, other imports: 1.1s, load scripts: 2.4s, create ui: 1.1s, gradio launch: 0.6s).
Applying attention optimization: xformers... done.
Model loaded in 19.2s (load weights from disk: 0.6s, create model: 1.3s, apply weights to model: 14.3s, apply half(): 1.6s, calculate empty prompt: 1.2s).
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: d3463ae7
Starting job task(66l5jhnueq76e3z)-0...
Video to extract: "C:\Users\chris\Projekte\H. C. BEHRENDTSEN\Physics Musikvideo\Physics input\bauhaus_test1-1_short.mov"
Extracting video (1 every 1) frames to C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\outputs\img2img-images\Deforum_20240316204329\inputframes...
Trying to extract frames from video with input FPS of 24.0. Please wait patiently.
Extracted 807 frames from video in 3.64 seconds!
Using 807 input frames from C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\outputs\img2img-images\Deforum_20240316204329\inputframes...
Using init_image from video: C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\outputs\img2img-images\Deforum_20240316204329\inputframes\bauhaus_test1-1_short000000000.jpg
Saving animation frames to:
C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\outputs\img2img-images\Deforum_20240316204329
Animation frame: 0/807
Seed: 1096329681
Prompt: tiny cute bunny, vibrant diffraction, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus
Neg Prompt: nsfw, nude
╭─────┬───┬───────┬─────┬──────┬──────┬──────┬────┬────╮
│Steps│CFG│Denoise│Angle│ Zoom │Tr C X│Tr C Y│Tr X│Tr Y│
├─────┼───┼───────┼─────┼──────┼──────┼──────┼────┼────┤
│ 25 │7.0│ 0.2 │ 0 │1.0025│ 0.5 │ 0.5 │ 0 │ 0 │
╰─────┴───┴───────┴─────┴──────┴──────┴──────┴────┴────╯
100%|████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:03<00:00, 1.44it/s]
Animation frame: 1/807 | 5/7263 [00:01<32:25, 3.73it/s]
Calculating Farneback optical flow for frames 0 to 1
Seed: 1096329682
Prompt: tiny cute bunny, vibrant diffraction, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus
Neg Prompt: nsfw, nude
╭─────┬───┬───────┬─────┬──────┬──────┬──────┬────┬────╮
│Steps│CFG│Denoise│Angle│ Zoom │Tr C X│Tr C Y│Tr X│Tr Y│
├─────┼───┼───────┼─────┼──────┼──────┼──────┼────┼────┤
│ 25 │7.0│ 0.35 │ 0 │1.0028│ 0.5 │ 0.5 │ 0 │ 0 │
╰─────┴───┴───────┴─────┴──────┴──────┴──────┴────┴────╯
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:03<00:00, 2.82it/s]
Animation frame: 2/807 | 14/7263 [00:07<42:28, 2.84it/s]
Calculating Farneback optical flow for frames 1 to 2
*START OF TRACEBACK*
Traceback (most recent call last):
File "C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\run_deforum.py", line 116, in run_deforum
render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root)
File "C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\render.py", line 433, in render_animation
flow = get_flow_for_hybrid_motion(frame_idx - 1, (args.W, args.H), inputfiles, hybrid_frame_path, prev_flow, anim_args.hybrid_flow_method, raft_model,
File "C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\hybrid_video.py", line 193, in get_flow_for_hybrid_motion
flow = get_flow_from_images(i1, i2, method, raft_model, prev_flow) # old single flow forward
File "C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\hybrid_video.py", line 353, in get_flow_from_images
return get_flow_from_images_Farneback(i1, i2, prev_flow)
File "C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\hybrid_video.py", line 411, in get_flow_from_images_Farneback
if preset == "fine":
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
*END OF TRACEBACK*
User friendly error message:
Error: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all(). Please, check your schedules/ init values.
Deforum progress: 0%| | 14/7263 [00:08<1:16:31, 1.58it/s]
saving custom settings to C:\Users\chris\Projekte\automatic1111\stable-diffusion-webui\deforum_settings.txtAdditional information
No response