Skip to content

Home

Jump to bottom
tintwotin edited this page Jul 5, 2025 · 7 revisions

Pallaidium

Pallaidium is a free, generative AI movie studio integrated directly into the Blender Video Sequence Editor (VSE). It empowers users to generate video, images, audio, and text from simple text prompts or existing media strips, streamlining the creative process within a single application.

This document provides comprehensive information for installing, using, understanding the features, and troubleshooting Pallaidium.

Index

Features

Pallaidium provides a comprehensive suite of generative AI tools directly inside Blender's VSE.

Core Generation Capabilities

  • Text-to-Media: Generate Video, Image, Speech, Music, and Sound Effects from a text prompt.
  • Media-to-Media: Perform Video-to-Video, Image-to-Image, and Image-to-Video transformations.
  • Utility Functions: Generate Text-from-Text (Prompt Rewriting), Image-from-Text (Captioning), and Video-from-Text.

Generation Matrix

This matrix provides a quick overview of the core generative capabilities, mapping input types to possible outputs.

Input Image Video Text Audio Music Speech
Text ✅ ✅ ✅ ✅ ✅ ✅
Image ✅ ✅ ✅
Video ✅ ✅ ✅

Workflow Enhancements

  • Batch Processing: Batch convert, refine, and upscale multiple strips at once.
  • Prompt Batching: Use VSE text strips for sequential prompt generation.
  • File Management: Set custom paths for models and generated files. A convenient Open Output Folder button is available in the UI.
  • UI/UX: A clean interface with a "Show Advanced Options" checkbox to reveal deeper settings. Get desktop notifications when rendering is complete.

Installation

Follow these steps carefully to install Pallaidium and its dependencies.

Requirements

Component Requirement
Operating System Windows. Limited support for Linux and macOS
Graphics Card CUDA-supported Nvidia card with at least 6 GB of VRAM
CUDA Version 12.4
Storage 20+ GB of free HDD space (each AI model requires 6+ GB)

Installation Steps

  1. Install Git: Download and install Git from https://git-scm.com/downloads, ensuring it is added to your system's PATH.
  2. Download the Add-on: Download the latest add-on zip file from https://github.com/tin2tin/Pallaidium/archive/refs/heads/main.zip.
  3. Run Blender as Administrator: You must right-click the Blender icon and select "Run as Administrator" on Windows to install dependencies correctly.
  4. Install the Add-on in Blender:
    • Open Blender (as Administrator).
    • Go to Preferences > Add-ons > Install.
    • Select the downloaded .zip file and enable the add-on.
  5. Uninstall and Reinstall Dependencies (Crucial Step):
    • In the add-on's preferences, click the "Uninstall Dependencies" button.
    • Restart Blender (as Administrator).
    • Return to the add-on preferences and click the "Install Dependencies" button.
  6. Final Restart: Restart your computer, then launch Blender as Administrator one more time before use.

Important Note: The first time you execute any model, 5-10 GB of data will be downloaded from HuggingFace. This only happens once per model.

Troubleshooting

If you encounter issues, consult this guide for common solutions.

Problem Description/Error Message Solution
Dependency/Module Errors Missing Python modules or installation failures. Use the blender_pip add-on to manually install packages: https://github.com/tin2tin/blender_pip
MSVC Error "WARNING: Failed to find MSVC" during installation. Install "Tools for Visual Studio" from Microsoft: https://aka.ms/vs/17/release/vs_BuildTools.exe
Out of Memory "CUDA is out of memory" error during generation. 1. Restart Blender to clear VRAM.
2. If you have low VRAM, try disabling "System Memory Fallback" in your NVIDIA control panel.
Broken/Distorted Renders Generated media appears corrupted or has incorrect dimensions. Ensure the resolution set in the panel matches the native resolution of the model (check the Model Card in Preferences).
Stuttering Playback Generated video strips play back slowly in the VSE. Select the strip, then go to Menu > Strip > Movie Strip > Set Render Size.
Installation Permissions Errors related to file writing or "access denied." Always run Blender as Administrator when installing or managing dependencies.

Usage

Accessing the Panel

The Pallaidium panel is located in Blender's Video Sequence Editor > Sidebar (N-Panel) > Generative AI tab.

Basic Workflow

  1. Select Output Type: Choose your desired media type: Image, Video, Audio, or Text.
  2. Select Model: Choose a specific AI model from the dropdown list.
  3. Configure Input Source:
    • From Text Prompts: Leave the Input dropdown on Prompts and enter your text.
    • From Existing Strips: Select one or more strips in the VSE timeline and set the Input dropdown to Strips.
  4. Adjust Parameters: Configure basic settings like prompts, resolution, and seed.
  5. Generate: Click Generate (for prompts) or Generate from Strips.

The generated media is automatically saved to the directory specified in the add-on preferences and added to your VSE timeline.

Audio Generation

Pallaidium can generate speech, music, and sound effects using various models. For high-quality text-to-speech, the Parler TTS model is available with the following voices:

  • Aaron, Alisa, Anna, Barbara, Bill, Brenda, Bruce, Carol, David, Eileen, Emily, Eric, Gary, James, Jason, Jenna, Jerry, Jon, Jordan, Joy, Karen, Laura, Lauren, Lea, Mike, Naomie, Patrick, Rebecca, Rick, Rose, Tom, Tina, Will, Yann.

Advanced Controls

Style Presets

Quickly apply a predefined style to your prompt. These presets are customizable by editing the prompts.json file in the addon's folder.

  • Default Presets: Cinematic, Photographic, Anime, Manga, Fantasy, Neon Punk, 3D Model, etc.

ControlNet

Use a source image to control the composition, pose, or structure of the generated output.

  • Models: Canny (edge detection), OpenPose (human pose), Scribble (line drawings).
  • Parameters: Adjust Conditioning Scale (strength), Guidance Start, and Guidance End for precise control.

LoRAs (Low-Rank Adaptation)

Apply one or more LoRA models to guide the generation towards a specific character, object, or artistic style. The influence of the LoRA can be adjusted with the LoRA Weight parameter.

IP Adapter (Image Prompt)

Use an input image as a prompt to transfer its style or facial characteristics to the generated output. The strength is controlled by the IP Adapter Scale.

ADetailer (Automatic Detailer)

Enable this feature to automatically detect objects (typically faces) in the generated image and run a second, targeted pass to improve their detail and coherence.

Key Parameters Explained

  • Quality Steps (Inference Steps): The number of steps the model takes to generate the media. Higher values can improve quality but increase render time.
  • Word Power (CFG Scale): How strongly the model adheres to your text prompt. Higher values mean stricter adherence.
  • Strip Power (Denoising Strength): In media-to-media generation, this controls how much the original source is altered. A low value preserves the original, while a high value gives the AI more creative freedom.

Supported Models

Pallaidium integrates a variety of state-of-the-art generative models from ex. HuggingFace.

Video Models

Model Name HuggingFace Link
FramePack [https://github.com/lllyasviel/FramePack]
SkyReels-V1-Hunyuan-I2V/T2V Tencent-Hunyuan/SkyReels-V1
HunyuanVideo Tencent-Hunyuan/Hunyuan-Video
LTX-Video lightricks/ltx-video
Stable Video Diffusion stabilityai/stable-video-diffusion-img2vid-xt
CogVideoX THUDM/CogVideoX-5b
Zeroscope V2 XL cerspense/zeroscope_v2_XL

Image Models

Model Name HuggingFace Link
Stable Diffusion XL (SDXL) stabilityai/stable-diffusion-xl-base-1.0
SDXL Lightning ByteDance/SDXL-Lightning
Stable Diffusion 3 Medium stabilityai/stable-diffusion-3-medium-diffusers
FLUX.1 Schnell/Dev black-forest-labs/FLUX.1-schnell
Relighting Kontext kontext-community/relighting-kontext-dev-lora-v3
Fluently-XL-Final fluently/Fluently-XL-Final
Shuttle Jaguar shuttleai/shuttle-jaguar
HunyuanDiT Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers
Kolors Kwai-Kolors/Kolors-diffusers
OpenDalleV1.1 dataautogpt3/OpenDalleV1.1
PixArt-Sigma PixArt-alpha/PixArt-Sigma_16bit
RealVisXL V4.0 SG161222/RealVisXL_V4.0
ProteusV0.4 dataautogpt3/ProteusV0.4
BLIP Diffusion Salesforce/blipdiffusion
OmniGen VectorSpaceLab/OmniGen
Chroma lodestones/Chroma
Lumina 2 Alpha-VLLM/Lumina-txl-2.0
Sana Efficient-Large-Model/Sana-1024

Audio Models

Model Name HuggingFace Link
Stable Audio Open stabilityai/stable-audio-open-1.0
MusicGen Stereo facebook/musicgen-stereo-medium
F5-TTS MyShell/F5-TTS
WhisperSpeech collabora/whisperspeech
MMAudio hustvl/MM-Audio
Chatterbox codem-solutions/chatterbox

Utility & ControlNet Models

Task Model Name HuggingFace Link
Background Removal BiRefNet_HR ZhengPeng7/BiRefNet_HR
Caption/Prompt Gen Florence-2-large microsoft/Florence-2-large
Prompt Rewriter MoviiGen1.1 ZuluVision/moviigen-1.1-prompt_rewriter
ControlNet Canny Canny SDXL diffusers/controlnet-canny-sdxl-1.0-small
ControlNet OpenPose OpenPose SDXL xinsir/controlnet-openpose-sdxl-1.0
ControlNet Scribble Scribble SDXL xinsir/controlnet-scribble-sdxl-1.0

Useful Add-ons

These companion add-ons, also developed by tin2tin, integrate seamlessly with Pallaidium to enhance the generative workflow.

Add-on Description Link
GPT4Blender Generate image prompts or stories using GPT4ALL. https://github.com/tin2tin/GPT4BLENDER
Text to Strip Convert text blocks into individual VSE strips for batch prompting. https://github.com/tin2tin/text_to_strip
Subtitle Editor A comprehensive tool for editing and managing text strips in the VSE. https://github.com/tin2tin/Subtitle_Editor
VSE Masking Tools Create and manage masks directly on top of clips in the Sequencer. https://github.com/tin2tin/vse_masking_tools
Add Rendered Strips Convert other strip types into movie strips for use as inputs. https://github.com/tin2tin/Add_Rendered_Strips
blender_pip A utility to install Python packages into Blender’s environment. https://github.com/tin2tin/blender_pip

Clone this wiki locally