Natalie Portman LoRA - Prompt: A beautiful lady, p0rtm4n, holding a smartphone using the camera to take selfie, in Paris, ultra-realistic, video realism, masterpiece, ultra high detail.
Step into the future of video creation where we now can use LoRAs together with the AI video model Hunyuan. Why? To keep character or object consistency in a video.
Hunyuan delivers great video quality but as with all AI models, consistency over multiple generations has always been an issue. LoRAs have always been the answer in still images, now it's the case with videos too. Let's dive into how you can use your own LoRAs with Hunyuan.
How to run Hunyuan with LoRA in ComfyUI
Installation guide
Custom Nodes
If there are red nodes in the workflow, it means that the workflow lacks the certain required nodes. Install the custom nodes in order for the workflow to work.
- Go to ComfyUI Manager > Click Install Missing Custom Nodes
- Check the list below if there's a list of custom nodes that needs to be installed and click the install.
Models
For this guide you'll need 6 models.
1. p0rtm4n-hunyuan-v1.1-vfx_ai.safetensors
2. hyvideo_Fastvideo_LoRA-fp8.safetensors
3. hunyuan_video_720_cfgstill_fp8_e4m3fn.safetensors
4. clip_l.safetensors
5. llava_llama3_fp8_scaled.safetensors
6. hunyuan_video_vae_bf16.safetensors
You can use other Hunyuan LoRAs from for example Civitai. Search by using Hunyuan LoRA and choose any of the results. Make sure that it's a Hunyuan specific LoRA by checking its base model.
Install the other models by following the steps below. In some cases, the models can't be found in the model manager and you'll need to copy the model's link and upload it (see list below).
- Go to ComfyUI Manager > Click Model Manager
- When you find the exact model that you're looking for, click install and make sure to press refresh when you are finished.
Model Path Source
Use the model path source if you prefer to install the models using model's link address and paste into ThinkDiffusion MyFiles using upload URL.
| Model Name | Model Link Address | ThinkDiffusion Upload Directory |
|---|---|---|
| p0rtm4n-hunyuan-v1.1-vfx_ai.safetensors | ...comfyui/models/loras/ |
|
| hyvideo_FastVideo_LoRA-fp8.safetensors | ...comfyui/models/loras/ |
|
| hunyuan_video_720_cfgdistill_fp8_e4m3fn.safetensors | ...comfyui/models/diffusion_models/ |
|
| clip_l.safetensors | ...comfyui/models/clip/ |
|
| llava_llama3_fp8_scaled.safetensors | ...comfyui/models/clip/ |
|
| hunyuan_video_vae_bf16.safetensors | ...comfyui/models/vae/ |
Step-by-step Workflow Guide
This workflow was pretty easy to set up and runs well from the default settings. Here are a few steps where you might want to take extra note.
| Steps | Recommended Nodes |
|---|---|
| 1. Write a Prompt Write a simple prompt in the first text box which consists of a subject and a description. While in the second box write your desired filter or effects to the video. It will append as one during the generation. |
 |
| 2. Set the Models |  |
| 3. Set the Size You can set the resolution up to 1280x720. To increase speed, lower the resolution. |
 |
| 4. Check the Generation Settings Just follow the settings as seen on the image. You can also set the settings based from the instructions from the source of the LoRA you use. |
 |
| 5. Check the Generated Video |  |
Examples
Anime Style LoRA - Prompt: In a serene meadow, a single dandelion releases its seeds into the breeze, each one embarking on a delicate journey under the open sky in anime style, ultra-realistic, video realism, masterpiece, ultra high detail.
Arnold Schwarzenegger LoRA - Prompt: A 1 man, ohwx person, takes a hearty bite of a loaded burger at a lively food truck festival.
Pixel Art LoRA - Prompt: In a cozy café corner, a pensive man with a thoughtful expression gazes out the window, his face reflecting the soft glow of the afternoon sun as he sips on a steaming cup of coffee in Pixel anime pixel art drawing, ultra-realistic, video realism, masterpiece, ultra high detail
If you’re having issues with slow hardware or installation of ComfyUI, you can try any of these workflows on a more powerful GPU in your browser with ThinkDiffusion.
If you enjoy ComfyUI and you want to test out creating awesome animations, then feel free to check out this AnimateDiff tutorial here. And have fun out there with your LoRAs!
Member discussion