Are you new to creating videos with AI? If so, you're in the right place! In this guide, we'll introduce you to CogVideoX. It is perfect for beginners who want to start making amazing videos from ANY still image. This Image2Video and Text2Video model can be used inside ComfyUI, a powerful interface that makes working with image-to-video model CogVideoX easy and fun. Let's get started!
What is CogVideoX AI?
CogVideoX AI is a custom model and node that has the ability to generate videos based on input using text or images. This makes it perfect for beginners who want to make videos without learning complex video editing software. There are several models for CogVideoX that can be used. But in this guide, we're going to use CogVideoX-5B-I2V, an open-source model created by THUDM which is the recommended model for image to video generation. The I2V stands for Image2Video.
ComfyUI simplifies the process by providing ready made workflows that you can drag & drop. You can focus on being creative without worrying about building complicated workflows, making the whole experience of using ComfyUI and CogVideoX much more accessible and fun.
How to Use CogVideoX in ComfyUI
One-Time Setup
Custom Nodes
If there are red nodes in the workflow, it means that the workflow lacks the certain required nodes. Install the custom nodes in order for the workflow to work.
- Go to ComfyUI Manager > Click Install Missing Custom Nodes
- Check the list below if there's a list of custom nodes that needs to be installed and click the install.
Models
Download the recommended models (see list below) using the ComfyUI manager and go to Install models. Refresh or restart the machine after the files have downloaded.
- Go to ComfyUI Manager > Click Install Models
- When you find the exact model that you're looking for, click install and make sure to press refresh when you are finished.
Model Path Source
Use the model path source if you prefer to install the models using model's link address and paste into ThinkDiffusion MyFiles using upload URL.
| Model Name | Model Link Address |
|---|---|
| t5xxl_fp8_e4m3fn.safetensors | |
| clip_l.safetensors | Pre-loaded |
Guide Table for Upload
Reminder
Procedures
Now that the hard work is out of the way, let's get creative. You need to follow the steps from top to bottom. The workflow is a one-click process after everything has been set up.
| Steps | Default Nodes |
|---|---|
| Load an Image |  |
| Set the Image Size (720x480) |  |
| Set the Model |  |
| Write a Prompt |  |
| Set the Settings as seen on the Image. Run the Prompt |  |
| Check the Video Output |  |
Reminders
CogVideoX Examples
Click the toggle arrow to view the generation settings below.
Shoes Display
Prompt: A quick 360-degree view of shoes on a white background, capturing its unique design.
Seed - 303553149755637
steps - 50, cfg - 6, denoise - 1
clip model - t5xxl_fp8
Animate the Anime
Prompt: An anime woman dressed in traditional Japanese attire holds a glowing sparkler, the soft light illuminating her serene expression and the delicate details of her kimono.
Seed - 184363993861098
steps - 50, cfg - 6, denoise - 1
clip model - t5xxl_fp8
Food Presentation
Prompt: A close-up view of a newly cooked ramen, featuring rich, savory broth, perfectly tender noodles, slices of succulent pork, a soft-boiled egg, fresh green onions, and a sprinkle of sesame seeds.
Seed - 470516029376101
steps - 50, cfg - 6, denoise - 1
clip model - t5xxl_fp8
Coffee Demo
Prompt: A barista carefully a coffee into a blue cup.
Seed - 1040357968793837
steps - 50, cfg - 6, denoise - 1
clip model - t5xxl_fp8
If you’re having issues with installation or slow hardware, you can try any of these workflows on a more powerful GPU in your browser with ThinkDiffusion.
If you enjoy ComfyUI and you want to test out creating awesome animations, then feel free to check out this AnimateDiff tutorial here. Happy creating!
Member discussion