In the world of 3d art, having the right platform integration can make all the difference.
Today we'll be using Hyper Stable Diffusion in ComfyUI and Blender. Blender is renowned for its capabilities in creating 3D models and images. You might wonder how to integrate ComfyUI and Blender. By the end, you'll have a clear understanding of how to leverage these two in real-time. Lets get started.
What is Hyper Stable Diffusion?
ByteDance has demonstrated its dedication to both speed and innovation with the introduction of Hyper-SD. It is designed to speed up generation time significantly.
With increased speed, it also ensures that images are sharper, more detailed and visually appealing.
The image above is a comparison between Hyper SD and SDXL Lightning using 1 step. Tests show that Hyper-SD has better quality and works faster than earlier models such as SDXL-Lightning.
What is Blender?
If you're looking for 3D software, one option is Blender. It is a free and open-source software for creating 3D models, animations and images. It allows users to build, shape and paint objects in three dimensions, making it a powerful tool for artists and designers. With Blender, you can create detailed characters, elaborate scenes and stunning effects for movies, video games and digital art. The software is supported by a passionate community that shares tips, tutorials and plugins to help each other improve their skills.
Blender and Hyper SD in ComfyUI
Ever wondered how we can integrate Blender into ComfyUI? ComfyUI can connect to Blender with the custom node MixLab. This seamless integration ensures consistent and reliable results, which are essential for any professional projects. We would like to credit Andrea Baioni for his outstanding work in the development of this ComfyUI workflow where he combines the Hyper SD base model, Blender and ComfyUI.
How to Use
ComfyUI - One-Time Setup
Custom Nodes
If there are red nodes in the workflow, it means that the workflow lacks the certain required nodes. Install the custom nodes in order for the workflow to work.
- Go to ComfyUI Manager > Click Install Missing Custom Nodes
- Check the list below if there's a list of custom nodes that needs to be installed and click the install.
Models
Download the recommended models (see list below) using the ComfyUI manager and go to Install models. Refresh or restart the machine after the files have downloaded.
- Go to ComfyUI Manager > Click Install Models
- When you find the exact model that you're looking for, click install and make sure to press refresh when you are finished.
Model Path Source
Use the model path source if you prefer to install the models using model's link address and paste into ThinkDiffusion MyFiles using upload URL.
| Model Name | Model Link Address |
|---|---|
| Hyper-SDXL-1step-Unet-Comfyui.fp16.safetensors | |
| depth_anything_vitl14.pth | Pre-loaded |
| OpenPoseXL2.safetensors | |
| sai_xl_depth_128lora.safetensors | |
| sai_xl_canny_128lora.safetensors | |
| t2i-adapter_diffusers_xl_lineart.safetensors |
Guide Table for Upload
Reminder
Blender - One-Time Setup
- Download Blender here https://www.blender.org/download/. Read about the Blender system requirements here https://www.blender.org/download/requirements/
- Install Blender onto your device
- Go here https://www.mixamo.com/#/?page=1&type=Character Choose 3D characters in Characters tab then Click the Animations tab to choose what kind of movement you want for your character.
- Download the Animated Character as FBX for Unity or you may choose any FBX version to test the compatibility of your version of Blender.
- Open Blender and delete the default object Cube. Click the Select tool and click the cube and hit delete.
- Import the .fbx file to Blender
- After importing, you can see the Blender interface with its imported 3D model that will be used for the HyperSD workflow. For more info about Blender, you can read here Getting Started with Blender manual.
- The ideal perspective view of a 3D model character that will be used for the workflow.
1. free3d.com
2. cgtrader.com
3. sketchfab.com
4. 3dexport.com
and more...
Procedures
Now that the hard work is out of the way, let's get creative. You need to follow the steps from top to bottom. The workflow is a one-click process after everything has been set up.
| Steps | Detailed Steps | Default Nodes |
|---|---|---|
| Start Screen Share | 1. Click Share Screen 2. In Pop-up dialog window, select Blender screen in its window tab. 3. Click the Set Area and create a box 4. Click Live Run to establish Live connection |
 |
| Set the Image Size |  |
|
| Set the Base Model |  |
|
| Write a Prompt | Write a prompt of what you want to be in your 3D model |  |
| Set the Model | Set the model as seen on the image. |  |
| Check the Settings and Run the Queue Prompt | Set the step only in 1 and sampler name to lcm only. |  |
| Check the Image Preview | Check the preview and you can test the 3d live preview by moving or rotating the view of the 3D model in blender. |  |
Reminders
Hyper SD + Blender Image Examples
| Blender | Text Prompt | ComfyUI |
|---|---|---|
 |
Superman, background is city buildings |  |
 |
Spiderman, walking, background scary dungeon |  |
 |
Assassin, jumping, background is forest |  |
Hyper SD + Blender Real-Time Examples
If you’re having issues with installation or slow hardware, you can try any of these workflows on a more powerful GPU in your browser with ThinkDiffusion.
If you enjoy ComfyUI and you want to test out creating awesome animations, then feel free to check out this AnimateDiff tutorial here. Happy memory creating!
Member discussion