Create cool trippy animations (using Deforum)

What is Deforum?

Deforum is a powerful tool that enables you to create animation videos using Stable Diffusion. In this tutorial, we will walk you through the process of making a Deforum video.

Understanding Deforum

• Deforum is an open-source and free software for making animations.
• It utilizes Stable Diffusion's image-to-image function to generate a sequence of images, which are then stitched together to form a video.
• By applying small transformations to each image frame, Deforum creates the illusion of a continuous video.

Keyframes Tab

• Locate the Motion tab at the bottom half of the page, and click on it.
• This is where you will set the camera parameters for your video.
• The "Max frames" option determines the number of frames in your video. Higher values result in longer videos.

Setting Prompts

• Switch to the Prompts tab.
• You will see a list of prompts, each with a corresponding number indicating the frame at which the prompt becomes effective.
• For example, the first prompt is effective from the beginning of the video, the second at the 30th frame, the third at the 60th frame, and so on.
• You can modify the prompts according to your desired effects. Shown below is the default prompts that you see when you first enter the prompts tab

Generating the Video

• Click on the "Generate" button to initiate the video generation process.
• Once the generation is complete, click the button above the "Generate" button to preview the video.
• To save the video to your local storage, click the three vertical dots in the bottom right corner, or locate the video in the output directory under the "img2img-images" folder.

Motion Settings

• Motion settings play a crucial role in Deforum. By modifying them along with prompts, you can create visually stunning effects.
• We will cover the two most commonly used animation modes: 2D and 3D.

2D Motion Settings

• 2D Zoom: Adjust the zoom function to zoom in or out of the image. Use values larger than 1 to zoom in and values less than 1 to zoom out. The further the value is from 1, the faster the zoom.
• 2D Angle: Rotate the images using the 2D angle setting. Positive values rotate the image counter clockwise, while negative values rotate it clockwise. Larger values result in faster rotation.
• 2D Translation X: Move the image sideways using the Translation X parameter. Positive values shift the image to the right, and negative values shift it to the left.
• 2D Translation Y: Adjust the Translation Y parameter to move the image up and down. Positive values move the image down, and negative values move it up.
• Transform Center: This setting allows you to change the focal point of zoom and rotation.

3D Motion Settings

3D motion serves as an alternative to 2D motion. Imagine holding a camera and freely moving and rotating it in any direction.

• 3D Translation X: Move the camera sideways using the Translation X parameter. Positive values move the camera to the right, while negative values move it to the left.
• 3D Translation Y: Adjust the Translation Y parameter to move the camera up and down. Positive values move the camera up, and negative values move it down.
• Translation Z: Similar to zoom in 2D motions, Translation Z controls the zoom effect in 3D.
• Rotation X, Rotation Y, Rotation Z: These parameters allow you to rotate the camera around the X, Y, and Z axes, respectively.

Motion Schedule

• Motion settings are defined using the following format: frame1:(value1), frame2:(value2), frame3:(value3) etc...…..
• Each entry consists of two numbers separated by a colon: the frame number when the motion takes effect, and the corresponding value enclosed in brackets.
• By combining different motion settings and adjusting them over time, you can create dynamic visual effects.

An example of creating our own 3D video with Deforum

Let's create our own video by following along:-

Within the Run tab I am going to

• set the resolution to 768 x 768
• the sampler method to Euler a
• And the number of steps to 35

Within the Keyframes tab I am going to set

• The animation mode to 3D
• The Border mode to Wrap
• And the max frames to 360

For motion settings we are going to use the following values:

• Translation X - 0:(0), 30:(15), 210:(15), 300:(0)
• Translation Y - 0:(0)
• Translation Z - 0:(0.2),60:(10),300:(15)
• Rotation 3D X - 0:(0),60: (0),90: (0.5),180: (0.5),300:(0.5)
• Rotation 3D Y - 0:(0).30:(-3.5).90:(0.5).180:(-2.8).300:(-2).420:(0)
• Rotation 3D Z - 0:(0,60:(0.2),90:0 ,180:(-0.5 ,300:(0)

• For the noise schedule on the noise tab, we can enter the following:

• For the anti blur tab we are going to adjust the Amount schedule to 0: (0.05)

• For the video output tab , we are going to upscale the video as follows:

• We are now ready to enter our prompts! here's what I used:

{
"O": "ultra detailed, 8k, photorealism, dramatic lighting, ultra detailed A robot is walking through a dystopian bleak society
with dark moody lighting",
"100": "ultra detailed, 8k, photorealism, dramatic lighting, ultra detailed Robots start to fight each other with flames and
smoke surrounding the robots",
"200": "ultra detailed, 8k, photorealism, dramatic lighting, ultra detailed Buildings start to collapse around the robots",
"300": "ultra detailed, 8k, photorealism, dramatic lighting, ultra detailed, derelict buildings"
}

• You can now click generate to create your video!

With these steps, you can create impressive videos using Stable Diffusion and Deforum. Remember to experiment with different settings and prompts to unleash your creativity.

If you’re having issues with installation or slow hardware, you can try any of these workflows on a more powerful GPU in your browser with ThinkDiffusion.

If you’d like to explore more animation techniques with SD, check out my post to using Infinite Zoom here. Let me know what you cool crazy animating techniques you're using, and happy creating!