What is ControlNet Scribble?
ControlNet Scribble refers to a technique or feature that allows users to provide manual annotations or markings on an image to guide the generation process.
With ControlNet Scribble, users can draw or scribble directly on the image to specify desired modifications, edits, or attributes they want to see in the generated output.
By leveraging ControlNet Scribble, users can have more fine-grained control over the generated images. The model takes into account the marked areas or annotations and uses them as additional guidance to influence the generated output. This allows users to actively participate in shaping the outcome and tailor it according to their preferences.
ControlNet Scribble is particularly useful when users want to make specific changes or adjustments to certain parts of an image, such as modifying the appearance of objects, altering the background, or emphasizing particular elements. It provides an interactive and intuitive way for users to communicate their desired modifications directly to the Stable Diffusion model.
When it comes to unleashing your artistic flair, Scribble Preprocessors are an absolute game-changer. They effortlessly transform ordinary pictures from hand-drawn scribbles, adding that authentic touch to your creations.
What Scribble pre-processors are available?
To find the Scribble pre-processors:
- First make sure you're on the txt2img tab.
- Scroll down, and expand out the ControlNet panel.
- You'll find the Pre-Processors inside the panel.
Option 1: Scribble HED
Harnessing the power of Holistically-Nested Edge Detection (HED), this pre-processor excels at generating outlines that closely resemble those hand-drawn by a skilled artist. ControlNet's experts recommend HED for recolouring and restyling images, infusing them with a unique artistic flair.
With its ability to produce coarse yet expressive scribble lines, HED lends a unique character to your artwork, adding an intriguing visual element that captures attention.
Option 2: Scribble Pidinet
If you're seeking crisp curves and straight edges, look no further than Pixel Difference network (Pidinet). With its ability to detect and emphasize clean lines, Pidinet grants your artwork a polished look while preserving essential details. It's perfect for capturing the board outline or any other intricate shapes without overwhelming intricacies.
When you need a touch of elegance, Pidinet steps in, delivering refined lines with minimal intricacies. It's the ideal choice for replicating board outlines or creating clean, sophisticated designs.
Option 3: Scribble xDoG
Enter the realm of Extended Difference of Gaussian (xDoG), a remarkable edge detection technique. By adjusting the xDoG threshold, you have the power to control the level of detail in your artwork. This versatile pre-processor allows you to strike the perfect balance, resulting in captivating scribbles that showcase just the right amount of intricacy.
Find the perfect balance of detail by fine-tuning the XDoG threshold. This exceptional pre-processor allows you to create scribbles that strike the right chord, capturing the essence of your artistic vision.
Examples of using the ControlNet Scribble pre-processors
Pre-Processor 1: Scribble HED
Let's say we tranform a hand drawing of an elephant using Scribble HED, we can
- (1) Select the control type to be Scribble,
- (2) Set the pre-processor to scribble_hed
- (3) and control_sd15_scribble as the model as shown below:
We can now:
- Upload our image to the single image tab within the ControlNet section
- (1) I have selected 'RealisticVisionV20' as the SD model
- (2) DPM++2M Karras as the sampling method
- (3) 40 sampling steps
- (4) And a CFG of 15.
- (5) Check the 'enable controlnet' checkbox
- (6) And check the 'pixel perfect' checkbox.
We can then enter our positive and negative prompts as shown below:
Positive Prompts:
An elephant with an Indian dressing on his trunk wearing feathers, (Extremely Detailed Oil Painting:1.2), glow effects, godrays, Hand drawn, render, 8k, octane render, cinema 4d, blender, dark, atmospheric 4k ultra detailed, cinematic sensual, Sharp focus, humorous illustration, big depth of field, Masterpiece, colors, 3d octane render, 4k, concept art, trending on artstation, hyperrealistic, Vivid colors, extremely detailed CG unity 8k wallpaper, trending on ArtStation, trending on CGSociety, Intricate, High Detail, dramatic
Negative Prompts:
Watermark, Text, censored, deformed, bad anatomy, disfigured, mutated, extra limb, ugly, missing limb, floating limbs, disconnected limbs, disconnected head, malformed hands, cropped, worst quality, low quality, mutation, poorly drawn
Et Voila!
Pre-Processor 2: Scribble Pidinet
Let's try a hand drawing of a bunny with Pidinet, we can:
- (1) Select the control type to be Scribble
- (2) The pre-processor to scribble_pidinet
- (3) And control_sd15_scribble.
We can now upload our image to the single image tab within the ControlNet section
- (1) I have selected 'RealisticVisionV20' as the SD model
- (2) DPM++2M Karras as the sampling method
- (3) 40 sampling steps
- (4) And a CFG of 15.
- (5) Check the 'enable controlnet' checkbox
- (6) And check the 'pixel perfect' checkbox.
Positive Prompts: A colourful bunny, (Extremely Detailed Oil Painting:1.2), glow effects, godrays, Hand drawn, render, 8k, octane render, cinema 4d, blender, dark, atmospheric 4k ultra detailed, cinematic sensual, Sharp focus, humorous illustration, big depth of field, Masterpiece, colors, 3d octane render, 4k, concept art, trending on artstation, hyperrealistic, Vivid colors, extremely detailed CG unity 8k wallpaper, trending on ArtStation, trending on CGSociety, Intricate, High Detail, dramatic
Negative Prompts: Watermark, Text, censored, deformed, bad anatomy, disfigured, mutated, extra limb, ugly, missing limb, floating limbs, disconnected limbs, disconnected head, malformed hands, cropped, worst quality, low quality, mutation, poorly drawn
There we go!
Pre-Processor3: Scribble xDog
One more, let's see what xDog does with a hand drawing of a butterfly:
- (1) Select the control type to be Scribble
- (2) The pre-processor to scribble_xDog
- (3) And control_sd15_scribble
We can now upload our image to the single image tab within the ControlNet section:
- (1) I have selected 'RealisticVisionV20' as the SD model
- (2) DPM++2M Karras as the sampling method
- (3) 40 sampling steps
- (4) And a CFG of 15.
- (5) Check the 'enable controlnet' checkbox
- (6) And check the 'pixel perfect' checkbox.
We can then enter our positive and negative prompts as shown below:
Positive Prompts: (extremely detailed CG unity 8k wallpaper), a beautiful butterfly, professional majestic oil painting by Ed Blinkley, Atey Ghailan, Studio Ghibly, by Jeremy Mann, Greg Manchess, Antonio Moro, trending on artstation, trending on CGSociety, Intricate, High Detail, Sharp focus, dramatic, photorealistic paniting art by MidJourney and Greg Rutowski
Negative Prompts: Watermark, Text, censored, deformed, bad anatomy, disfigured, mutated, extra limb, ugly, missing limb, floating limbs, disconnected limbs, disconnected head, malformed hands, cropped, worst quality, low quality, mutation, poorly drawn
Tada!
Increasing the Quality Even Further
There tons of ways to push the image quality of your image even further, especially once you've established general settings that you like.
Enhance Faces & Hands
Open the ADetailer dropdown and click "Enable ADetailer". This will enhance the faces by default. By clicking on the 2nd tab, you can also enhance hands as well. More details on our dedicated tutorial here.
Enhance the image resolution
You can upscale the image while it's generating, which gives the AI the opportunity to use your image as a base to then create a larger more detailed version in a 2nd step. Simply click the "Hires Fix"and it will upscale your image 2x by default.
Open the dropdown and you'll find various settings to scale your image. Play with various upscaling models to find the one that best suits your image. Latent works well as a default Upscaling model.
Denoising strength changes how much variation you'd like to see between the original image and the upscaled result.
Wrap Up
Remember, for optimal results, all these pre-processors work seamlessly in conjunction with the Scribble Control Model, enabling you to unlock your artistic potential and bring your visions to life and reflect your unique artistic style.
If you’re having issues with installation or slow hardware, you can try any of these workflows on a more powerful GPU in your browser with ThinkDiffusion.
If you’d like to have more control over your characters by posing with a stick figure instead of drawing, check out my post to using OpenPose here. Happy scribbling!
Member discussion