How to choose Preprocessor?

The first step of using ControlImage is to choose a preprocessor.

Canny

Canny edge detector is a general-purpose, old-school edge detector. It extracts the outlines of an image. It is useful for retaining the composition of the original image.

Select canny in the Preprocessor menus to use.

The generated images will follow the outlines.

MLSD

M-LSD (Mobile Line Segment Detection) is a straight-line detector. It is useful for extracting outlines with straight edges like interior designs, buildings, street scenes, picture frames, and paper edges.

Curves will be ignored.

Reference

Reference is a new set of preprocessors that lets you generate images similar to the reference image. The Diffusion model and the prompt will still influence the images.

Reference preprocessors do NOT use a control model. You only need to select the preprocessor but not the model. 

The Reference preprocessor NovArch AI is using is Reference only, where it links the reference image directly to the attention layers.

Select Reference preprocessors to use.

Below is an example.

Scribble

Scribble preprocessor turn a picture into a scribble, like those drawn by hand.

• Scribble HED: Holistically-Nested Edge Detection (HED) is an edge detector good at producing outlines like an actual person would. According to ControlNet’s authors, HED is suitable for recoloring and restyling an image.

• Scribble Pidinet: Pixel Difference network (Pidinet) detects curves and straight edges. Its result is similar to HED but usually results in cleaner lines with fewer details.

• Scribble xdog: EXtended Difference of Gaussian (XDoG) is an edge detection method technique. It is important to adjust the xDoG threshold and observe the preprocessor output.

NovArch AI uses Scribble Pidinet for Architectural Visualization.

Scribble Pidinet

Pidinet tends to produce coarse lines with little detail. It’s good for copying the board outline without fine details.

Use the Scribble preprocessor to turn hand-drawn rough sketches into beautiful renderings.

QrCode Preprocessor

This model is made to generate creative QR codes that still scan. Keep in mind that not all generated codes might be readable, but you can try different parameters and prompts to get the desired results. 

How to Use

• Condition: QR codes are passed as condition images with a module size of 16px. Use a higher error correction level to make it easier to read (sometimes a lower level can be easier to read if smaller in size). Use a gray background for the rest of the image to make the code integrate better.

• Prompts: Use a prompt to guide the QR code generation. The output will highly depend on the given prompt. Some seem to be really easily accepted by the qr code process, some will require careful tweaking to get good results.

• ControlImage guidance scale: Set the ControlImage guidance scale value:

  • High values: The generated QR code will be more readable.

  • Low values: The generated QR code will be more creative.

Tips

• For an optimally readable output, try generating multiple QR codes with similar parameters, then choose the best ones.

• Use the Image-to-Image feature to improve the readability of a generated QR code:

  • Decrease the denoising strength to retain more of the original image.

  • Increase the Control Weight value for better readability. A typical workflow for "saving" a code would be: Max out the guidance scale and minimize the denoising strength, then bump the strength until the code scans.

Example Outputs

Here are some examples of creative, yet scannable QR codes produced by QrCode preprocessor:

Feel free to experiment with prompts, parameters, and the InPaint feature to achieve the desired QR code output. 

Good luck and have fun!