What is ControlImage, and how it works?

ControlImage is a neural network model for controlling NovArch AI models. You can use ControlImage along with any NovArch AI models.

The most basic form of using ControlImage models is text-to-image. It uses text prompts as the conditioning to steer image generation so that you generate images that match the text prompt.

ControlImage adds one more conditioning in addition to the text prompt. The extra conditioning can take many forms in ControlImage.

Let me show you what ControlImage can do: Controlling image generation with (1) edge detection (canny).

Edge detection example

As illustrated below, ControlImage takes an additional input image and detects its outlines using the Canny edge detector. An image containing the detected edges is then saved as a control map. It is fed into the ControlImage model as an extra conditioning to the text prompt.

NovArch AI ControlImage with Canny edge conditioning.

The process of extracting specific information (edges in this case) from the input image is called Annotation (in the research docs) or Preprocessing (in the ControlImage extension).