1

I want to design a prompt where I already have the main thing in the image fixed, but I need to have a text included in the image with exact word(s) I put in there. For example, this is my prompt.

a flying bird, text "Bird" in the background

The result only generates a flying bird. No text in the image, let alone the exact word.

How to generate exact word(s) in generated image?

I am using this tool.

1

1 Answer 1

2

ControlNets and T2I adapters both allow you to condition generation on some kind of image input rather than just a text prompt. Best option for exact text would likely be the ControlNet or T2I adapter for edges:

These are available for Stable Diffusion 1.5, which is used by the linked web demo, and also Stable Diffusion XL. You could use the ControlNet/T2I adapter scripts in their repos, but I'd recommend using ComfyUI - or Automatic1111's web UI with its ControlNet extension.

You can first generate an image of a bird as normal:

Bird
(image contains ComfyUI workflow graph metadata - can be downloaded and dragged into the UI)

Then filter it with a canny edge detector. Below I used the "Image Canny Filter" from WAS Node Suite, but any method should work:

Bird canny edges
(image contains ComfyUI workflow graph metadata)

Repeat for some text "Bird"

"Bird" text canny edges

Combine the two:

Combined canny edges

Then, as shown below, feed this alongside the text prompt into the ControlNet/T2I Adapter to adjust the conditioning:

ComfyUI workflow

Result:

Generated image of bird flying in front of "Bird" text
(image contains ComfyUI workflow graph metadata)

1
  • 1
    This feels like doing the photoshop work to me. I'll wait for some time to see if anyone can answer with just prompt; otherwise, I will accept this answer later.
    – holydragon
    Commented Aug 8, 2023 at 2:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.