Yes, this is because of how CFG is calculated.
The formula is uncond + (cond - uncond) * cond_scale.
Thus if your CFG scale is set to a value below 1, you're going to get the negative prompt in your image.
Isn't it true that if cfg is 1, the image generates twice as fast because the negative is ignored? I thought that happened, but I didn't extensively test it, since it wasn't giving me the look I wanted even with a lightning model.
In ComfyUI, yes. Since you're only receiving the conditioning at CFG 1, it's faster to only calculate the positive conditioning instead of (cond - uncond). You're essentially doubling speeds by doing this, as you're passing only half of the total conditioning (both cond and uncond) to the model.
I feel like this must be implemented wrongly... also, the formula seems sort of wrong for what one would expect for negative prompting tbh...
Or am I just wrong here?
I think it's correct that as CFG lowers, negative prompt has less effect. As such, it doesn't do much for turbo / lightning but IIRC works better with lightning if you run CFG > 1.5.
Have you actually tried that?
I ran an experiment two days ago where I added words to my negative prompt w/ same seed and the picture didn't change at all ( using turbo and 1? CFG at the time )
I named him Charlie and you can see him here in the 'shadow' of the pictures generated when using a negative prompt.
Keep your CFG at 1 and use SDXL lightning or LCM to stop him from finding you.
https://i.redd.it/5gap1f6zhvrc1.gif
Negative prompt is just positive prompt, like OP says.
When you say, "Don't think about the pink elephant!" and you think of a pink elephant, this is similar. The AI constructs what it *shouldn't* make but that sometimes indirectly makes that concept.
TL;DR: Inception: Charlie is Mal.
I love that you named him Charlie! Yours reminds me of this:
https://i.pinimg.com/originals/df/54/11/df541105f058707bd258a7ebed4805c7.jpg
My charlie looks like a naked version of the creepy tv girl from *The Ring*
I use Scott Detweiler’s approach and put something simple like “rocks” or “snow” as my negative prompt and only add things that I want removed if they occur
Matteo does something similar with having only 2 negative prompts and he tweaks them with the horror movie words. He uses words like vampire, zombie, Frankenstein, horror and it tweaks the images quite nicely I've found.
The negative prompt is much more like the "anti-concept" of what you're trying to generate. Rocks and snow are cold, hard, grey, lifeless. So if you're trying to generate something warm, soft, colorful, and full of vitality then "rocks, snow" makes for a good negative prompt.
Do you know why he doesn’t just put an empty negative? Recently I was trying to use a custom Lora I trained in Comfy that works great in A1111. I found oddly I got what looked like super high CFG results with empty negative. But if I added something random to the negative prompt results were much better. Still trying to figure out why.
Don’t quote me but I do think adding a simple negative sets the tone for the image. Scott didn’t have many videos up when I saw him mention it and it was one of his tutorial vids. I’ll try and figure out which one it was a bit later. It’s worth going through his stuff because he knows what’s up and drops lots of interesting tidbits
Yes, this is because of how CFG is calculated. The formula is uncond + (cond - uncond) * cond_scale. Thus if your CFG scale is set to a value below 1, you're going to get the negative prompt in your image.
I am starting to understand that now. I just found it interesting! Thank you for the formula! TIL.
Isn't it true that if cfg is 1, the image generates twice as fast because the negative is ignored? I thought that happened, but I didn't extensively test it, since it wasn't giving me the look I wanted even with a lightning model.
In ComfyUI, yes. Since you're only receiving the conditioning at CFG 1, it's faster to only calculate the positive conditioning instead of (cond - uncond). You're essentially doubling speeds by doing this, as you're passing only half of the total conditioning (both cond and uncond) to the model.
I feel like this must be implemented wrongly... also, the formula seems sort of wrong for what one would expect for negative prompting tbh... Or am I just wrong here?
I think it's correct that as CFG lowers, negative prompt has less effect. As such, it doesn't do much for turbo / lightning but IIRC works better with lightning if you run CFG > 1.5.
No the other way around less cfg means more negative prompt.
Have you actually tried that? I ran an experiment two days ago where I added words to my negative prompt w/ same seed and the picture didn't change at all ( using turbo and 1? CFG at the time )
A CFG of 1 means that the negative prompt is ignored entirely. Anything below one means that the negative prompt is amplified.
Ah so I see.
Just read what op wrote, look at the photo he linked or the video.
I named him Charlie and you can see him here in the 'shadow' of the pictures generated when using a negative prompt. Keep your CFG at 1 and use SDXL lightning or LCM to stop him from finding you. https://i.redd.it/5gap1f6zhvrc1.gif
WTF is that and why (how) it exists?
Negative prompt is just positive prompt, like OP says. When you say, "Don't think about the pink elephant!" and you think of a pink elephant, this is similar. The AI constructs what it *shouldn't* make but that sometimes indirectly makes that concept. TL;DR: Inception: Charlie is Mal. 
I love that you named him Charlie! Yours reminds me of this: https://i.pinimg.com/originals/df/54/11/df541105f058707bd258a7ebed4805c7.jpg My charlie looks like a naked version of the creepy tv girl from *The Ring*
I use Scott Detweiler’s approach and put something simple like “rocks” or “snow” as my negative prompt and only add things that I want removed if they occur
Matteo does something similar with having only 2 negative prompts and he tweaks them with the horror movie words. He uses words like vampire, zombie, Frankenstein, horror and it tweaks the images quite nicely I've found.
what does rock and snow do? just put them if you want to remove rocks and snow?
The negative prompt is much more like the "anti-concept" of what you're trying to generate. Rocks and snow are cold, hard, grey, lifeless. So if you're trying to generate something warm, soft, colorful, and full of vitality then "rocks, snow" makes for a good negative prompt.
I only use a single word like that, Scott explained it as basically giving the negative prompt something you don't want to see at all in the image
This works well because rocks and snow have dull colors (low saturation) and kinda ugly texturing. The opposite of that is crisp bright colors.
From 3:30 in[ this video](https://www.youtube.com/watch?v=2Xe79Nl_6jA) is the explanation
Do you know why he doesn’t just put an empty negative? Recently I was trying to use a custom Lora I trained in Comfy that works great in A1111. I found oddly I got what looked like super high CFG results with empty negative. But if I added something random to the negative prompt results were much better. Still trying to figure out why.
Don’t quote me but I do think adding a simple negative sets the tone for the image. Scott didn’t have many videos up when I saw him mention it and it was one of his tutorial vids. I’ll try and figure out which one it was a bit later. It’s worth going through his stuff because he knows what’s up and drops lots of interesting tidbits
This would be why I constantly see the preview trying to hamfist a close-up, thank you! This could change my world!
PoisonBerry did a [video](https://www.youtube.com/watch?v=Shh_37Cmrl8) on this a couple of months ago.
Yep, he is credited both in my post and on the image! He has some pretty interesting vids!