Lycoris/Locon Latent generation mode

[cpappasai]

Hi everyone, just making my first post to this project, because wow it's certainly effective and I want to give feedback on any bugs that might assist with development. I just noticed when I was trying it that when you apply a to the latent generation method for reigonal prompter, it does actually work; however, it seems that in post no matter what you do the lycoris will stay a part of your image generations even if the settings were entirely blank. You could type "flower" or something, and it'd have the same style without it being located in the prompt. I was curious if this were intentional behavior or something that can be fixed. Has anyone else had the same experience?

[Symbiomatrix]

Hello.
Please clarify, do you mean that the lyco bleeds to other regions, or that it appears even if you turn off regions altogether?
The former is a known issue. There should not be any lasting effects, if there are that's a severe bug, could be that an override is not getting cleared.
Can you test the following:

1. Fixed seed and parameters, generate an image normally.
2. Generate a regioned image with some lyco in latent.
3. Generate image 1 with the same settings - is it identical or different?

If it differs, please show the results and point me to the model.

Edit: Might also be useful to turn on debug mode (whilst regional is still deactivated) - if any hooks are still attached, they might print something which pinpoints which ones exactly remain.

[cpappasai]

Ah, perhaps I wasn't clear, my apologies. The ladder is what I am referring to the process being the following I have recreated a few times,

• same seed, same parameters. I have tested with multiple lycoris and that is the factor that will cause the change. I have used a thicker outline lycoris I trained for this example, but it's applied to others. To recreate this I had done the following steps,

• Load Stable Diffusion

• type "flower" and generate an image (first image)

• go into the regional prompt detector, set to LATENT

• activate lycoris on any generation

• turn off regional prompt

• try the same initial generation same seed parameters

• get different result (exact same result as original image with lycoris applied), this effect stacks if you were to repeat the process of generating with latent and then turning off

I have tested this on anylora, expmix, and meinamix thus far, all yielding the same result. One thing I'd also like to note is that seemingly high res fix also breaks as well, but only if a LORA is applied. In regards to just attention mode, lyco works fine, but high res fix only works up till the first column. Anyway, I do hope this information was useful!

[Symbiomatrix]

This is very interesting. You're saying you haven't even added BREAK / ADDX during the regional activation phase, still the prompt is just "flower"? That should have unloaded everything instantly, before processing even began.
Please clarify what you mean by "activate lycoris on any generation". The way I'm familiar with, lora / lyco are supposed to be added to the prompt using <lora:loraname:weight>. Are you perhaps using the extension version activation, with the dropdown list? Which one exactly?
And in steps 5-6, do you also deactivate the lyco, before or after deactivating regional?
As for hires, it's been a can of worms since release 1.3 and there are several open issues (hako took another stab at it the other day and it might be fixed now), so we'll leave that setting out from testing.
Edit: Actually, seems there's no self deactivation on lack of regions anymore. Wonder where it went.
Edit: I think I see this, "Add lycoris to prompt" in the latest version. Way behind on the versions. Anyway, the dropdown style is problematic in general, since there's no positional information about it - dunno to which region it should be associated.

[cpappasai]

My apologies if I wasn't very clear, but I meant that any generation after utilizing a lycoris in the prompt for a latent generation will apply no matter what even if it were in img2img, you could go to txt2img and still have it imprinted on the result. This only occurs on latent generations and not attention, also for reference I am using the extension activation with a1111-sd-webui-lycoris, in regards to steps 5-6. Once the initial generation is completed I have completed removed it from the prompt, it just is an additive until I reset SD.

Side note, outside of that though it's really impressed me, it's worked a lot better for me than latent couple extension in terms of accurate prompt outputs. Just figured I'd update you either way on the chance it assists with troubleshooting!

[Symbiomatrix]

I was unable to reproduce this in a controlled manner on several models. Since then, I've transitioned to all models running as lycos, and have now encountered this degradation in the wild. Unfortunately, a debug shows no sign of the lora forward having been overridden permanently, nor lycos & loras being prematurely loaded.
Narrows it down to txt2img only, combination of common & row, transitioning from no regions - attention - latent and back, weight adjustment, specific models, multiple models, clip skip 1-2.
Edit: I can reproduce this very minimally: a simple two region prompt, high weight on a lora (eg lykon's migurdia) in lyco folder, run latent twice, then switch to attention to get complete corruption. I think this might have to do with #141; we're loading the old extra network loras. which don't detect nor handle non loras, whilst lyco extension does its own thing in the background, and the regioning might be interrupting its loading process somehow. It's possible that cooperating with lyco directly as I proposed yonder might help, or it might require further cleanup. Anyway, don't have time to check something this heavy for a few weeks.