-
Notifications
You must be signed in to change notification settings - Fork 146
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
7 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,23 +2,22 @@ | |
ComfyUI implementation of https://github.com/layerdiffusion/LayerDiffuse. | ||
|
||
## Installation | ||
Download the repository and unpack into the custom_nodes folder in the ComfyUI installation directory. | ||
Download the repository and unpack it into the custom_nodes folder in the ComfyUI installation directory. | ||
|
||
Or clone via GIT, starting from ComfyUI installation directory: | ||
```bash | ||
cd custom_nodes | ||
git clone [email protected]:huchenlei/ComfyUI-layerdiffuse.git | ||
``` | ||
|
||
Run `pip install -r requirements.txt` to install python dependencies. You might experience version conflict on diffusers if you have other extensions | ||
that depends on other versions of diffusers. In this case, it is recommended to setup separate Python venvs. | ||
Run `pip install -r requirements.txt` to install python dependencies. You might experience version conflict on diffusers if you have other extensions that depend on other versions of diffusers. In this case, it is recommended to set up separate Python venvs. | ||
|
||
## Workflows | ||
### [Generate foreground](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_fg_example_rgba.json) | ||
![rgba](https://github.com/huchenlei/ComfyUI-layerdiffuse/assets/20929282/5e6085e5-d997-4a0a-b589-257d65eb1eb2) | ||
|
||
### [Generate foreground (RGB + alpha)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_fg_example.json) | ||
If you want more control of getting RGB image and alpha channel mask separately, you can use this workflow. | ||
If you want more control of getting RGB images and alpha channel mask separately, you can use this workflow. | ||
![readme1](https://github.com/huchenlei/ComfyUI-layerdiffuse/assets/20929282/4825b81c-7089-4806-bce7-777229421707) | ||
|
||
### [Blending (FG/BG)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_example.json) | ||
|
@@ -33,20 +32,20 @@ Blending given BG | |
|
||
### [Extract BG from Blended + FG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_bg.json) | ||
[Forge impl's sanity check](https://github.com/layerdiffuse/sd-forge-layerdiffuse#sanity-check) sets `Stop at` to 0.5 to get better quality BG. | ||
This workflow might be inferior comparing to other object removal workflows. | ||
This workflow might be inferior compared to other object removal workflows. | ||
![diff_fg](https://github.com/huchenlei/ComfyUI-layerdiffuse/assets/20929282/05a10add-68b0-473a-acee-5853e4720322) | ||
|
||
### [Extract BG from Blended + FG (Stop at 0.5)](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_diff_bg_stop_at.json) | ||
In [SD Forge impl](https://github.com/layerdiffuse/sd-forge-layerdiffuse), there is a `stop at` param that determines when | ||
layer diffuse should stop in the denosing process. In the background, what this param does is unapply the LoRA and c_concat cond after a certain step | ||
threshold. This is hard/risky to implement directly in ComfyUI as it requires manually load a model that has every changes except the layer diffusion | ||
layer diffuse should stop in the denoising process. In the background, what this param does is unapply the LoRA and c_concat cond after a certain step | ||
threshold. This is hard/risky to implement directly in ComfyUI as it requires manually loading a model that has every change except the layer diffusion | ||
change applied. A workaround in ComfyUI is to have another img2img pass on the layer diffuse result to simulate the effect of `stop at` param. | ||
![diff_fg_stop_at](https://github.com/huchenlei/ComfyUI-layerdiffuse/assets/20929282/e383c9d3-2d47-40c2-b764-b0bd48243ee8) | ||
|
||
|
||
### [Generate FG from BG combined](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_fg_all.json) | ||
Combines previous workflows to generate blended and FG given BG. We found that there are some color variations in the extracted FG. Need to confirm | ||
with layer diffusion authors on whether this is expected. | ||
with layer diffusion authors whether this is expected. | ||
![fg_all](https://github.com/huchenlei/ComfyUI-layerdiffuse/assets/20929282/f4c18585-961a-473a-a616-aa3776bacd41) | ||
|
||
### [2024-3-9] [Generate FG + Blended given BG](https://github.com/huchenlei/ComfyUI-layerdiffuse/blob/main/examples/layer_diffusion_cond_joint_bg.json) | ||
|