Skip to content

Commit

Permalink
Florence2 nodes support PromptGen-v2.0
Browse files Browse the repository at this point in the history
  • Loading branch information
chflame163 committed Nov 8, 2024
1 parent edceb07 commit 57d737f
Show file tree
Hide file tree
Showing 4 changed files with 16 additions and 2 deletions.
1 change: 1 addition & 0 deletions README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ Please try downgrading the ```protobuf``` dependency package to 3.20.3, or set e

<font size="4">**If the dependency package error after updating, please double clicking ```repair_dependency.bat``` (for Official ComfyUI Protable) or ```repair_dependency_aki.bat``` (for ComfyUI-aki-v1.x) in the plugin folder to reinstall the dependency packages. </font><br />

* Florence2 nodes support base-PromptGen-v2.0 and large-PromptGen-v2.0, Download ```base-PromptGen-v2.0``` and ```large-PromptGen-v2.0``` two folder from [huggingface](https://huggingface.co/chflame163/ComfyUI_LayerStyle/tree/main/ComfyUI/models/florence2) or [BaiduNetdisk](https://pan.baidu.com/s/1BVvXt3N7zrBnToyF-GrC_A?pwd=xm0x) and copy to ```ComfyUI/models/florence2``` folder.
* [SAM2Ultra](#SAM2Ultra) and ObjectDetector nodes support image batch.
* [SAM2Ultra](#SAM2Ultra) and [SAM2VideoUltra](#SAM2VideoUltra) nodes add support for SAM2.1 model, including [kijai](https://github.com/kijai)'s FP16 model. Download model files from [BaiduNetdisk](https://pan.baidu.com/s/1xaQYBA6ktxvAxm310HXweQ?pwd=auki) or [huggingface.co/Kijai/sam2-safetensors](https://huggingface.co/Kijai/sam2-safetensors/tree/main) and copy to ```ComfyUI/models/sam2``` folder.
* Commit [JoyCaption2Split](#JoyCaption2Split) and [LoadJoyCaption2Model](#LoadJoyCaption2Model) nodes, Sharing the model across multiple JoyCaption2 nodes improves efficiency.
Expand Down
1 change: 1 addition & 0 deletions README_CN.MD
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ If this call came from a _pb2.py file, your generated code is out of date and mu
## 更新说明
<font size="4">**如果本插件更新后出现依赖包错误,请双击运行插件目录下的```install_requirements.bat```(官方便携包),或 ```install_requirements_aki.bat```(秋叶整合包) 重新安装依赖包。

* Florence2 节点更新支持 base-PromptGen-v2.0 和 large-PromptGen-v2.0 模型。从 [huggingface](https://huggingface.co/chflame163/ComfyUI_LayerStyle/tree/main/ComfyUI/models/florence2)[BaiduNetdisk](https://pan.baidu.com/s/1BVvXt3N7zrBnToyF-GrC_A?pwd=xm0x) 下载 ```base-PromptGen-v2.0``````large-PromptGen-v2.0``` 两个文件夹并复制到 ```ComfyUI/models/florence2```
* [SAM2Ultra](#SAM2Ultra) 及 ObjectDetector 节点支持图像批次。
* [SAM2Ultra](#SAM2Ultra)[SAM2VideoUltra](#SAM2VideoUltra) 节点增加支持SAM2.1模型,包括[kijai](https://github.com/kijai)量化版fp16模型。请从请从[百度网盘](https://pan.baidu.com/s/1xaQYBA6ktxvAxm310HXweQ?pwd=auki) 或者 [huggingface.co/Kijai/sam2-safetensors](https://huggingface.co/Kijai/sam2-safetensors/tree/main)下载模型文件并复制到```ComfyUI/models/sam2```文件夹。
* 添加 [JoyCaption2Split](#JoyCaption2Split)[LoadJoyCaption2Model](#LoadJoyCaption2Model) 节点,在多个JoyCaption2节点时共用模型提高效率。
Expand Down
14 changes: 13 additions & 1 deletion py/florence2_ultra.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@
"CogFlorence-2-Large-Freeze": "thwri/CogFlorence-2-Large-Freeze",
"CogFlorence-2.1-Large": "thwri/CogFlorence-2.1-Large",
"base-PromptGen-v1.5":"MiaoshouAI/Florence-2-base-PromptGen-v1.5",
"large-PromptGen-v1.5":"MiaoshouAI/Florence-2-large-PromptGen-v1.5"
"large-PromptGen-v1.5":"MiaoshouAI/Florence-2-large-PromptGen-v1.5",
"base-PromptGen-v2.0":"MiaoshouAI/Florence-2-base-PromptGen-v2.0",
"large-PromptGen-v2.0":"MiaoshouAI/Florence-2-large-PromptGen-v2.0"
}

def fixed_get_imports(filename) -> list[str]:
Expand Down Expand Up @@ -322,6 +324,14 @@ def process_image(model, processor, image, task_prompt, max_new_tokens, num_beam
task_prompt = '<MIXED_CAPTION>'
result = run_example(model, processor, task_prompt, image, max_new_tokens, num_beams, do_sample)
return result[task_prompt], None
elif task_prompt == 'mixed caption plus(PromptGen 2.0)':
task_prompt = '<MIXED_CAPTION_PLUS>'
result = run_example(model, processor, task_prompt, image, max_new_tokens, num_beams, do_sample)
return result[task_prompt], None
elif task_prompt == 'analyze(PromptGen 2.0)':
task_prompt = '<<ANALYZE>>'
result = run_example(model, processor, task_prompt, image, max_new_tokens, num_beams, do_sample)
return result[task_prompt], None

else:
return "", None # Return empty string and None for unknown task prompts
Expand Down Expand Up @@ -533,6 +543,8 @@ def INPUT_TYPES(s):
'description',
'generate tags(PromptGen 1.5)',
'mixed caption(PromptGen 1.5)',
'mixed caption plus(PromptGen 2.0)',
'analyze(PromptGen 2.0)',
"object detection",
"dense region caption",
"region proposal",
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[project]
name = "comfyui_layerstyle"
description = "A set of nodes for ComfyUI it generate image like Adobe Photoshop's Layer Style. the Drop Shadow is first completed node, and follow-up work is in progress."
version = "1.0.88"
version = "1.0.89"
license = "MIT"
dependencies = ["numpy", "pillow", "torch", "matplotlib", "Scipy", "scikit_image", "scikit_learn", "opencv-contrib-python", "pymatting", "segment_anything", "timm", "addict", "yapf", "colour-science", "wget", "mediapipe", "loguru", "typer_config", "fastapi", "rich", "google-generativeai", "diffusers", "omegaconf", "tqdm", "transformers", "kornia", "image-reward", "ultralytics", "blend_modes", "blind-watermark", "qrcode", "pyzbar", "transparent-background", "huggingface_hub", "accelerate", "bitsandbytes", "torchscale", "wandb", "hydra-core", "psd-tools", "inference-cli[yolo-world]", "inference-gpu[yolo-world]", "onnxruntime", "peft", "iopath"]

Expand Down

0 comments on commit 57d737f

Please sign in to comment.