[FEATURE REQUEST] Support for Flux.1 ControlNet node and Lora in BizyAir

[syl1130]

Hello @doombeaker , I request to add the following enhanced features and support for models such as nodes. I really like BizyAir and am actively promoting it to more users. I hope it will continue to improve.

ControlNet

FLUX.1-dev ControlNet [XLabs Canny] Repository

• Repository URL: https://huggingface.co/XLabs-AI/flux-controlnet-canny-v3

• FLUX.1-dev ControlNet [XLabs Canny] Description and Use Case
  This repository provides a Depth ControlNet checkpoint for FLUX.1-dev model by Black Forest Labs

FLUX.1-dev ControlNet [XLabs depth] Repository

• Repository URL: https://huggingface.co/XLabs-AI/flux-controlnet-depth-v3

• FLUX.1-dev ControlNet [XLabs depth] Description and Use Case
  This repository provides a Canny ControlNet checkpoint for FLUX.1-dev model by Black Forest Labs

Inpaint (Fill, Expand, Add/Remove Object) FLUX.1-dev ControlNet Inpainting - Beta Repository

• Repository URL: https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta

• FLUX.1-dev ControlNet Inpainting - Beta Description and Use Case

Our latest inpainting model brings significant improvements compared to the previous version:

1024 Resolution Support: Capable of directly processing and generating 1024x1024 resolution images without additional upscaling steps, providing higher quality and more detailed output results.
Enhanced Detail Generation: Fine-tuned to capture and reproduce finer details in inpainted areas.
Improved Prompt Control: Offers more precise control over generated content through enhanced prompt interpretation.

[FLUX.1-Turbo-Alpha] 8-step Lora Repository

• Repository URL: https://huggingface.co/alimama-creative/FLUX.1-Turbo-Alpha

• [FLUX.1-Turbo-Alpha] 8-step Lora Description and Use Case

Improve image efficiency and increase speed by 50%:

This checkpoint is a 8-step distilled Lora, trained based on FLUX.1-dev model. We use a multi-head discriminator to improve the distill quality. Our model can be used for T2I, inpainting controlnet and other FLUX related models. The recommended guidance_scale=3.5 and lora_scale=1.

Flux IP-Adapter Stylization Repository

• Repository URL: https://github.com/logtd/ComfyUI-Fluxtapoz

• Flux IP-Adapter Stylization Description and Use Case
  Rectified Flow Inversion (Unsampling from RF Inversion)
  Update [2024.10.16]
  Stylization now works! rf_inversion_stylization

It can also be used to mix or style images (although I'm still working out the settings for this)

[doombeaker]

You can upload the loras and use it by you own, please refer to https://siliconflow.github.io/BizyAir/model-host/introduce.html to get details.

flux controlnet will be supported in a few weeks, you can upload your own controlnet and use it by BizyAir then.

RF Inversion is not on the to do list so far.

Thank you for your advice!

[syl1130]

thank you for your support

[jasoncow007]

You can upload the loras and use it by you own, please refer to https://siliconflow.github.io/BizyAir/model-host/introduce.html to get details.

flux controlnet will be supported in a few weeks, you can upload your own controlnet and use it by BizyAir then.

I tried the controlnet apply node works well with samplercustomadvanced, but get error while using Ksampler

Unexpected error accessing result["data"]["payload"]. Result: {'type': 'error', 'message': 'Error processing task e5b421cd-d6d8-4057-8dcb-36bc40ab5904: Exceeded limit for instance UNETLoader count <= 0.', 'details': 'Traceback (most recent call last):\n File "/workspace/comfybridge/comfybridge/bizyair/schedulers.py", line 287, in process_request\n result = await self._execute_task(\n File "/workspace/comfybridge/comfybridge/bizyair/schedulers.py", line 341, in _execute_task\n return await loop.run_in_executor(self.pool, executor.execute)\n File "/opt/conda/lib/python3.10/concurrent/futures/thread.py", line 58, in run\n result = self.fn(*self.args, **self.kwargs)\n File "/workspace/comfybridge/comfybridge/bizyair/schedulers.py", line 80, in execute\n return self.execute_func(*self.args, **self.kwargs)\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/utils/cuda_memory_utils.py", line 22, in wrapper\n result = func(*args, **kwargs)\n File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context\n return func(*args, **kwargs)\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/server.py", line 124, in run\n user_params: ServerFormat = convert_to_server_format(\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/transpiler/transpiler.py", line 81, in convert_to_server_format\n prompt, node_id_registry = allocator.generate_workflow_with_allocated_ids(\n File "/workspace/comfybridge/comfybridge/plugins/comfy_pipeline/transpiler/node_id_allocator.py", line 105, in generate_workflow_with_allocated_ids\n raise RuntimeError(\nRuntimeError: Exceeded limit for instance UNETLoader count <= 0\n', 'extra_info': None, 'data_status': 'COMPLETED', 'start_time': 1731129655.359425, 'end_time': 1731129655.4180846, 'task_id': 'e5b421cd-d6d8-4057-8dcb-36bc40ab5904', 'upload_to_s3': None}

[ZhaoXinyi]

When will the pose be supported?