Issue with Load Checkpoint Path when using segment_anything_ultra_v2 node

[SagarAngadi1]

I have used the SegmentAnythingUltra V2 node in one of my workflows and I am hosting that workflow in 'Baseten' platform to get it as an API endpoint, the custom nodes have loaded properly but when I ran the workflow then it is giving the below error, there is some issue with sam_model and checkpoint path that is provided there, I have downloaded all the model files as said in the readme file, can you please look into this:

I had raised this issue in COmfyUI_LayerStyle_Advance repo but thought to raise it here as well.

[chflame163]

I am unable to reproduce the issue. It seems that there was an error loading the model. Perhaps we should check if the model file is damaged?

[SagarAngadi1]

Yeah actually the error originates from trying to load a checkpoint file (sam_checkpoint_path) using torch.load(). The file seems corrupted or incompatible with the model. I gave the error logs to chatgpt to analyze it and this is what it says, I have pasted it below:

The error indicates an issue with loading the model checkpoint (torch.load(f)) in the Segment Anything function. The specific message, "invalid load key, 'v'", suggests that the checkpoint file may be either corrupted, incorrectly formatted, or incompatible with the code expecting it.

Here’s a detailed breakdown and steps to resolve the issue:

---

Cause Analysis

1. Checkpoint Format Issue:

   • The checkpoint file being loaded is likely not in the expected format (PyTorch state_dict or full model serialization).

2. Path Issue:

   • The checkpoint file path (sam_checkpoint_path) might be pointing to an incorrect or corrupted file.

3. Compatibility Issue:

   • The model's loading logic might require a specific version of the torch library or the checkpoint file.

---

Steps to Resolve

Step 1: Verify Checkpoint File

• Confirm the integrity and format of the checkpoint file (sam_checkpoint_path):

  • Ensure the checkpoint file is valid and in PyTorch's expected format (.pt or .pth).
  • Test loading it locally using a simple script:

    import torch
    checkpoint = torch.load("path_to_checkpoint.pth")
    print(checkpoint.keys())  # Ensure this works

• If the file is corrupted or invalid, re-download or regenerate the checkpoint.

---

Step 2: Check the sam_model_registry and model_type

• Ensure sam_model_registry contains the correct function mapping for the model_type being passed.
• Validate the sam_checkpoint_path:
  • It should point to the correct file for the specified model_type.

---

Step 3: Debug the Issue in Isolation

• Test the loading logic outside the custom node:

  from segment_anything import sam_model_registry
  
  model_type = "vit_h"
  sam_checkpoint_path = "path_to_checkpoint.pth"
  sam = sam_model_registry[model_type](checkpoint=sam_checkpoint_path)
  print("Model loaded successfully.")

• If this fails, the issue lies with the checkpoint or the segment_anything library.

---

Step 4: Update segment_anything Library and PyTorch

• Ensure you are using the latest compatible versions:

  pip install --upgrade torch segment-anything

• Check the library's documentation for the correct versions required.

---

Step 5: Validate Custom Node Implementation

• Ensure the custom node passes the correct parameters (sam_model, sam_checkpoint_path, model_type).
• Check for hardcoded paths or incorrect mappings.

---

Step 6: Retry with a Fresh Environment

• If none of the above works, recreate the environment to rule out dependency conflicts:

  pip install torch segment-anything

---

[SagarAngadi1]

I am unable to reproduce the issue. It seems that there was an error loading the model. Perhaps we should check if the model file is damaged?

How to check if the model file is damaged? If it is damaged then how to resolve this issue, please let me know.

[chflame163]

By calculating the SHA256 value of the file and comparing it with the SHA256 value of the source file on the download page, we can determine if it is damaged. If the file is damaged, it needs to be downloaded again

[SagarAngadi1]

Okay, I will download the model file again, I have downloaded the model files from your hugging face repo which you have given in the instructions, is there any possibility of damage in the model files in your hugging face repo itself? Or can I just redownload the model file to solve the issue? Or is there anything changed in how we download the model files since you separated some nodes into another repo and the node I use 'segmentanythingultra V2' is from the ComfyUI_LayerStyle_Advance repo

[chflame163]

Or is there anything changed in how we download the model files since you separated some nodes into another repo and the node I use 'segmentanythingultra V2' is from the ComfyUI_LayerStyle_Advance repo

The model file has not undergone any changes