Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support new hf.co/ format #377

Closed
wants to merge 1 commit into from
Closed

Support new hf.co/ format #377

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions ramalama/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -688,13 +688,38 @@ def dry_run(args):
print()


def convert_model_format(input_string):
# Split the input string to extract the relevant parts
parts = input_string.split('/')
if len(parts) != 3 or not parts[2]:
raise ValueError("Input string is not in the expected format")

username = parts[1]
model_info = parts[2]

# Check if quantization is specified, else default to Q4_K_M
if ':' in model_info:
model_name, quantization = model_info.split(':')
else:
model_name = model_info
quantization = 'Q4_K_M' # Default quantization scheme

# Construct the new format
return f"hf://{username}/{model_name}/{model_name}-{quantization}.gguf"
Copy link
Contributor

@swarajpande5 swarajpande5 Oct 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ericcurtin I would like to take this up, the new format which is being returned might not always be correct.
For example, running the following,

ramalama pull hf.co/instructlab/merlinite-7b-lab-GGUF

will use the following new format,

hf://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-GGUF-Q4_K_M.gguf

And the above path does not exist.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... Maybe there's a REST API to get the real filename if you pass it instructlab/merlinite-7b-lab-GGUF and Q4_K_M or something like that but I'm really guessing...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there should be one to get the complete path. I'll explore this :)



def New(model, args):
if model.startswith("huggingface://") or model.startswith("hf://"):
return Huggingface(model)
if model.startswith("huggingface.co/") or model.startswith("hf.co/"):
model = convert_model_format(model)
return Huggingface(model)
if model.startswith("ollama://"):
return Ollama(model)
if model.startswith("oci://") or model.startswith("docker://"):
return OCI(model, args.engine)
if model.startswith("https://"):
raise NotImplementedError("https support is to be implemented")

transport = os.getenv("RAMALAMA_TRANSPORT")
if transport == "huggingface":
Expand Down
Loading