Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

task: Ichigo LLM v0.5 Training #122

Open
2 of 5 tasks
Tracked by #116
hahuyhoang411 opened this issue Nov 19, 2024 · 3 comments
Open
2 of 5 tasks
Tracked by #116

task: Ichigo LLM v0.5 Training #122

hahuyhoang411 opened this issue Nov 19, 2024 · 3 comments
Assignees
@tuanlda78202 @bachvudinh
Labels
P1: important Important feature / fix
Milestone
Ichigo v0.5

Comments

@hahuyhoang411
Copy link
Contributor

hahuyhoang411 commented Nov 19, 2024 •
edited by bachvudinh
Loading

Goal

Experiment on Ichigo model on multilingual.

Tasklist
@hahuyhoang411 hahuyhoang411 mentioned this issue Nov 19, 2024
Open
7 tasks
@hahuyhoang411 hahuyhoang411 changed the title Ichigo Training (Issue: ) task: Ichigo Training Nov 19, 2024
@hahuyhoang411 hahuyhoang411 added the P1: important Important feature / fix label Nov 19, 2024
@hahuyhoang411
Copy link
Contributor Author

Lora for pretraining ref: https://unsloth.ai/blog/contpretraining

@tuanlda78202
Copy link
Contributor

Paired with @bachvudinh to understand the fully trained original Ichigo pipeline.

@github-project-automation github-project-automation bot moved this to Investigating in Jan & Cortex Nov 22, 2024
@tikikun tikikun moved this from Investigating to In Progress in Jan & Cortex Nov 25, 2024
@dan-homebrew
Copy link
Contributor

@hahuyhoang411 - Can we combine this into #116

@hahuyhoang411 hahuyhoang411 changed the title task: Ichigo Training task: Ichigo LLM v0.5 Training Dec 1, 2024
@hahuyhoang411 hahuyhoang411 moved this from In Progress to Scheduled in Jan & Cortex Dec 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tuanlda78202 tuanlda78202

@bachvudinh bachvudinh

Labels
P1: important Important feature / fix
Projects
Jan & Cortex
Status: Scheduled
Milestone
Ichigo v0.5
Development

No branches or pull requests
4 participants
@hahuyhoang411 @tuanlda78202 @bachvudinh @dan-homebrew