Skip to content

Commit

Permalink
Update leaderboard.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mtian8 authored Nov 4, 2024
1 parent bee79bb commit 9264a4b
Showing 1 changed file with 19 additions and 2 deletions.
21 changes: 19 additions & 2 deletions docs/leaderboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,24 @@

<center>

Under construction
# SciCode Leaderboard

| Model | Main Problem Resolve Rate |
|------------------------|---------------------------|
| OpenAI o1-preview | 7.7% |
| Claude3.5-Sonnet | 4.6% |
| Deepseek-Coder-v2 | 3.1% |
| GPT-4o | 1.5% |
| GPT-4-Turbo | 1.5% |
| OpenAI o1-mini | 1.5% |
| Gemini 1.5 Pro | 1.5% |
| Claude3-Opus | 1.5% |
| Claude3-Sonnet | 1.5% |
| Qwen2-72B-Instruct | 1.5% |
| Llama-3.1-405B-Instruct| 0% |
| Llama-3.1-70B-Instruct | 0% |
| Mixtral-8x22B-Instruct | 0% |
| Llama-3-70B-Chat | 0% |

<!-- Once you've added the results to the submission repository,
bring back the table here -->
Expand All @@ -11,4 +28,4 @@ Under construction
</center>

!!! tip "How to submit"
Want to submit your own model? Head over to the [documentation](docs/index.md).
Want to submit your own model? Head over to the [documentation](docs/index.md).

0 comments on commit 9264a4b

Please sign in to comment.