[BFCL] Add New Model watt-tool-8B and watt-tool-70B (#847)

This PR adds the model `watt-ai/watt-tool-8B` and `watt-ai/watt-tool-70B` to the leaderboard.
ShishirPatil · Dec 21, 2024 · 1792fb1 · 1792fb1
1 parent 68a0500
commit 1792fb1
Show file tree

Hide file tree

Showing 4 changed files with 17 additions and 0 deletions.
diff --git a/berkeley-function-call-leaderboard/CHANGELOG.md b/berkeley-function-call-leaderboard/CHANGELOG.md
@@ -2,6 +2,7 @@
 
 All notable changes to the Berkeley Function Calling Leaderboard will be documented in this file.
 
+- [Dec 21, 2024] [#847](https://github.com/ShishirPatil/gorilla/pull/847): Add new model `watt-ai/watt-tool-8B` and `watt-ai/watt-tool-70B` to the leaderboard.
 - [Dec 20, 2024] [#842](https://github.com/ShishirPatil/gorilla/pull/842): Add the following new models to the leaderboard:
   - `Qwen/Qwen2.5-0.5B-Instruct`
   - `Qwen/Qwen2.5-3B-Instruct`

diff --git a/berkeley-function-call-leaderboard/SUPPORTED_MODELS.md b/berkeley-function-call-leaderboard/SUPPORTED_MODELS.md
@@ -89,6 +89,8 @@ Below is a comprehensive table of models supported for running leaderboard evalu
 |openbmb/MiniCPM3-4B 💻| Prompt|
 |THUDM/glm-4-9b-chat 💻| Function Calling|
 |Team-ACE/ToolACE-8B 💻| Function Calling|
+|watt-ai/watt-tool-8B 💻| Function Calling|
+|watt-ai/watt-tool-70B 💻| Function Calling|
 
 ---
 

diff --git a/berkeley-function-call-leaderboard/bfcl/eval_checker/model_metadata.py b/berkeley-function-call-leaderboard/bfcl/eval_checker/model_metadata.py
@@ -745,6 +745,18 @@
         "openbmb",
         "Apache-2.0",
     ],
+    "watt-ai/watt-tool-8B": [
+        "watt-tool-8B (FC)",
+        "https://huggingface.co/watt-ai/watt-tool-8B/",
+        "Watt AI Lab",
+        "Apache-2.0",
+    ],
+    "watt-ai/watt-tool-70B": [
+        "watt-tool-70B (FC)",
+        "https://huggingface.co/watt-ai/watt-tool-70B/",
+        "Watt AI Lab",
+        "Apache-2.0",
+    ],
     "BitAgent/GoGoAgent": [
         "GoGoAgent",
         "https://gogoagent.ai",

diff --git a/berkeley-function-call-leaderboard/bfcl/model_handler/handler_map.py b/berkeley-function-call-leaderboard/bfcl/model_handler/handler_map.py
@@ -142,6 +142,8 @@
     "Team-ACE/ToolACE-8B": LlamaHandler,
     "openbmb/MiniCPM3-4B": MiniCPMHandler,
     "openbmb/MiniCPM3-4B-FC": MiniCPMFCHandler,
+    "watt-ai/watt-tool-8B": LlamaHandler,
+    "watt-ai/watt-tool-70B": LlamaHandler,
     "deepseek-ai/DeepSeek-V2.5": DeepseekCoderHandler,
     "deepseek-ai/DeepSeek-Coder-V2-Instruct-0724": DeepseekCoderHandler,
     "deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct": DeepseekCoderHandler,