Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

建议优化一下测试用例 #3

Open
klb3713 opened this issue Nov 9, 2023 · 2 comments
Open

建议优化一下测试用例 #3

klb3713 opened this issue Nov 9, 2023 · 2 comments

Comments

@klb3713
Copy link

klb3713 commented Nov 9, 2023 •
edited
Loading

恕我直言，从给的例子来看，这个评测和真实agent开发的情况离得太远了
给的例子，大部分更像是在评测『创作』能力，或者更形象点，有的像教小白编程的问题……

agent是应用，agent需要的大模型能力主要是面向开发者的，建议参考openai 11月发布会的新功能，构造更接近真实场景的用例
@klb3713 klb3713 changed the title 恕我直言 建议优化一下测试用例 Nov 9, 2023
@brightmart
Copy link
Member

感谢反馈。
第一阶段测试的是，LLM作为agent需要具备的基础核心能力，从工具使用、任务规划到长短期记忆。

@brightmart
Copy link
Member

如果你对LLM agent方面感兴趣，可以阅读相关材料：LLM Powered Autonomous Agents

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@klb3713 @brightmart