Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] enhance list_files function #252

Merged
merged 1 commit into from
Oct 18, 2024

Conversation

leeeizhang
Copy link
Collaborator

@leeeizhang leeeizhang commented Oct 18, 2024

from mle.function import list_files
print(list_files("./", max_depth=3, max_items=5)) # its return will never exceed (max_items ^ max_depth) + 1 lines
/root/workspace/MLE-agent
│── dist/  (0 directories, 1 files)
│   └── mle_agent-0.3.1-py3-none-any.whl (303,703 bytes)
│── assets/  (0 directories, 1 files)
│   └── kaia_llama.webp (93,848 bytes)
│── web/  (5 directories, 11 files)
│   │── app/  (1 directories, 3 files)
│   │── .next/  (4 directories, 5 files)
│   │── public/  (3 directories, 2 files)
│   │── src/  (6 directories, 0 files)
│   │── node_modules/  (28 directories, 1 files)
│   │── next.config.mjs (92 bytes)
│   │── postcss.config.mjs (135 bytes)
│   │── tsconfig.json (26 lines)
│   │── .gitignore (391 bytes)
│   │── package-lock.json (7963 lines)
│   └── ... (remaining: 6 files)
│── .github/  (1 directories, 4 files)
│   │── workflows/  (0 directories, 3 files)
│   │── ISSUE_TEMPLATE.md (24 lines)
│   │── PULL_REQUEST_TEMPLATE.md (21 lines)
│   │── OPEN_SOURCE_LICENSES.md (0 lines)
│   └── CODEOWNERS (12 bytes)
│── mle_agent.egg-info/  (0 directories, 7 files)
│   │── SOURCES.txt (73 lines)
│   │── requires.txt (21 lines)
│   │── entry_points.txt (3 lines)
│   │── top_level.txt (2 lines)
│   │── dependency_links.txt (1 lines)
│   └── ... (remaining: 2 files)
│── .../ (remaining: 5 directories)
│── CONTRIBUTING.md (75 lines)
│── setup.py (1,959 bytes)
│── .gitignore (3,132 bytes)
│── requirements.txt (22 lines)
│── setup.cfg (85 bytes)
└── ... (remaining: 3 files)

Before submitting this PR, please make sure you have:

  • confirmed all checks still pass OR confirm CI build passes.
  • verified that any code or assets from external sources are properly credited in comments and/or in
    the credit file.

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels Oct 18, 2024
@leeeizhang leeeizhang changed the title [WIP] enhance the list_files function [MRG] enhance the list_files function Oct 18, 2024
@leeeizhang leeeizhang changed the title [MRG] enhance the list_files function [MRG] enhance list_files function Oct 18, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Oct 18, 2024
@huangyz0918 huangyz0918 merged commit 3ceb69d into MLSysOps:main Oct 18, 2024
3 checks passed
@huangyz0918
Copy link
Member

Nice one @leeeizhang !

@leeeizhang leeeizhang deleted the lei/enhance-list-files branch October 18, 2024 16:45
@huangyz0918
Copy link
Member

You need to change the function call schema as well since the parameters are changed @leeeizhang
Make sure the code has been tested before submitting

@huangyz0918
Copy link
Member

huangyz0918 commented Oct 18, 2024

Another thing is that we don't know if the tree-printing structure is a good format for LLMs to understand, previously I just returned abs paths in a list. We need more tests for this format (personally I don't think this is a good format, so this PR may result in a negative optimization)

@huangyz0918
Copy link
Member

huangyz0918 commented Oct 18, 2024

The function contains bugs:

[MLE FUNC CALL]:  list_files
Traceback (most recent call last):
  File "/Users/huangyz0918/miniconda3/envs/mle/bin/mle", line 33, in <module>
    sys.exit(load_entry_point('mle-agent', 'console_scripts', 'mle')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/miniconda3/envs/mle/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/miniconda3/envs/mle/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/miniconda3/envs/mle/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/miniconda3/envs/mle/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/miniconda3/envs/mle/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/cli.py", line 155, in kaggle
    return workflow.auto_kaggle(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/workflow/kaggle.py", line 109, in auto_kaggle
    code_report = coder.debug(coding_task, debug_report)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/agents/coder.py", line 200, in debug
    text = self.model.query(
           ^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/model/__init__.py", line 39, in query
    return self.model.query(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/model/openai.py", line 66, in query
    result = get_function(function_name)(**arguments)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/function/files.py", line 141, in list_files
    return f"{Path(path).absolute()}\n" + list_directories(path, max_depth - 1, max_items)
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/huangyz0918/Desktop/MLE-agent/mle/function/files.py", line 78, in list_directories
    for item in os.listdir(path):

I am going to revert this function and then re-think how to improve the preview things

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants