Sample test-code candidates based on current open file #415
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, all tests from a project where incorporated as test-code candidates for the LLM, however, this can be slow for larger projects. This change uses the currently open file, determines the test methods that call code in the currently open file, and only use those as candidates for the LLM. As a fallback, if none could be found, all tests from the project will be candidates.
The core algorithm is still not tested because I do not see how I could get the required objects in the necessary state. The unfortunate thing about the implemented selection algorithms is that they rely on an open editor for a normal execution, which seems to be hard to set up for a unit test. Any feedback and help on this is highly appreciated!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes made
The test-sample provider for LLM with these changes now only shows tests that call one or more methods in the currently open source file. If none where found, as a fallback, it still provides all test samples to the user for further selection.
Why is merge request needed
The current test-sample provider for LLMs runs over the full project and searches for all tests that could be used as samples. On the one hand, this might be slow for larger projects, on the other hand, tests for other classes than the currently open one (in the editor) might not contain anything that can be beneficial for the LLM, because the other code might do something completely different.
The idea of this merge request is to only provide test cases as samples that call the currently open file because they obviously are somehow related the the current file and they might provide beneficial information to the LLM. Besides, this makes the list of potential test samples presented to the user shorter and more relevant, which allows them to choose a sample easier.
Other notes
This merge request is part of the assignment I've received from @pderakhshanfar.
What is missing?
The PR is open for discussion. I am particularly unhappy with the stage of testing—the main algorithms are not tested—however, I did not find a way to properly set up the required environment, since the test-sample selection algorithms require information from basically a running IDE.