Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Misc fixes 202412 #19341

Merged
merged 7 commits into from
Dec 18, 2024
Merged

Misc fixes 202412 #19341

merged 7 commits into from
Dec 18, 2024

Conversation

nsoranzo
Copy link
Member

End-of-the-year clean up of my git stash :)

See individual commit messages for details.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@nsoranzo nsoranzo added kind/bug kind/refactoring cleanup or refactoring of existing code, no functional changes labels Dec 17, 2024
@github-actions github-actions bot added this to the 25.0 milestone Dec 17, 2024
@nsoranzo
Replace deprecated example parameter of fastapi.Path()
9e2a3cc 
See https://fastapi.tiangolo.com/reference/parameters/#fastapi.Path
@nsoranzo
Fail earlier if archive download failed
b1cf76c 
Simplify debugging this traceback (which was just an intermittent
network error):

https://github.com/galaxyproject/galaxy/actions/runs/12374234876/job/34536295999

```
______________________ ERROR at setup of test_build_index ______________________

cls = <class 'tarfile.TarFile'>, name = None, mode = 'r'
fileobj = <gzip on 0x7f349cee7d00>, compresslevel = 9, kwargs = {}
GzipFile = <class 'gzip.GzipFile'>

    @classmethod
    def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
        """Open gzip compressed tar archive name for reading or writing.
           Appending is not allowed.
        """
        if mode not in ("r", "w", "x"):
            raise ValueError("mode must be 'r', 'w' or 'x'")

        try:
            from gzip import GzipFile
        except ImportError:
            raise CompressionError("gzip module is not available")

        try:
            fileobj = GzipFile(name, mode + "b", compresslevel, fileobj)
        except OSError:
            if fileobj is not None and mode == 'r':
                raise ReadError("not a gzip file")
            raise

        try:
>           t = cls.taropen(name, mode, fileobj, **kwargs)

/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:1854:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:1831: in taropen
    return cls(name, mode, fileobj, **kwargs)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:1694: in __init__
    self.firstmember = self.next()
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:2578: in next
    tarinfo = self.tarinfo.fromtarfile(self)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:1282: in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/gzip.py:292: in read
    return self._buffer.read(size)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/_compression.py:68: in readinto
    data = self.read(len(byte_view))
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/gzip.py:479: in read
    if not self._read_gzip_header():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <gzip._GzipReader object at 0x7f349cee79a0>

    def _read_gzip_header(self):
        magic = self._fp.read(2)
        if magic == b'':
            return False

        if magic != b'\037\213':
>           raise BadGzipFile('Not a gzipped file (%r)' % magic)
E           gzip.BadGzipFile: Not a gzipped file (b'<!')

/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/gzip.py:427: BadGzipFile

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="module")
    def community_file_dir():
        extracted_archive_dir = tempfile.mkdtemp()
        b = BytesIO(requests.get(URL).content)
>       tarfile.open(fileobj=b, mode="r:gz").extractall(extracted_archive_dir)

test/unit/tool_shed/test_shed_index.py:30:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:1801: in open
    return func(name, filemode, fileobj, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

cls = <class 'tarfile.TarFile'>, name = None, mode = 'r'
fileobj = <gzip on 0x7f349cee7d00>, compresslevel = 9, kwargs = {}
GzipFile = <class 'gzip.GzipFile'>

    @classmethod
    def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
        """Open gzip compressed tar archive name for reading or writing.
           Appending is not allowed.
        """
        if mode not in ("r", "w", "x"):
            raise ValueError("mode must be 'r', 'w' or 'x'")

        try:
            from gzip import GzipFile
        except ImportError:
            raise CompressionError("gzip module is not available")

        try:
            fileobj = GzipFile(name, mode + "b", compresslevel, fileobj)
        except OSError:
            if fileobj is not None and mode == 'r':
                raise ReadError("not a gzip file")
            raise

        try:
            t = cls.taropen(name, mode, fileobj, **kwargs)
        except OSError:
            fileobj.close()
            if mode == 'r':
>               raise ReadError("not a gzip file")
E               tarfile.ReadError: not a gzip file

/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/tarfile.py:1858: ReadError
------------------------------ Captured log setup ------------------------------
DEBUG    urllib3.connectionpool:connectionpool.py:1022 Starting new HTTPS connection (1): github.com:443
DEBUG    urllib3.connectionpool:connectionpool.py:475 https://github.com:443 "GET /mvdbeek/toolshed-test-data/blob/master/toolshed_community_files.tgz?raw=true HTTP/1.1" 503 54894
```
@nsoranzo
Small refactorings
2c3c416
@nsoranzo
Fix validation of config files
0f5df5f
@nsoranzo
Add entry_points to the schema template
6546205
@nsoranzo
Reuse Tabular.displayable() in the inherited method
b100c1e 
Follow up on galaxyproject#18547 .
@nsoranzo nsoranzo force-pushed the misc_fixes_202412 branch 2 times, most recently from 54557fe to 7b6b526 Compare December 17, 2024 20:27
@nsoranzo
Fix test_singularity_container_test test
4e650b4 
(which started failing since ubuntu-latest was moved to 24.04)
by upgrading apptainer to >=1.3.3 to include
apptainer/apptainer#2262 .

Fix:

```
>       assert "samtools:1.0--1" in results["passed"], results
E       AssertionError:
E       {'failed': [{'commands': ['python -c "import pyBigWig; assert(pyBigWig.numpy '
E                                 '== 1); assert(pyBigWig.remote == 1)"'],
E                    'container': 'pybigwig:0.3.22--py36h54a71a5_0',
E                    'errors': [{'command': 'python -c "import pyBigWig; '
E                                           'assert(pyBigWig.numpy == 1); '
E                                           'assert(pyBigWig.remote == 1)"',
E                                'output': '\x1b[91mERROR  : Could not write info to '
E                                          'setgroups: Permission denied\n'
E                                          '\x1b[0m\x1b[91mERROR  : Error while '
E                                          'waiting event for user namespace mappings: '
E                                          'no event received\n'
E                                          '\x1b[0m'},
E                               {'import': 'pyBigWig',
E                                'output': '\x1b[91mERROR  : Could not write info to '
E                                          'setgroups: Permission denied\n'
E                                          '\x1b[0m\x1b[91mERROR  : Error while '
E                                          'waiting event for user namespace mappings: '
E                                          'no event received\n'
E                                          '\x1b[0m'}],
E                    'import_lang': 'python -c',
E                    'imports': ['pyBigWig']},
E                   {'commands': ['samtools view --help 2>&1 | grep Notes > /dev/null'],
E                    'container': 'samtools:1.0--1',
E                    'errors': [{'command': 'samtools view --help 2>&1 | grep Notes > '
E                                           '/dev/null',
E                                'output': '\x1b[91mERROR  : Could not write info to '
E                                          'setgroups: Permission denied\n'
E                                          '\x1b[0m\x1b[91mERROR  : Error while '
E                                          'waiting event for user namespace mappings: '
E                                          'no event received\n'
E                                          '\x1b[0m'}],
E                    'import_lang': 'python -c'}],
E        'notest': ['yasm:1.3.0--0'],
E        'passed': []}
E       assert 'samtools:1.0--1' in []
```

Also:
- Improve assertion messages to facilitate debugging
@nsoranzo
Copy link
Member Author

Test failures unrelated.

@nsoranzo nsoranzo requested a review from a team December 18, 2024 01:19
@davelopez
Copy link
Contributor

Would it be interesting to backport any of the fixes? 😇

@nsoranzo
Copy link
Member Author

Would it be interesting to backport any of the fixes? 😇

Commit 4e650b4 yes, not sure about the others, but let me know if you have some in mind.

@davelopez
Copy link
Contributor

Yes, that looks like an interesting one! Thanks! 🙏

@mvdbeek
Copy link
Member

mvdbeek commented Dec 18, 2024

Thank you!

@mvdbeek mvdbeek merged commit b74d24f into galaxyproject:dev Dec 18, 2024
54 of 58 checks passed
@nsoranzo nsoranzo deleted the misc_fixes_202412 branch December 18, 2024 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mvdbeek mvdbeek mvdbeek approved these changes

Assignees
No one assigned
Labels
area/testing area/tool-framework kind/bug kind/refactoring cleanup or refactoring of existing code, no functional changes
Projects
None yet
Milestone
25.0
Development

Successfully merging this pull request may close these issues.
3 participants
@nsoranzo @davelopez @mvdbeek