Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extractcode FileNotFoundError if using replace-originals option #31

Open
Smascer opened this issue Aug 9, 2021 · 1 comment
Open

Extractcode FileNotFoundError if using replace-originals option #31

Smascer opened this issue Aug 9, 2021 · 1 comment

Comments

@Smascer
Copy link

Smascer commented Aug 9, 2021

If extractcodes tries to extract and get some errors and afterwards tries to clean, up you will get error with filenotfound exception.
e.g. again gcc magic issue6550.gz archive
here traces without replace-originals

λ extractcode --verbose D:\test-extractcode\test.zip
Extracting archives...
Extracting: test.zip
Extracting: test.zip
Extracting: zweite_ebene.zip
Extracting: issue6550.gz
ERROR extracting: D:/test-extractcode/test.zip-extract/zweite_ebene.zip-extract/issue6550.gz: Error -3 while decompressing data: too many length or distance symbols
Extracting done.

with replace originals

λ extractcode --verbose --replace-originals D:\test-extractcode\test.zip
Extracting archives...
Extracting: test.zip
Extracting: test.zip
Extracting: zweite_ebene.zip
Extracting: issue6550.gz
Extracting: issue6550.gz
Traceback (most recent call last):
  File "C:\WS\tools\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\WS\tools\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\WS\tools\scancode-toolkit-21.3.31\Scripts\extractcode.exe\__main__.py", line 7, in <module>
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\commoncode\cliutils.py", line 87, in main
    **extra,
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\extractcode\cli.py", line 184, in extractcode
    for xev in extraction_events:
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\click\_termui_impl.py", line 259, in next
    rv = next(self.iter)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\extractcode\api.py", line 42, in extract_archives
    ignore_pattern=ignore_pattern
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\extractcode\extract.py", line 142, in extract
    fileutils.copytree(target, source)
  File "c:\ws\tools\scancode-toolkit-21.3.31\lib\site-packages\commoncode\fileutils.py", line 403, in copytree
    names = os.listdir(src)
FileNotFoundError: [WinError 3] Das System kann den angegebenen Pfad nicht finden: 'D:\\test-extractcode\\test.zip-extract\\zweite_ebene.zip-extract\\issue6550.gz-extract'

can be fixed in the extract.py in method extract to skip moving file when event has errors.
I mean here for example:

    for event in extract_events:
        yield event
        if replace_originals:
            processed_events_append(event)
@pombredanne
Copy link
Member

Thanks. This likely is similar to aboutcode-org/scancode-toolkit#2723
This is a test file for this https://golang.org/src/compress/gzip/testdata/issue6550.gz.base64

pombredanne added a commit that referenced this issue Oct 8, 2021
These archive should not crash extraction when using --replace-originals

Reported-by: Smascer @Smascer
Reported-by: Bryan Sutula @sutula
Reference: #31
Reference: aboutcode-org/scancode-toolkit#2723
Signed-off-by: Philippe Ombredanne <[email protected]>
pombredanne added a commit that referenced this issue Oct 8, 2021
These archive should not crash extraction when using --replace-originals

Reported-by: Smascer @Smascer
Reported-by: Bryan Sutula @sutula
Reference: #31
Reference: aboutcode-org/scancode-toolkit#2723
Signed-off-by: Philippe Ombredanne <[email protected]>
pombredanne added a commit that referenced this issue Oct 8, 2021
This is cleaner than to do tis in the main loop.
Also refine and format the documentation.

Reported-by: Bryan Sutula @sutula
Reference: #31
Reference: aboutcode-org/scancode-toolkit#2723
Signed-off-by: Philippe Ombredanne <[email protected]>
pombredanne pushed a commit that referenced this issue Oct 8, 2021
pombredanne pushed a commit that referenced this issue Oct 8, 2021
Check for deps in local thirdparty directory #31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants