ERROR: failed to setup plugin: scan:licenses

[dineshr93]

System configuration

For bug reports, it really helps us to know:

• Ubuntu 64 bit run in virtual box 6.0.4
  scan code version 3.0.2
• installation method source download.

I was just trying to install in Ubuntu-18.04.1-desktop-amd64.iso in virtualbox 64 bit. I get below error

dinesh@dinesh-VirtualBox:~/Downloads/scan/scancode-toolkit$ ./scancode --json-pp - --license --package --verbose  samples
 Setup plugin: output:json-pp...
 Setup plugin: scan:packages...
 Setup plugin: scan:licenses...
Removing temporary files...done.
ERROR: failed to setup plugin: scan:licenses:
Traceback (most recent call last):
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/scancode/cli.py", line 700, in run_scan
    plugin.setup(**kwargs)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/plugin_license.py", line 119, in setup
    get_index(return_value=False)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/cache.py", line 67, in get_index
    _LICENSES_BY_KEY_INDEX = get_cached_index(cache_dir, check_consistency)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/cache.py", line 244, in get_cached_index
    idx = LicenseIndex(rules, _spdx_tokens=spdx_tokens)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/index.py", line 234, in __init__
    self._add_rules(rules, _ranked_tokens, _spdx_tokens)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/index.py", line 407, in _add_rules
    rules_automaton_add(tids=rule_token_ids, rid=rid)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/match_aho.py", line 81, in add_sequence
    automaton.add_word(tokens, set([value]))
SystemError: error return without exception set

dinesh@dinesh-VirtualBox:~/Downloads/scan/scancode-toolkit$ 

[dineshr93]

I tried again now it is throwing different error

dinesh@dinesh-VirtualBox:~/Downloads/scan/scancode-toolkit$ ./scancode --license --copyright --json-pp scancode_result.json samples
Setup plugins...
Removing temporary files...done.
ERROR: failed to setup plugin: scan:licenses:
Traceback (most recent call last):
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/scancode/cli.py", line 700, in run_scan
    plugin.setup(**kwargs)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/plugin_license.py", line 119, in setup
    get_index(return_value=False)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/cache.py", line 67, in get_index
    _LICENSES_BY_KEY_INDEX = get_cached_index(cache_dir, check_consistency)
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/cache.py", line 247, in get_cached_index
    ifc.write(idx.dumps())
  File "/home/dinesh/Downloads/scan/scancode-toolkit/src/licensedcode/index.py", line 738, in dumps
    return pickle.dumps(self, protocol=cPickle.HIGHEST_PROTOCOL)
  File "/usr/lib/python2.7/pickle.py", line 1380, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 425, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 554, in save_tuple
    save(element)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 687, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 606, in save_list
    self._batch_appends(iter(obj))
  File "/usr/lib/python2.7/pickle.py", line 639, in _batch_appends
    save(x)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 687, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 411, in save_reduce
    self.memoize(obj)
  File "/usr/lib/python2.7/pickle.py", line 247, in memoize
    self.memo[id(obj)] = memo_len, obj
MemoryError

[pombredanne]

@dineshr93 thank you again and sorry for the issue. This is surprising and you should not have any issue. Did you install the required packages as explained in the install doc? How much RAM do you have in this VM?

[dineshr93]

@pombredanne. Yes I installed all the required dependencies like python 2.7 and other packages. I have allocated 1.5gb RAM. I got this same error in normal Windows 10 yesterday for which I created the issue ticket here. Now, I tried using linux using vm. It threw the same error

[DragonesFist]

I got the same error while trying to install in Linux environment in ec2 but successfully installed in Windows 10. Is there anyone who solved this?

[pombredanne]

I am closing this for now and please reopen if you still experience the same issue. We have since migrated to Python 3
In all cases using 1.5GB of RAM is not enough as this is below the current minimal required RAM of 4GB https://scancode-toolkit.readthedocs.io/en/latest/getting-started/install.html?highlight=RAM#system-requirements

I am closing this and we will track instead a doc update and refresh in #2835