Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize Path to JSON when using FileStore #76

Open
WardLT opened this issue Oct 3, 2022 · 1 comment
Open

Serialize Path to JSON when using FileStore #76

WardLT opened this issue Oct 3, 2022 · 1 comment

Comments

@WardLT
Copy link
Collaborator

WardLT commented Oct 3, 2022

We get an "unserializable type" error if you use a Path when initializing a FileStore.

  File "/home/lward/miniconda3/envs/voc/lib/python3.10/site-packages/colmena/redis/queue.py", line 397, in send_inputs
    self.outbound.put(result.json(), topic=topic)
  File "/home/lward/miniconda3/envs/voc/lib/python3.10/site-packages/colmena/models.py", line 218, in json
    return json.dumps(data, default=proxy_json_encoder)
  File "/home/lward/miniconda3/envs/voc/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/home/lward/miniconda3/envs/voc/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/lward/miniconda3/envs/voc/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/home/lward/miniconda3/envs/voc/lib/python3.10/site-packages/colmena/proxy.py", line 46, in proxy_json_encoder
    raise TypeError(f'Unserializable type: {type(proxy)}')
TypeError: Unserializable type: <class 'pathlib.PosixPath'>
@gpauloski
Copy link
Collaborator

Is it the proxystore_kwargs where the Path object is that fails?

proxystore_kwargs: Optional[Dict] = Field(None, description="Kwargs to reinitialize ProxyStore backend")

It's an easy cast to str when initializing the FileStore. That happens on the user-side, though, and not the library-side so maybe we need to extend the warnings/errors in proxy_json_encoder to note args to ProxyStore Stores need to be JSON serializable if the user selects JSON serialization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants