Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pickle protocol semantics to the simplekv interface #107

Open
fjetter opened this issue Sep 14, 2020 · 0 comments
Open

Add pickle protocol semantics to the simplekv interface #107

fjetter opened this issue Sep 14, 2020 · 0 comments

Comments

@fjetter
Copy link
Contributor

fjetter commented Sep 14, 2020

Some of the simplekv stores implement the pickle protocol and allow the safe serialization of the store (e.g. AzureBlockBlobStore see here. In this case, the pickle roundtrip restores the store both safely and semantically correct.

Some stores will simply fail when trying to be serialized, for instance (taken from the test suite, I guess there are more examples)

import pickle
from simplekv.cache import CacheDecorator
from simplekv.memory import DictStore
store = CacheDecorator(DictStore(), DictStore())
try:
    pickle.loads(pickle.dumps(store))
except RuntimeError: # max recursion depth exceeded
    pass

will cause a recursion depth runtime error.

And for yet other stores, they can theoretically be safely serialized but it isn't sensible to do so, for instance the memory store will not point to the same dictionary anymore

from simplekv.memory import DictStore
import pickle

store = DictStore()
store_roundtrip = pickle.loads(pickle.dumps(store))

store.put('key', b'key')
assert 'key' in store
assert not 'key' in store_roundtrip

For a user it is not immediately possible to identify in which category a given store belongs.

It would be a great addition to the simplekv interface to expose whether or not the store can be safely and sensibly serialised or not.

I would therefore propose an extension to the simplekv interface such that simplekv stores shall implement the pickle protocol from now on. The base class KeyValueStore would, by default, raise a NotImplementError which gives a generic exception with a short explanation about the culprits of serialising the stores.

On top of this, there is either a mixin or an attribute available which indicates that the given store can be safely serialized to allow the users to adapt their code depending on whether this is supported or not. This may be omitted if the stores themselves raise an appropriate NotImplementedError since users may check

pickleable = False
try:
    pickle.dumps(store)
    pickleable = True
except NotImplementError:
    pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant