Consider alternative to Pickle queues?

[claudijd]

@moz-hwine mentioned on a #risk thread that...

1.) Pickle queues can be a security hole
2.) JSON is much better suited for caching data

So capturing it so it's not lost feedback.

[claudijd]

@moz-hwine would you mind pointing to a proof of concept or example of using JSON as a queuing/rotation capability like we're using pickle today?

Basically, we need a way to rotate through team members and make an assignment per run. We also need the script to be able to recognize that the config list has changed, restore to preserve the order, and re-write the new rotation for future runs. The existing assigner.py works this way with pickle, and I welcome feedback on how to do it better/safer.

[gdestuynder]

it doesnt really matter for local data (theres no "security hole" to my knowledge - pickles are eval'ed hence dangerous if used with untrusted data, which isnt the case here)

that said, you can also use a sqlitedb - or with json/yaml you just rewrite the file (please do not rewrite the config file though, config files are to be read-only)

[hwine]

or even tinydb: you get query access, while data is stored as JSON. (i.e. very accessible from jq, etc)

[claudijd]

I'm not seeing a clear benefit to making this change based on the context above. Closing for now.

[gdestuynder]

note that if moved to lambda this will have to change regardless

[claudijd]

Agreed, reopening for that sake.