Utilize Schemas from Spec Repository

[Teque5]

Now that the specification repository is using it's own schemas to generate the documentation, we can now use those schemas in the validator instead of having our own.

I dislike git submodule, but I don't see a better way. Since the spec schemas is based upon the one here I suspect this will be an easy transition.

Any alternatives to git submodule? If no, we can try to get this working during grcon next week.

[777arc]

One alternative to git submodule is for the github action that publishes to pypi to clone the spec repo as part of building the package, and then for folks doing local dev there can be a line in the install instructions that involves cloning it. There may also be some pyproject.toml way to do it, I know that you can specify a dependency by giving it the github url but we're not talking about a python package, it just needs the json schema. The other thing is that json schemas are supposed to be available via url, so if we standardize the url its available at we can hardcode it in this repo.

[Teque5]

I like the idea of having the URL, but will there be a separate URL for each version of the spec? I wouldn't want the spec to change and break the python module. At least with submodule you explicitly pin a commit value.

[777arc]

Alright I just tried something on the main repo to see if it would work- what we can do is use githubs release system and the fact you can get to any raw file using the tag in the URL, to deal with version control of the json schema without going out of our way. Eg the latest one is- https://raw.githubusercontent.com/sigmf/SigMF/v1.2.2/sigmf-schema.json

[Teque5]

Reminder to reconsider #58 and associated pr #61 after this issue closes.

[Teque5]

Plan A

1. include this in the __init__.py:

__specification__ = "1.2.3"
_schema_base_url = "https://raw.githubusercontent.com/sigmf/SigMF/v{specification}/{name}-schema.json"

2. when schema is needed, get_schema will look in module path (as it does now). If it doesn't find it then it will use pooch to cache it into the python's cache location and use that:

from pathlib import Path
import pooch

schema_path = Path(pooch.retrieve(
    url=_schema_base_url.format(**dict(specification=__specification__, name='sigmf')),
    known_hash=None,
))

3. Ensure that during packaging for pip we copy the file from schema_path into sigmf/ before wheel is compiled. That way all it's distributed through pypi.

Why this way?

good

• If developers download from github && have internet connection, they always have the latest spec. This is most people who use pip install ., but not everyone.
• If users download from pypi they have the spec associated with the module. pip install sigmf

bad

• If developers download the source from github, then try to use it without an internet connection, whey won't have the spec. Is there a way to solve this problem?

Plan B

1. Simply copy any .json files from the SigMF repo into this one when specification changes.

good

• simple
• works for all developers & users

bad

• we are checking bigger and bigger blobs into the repo

[777arc]

For Plan B, I wouldn't be concerned with the size of the JSON, the schema is super minimal and we would only need to copy it when the major or minor version changes which is like a few times a year, most changes are patch-level due to descriptions being tweaked. The main downside to B is just the manual aspect of having to grab the latest, but if we want the ability to handle multiple versions down the road, I like the idea of just copying them in and that way we always have the older versions.

[777arc]

So yeah I like B, we just need a standard directory and file naming convention to use in this repo for storing the jsons

[Teque5]

I'll get to Plan B next week... too many hours on advent of code lately.