-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open Knowledge Validator #50
Open
ElijahAhianyo
wants to merge
13
commits into
helpfulengineering:main
Choose a base branch
from
ElijahAhianyo:elijah/ok-validator
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
f0d561e
WIP: Open Knowledge Validator
ElijahAhianyo 1577afc
fix typo
ElijahAhianyo acf5cd1
add more validation checks
ElijahAhianyo 502cd3c
test files initial commit
ElijahAhianyo 671228c
refactor and added tests
ElijahAhianyo 4e1fb83
python 3.7 support for tests
ElijahAhianyo ecb2283
add conftest
ElijahAhianyo 12711ae
more tests
ElijahAhianyo 35496a2
remove dead code
ElijahAhianyo 2a66b7e
update docstrings
ElijahAhianyo b0240a7
use pydantic-yaml
ElijahAhianyo dd7161f
add pyproject file and readme
ElijahAhianyo bc695d3
update pyproject file
ElijahAhianyo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
## Open Knowledge Validator | ||
|
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
[tool.poetry] | ||
name="okvalidator" | ||
version="0.1.0" | ||
description="An Open Knowledge validator" | ||
authors = [ | ||
"Helpful Engineering", | ||
] | ||
readme="README.md" | ||
|
||
repository = "https://github.com/helpfulengineering/project-data-platform/tools/ok_validator" | ||
|
||
classifiers = [ | ||
"Intended Audience :: Developers", | ||
"Topic :: Software Development :: Open Knowledge :: Supply Chain", | ||
"License :: OSI Approved :: MIT License", | ||
"Programming Language :: Python :: 3", | ||
"Programming Language :: Python :: 3.7", | ||
"Programming Language :: Python :: 3.8", | ||
"Programming Language :: Python :: 3.9", | ||
"Programming Language :: Python :: 3.10", | ||
"Programming Language :: Python :: 3.11", | ||
"Programming Language :: Python :: 3 :: Only", | ||
] | ||
|
||
|
||
[tool.poetry.dependencies] | ||
python = "^3.7" | ||
pyyaml = "6.0" | ||
|
||
[tool.poetry.group.dev.dependencies] | ||
pytest = "^7.2.1" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
from pydantic import BaseModel | ||
|
||
from .validate import OKValidator | ||
|
||
|
||
class OKH(BaseModel, extra="allow"): | ||
title: str | ||
bom: list[str] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. as per https://okh.makernet.org/form, I think bom field is a string, not an array of strings |
||
|
||
|
||
class OKHValidator(OKValidator): | ||
def __init__(self): | ||
super().__init__(OKH) |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
"""Open knowledge validator""" | ||
import json | ||
from pathlib import Path | ||
from typing import Type, TypeVar, Union | ||
from pydantic import BaseModel | ||
from pydantic_yaml import parse_yaml_file_as, parse_yaml_raw_as | ||
|
||
T = TypeVar("T", bound=BaseModel) | ||
|
||
|
||
class OKValidator: | ||
def __init__(self, model_type: Type[T]): | ||
self.model_type = model_type | ||
|
||
def validate(self, src: Union[str, Path, dict], raise_exception=False) -> bool: | ||
""" | ||
Validate the YAML source, which can be a file path, YAML content string, | ||
Path object, or YAML dictionary. | ||
|
||
Args: | ||
src: The string path or parsed yaml contents represented as dictionary. | ||
raise_exception: Raises an exception if true otherwise returns a boolean result. | ||
""" | ||
if not isinstance(src, Union[str, dict, Path]): | ||
return self.return_value_or_error( | ||
ValueError( | ||
"`src` should be one of the following: a string path, " | ||
"a Path object, or Yaml dict", | ||
), | ||
raise_exception, | ||
) | ||
try: | ||
if isinstance(src, dict): | ||
parse_yaml_raw_as(self.model_type, json.dumps(src)) | ||
else: | ||
parse_yaml_file_as(self.model_type, src) | ||
return True | ||
except Exception as err: | ||
return self.return_value_or_error(err, raise_exception) | ||
|
||
@staticmethod | ||
def return_value_or_error(error: Exception, raise_exception: bool = False): | ||
"""Return a bool or raise an exception. | ||
|
||
Args: | ||
error: Exception to raise. | ||
raise_exception: If set to true, the provided exception will be raised. | ||
""" | ||
if not raise_exception: | ||
return False | ||
raise error |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
import pytest | ||
|
||
from tools.ok_validator.src.okh import OKHValidator | ||
|
||
|
||
@pytest.fixture | ||
def okh_string(): | ||
return """ | ||
title: mock okh title. | ||
description: mock description for okh. | ||
bom: | ||
- mock bom field. | ||
""" | ||
|
||
|
||
@pytest.fixture | ||
def okh_yaml_file(tmp_path, okh_string): | ||
okh_file = tmp_path / "okh.yaml" | ||
okh_file.touch() | ||
okh_file.write_text(okh_string) | ||
|
||
return okh_file | ||
|
||
|
||
@pytest.fixture | ||
def okh_dict(): | ||
return { | ||
"title": "mock okh title.", | ||
"description": "mock description for okh.", | ||
"bom": ["mock bom field."], | ||
} | ||
|
||
|
||
@pytest.fixture | ||
def okh_dict_partial(): | ||
return {"title": "mock okh title."} | ||
|
||
|
||
@pytest.fixture | ||
def okh_validator(): | ||
return OKHValidator() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
"""Unit tests for okh validator""" | ||
import pytest | ||
|
||
|
||
@pytest.mark.parametrize("src_fixture", ["okh_yaml_file", "okh_dict"]) | ||
def test_okh_validate(okh_validator, src_fixture, request): | ||
"""Test that the OKH validator return the correct boolean | ||
on validate. | ||
|
||
Args: | ||
okh_validator: OKHValidator instance. | ||
src_fixture: src dict or file containing all required OKH fields. | ||
""" | ||
assert okh_validator.validate(request.getfixturevalue(src_fixture)) | ||
|
||
|
||
def test_okh_validate_partial_fields(okh_validator, okh_dict_partial): | ||
"""Test that the OKH validator return the correct boolean | ||
on validate. | ||
|
||
Args: | ||
okh_validator: OKHValidator instance. | ||
okh_dict_partial: Yaml dict with missing required OKH fields. | ||
""" | ||
assert not okh_validator.validate(okh_dict_partial) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is pydantic included in the pyproject toml file?