Skip to content

Commit

Permalink
Upgrade versions
Browse files Browse the repository at this point in the history
Signed-off-by: Johannes Ott <[email protected]>

Upgrade all python and angular versions

Signed-off-by: Johannes Ott <[email protected]>

Upgrade docker build

Signed-off-by: Johannes Ott <[email protected]>
  • Loading branch information
DerOetzi committed May 13, 2024
1 parent 2670f9c commit 35c61f0
Show file tree
Hide file tree
Showing 10 changed files with 4,768 additions and 4,630 deletions.
52 changes: 36 additions & 16 deletions .github/workflows/build_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
env:
REGISTRY_GITHUB: ghcr.io
IMAGE_NAME: ${{ github.repository }}
PACKAGE_BRANCHES: '["main"]'

jobs:
variables:
Expand All @@ -18,15 +19,15 @@ jobs:
runs-on: "ubuntu-latest"
steps:
- name: Setting global variables
uses: actions/github-script@v6
uses: actions/github-script@v7
id: var
with:
script: |
core.setOutput('ref_name', '${{ github.ref_name }}'.toLowerCase().replaceAll(/[/.]/g, '-').trim('-'));
build-ui:
runs-on: ubuntu-latest
needs: variables
needs: [variables]
env:
REF_NAME: ${{ needs.variables.outputs.ref_name }}
steps:
Expand All @@ -49,7 +50,7 @@ jobs:
working-directory: ui

- name: Save angular artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: ui-artifacts-${{ env.REF_NAME }}
path: |
Expand All @@ -67,17 +68,32 @@ jobs:
matrix:
python-version:
- '3.10'
- '3.11'
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}


- name: Cache Python packages
uses: actions/cache@v4
with:
path: |
~/.cache/pip
!~/.cache/pip/log
key: ${{ runner.os }}-python-${{ matrix.python-version }}-pip-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-python-${{ matrix.python-version }}-pip-
${{ runner.os }}-python-${{ matrix.python-version }}-
${{ runner.os }}-python-
${{ runner.os }}-pip-
- name: Download UI
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: ui-artifacts-${{ env.REF_NAME }}
path: core/learninghouse/ui
Expand All @@ -96,20 +112,20 @@ jobs:
working-directory: core

- name: Save python artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: python-artifacts-${{ matrix.python-version }}-${{ env.REF_NAME }}
path: |
core/dist
- name: Publish package to pypi
if: ${{github.event_name == 'release' && matrix.python-version == '3.10'}}
if: ${{github.event_name == 'release' && matrix.python-version == '3.11'}}
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: core/dist/

- name: Publish assets to github
if: ${{github.event_name == 'release'}}
if: ${{github.event_name == 'release' && matrix.python-version == '3.11'}}
uses: svenstaro/upload-release-action@v2
with:
repo_token: ${{ github.token }}
Expand All @@ -127,15 +143,19 @@ jobs:
- name: Checkout
uses: actions/checkout@v4

- name: Load learninghouse wheel
uses: actions/download-artifact@v3
- name: Copy core learninghouse to docker
run: cp -r core/* docker/

- name: Download UI
uses: actions/download-artifact@v4
with:
name: python-artifacts-3.10-${{ env.REF_NAME }}
path: docker/container
name: ui-artifacts-${{ env.REF_NAME }}
path: core/learninghouse/ui

- name: Generate static _version.py
run: python freeze_version.py
working-directory: docker

- name: Copy requirements.txt for core
run: cp core/requirements.txt docker/container/

- name: Set up QEMU
uses: docker/setup-qemu-action@master
with:
Expand Down Expand Up @@ -182,7 +202,7 @@ jobs:
with:
builder: ${{ steps.buildx.outputs.name }}
context: docker
push: ${{github.event_name == 'release' || (github.event_name == 'push' && github.ref_name == 'main')}}
push: ${{github.event_name == 'release' || (github.event_name == 'push' && contains(fromJSON(env.PACKAGE_BRANCHES), github.ref_name))}}
tags: ${{github.event_name == 'release' && steps.meta_release.outputs.tags || steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

Expand Down
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,16 +203,18 @@ curl --location --request PUT 'http://localhost:5000/api/brain/darkness/training
--header 'Content-Type: application/json' \
--header 'X-LEARNINGHOUSE-API-KEY: YOURSECRETKEY' \
--data-raw '{
"azimuth": 321.4441223144531,
"elevation": -19.691608428955078,
"rain_gauge": 0.0,
"pressure": 971.0,
"pressure_trend_1h": "falling",
"temperature_outside": 23.0,
"temperature_trend_1h": "rising",
"light_state": false,
"darkness": true
}'
"dependent_value": true,
"sensors_data": {
"azimuth": 321.4441223144531,
"elevation": -19.691608428955078,
"rain_gauge": 0.0,
"pressure": 971.0,
"pressure_trend_1h": "falling",
"temperature_outside": 23.0,
"temperature_trend_1h": "rising",
"light_state": false
}
}'
```

You can send either a field `timestamp` with your dataset containing a UNIX-Timestamp or the service will add this information with its current time. The service generates some further time-relevant fields inside the training dataset that you can also use as `features`. These are `month_of_year`, `day_of_month`, `day_of_week`, `hour_of_day`, and `minute_of_hour`.
Expand Down
25 changes: 25 additions & 0 deletions core/freeze_version.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import json

import versioneer


def generate_static_version_file():
versions = versioneer.get_versions()
version_string = f"""# This file was generated automatically
import json
versions_json = '''{json.dumps(versions, indent=4)}'''
def get_versions() -> dict[str | bool | None]:
return json.loads(versions_json)
"""
with open("learninghouse/_version.py", "w", encoding="utf-8") as file:
file.write(version_string)

print("Generated learninghouse/_version.py")
print(f"Version: {versions['version']}")


if __name__ == "__main__":
generate_static_version_file()
3 changes: 2 additions & 1 deletion core/learninghouse/services/brain.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ def request(
if dependent_value is not None:
trainings_data[name] = dependent_value
else:
# Todo this is never thrown because of if 86
raise BrainBadRequest("Missing dependent variable!")

if trainings_data is None:
Expand Down Expand Up @@ -162,7 +163,7 @@ def prediction(cls, name: str, request_data: Dict[str, Any]):
try:
brain = cls.load_brain(name)
if not brain.actual_versions:
raise BrainNotActual(name, brain.version)
raise BrainNotActual(name, brain.versions)

request_data = DatasetPreprocessing.add_time_information(
request_data)
Expand Down
8 changes: 4 additions & 4 deletions core/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
fastapi==0.103.1
fastapi==0.110.2
uvicorn[standard]==0.23.2
python-jose[cryptography]==3.3.0
passlib==1.7.4
pydantic==2.3.0
pydantic==2.7.0
pydantic-settings==2.0.3
loguru==0.7.2
numpy==1.25.2
pandas==2.1.0
numpy==1.26.4
pandas==2.2.2
scikit-learn==1.3.0
154 changes: 154 additions & 0 deletions docker/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
.pylintrc
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.env.dev.example
.env.example
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

docker-compose.yml
.vscode
.github/
.env.example
.git/
.gitignore
.gitattributes
setup.cfg
setup.py
doc
versioneer.py
freeze_version.py
Dockerfile
.dockerignore
MANIFEST.in
Loading

0 comments on commit 35c61f0

Please sign in to comment.