Skip to content

Commit

Permalink
feat: kubernetes resources added. linting fixed (#46)
Browse files Browse the repository at this point in the history
  • Loading branch information
marcosschroh authored Feb 18, 2020
1 parent 3eba7f2 commit 1d586f1
Show file tree
Hide file tree
Showing 37 changed files with 371 additions and 303 deletions.
14 changes: 4 additions & 10 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,8 @@ install: pip install -r requirements.txt

matrix:
include:
- name: test python 3.6
python: 3.6
script: ./scripts/test.sh
- name: test python 3.7
- name: Test
python: 3.7
script: ./scripts/test.sh
- name: lint
script: ./scripts/lint.sh
- name: Docker image
script: ./scripts/test_docker.sh

script: ./scripts/test
- name: Test Docker image
script: ./scripts/test_docker
37 changes: 20 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ Cookiecutter Faust
[![Build Status](https://travis-ci.org/marcosschroh/cookiecutter-faust.svg?branch=master)](https://travis-ci.org/marcosschroh/cookiecutter-faust)
[![GitHub license](https://img.shields.io/github/license/marcosschroh/cookiecutter-faust.svg)](https://github.com/marcosschroh/cookiecutter-faust/blob/feature/add-license-and-remove-network-after-clean/LICENSE)


Table of Contents
-----------------

Expand All @@ -18,16 +17,15 @@ Table of Contents
Features
--------

* For Faust 1.9.0
* Python 3.7
* Docker and docker-compose support
* Useful commands included in Makefile
* project skeleton is defined as a medium/large project according to [faust layout](https://faust.readthedocs.io/en/latest/userguide/application.html#projects-and-directory-layout)
* The `setup.py` has the entrypoint to resolve the [entrypoint problem](https://faust.readthedocs.io/en/latest/userguide/application.html#problem-entrypoint)

- Python 3.7+
- Docker and docker-compose for development
- Useful commands included in Makefile
- Project skeleton is defined as a medium/large project according to [faust layout](https://faust.readthedocs.io/en/latest/userguide/application.html#projects-and-directory-layout)
- The `setup.py` has the entrypoint to resolve the [entrypoint problem](https://faust.readthedocs.io/en/latest/userguide/application.html#problem-entrypoint)
- Kubernetes manifests included

Usage
------
-----

Let's pretend you want to create a Faust project called "super faust".

Expand Down Expand Up @@ -71,7 +69,7 @@ include_page_view_tutorial [n]: y
worker_port [6066]:
kafka_server_environment_variable [KAFKA_BOOTSTRAP_SERVER]:
include_codec_example [y]:
Select faust_loglevel:
Select log_level:
1 - CRITICAL
2 - ERROR
3 - WARNING
Expand Down Expand Up @@ -126,7 +124,6 @@ git push -u origin master

Now take a look at your repo. Don't forget to carefully look at the generated README. Awesome, right?


Useful Commands
---------------

Expand All @@ -140,7 +137,6 @@ Useful Commands
| `make send-page-view-event payload='{a payload}'`| Send event to a page view application | -- | `make send-page-view-event payload='{"id": "foo", "user": "bar"}'` |
| `make list-agents`| List faust agents| --- | |


Settings
--------

Expand Down Expand Up @@ -174,15 +170,15 @@ LOGGING = {
},
'handlers': {
'console': {
'level': '{{cookiecutter.faust_loglevel}}',
'level': '{{cookiecutter.log_level}}',
'class': 'logging.StreamHandler',
'formatter': 'default',
},
},
'loggers': {
'your_project_slug': {
'handlers': ['console'],
'level': '{{cookiecutter.faust_loglevel}}',
'level': '{{cookiecutter.log_level}}',
},
},
}
Expand Down Expand Up @@ -215,10 +211,17 @@ Useful `ENVIRONMENT` variables that you may change:
| SCHEMA_REGISTRY_SERVER_PORT | Schema registry server port | `8081` |
| SCHEMA_REGISTRY_URL | Schema Registry Server url | `http://schema-registry:8081` |

Development
-----------

Run tests:

```bash
./scripts/test
```

Run tests
----------
Run code linting

```bash
./scripts/test.sh
./scripts/lint
```
2 changes: 1 addition & 1 deletion cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"worker_port": 6066,
"kafka_server_environment_variable": "KAFKA_BOOTSTRAP_SERVER",
"include_codec_example": "y",
"faust_loglevel": [
"log_level": [
"INFO",
"CRITICAL",
"ERROR",
Expand Down
22 changes: 11 additions & 11 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
cookiecutter==1.6.0
sh==1.12.14
binaryornot==0.4.4
cookiecutter
sh
binaryornot

# Code quality
# ------------------------------------------------------------------------------
black==19.10b0
flake8==3.7.9
isort==4.3.21
mypy==0.761
black
flake8
isort
mypy

# Testing
# ------------------------------------------------------------------------------
pytest==5.3.2
pytest_cases==1.6.2
pytest-cookies==0.3.0
pytest-asyncio==0.10.0
pytest
pytest_cases
pytest-cookies
pytest-asyncio
9 changes: 9 additions & 0 deletions scripts/lint
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh -e

export PREFIX=""
if [ -d 'venv' ] ; then
export PREFIX="venv/bin/"
fi

${PREFIX}isort --multi-line=3 --trailing-comma --force-grid-wrap=0 --combine-as --line-width 120 --recursive --apply src tests
${PREFIX}black src tests
13 changes: 0 additions & 13 deletions scripts/lint.sh

This file was deleted.

2 changes: 1 addition & 1 deletion scripts/publish.sh → scripts/publish
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/sh -e

VERSION=`cat setup.py | grep '__version__ =' | sed 's/__version__ = //' | sed 's/"//g'`

Expand Down
8 changes: 8 additions & 0 deletions scripts/test
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh -e

export PREFIX=""
if [ -d 'venv' ] ; then
export PREFIX="venv/bin/"
fi

${PREFIX}pytest ${1-"./tests"}
13 changes: 0 additions & 13 deletions scripts/test.sh

This file was deleted.

10 changes: 5 additions & 5 deletions scripts/test_docker.sh → scripts/test_docker
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
#!/bin/sh
# sh tests/test_docker.sh
#!/bin/sh -e

set -o errexit
project_name="my_awesome_faust_project"

# install test requirements
pip install faust==1.9.0 simple-settings==0.16.0
pip install faust simple-settings
pip install -r ./requirements.txt

# create a cache directory
Expand All @@ -18,9 +17,10 @@ cd ${project_name}

docker-compose stop
echo yes | docker-compose rm
docker network rm ${project_name} | true

./scripts/test.sh
docker network rm ${project_name} || true

./scripts/test

# Listing the agents to prove that the project generated is working
docker-compose run -e SIMPLE_SETTINGS=${project_name}.settings ${project_name} faust -A ${project_name}.app agents
6 changes: 4 additions & 2 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
[flake8]
max-line-length = 100
max-line-length = 120
exclude = .tox,.git,docs
select = B,E,F,W,C,W504,B902,B903,B950
ignore = E203,E231,E902,W503

[pycodestyle]
max-line-length = 100
max-line-length = 120
exclude = .tox,.git,docs

[mypy]
Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
download_url="",
packages=[],
include_package_data=True,
license="GPLv3",
license="MIT",
classifiers=[
"Framework :: Faust :: 1.9.0",
"Framework :: Faust",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Topic :: Software Development",
],
keywords=(
Expand Down
8 changes: 4 additions & 4 deletions tests/test_cookiecutter_generation.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
RE_OBJ = re.compile(PATTERN)

YN_CHOICES = ["y", "n"]
FAUST_LOGLEVEL = ["CRITICAL", "ERROR", ]
log_level = ["CRITICAL", "ERROR", ]
CI_PROVIDERS = ["travis", "none", ]
WORKER_PORT = [6066, 8000, 8080]
KAFKA_SERVER = ["KAFKA_BOOTSTRAP_SERVER", "KAFKA_SERVER"]
Expand All @@ -37,7 +37,7 @@ def context():
"include_page_view_tutorial", YN_CHOICES, ids=lambda yn: f"page_tutorial:{yn}"
)
@pytest.mark.parametrize(
"faust_loglevel", FAUST_LOGLEVEL, ids=["CRITICAL", "ERROR", ]
"log_level", log_level, ids=["CRITICAL", "ERROR", ]
)
@pytest.mark.parametrize("worker_port", WORKER_PORT, ids=lambda yn: f"worker_port:{yn}")
@pytest.mark.parametrize(
Expand All @@ -60,7 +60,7 @@ def context_combination(
use_docker,
include_docker_compose,
include_page_view_tutorial,
faust_loglevel,
log_level,
worker_port,
kafka_server_environment_variable,
include_codec_example,
Expand All @@ -74,7 +74,7 @@ def context_combination(
"use_docker": use_docker,
"include_docker_compose": include_docker_compose,
"include_page_view_tutorial": include_page_view_tutorial,
"faust_loglevel": faust_loglevel,
"log_level": log_level,
"worker_port": worker_port,
"kafka_server_environment_variable": kafka_server_environment_variable,
"include_codec_example": include_codec_example,
Expand Down
Loading

0 comments on commit 1d586f1

Please sign in to comment.