Merge pull request #54 from EOEPCA/release/v0.3

Release/v0.3

.travis.yml

@@ -11,6 +11,15 @@ jobs:
       install: pip3 install -r src/requirements.txt
       script: python3 -m unittest tests/testMongo.py
 
+    - stage: Static Code Analysis    
+      language: java
+      if: branch = develop
+      before_install:
+        - wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip
+        - unzip sonar-scanner-cli-3.3.0.1492-linux.zip -d /tmp
+      script: /tmp/sonar-scanner-3.3.0.1492-linux/bin/sonar-scanner -Dsonar.projectKey=$SONARQUBE_PROJECT_KEY -Dsonar.sources=. -Dsonar.host.url=$SONARQUBE_URL -Dsonar.login=$SONAR_TOKEN
+
+
     - stage: container creation and publishing
       install: skip
       script: travis/containerCreation.sh um-pep-engine

Dockerfile

@@ -8,8 +8,14 @@ RUN pip install -r requirements.txt
 # Add requirements, code
 COPY src/ /
 
+
+RUN ln -sf /management_tools_script.sh /bin/management_tools
+RUN chmod +x /management_tools.py
+RUN chmod +x /management_tools_script.sh
+
 # Declare and expose service listening port
 EXPOSE 5566/tcp
+EXPOSE 5576/tcp
 
 # Declare entrypoint of that exposed service
 ENTRYPOINT ["python3", "./main.py"]

README.md

@@ -30,15 +30,6 @@
 - [Table of Contents](#table-of-contents)
   - [Built With](#built-with)
 - [Getting Started](#getting-started)
-  - [Prerequisites](#prerequisites)
-  - [Installation](#installation)
-- [Dependencies](#dependencies)
-- [Configuration](#configuration)
-- [Usage & functionality](#usage--functionality)
-- [Developer documentation](#developer-documentation)
-  - [Demo functionality](#demo-functionality)
-  - [Endpoints](#endpoints)
-  - [Resources cache](#resources-cache)
 - [Roadmap](#roadmap)
 - [Contributing](#contributing)
 - [License](#license)
@@ -68,115 +59,9 @@ This is an example of how to list things you need to use the software and how to
 - [Docker](https://www.docker.com/)
 - [Python](https://www.python.org//)
 
-### Installation
+### Usage, Deployment and Configuration
 
-1. Get into EOEPCA's development environment
-
-```sh
-vagrant ssh
-```
-
-3. Clone the repo
-
-```sh
-git clone https://github.com/EOEPCA/um-pep-engine.git
-```
-
-4. Change local directory
-
-```sh
-cd um-pep-engine
-```
-## Dependencies
-The PEP is written and tested for python 3.6.9, and has all dependencies listed in src/requirements.txt
-
-## Configuration
-
-The PEP gets all its configuration from the file located under `config/config.json`.
-The parameters that are accepted, and their meaning, are as follows:
-- **realm**: 'realm' parameter answered for each UMA ticket. Default is "eoepca"
-- **auth_server_url**: complete url (with "https") of the Authorization server.
-- **proxy_endpoint**: "/path"-formatted string to indicate where the reverse proxy should listen. The proxy will catch any request that starts with that path. Default is "/pep"
-- **service_host**: Host for the proxy to listen on. For example, "0.0.0.0" will listen on all interfaces
-- **service_port**: Port for the proxy to listen on. By default, **5566**. Keep in mind you will have to edit the docker file and/or kubernetes yaml file in order for all the prot forwarding to work.
-- **s_margin_rpt_valid**: An integer representing how many seconds of "margin" do we want when checking RPT. For example, using **5** will make sure the provided RPT is valid now AND AT LEAST in the next 5 seconds.
-- **check_ssl_certs**: Toggle on/off (bool) to check certificates in all requests. This should be forced to True in a production environment
-- **use_threads**: Toggle on/off (bool) the usage of threads for the proxy. Recommended to be left as True.
-- **debug_mode**: Toggle on/off (bool) a debug mode of Flask. In a production environment, this should be false.
-- **resource_server_endpoint**: Complete url (with "https" and any port) of the Resource Server to protect with this PEP.
-- **client_id**: string indicating a client_id for an already registered and configured client. **This parameter is optional**. When not supplied, the PEP will generate a new client for itself and store it in this key inside the JSON.
-- **client_secret**: string indicating the client secret for the client_id. **This parameter is optional**. When not supplied, the PEP will generate a new client for itself and store it in this key inside the JSON.
-
-## Usage & functionality
-
-Use directly from docker with
-```sh
-docker run --publish <configured-port>:<configured-port> <docker image>
-```
-Where **configured-port** is the port configured inside the config.json file inside the image. The default image is called **eoepca/um-pep-engine:latest**.
-
-If this is running in a development environment without proper DNS setup, add the following to your docker run command:
-```sh
---add-host <auth-server-dns>:<your-ip>
-```
-
-When launched, the PEP will answer to all requests that start with the configured path. These answers will come in the form of UUMA tickets (if there are no RPT provided, or an invalid one is used).
-In case the request is accompained by an "Authorization: Bearer <valid_RPT>", the PEP will make a request to the resource server, for the resource located exactly at the path requested (minus the configured at config), and return the resource's server answer.
-
-Examples, given the example values of:
-- path configured: "/pep"
-- PEP is at pep.domain.com/pep
-- Resource server is at remote.server.com
-
-| Token | Request to PEP | PEP Action | PEP answer |
-|-------|---------|------------|--------------|
-| No RPT | pep.domain.com | None (request does not get to PEP endpoint) | None (the PEP doesn't see this request) |
-| No RPT | pep.domain.com/pep/thing | Generate ticket for "/thing" | 401 + ticket |
-| Valid RPT for "/thing" | pep.domain.com/pep/thing | Request to remote.server.com/thing | Contents of remote.server.com/thing |
-| Valid RPT for "/thing" | pep.domain.com/pep/different | Generate ticket for "/different" | 401 + ticket |
-| INVALID RPT for "/thing" | pep.domain.com/pep/thing | Generate ticket for "/thing" | 401 + ticket |
-| No RPT | pep.domain.com/pep/thing/with/large/path | Generate ticket for "/thing/with/large/path" | 401 + ticket |
-| Valid RPT for "/thing/with/large/path" | pep.domain.com/pep/thing/with/large/path | Request to remote.server.com/thing/with/large/path | Contents of remote.server.com/thing/with/large/path |
-
-## Developer documentation
-
-The API will expose an endpoint to interact with the resources.
-The main endpoints for the resource operations exposed by the API are now secured with OAuth/OIDC, it would accept both OAuth and JWT in order to authorize the user and both are expected on the header.
-This check will retrieve the UUID for the user and insert it on the data model of the resource storage, so when any call is made against a resource, the API will double check if the UUID of the requester matches the one associated to the resource in order to operate against it.
-
---------
-
-Testing and Demo for the validation with OAuth/OIDC:
-
-Execute the `test_validation_token.py` in `um-pep-engine/tests/`
-
-### Demo functionality
-
-At the moment, the PEP will auto register a resource for the sake of demoing it's capabilities, using the `create` function of the UMA handler. This can be deleted if unwanted, or expanded to dinamically register resources. Note that the UMA library used allows for full control over resources (create, delete, etc) and could be used to help in that functionality expansion.
-
-### Test functionality
-
-In order to test the PEP engine at the moment first you have reach this prerequisites:
-
-- Register a client and a user inside the gluu instance and update the test_settings.json
-- Disable current UMA Policies and set inside JSONConfig > OxAuth umaGrantAccessIfNoPolicies to true
-
-### Endpoints
-
-The PEP uses the following endpoints from a "Well Known Handler", which parses the Auth server's "well-known" endpoints:
-
-- OIDC_TOKEN_ENDPOINT
-- UMA_V2_RESOURCE_REGISTRATION_ENDPOINT
-- UMA_V2_PERMISSION_ENDPOINT
-- UMA_V2_INTROSPECTION_ENDPOIN
-
-### Resources Repository
-
-
-When a resource is registered, the name and id are stored as a document into a Mongodb database as a sidecar container sharing data through a persistent storage volume.
-The pod runs the pep-engine image and the mongo image exposing the default mongo port (27017) where communicates the service and keeps it alive for the pep-engine container to query the database.
-
-A local MongoDB service can be used to test the repo since the main script would listen the port 27017
+The full getting started guide starts in the Wiki home page
 
 ## Roadmap
 

charts/pep-engine/Chart.yaml

@@ -0,0 +1,24 @@
+apiVersion: v1
+name: pep-engine
+description: A Helm chart for PEP Engine
+maintainers:
+    - name: eoepca
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+appVersion: 1.0

charts/pep-engine/templates/_helpers.tpl

@@ -0,0 +1,32 @@
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "login-service.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "login-service.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "login-service.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+

charts/pep-engine/templates/ingress.yaml

@@ -0,0 +1,20 @@
+
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+  name: gluu-ingress-pep-engine
+
+  annotations:
+    kubernetes.io/ingress.class: nginx
+    nginx.ingress.kubernetes.io/ssl-redirect: "false"
+    nginx.ingress.kubernetes.io/rewrite-target: /$2
+spec:
+  rules:
+  - host: {{ .Values.global.ep | quote }}
+    http:
+      paths:
+      - path: /secure(/|$)(.*)
+        backend:
+          serviceName: pep-engine
+          servicePort: 5566
+

charts/pep-engine/templates/pep-cm.yml

@@ -0,0 +1,22 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: pep-cm
+data:
+  PEP_REALM: {{ .Values.global.realm | quote }}
+  PEP_AUTH_SERVER_URL: {{ .Values.global.domain | quote }}
+  PEP_SERVICE_HOST: {{ .Values.global.serviceHost | quote }}
+  PEP_SERVICE_PORT: {{ .Values.global.servicePort | quote }}
+  PEP_S_MARGIN_RPT_VALID: {{ .Values.global.margin | quote }}
+  PEP_CHECK_SSL_CERTS: {{ .Values.global.sslCerts | quote }}
+  PEP_USE_THREADS: {{ .Values.global.useThreads | quote }}
+  PEP_DEBUG_MODE: {{ .Values.global.debugMode | quote }}
+  PEP_RESOURCE_SERVER_ENDPOINT: {{ .Values.global.resourceServer | quote }}
+  PEP_API_RPT_UMA_VALIDATION: {{ .Values.global.umaValidation | quote }}
+  PEP_RPT_LIMIT_USES: {{ .Values.global.limitUses | quote }}
+  PEP_PDP_URL: {{ .Values.global.pdpUrl | quote }}
+  PEP_PDP_PORT: {{ .Values.global.pdpPort | quote }}
+  PEP_PDP_POLICY_ENDPOINT: {{ .Values.global.pdpPolicy | quote }}
+  PEP_VERIFY_SIGNATURE: {{ .Values.global.verifySignature | quote }}
+  PEP_PROXY_SERVICE_PORT: {{ .Values.global.proxyServicePort | quote }}
+  PEP_RESOURCES_SERVICE_PORT: {{ .Values.global.resourcesServicePort | quote }}

charts/pep-engine/templates/pep-deployment.yml

@@ -0,0 +1,52 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ .Values.global.pep }}
+  labels:
+    app: {{ .Values.global.pep }}
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: {{ .Values.global.pep }}
+  template:
+    metadata:
+      labels:
+        app: {{ .Values.global.pep }}
+    spec:
+      containers:
+      - name: {{ .Values.global.pep }}
+        imagePullPolicy: {{ .Values.image.imagePullPolicy }}
+        image: {{ .Values.image.image }}
+        ports:
+        - name: http-pep
+          containerPort: 5566
+          protocol: TCP
+        - name: https-pep
+          containerPort: 443
+          protocol: TCP
+        envFrom:
+        - configMapRef:
+            name: pep-cm
+      - name: mongo
+        imagePullPolicy: {{ .Values.image.imagePullPolicy }}
+        image: mongo
+        ports:
+        - name: http-rp
+          containerPort: 27017
+          protocol: TCP
+        envFrom:
+        - configMapRef:
+            name: pep-cm
+        volumeMounts:
+        - mountPath: /data/db/
+          sub_path: pep-engine/db/resource
+          name: eoepca-pep-pv-host
+      hostAliases:
+      - ip: {{ .Values.global.nginxIp }}
+        hostnames:
+        - {{ .Values.global.ep }}
+      volumes:
+        - name: eoepca-pep-pv-host
+          persistentVolumeClaim:
+            claimName: eoepca-pep-pvc

charts/pep-engine/templates/pep-service.yml

@@ -0,0 +1,21 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ .Values.global.pep }}
+  labels:
+    app: {{ .Values.global.pep }}
+spec:
+  type: NodePort
+  ports:
+  - port: 5566
+    name: http-pep
+    targetPort: 5566
+    protocol: TCP
+    nodePort: 31707
+  - port: 1025
+    name: https-pep
+    targetPort: 443
+    protocol: TCP
+  selector:
+    app: {{ .Values.global.pep }}
+

charts/pep-engine/templates/pv.yaml

@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: eoepca-pep-pv-host
+  labels:
+    eoepca_type: userman
+spec:
+  capacity:
+    storage: {{ .Values.persistence.dbStorageSize }}
+  accessModes:
+    - {{ .Values.persistence.accessModes }}
+  hostPath:
+    path: "/kubedata/userman"
+    type: {{ .Values.persistence.type }}

charts/pep-engine/templates/pvc.yaml

@@ -0,0 +1,21 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: eoepca-pep-pvc
+  namespace: {{ .Release.Namespace }}
+  labels:
+    eoepca_type: userman
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  accessModes:
+    - {{ .Values.persistence.accessModes }}
+  capacity:
+    storage: {{ .Values.persistence.dbStorageSize }}
+  resources:
+    requests:
+      storage: {{ .Values.persistence.dbStorageSize }}
+  selector:
+    matchLabels:
+      eoepca_type: userman