Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[radar-fitbit-connector] Allow config of application intervals #303

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[radar-fitbit-connector] Allow config of application intervals #303

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions charts/radar-fitbit-connector/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ A Helm chart for RADAR-base fitbit connector. This application collects data fro
| service.type | string | `"ClusterIP"` | Kubernetes Service type |
| service.port | int | `8083` | radar-fitbit-connector port |
| resources.requests | object | `{"cpu":"100m","memory":"1Gi"}` | CPU/Memory resource requests |
| heapOptions | string | `"-Xms256m -Xmx768m"` | |
| persistence.enabled | bool | `true` | Enable persistence using PVC |
| persistence.accessMode | string | `"ReadWriteOnce"` | PVC Access Mode for radar-fitbit-connector volume |
| persistence.size | string | `"5Gi"` | PVC Storage Request for radar-fitbit-connector volume |
Expand Down Expand Up @@ -83,3 +84,7 @@ A Helm chart for RADAR-base fitbit connector. This application collects data fro
| oauthClientSecret | string | `"secret"` | OAuth2 client secret from Management Portal |
| managementportal_url | string | `"http://management-portal:8080/managementportal"` | URL of Management Portal. This will be used to create URLs to access Management Portal |
| includeIntradayData | bool | `true` | Set to true, if intraday access data should be collected by the connector. This will be set in connector.properties. |
| rest_source_poll_interval_ms | int | `60000` | How often to poll the source URL. Only use to speed up processing times during e2e testing. |
| fitbit_user_poll_interval | int | `5000` | Polling interval per Fitbit user per request route in seconds. Only use to speed up processing times during e2e testing. |
| application_loop_interval_ms | int | `300000` | How often to perform the main application loop (only controls how often to poll for new user registrations). Only use to speed up processing times during e2e testing. |
| user_cache_refresh_interval_ms | int | `3600000` | How often to invalidate the cache and poll for new user registrations. Only use to speed up processing times during e2e testing. |
5 changes: 4 additions & 1 deletion charts/radar-fitbit-connector/templates/configmap-properties.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ data:
connector.class=org.radarbase.connect.rest.fitbit.FitbitSourceConnector
tasks.max={{ .Values.connector_num_tasks }}
rest.source.base.url={{ .Values.fitbit_api_url }}
rest.source.poll.interval.ms=5000
rest.source.poll.interval.ms={{ .Values.rest_source_poll_interval_ms | int }}
rest.source.request.generator.class=org.radarbase.connect.rest.fitbit.request.FitbitRequestGenerator
fitbit.api.client={{ .Values.fitbit_api_client }}
fitbit.api.secret={{ .Values.fitbit_api_secret }}
Expand All @@ -20,6 +20,9 @@ data:
fitbit.user.repository.client.id={{ .Values.oauthClientId }}
fitbit.user.repository.client.secret={{ .Values.oauthClientSecret }}
fitbit.user.repository.oauth2.token.url={{ .Values.managementportal_url }}/oauth/token
fitbit.user.poll.interval={{ .Values.fitbit_user_poll_interval | int }}
application.loop.interval.ms={{ .Values.application_loop_interval_ms | int }}
user.cache.refresh.interval.ms={{ .Values.user_cache_refresh_interval_ms | int}}
{{- if and .Values.kafka_wait.enabled .Values.kafka_wait.properties }}
kafka-wait.properties: |
{{ .Values.kafka_wait.properties | indent 4 }}
Expand Down
2 changes: 1 addition & 1 deletion charts/radar-fitbit-connector/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ spec:
- name: CONNECTOR_PROPERTY_FILE_PREFIX
value: "source-fitbit/source-fitbit"
- name: KAFKA_HEAP_OPTS
value: "-Xms256m -Xmx768m"
value: {{ .Values.heapOptions }}
- name: KAFKA_BROKERS
value: "{{ .Values.kafka_num_brokers }}"
- name: CONNECT_LOG4J_LOGGERS
Expand Down
16 changes: 16 additions & 0 deletions charts/radar-fitbit-connector/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ resources:
cpu: 100m
memory: 1Gi

heapOptions: "-Xms256m -Xmx768m"

persistence:
# -- Enable persistence using PVC
enabled: true
Expand Down Expand Up @@ -219,3 +221,17 @@ oauthClientSecret: secret
managementportal_url: http://management-portal:8080/managementportal
# -- Set to true, if intraday access data should be collected by the connector. This will be set in connector.properties.
includeIntradayData: true


# -- How often to poll the source URL.
# Only use to speed up processing times during e2e testing.
rest_source_poll_interval_ms: 60000
# -- Polling interval per Fitbit user per request route in seconds.
# Only use to speed up processing times during e2e testing.
fitbit_user_poll_interval: 5000
# -- How often to perform the main application loop (only controls how often to poll for new user registrations).
# Only use to speed up processing times during e2e testing.
application_loop_interval_ms: 300000
# -- How often to invalidate the cache and poll for new user registrations.
# Only use to speed up processing times during e2e testing.
user_cache_refresh_interval_ms: 3600000
Loading