Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Influxdb #1

Merged
merged 50 commits into from
Jun 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
2b1f6e0
#203
mal5305 May 21, 2021
01b7d2c
Merge pull request #205 from mal5305/#203
samwiseg0 May 21, 2021
c47b7c2
Update docker compose to specify influxdb:1.8.4
samwiseg0 Jun 3, 2021
c6b8dde
Update requirements to use urllib3==1.26.5
samwiseg0 Jun 3, 2021
57028cd
updated to support Radarr and Sonarr V3 Api
RobinDadswell Sep 17, 2021
e3f2ee9
Merge pull request #214 from RobinDadswell/RadarrAndSonarrV3Api
samwiseg0 Sep 20, 2021
7a8c5a3
bump requirements for requests
samwiseg0 Jan 15, 2022
2607584
Fix Sonarr & Radarr V3 API /queue endpoint (#220)
tigattack Jan 15, 2022
533ec10
Fix lint issues
samwiseg0 Jan 15, 2022
63746dd
More lint fixes
samwiseg0 Jan 15, 2022
9498a83
Update Sonarr structures
samwiseg0 Jan 15, 2022
73b2686
Add Overseerr Support (#210)
samwiseg0 Jan 15, 2022
d4c8037
Remove duplicate structures
samwiseg0 Jan 15, 2022
5a76459
update changelog to reflect v1.7.7 changes
samwiseg0 Jan 15, 2022
436a682
Add IP data to tautulli #202
samwiseg0 Jan 15, 2022
9508c3c
add missing ip address in tautulli
samwiseg0 Jan 15, 2022
870c6cd
Fixed: Streamlined API calls to Radarr and Sonarr (#221)
RobinDadswell Jan 17, 2022
463f37e
Fixed: Sonarr Data pull issues (#222)
RobinDadswell Jan 17, 2022
7a4cf59
Fix Sonarrr calendar
samwiseg0 Jan 17, 2022
5e8c8eb
Update lidarr structure (#225)
StewartThomson Jan 18, 2022
08c4969
Clean up request totals. Upstream change sct/overseerr#2426
samwiseg0 Jan 19, 2022
756b89b
Cleanup blank space
samwiseg0 Jan 19, 2022
908cfb1
Fix requested_date syntax.
samwiseg0 Jan 19, 2022
e4b9926
Fix requested_date for Overseerr tv and movie
samwiseg0 Jan 19, 2022
d1b47e0
Fix overseerr config refernces
samwiseg0 Jan 19, 2022
442b518
Fix overseerr structures
samwiseg0 Jan 19, 2022
e292002
Update intparser to accommodate changes to config structure
samwiseg0 Jan 19, 2022
8853599
Cleanup overseerr data collection
samwiseg0 Jan 19, 2022
3c70ecb
Fix SERVICES_ENABLED in varken.py to acomidate overseerr
samwiseg0 Jan 19, 2022
707c4a2
Fixed: Sonarr/Lidarr Queues (#227)
RobinDadswell Jan 19, 2022
06c4777
Change sonarr queue structures to str
samwiseg0 Jan 19, 2022
62749f2
Fixed: Multipage queue fetching
samwiseg0 Jan 21, 2022
45f9a20
Update historical tautulli import (#226)
StewartThomson Jan 21, 2022
518ea6c
Fixed: Sonarr perams ordering
samwiseg0 Jan 21, 2022
752073d
Fixed: Proper warnings for missing data in sonarr and radarr
samwiseg0 Jan 21, 2022
a60e41e
Added: Overseerr ENVs to docker compose.
samwiseg0 Jan 21, 2022
2f7f01e
Added: Logging to empty/no data returns
samwiseg0 Jan 21, 2022
7318085
Update Sonarr & Lidarr Structs to match latest API changes (#231)
Cajs Mar 7, 2022
2c81fe5
Fix typo in docker yml
samwiseg0 Apr 1, 2022
5f8af8e
Rename example url for overseerr in docker yml
samwiseg0 Apr 1, 2022
f3960d2
Update radarr structures to inclue originalLanguage
samwiseg0 Apr 21, 2022
7b4b445
Update radarr structures to include addOptions
samwiseg0 Apr 21, 2022
0a927d0
Update radarr structures to include popularity
samwiseg0 Apr 29, 2022
817c7a5
fix(ombi): Update structures.py (#238)
MDHMatt Jul 12, 2022
2cc26c1
feat(docker): remove envs from example
samwiseg0 Jul 17, 2022
cfc5c69
fix(logging): remove depreciation warning. Var for debug mode (#240)
MDHMatt Jul 17, 2022
2527b32
Support InfluxDB 2.x as addition to 1.8
Dinnerbone Jul 17, 2022
1e71ec5
Document that influxdb 2.x is supported
Dinnerbone Jul 17, 2022
aa9f259
Include influxdb username/password for v2 server support
Dinnerbone Jul 17, 2022
052be06
Support an optional v prefix for influxdb version strings
Dinnerbone Jul 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
# Change Log

## [v1.7.7](https://github.com/Boerderij/Varken/tree/v1.7.7) (2020-12-21)
[Full Changelog](https://github.com/Boerderij/Varken/compare/1.7.6...v1.7.7)

**Implemented enhancements:**
- \[Enhancement\] Ombi 4.0 compatibility [\#186](https://github.com/Boerderij/Varken/issues/186)
([samwiseg0](https://github.com/samwiseg0))

**Merged pull requests:**

- v1.7.7 Merge [\#191](https://github.com/Boerderij/Varken/pull/191)
([DirtyCajunRice](https://github.com/DirtyCajunRice))
- Type Error fix [\#177](https://github.com/Boerderij/Varken/pull/177)
([derek-miller](https://github.com/derek-miller))

**Fixed bugs:**

- \[BUG\] Influxdb exit code [\#174](https://github.com/Boerderij/Varken/issues/174)
([samwiseg0](https://github.com/samwiseg0))

**Notes:**
- Now built via github actions
- Available on ghcr, quay.io, and dockerhub
- Nightly builds done to accommodate dependabot MRs

## [v1.7.6](https://github.com/Boerderij/Varken/tree/v1.7.6) (2020-01-01)
[Full Changelog](https://github.com/Boerderij/Varken/compare/1.7.5...v1.7.6)

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ ecosystem into InfluxDB using Grafana for a frontend
Requirements:
* [Python 3.6.7+](https://www.python.org/downloads/release/python-367/)
* [Python3-pip](https://pip.pypa.io/en/stable/installing/)
* [InfluxDB 1.8.x](https://www.influxdata.com/)
* [InfluxDB 1.8.x or 2.x](https://www.influxdata.com/)
* [Grafana](https://grafana.com/)

<p align="center">
Expand Down Expand Up @@ -50,7 +50,7 @@ Please read [Asking for Support](https://wiki.cajun.pro/books/varken/chapter/ask

### InfluxDB
[InfluxDB Installation Documentation](https://wiki.cajun.pro/books/varken/page/influxdb-d1f)
Note: Only v1.8.x is currently supported.
Note: Only v1.8.x or v2.x are supported.

Influxdb is required but not packaged as part of Varken. Varken will create
its database on its own. If you choose to give varken user permissions that
Expand Down
21 changes: 18 additions & 3 deletions Varken.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
import platform
import schedule
import distro
from time import sleep
from queue import Queue
from sys import version
from threading import Thread
from os import environ as env
from os import access, R_OK, getenv
from distro import linux_distribution
from os.path import isdir, abspath, dirname, join
from argparse import ArgumentParser, RawTextHelpFormatter
from logging import getLogger, StreamHandler, Formatter, DEBUG


# Needed to check version of python
from varken import structures # noqa
from varken.ombi import OmbiAPI
from varken.overseerr import OverseerrAPI
from varken.unifi import UniFiAPI
from varken import VERSION, BRANCH, BUILD_DATE
from varken.sonarr import SonarrAPI
Expand All @@ -27,7 +29,7 @@
from varken.varkenlogger import VarkenLogger


PLATFORM_LINUX_DISTRO = ' '.join(x for x in linux_distribution() if x)
PLATFORM_LINUX_DISTRO = ' '.join(distro.id() + distro.version() + distro.name())


def thread(job, **kwargs):
Expand Down Expand Up @@ -156,6 +158,18 @@ def thread(job, **kwargs):
at_time = schedule.every(server.issue_status_run_seconds).seconds
at_time.do(thread, OMBI.get_issue_counts).tag("ombi-{}-get_issue_counts".format(server.id))

if CONFIG.overseerr_enabled:
for server in CONFIG.overseerr_servers:
OVERSEER = OverseerrAPI(server, DBMANAGER)
if server.get_request_total_counts:
at_time = schedule.every(server.request_total_run_seconds).seconds
at_time.do(thread, OVERSEER.get_request_counts).tag("overseerr-{}-get_request_counts"
.format(server.id))
if server.num_latest_requests_to_fetch > 0:
at_time = schedule.every(server.num_latest_requests_seconds).seconds
at_time.do(thread, OVERSEER.get_latest_requests).tag("overseerr-{}-get_latest_requests"
.format(server.id))

if CONFIG.sickchill_enabled:
for server in CONFIG.sickchill_servers:
SICKCHILL = SickChillAPI(server, DBMANAGER)
Expand All @@ -171,7 +185,8 @@ def thread(job, **kwargs):

# Run all on startup
SERVICES_ENABLED = [CONFIG.ombi_enabled, CONFIG.radarr_enabled, CONFIG.tautulli_enabled, CONFIG.unifi_enabled,
CONFIG.sonarr_enabled, CONFIG.sickchill_enabled, CONFIG.lidarr_enabled]
CONFIG.sonarr_enabled, CONFIG.sickchill_enabled, CONFIG.lidarr_enabled,
CONFIG.overseerr_enabled]
if not [enabled for enabled in SERVICES_ENABLED if enabled]:
vl.logger.error("All services disabled. Exiting")
exit(1)
Expand Down
15 changes: 14 additions & 1 deletion data/varken.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ sonarr_server_ids = 1,2
radarr_server_ids = 1,2
lidarr_server_ids = false
tautulli_server_ids = 1
ombi_server_ids = 1
ombi_server_ids = false
overseerr_server_ids = 1
sickchill_server_ids = false
unifi_server_ids = false
maxmind_license_key = xxxxxxxxxxxxxxxx
Expand All @@ -15,6 +16,7 @@ ssl = false
verify_ssl = false
username = root
password = root
org = -

[tautulli-1]
url = tautulli.domain.tld:8181
Expand Down Expand Up @@ -95,6 +97,17 @@ request_total_run_seconds = 300
get_issue_status_counts = true
issue_status_run_seconds = 300

[overseerr-1]
url = overseerr.domain.tld
apikey = xxxxxxxxxxxxxxxx
ssl = false
verify_ssl = false
get_request_total_counts = true
request_total_run_seconds = 30
get_latest_requests = true
num_latest_requests_to_fetch = 10
num_latest_requests_seconds = 30

[sickchill-1]
url = sickchill.domain.tld:8081
apikey = xxxxxxxxxxxxxxxx
Expand Down
91 changes: 3 additions & 88 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ services:
influxdb:
hostname: influxdb
container_name: influxdb
image: influxdb
image: influxdb:1.8
networks:
- internal
volumes:
Expand All @@ -22,91 +22,6 @@ services:
- /path/to/docker-varken/config-folder:/config
environment:
- TZ=America/Chicago
- VRKN_GLOBAL_SONARR_SERVER_IDS=1,2
- VRKN_GLOBAL_RADARR_SERVER_IDS=1,2
- VRKN_GLOBAL_LIDARR_SERVER_IDS=false
- VRKN_GLOBAL_TAUTULLI_SERVER_IDS=1
- VRKN_GLOBAL_OMBI_SERVER_IDS=1
- VRKN_GLOBAL_SICKCHILL_SERVER_IDS=false
- VRKN_GLOBAL_UNIFI_SERVER_IDS=false
- VRKN_GLOBAL_MAXMIND_LICENSE_KEY=xxxxxxxxxxxxxxxx
- VRKN_INFLUXDB_URL=influxdb.domain.tld
- VRKN_INFLUXDB_PORT=8086
- VRKN_INFLUXDB_SSL=false
- VRKN_INFLUXDB_VERIFY_SSL=false
- VRKN_INFLUXDB_USERNAME=root
- VRKN_INFLUXDB_PASSWORD=root
- VRKN_TAUTULLI_1_URL=tautulli.domain.tld:8181
- VRKN_TAUTULLI_1_FALLBACK_IP=1.1.1.1
- VRKN_TAUTULLI_1_APIKEY=xxxxxxxxxxxxxxxx
- VRKN_TAUTULLI_1_SSL=false
- VRKN_TAUTULLI_1_VERIFY_SSL=false
- VRKN_TAUTULLI_1_GET_ACTIVITY=true
- VRKN_TAUTULLI_1_GET_ACTIVITY_RUN_SECONDS=30
- VRKN_TAUTULLI_1_GET_STATS=true
- VRKN_TAUTULLI_1_GET_STATS_RUN_SECONDS=3600
- VRKN_SONARR_1_URL=sonarr1.domain.tld:8989
- VRKN_SONARR_1_APIKEY=xxxxxxxxxxxxxxxx
- VRKN_SONARR_1_SSL=false
- VRKN_SONARR_1_VERIFY_SSL=false
- VRKN_SONARR_1_MISSING_DAYS=7
- VRKN_SONARR_1_MISSING_DAYS_RUN_SECONDS=300
- VRKN_SONARR_1_FUTURE_DAYS=1
- VRKN_SONARR_1_FUTURE_DAYS_RUN_SECONDS=300
- VRKN_SONARR_1_QUEUE=true
- VRKN_SONARR_1_QUEUE_RUN_SECONDS=300
- VRKN_SONARR_2_URL=sonarr2.domain.tld:8989
- VRKN_SONARR_2_APIKEY=yyyyyyyyyyyyyyyy
- VRKN_SONARR_2_SSL=false
- VRKN_SONARR_2_VERIFY_SSL=false
- VRKN_SONARR_2_MISSING_DAYS=7
- VRKN_SONARR_2_MISSING_DAYS_RUN_SECONDS=300
- VRKN_SONARR_2_FUTURE_DAYS=1
- VRKN_SONARR_2_FUTURE_DAYS_RUN_SECONDS=300
- VRKN_SONARR_2_QUEUE=true
- VRKN_SONARR_2_QUEUE_RUN_SECONDS=300
- VRKN_RADARR_1_URL=radarr1.domain.tld
- VRKN_RADARR_1_APIKEY=xxxxxxxxxxxxxxxx
- VRKN_RADARR_1_SSL=false
- VRKN_RADARR_1_VERIFY_SSL=false
- VRKN_RADARR_1_QUEUE=true
- VRKN_RADARR_1_QUEUE_RUN_SECONDS=300
- VRKN_RADARR_1_GET_MISSING=true
- VRKN_RADARR_1_GET_MISSING_RUN_SECONDS=300
- VRKN_RADARR_2_URL=radarr2.domain.tld
- VRKN_RADARR_2_APIKEY=yyyyyyyyyyyyyyyy
- VRKN_RADARR_2_SSL=false
- VRKN_RADARR_2_VERIFY_SSL=false
- VRKN_RADARR_2_QUEUE=true
- VRKN_RADARR_2_QUEUE_RUN_SECONDS=300
- VRKN_RADARR_2_GET_MISSING=true
- VRKN_RADARR_2_GET_MISSING_RUN_SECONDS=300
- VRKN_LIDARR_1_URL=lidarr1.domain.tld:8686
- VRKN_LIDARR_1_APIKEY=xxxxxxxxxxxxxxxx
- VRKN_LIDARR_1_SSL=false
- VRKN_LIDARR_1_VERIFY_SSL=false
- VRKN_LIDARR_1_MISSING_DAYS=30
- VRKN_LIDARR_1_MISSING_DAYS_RUN_SECONDS=300
- VRKN_LIDARR_1_FUTURE_DAYS=30
- VRKN_LIDARR_1_FUTURE_DAYS_RUN_SECONDS=300
- VRKN_LIDARR_1_QUEUE=true
- VRKN_LIDARR_1_QUEUE_RUN_SECONDS=300
- VRKN_OMBI_1_URL=ombi.domain.tld
- VRKN_OMBI_1_APIKEY=xxxxxxxxxxxxxxxx
- VRKN_OMBI_1_SSL=false
- VRKN_OMBI_1_VERIFY_SSL=false
- VRKN_OMBI_1_GET_REQUEST_TYPE_COUNTS=true
- VRKN_OMBI_1_REQUEST_TYPE_RUN_SECONDS=300
- VRKN_OMBI_1_GET_REQUEST_TOTAL_COUNTS=true
- VRKN_OMBI_1_REQUEST_TOTAL_RUN_SECONDS=300
- VRKN_OMBI_1_GET_ISSUE_STATUS_COUNTS=true
- VRKN_OMBI_1_ISSUE_STATUS_RUN_SECONDS=300
- VRKN_SICKCHILL_1_URL=sickchill.domain.tld:8081
- VRKN_SICKCHILL_1_APIKEY=xxxxxxxxxxxxxxxx
- VRKN_SICKCHILL_1_SSL=false
- VRKN_SICKCHILL_1_VERIFY_SSL=false
- VRKN_SICKCHILL_1_GET_MISSING=true
- VRKN_SICKCHILL_1_GET_MISSING_RUN_SECONDS=300
depends_on:
- influxdb
restart: unless-stopped
Expand All @@ -118,7 +33,7 @@ services:
- internal
ports:
- 3000:3000
volumes:
volumes:
- /path/to/docker-grafana/config-folder:/config
environment:
- GF_PATHS_DATA=/config/data
Expand All @@ -128,4 +43,4 @@ services:
depends_on:
- influxdb
- varken
restart: unless-stopped
restart: unless-stopped
5 changes: 3 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
# Potential requirements.
# pip3 install -r requirements.txt
#---------------------------------------------------------
requests==2.21
requests==2.25.1
geoip2==2.9.0
influxdb==5.2.0
influxdb-client==1.30.0
schedule==0.6.0
distro==1.4.0
urllib3==1.24.2
urllib3==1.26.5
2 changes: 1 addition & 1 deletion utilities/historical_tautulli_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
DBMANAGER = DBManager(CONFIG.influx_server)

if CONFIG.tautulli_enabled:
GEOIPHANDLER = GeoIPHandler(DATA_FOLDER)
GEOIPHANDLER = GeoIPHandler(DATA_FOLDER, CONFIG.tautulli_servers[0].maxmind_license_key)
for server in CONFIG.tautulli_servers:
TAUTULLI = TautulliAPI(server, DBMANAGER, GEOIPHANDLER)
TAUTULLI.get_historical(days=opts.days)
1 change: 1 addition & 0 deletions varken.xml
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,6 @@
<Labels/>
<Config Name="PGID" Target="PGID" Default="" Mode="" Description="Container Variable: PGID" Type="Variable" Display="always" Required="true" Mask="false">99</Config>
<Config Name="PUID" Target="PUID" Default="" Mode="" Description="Container Variable: PUID" Type="Variable" Display="always" Required="true" Mask="false">100</Config>
<Config Name="Debug" Target="DEBUG" Default="False" Mode="" Description="Turn Debug on or off" Type="Variable" Display="always" Required="false" Mask="false">False</Config>
<Config Name="Varken DataDir" Target="/config" Default="" Mode="rw" Description="Container Path: /config" Type="Path" Display="advanced-hide" Required="true" Mask="false">/mnt/user/appdata/varken</Config>
</Container>
Loading