Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed many problem and added systemd unit, docker-compose file and k8s deployment #1

Merged
merged 1 commit into from
Feb 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions .github/workflows/default.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,32 @@
---
name: Default

on: push
on:
push:
branches:
- 'dev'
paths-ignore:
- ".github/**"
- "deploy/**"
- "testing/**"
- "README*"
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
inputs:
reason:
description: 'Launch reason'
required: true
default: 'Run tests'

jobs:
test:
runs-on: ubuntu-latest
container: lesovsky/pgscv-test-runner:0.0.9
container: cherts/pgscv-test-runner:1.0.0

steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Prepare test environment
run: prepare-test-environment.sh
- name: Run lint
Expand Down
28 changes: 16 additions & 12 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,19 @@
name: Release

on:
workflow_dispatch:
push:
tags:
- '*'
- "v0.*"
- "v1.*"

jobs:
test:
runs-on: ubuntu-latest
container: lesovsky/pgscv-test-runner:0.0.9
container: cherts/pgscv-test-runner:1.0.0
steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Prepare test environment
run: prepare-test-environment.sh
- name: Run lint
Expand All @@ -25,30 +27,32 @@ jobs:
needs: test
steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build image
run: make docker-build
- name: Log in to Docker Hub
run: docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_PASSWORD }}
run: docker login -u ${{ secrets.DOCKERHUB_USERNAME }} -p ${{ secrets.DOCKERHUB_TOKEN }}
- name: Push image to Docker Hub
run: make docker-push

goreleaser:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v2
- name: Run checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-go@v2
- name: Run setup Go
uses: actions/setup-go@v4
with:
go-version: 1.18
- uses: goreleaser/goreleaser-action@v2
go-version: 1.22
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v5
with:
version: latest
args: release --rm-dist
args: release --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

GITHUB_TOKEN: ${{ secrets.CUSTOM_GITHUB_TOKEN }}
33 changes: 29 additions & 4 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ builds:
main: ./cmd
goarch:
- amd64
- arm64
goos:
- linux
env:
Expand All @@ -17,15 +18,39 @@ builds:

archives:
- builds: [pgscv]
builds_info:
group: root
owner: root
wrap_in_directory: false
files:
- LICENSE
- src: "deploy/pgscv.yaml"
strip_parent: true
info:
owner: root
group: root
mode: 0640
- src: "deploy/pgscv.service"
strip_parent: true
info:
owner: root
group: root
mode: 0644
- src: "deploy/pgscv.default"
strip_parent: true
info:
owner: root
group: root
mode: 0644

changelog:
sort: asc

nfpms:
- vendor: pgscv
homepage: https://github.com/lesovsky/pgscv
maintainer: Alexey Lesovsky
description: pgSCV is a multi-purpose monitoring agent and metrics exporter
homepage: https://github.com/cherts/pgscv
maintainer: Mikhail Grigorev
description: pgSCV - PostgreSQL ecosystem metrics collector.
license: BSD-3
formats: []
bindir: /usr/bin
bindir: /usr/sbin
16 changes: 10 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
# stage 1
# __release_tag__ golang 1.18 was released 2022-03-15
FROM golang:1.18 as build
# stage 1
# __release_tag__ golang 1.22 was released 2024-02-06
FROM golang:1.22 as build
LABEL stage=intermediate
WORKDIR /app
COPY . .
RUN make build

# stage 2: scratch
# __release_tag__ alpine 3.13 was released 2021-02-18
FROM alpine:3.13 as dist
# __release_tag__ alpine 3.19.1 was released 2024-01-26
FROM alpine:3.19.1 as dist
COPY --from=build /app/bin/pgscv /bin/pgscv
CMD ["pgscv"]
#COPY docker_entrypoint.sh /bin/
EXPOSE 9890
ENTRYPOINT ["/bin/pgscv"]
#ENTRYPOINT ["/bin/docker_entrypoint.sh"]
CMD ["--log-level=info"]
12 changes: 10 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
DOCKER_ACCOUNT = lesovsky
DOCKER_ACCOUNT = cherts
APPNAME = pgscv

TAG=$(shell git describe --tags --abbrev=0)
Expand All @@ -9,7 +9,7 @@ LDFLAGS = -a -installsuffix cgo -ldflags "-X main.appName=${APPNAME} -X main.git

.PHONY: help \
clean lint test race \
build migrate docker-build docker-push deploy
build docker-build docker-push go-update

.DEFAULT_GOAL := help

Expand All @@ -21,10 +21,18 @@ clean: ## Clean
rm -f ./bin/${APPNAME} ./bin/${APPNAME}.tar.gz ./bin/${APPNAME}.version ./bin/${APPNAME}.sha256
rmdir ./bin

go-update: # Update go mod
go mod tidy -compat=1.22
go get -u ./cmd
go mod download
go get -u ./cmd
go mod download

dep: ## Get the dependencies
go mod download

lint: ## Lint the source files
go env -w GOFLAGS="-buildvcs=false"
golangci-lint run --timeout 5m -E golint -e '(struct field|type|method|func) [a-zA-Z`]+ should be [a-zA-Z`]+'
gosec -quiet ./...

Expand Down
86 changes: 61 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
# pgSCV - PostgreSQL ecosystem metrics collector.
# pgSCV - PostgreSQL ecosystem metrics collector

[По-русски / In Russian](README.ru.md)

### pgSCV
- [collects](https://github.com/lesovsky/pgscv/wiki/Collectors) a lot of stats about PostgreSQL environment.
- [collects](https://github.com/cherts/pgscv/wiki/Collectors) a lot of stats about PostgreSQL environment.
- exposes metrics through the HTTP `/metrics` endpoint in [Prometheus metrics exposition format](https://prometheus.io/docs/concepts/data_model/).

**IMPORTANT NOTES**
pgSCV moved from 'weaponry' to 'lesovsky' GitHub account. From version 0.8.0 all features required for Weaponry will be removed:
- auto-discovery (all Postgres, Pgbouncer services have to be defined explicitly, by configuration file or environment variables)
- bootstrap, uninstall and auto-update
- push metrics to remote service
- Patroni support (because it has this feature built-in)
This project is a continuation of the development of the original pgSCV by [Lesovsky Alexey](https://github.com/lesovsky)

### Features
- **Supported services:** support collecting metrics of PostgreSQL and Pgbouncer.
Expand All @@ -22,45 +20,83 @@ pgSCV moved from 'weaponry' to 'lesovsky' GitHub account. From version 0.8.0 all
block devices, network interfaces, filesystems, users, databases, etc.

### Requirements
- can run on Linux only; can connect to remote services running on other OS/PaaS.
- requisites for connecting to the services, such as login and password.
- database user should have privileges for executing stats functions and reading views.
For more details see [security considerations](https://github.com/lesovsky/pgscv/wiki/Security-considerations).
- Can run on Linux only; can connect to remote services running on other OS/PaaS.
- Requisites for connecting to the services, such as login and password.
- Database user should have privileges for executing stats functions and reading views.
For more details see [security considerations](https://github.com/cherts/pgscv/wiki/Security-considerations).

### Quick start
Download the archive from [releases](https://github.com/lesovsky/pgscv/releases). Unpack the archive. Start pgSCV under `postgres` user.
Download the archive from [releases](https://github.com/cherts/pgscv/releases). Unpack the archive. Create minimum config file. Start pgSCV systemd service under `postgres` user.

```bash
curl -s -L https://github.com/cherts/pgscv/releases/download/v0.8.1/pgscv_0.8.1_linux_amd64.tar.gz -o - | tar xzf - -C /tmp && \
mv /tmp/pgscv.yaml /etc && \
mv /tmp/pgscv.service /etc/systemd/system && \
mv /tmp/pgscv.default /etc/default/pgscv && \
mv /tmp/pgscv /usr/sbin && \
chown postgres:postgres /etc/pgscv.yaml && \
systemctl daemon-reload && \
systemctl enable pgscv --now
```

```shell
wget https://github.com/lesovsky/pgscv/releases/download/v0.7.5/pgscv_0.7.5_linux_amd64.tar.gz
tar xvzf pgscv_0.7.5_linux_amd64.tar.gz
sudo -u postgres ./pgscv
or using Docker, use `DATABASE_DSN` for setting up a connection to PostgreSQL:
```bash
docker pull cherts/pgscv:latest
docker run -ti -d --name pgscv \
-e PGSCV_LISTEN_ADDRESS=0.0.0.0:9890 \
-e PGSCV_DISABLE_COLLECTORS="system" \
-e DATABASE_DSN="postgresql://postgres:password@dbhost:5432/postgres" \
-p 9890:9890 \
--restart=always \
cherts/pgscv:latest
```

or using Docker, use `DATABASE_DSN` for setting up a connection to Postgres:
or using Docker, save `deploy/pgscv.yaml` config file to local directory /etc/pgscv:
```bash
docker pull cherts/pgscv:latest
docker run -ti -d --name pgscv \
-v /etc/pgscv:/etc/app \
-p 9890:9890 \
--restart=always \
cherts/pgscv:latest \
--config-file=/etc/app/pgscv.yaml
```
docker pull lesovsky/pgscv:latest
docker run -ti -e PGSCV_LISTEN_ADDRESS=0.0.0.0:9890 -e PGSCV_DISABLE_COLLECTORS="system" -e DATABASE_DSN="postgresql://postgres@dbhost/postgres" -p 9890:9890 lesovsky/pgscv:latest

or using Docker-compose, edit file `docker-compose.yaml` for setting up a connection to PostgreSQL:
```bash
mkdir ~/pgscv
curl -s -L https://raw.githubusercontent.com/CHERTS/pgscv/master/deploy/docker-compose.yaml -o ~/pgscv/docker-compose.yaml && cd ~/pgscv
docker-compose up -d
```

When pgSCV has been started it is ready to accept HTTP requests at `http://127.0.0.1:9890/metrics`.

or using k8s
```bash
curl -s -L https://raw.githubusercontent.com/CHERTS/pgscv/master/deploy/deployment.yaml -o ~/deployment.yaml
kubectl apply -f ~/deployment.yaml
```

### Complete setup
Checkout complete setup [guide](https://github.com/lesovsky/pgscv/wiki/Setup-for-regular-users).
Checkout complete setup [guide](https://github.com/cherts/pgscv/wiki/Setup-for-regular-users).

### Documentation
For further documentation see [wiki](https://github.com/lesovsky/pgscv/wiki).
For further documentation see [wiki](https://github.com/cherts/pgscv/wiki).

### Support and feedback
If you need help using pgSCV feel free to open discussion or create an [issue](https://github.com/lesovsky/pgscv/issues)
If you need help using pgSCV feel free to open discussion via [email]([email protected]) or Telegram [@cherts](https://t.me/cherts) or create an [issue](https://github.com/cherts/pgscv/issues)

### Development and contribution
To help development you are encouraged to:
- provide [suggestion/feedback](https://github.com/lesovsky/pgscv/discussions) or [issue](https://github.com/lesovsky/pgscv/issues)
- provide feedback via [email]([email protected]) or Telegram [@cherts](https://t.me/cherts) or create an [issue](https://github.com/cherts/pgscv/issues)
- pull requests for new features
- star the project

### Authors
- [Lesovsky Alexey](https://github.com/lesovsky)
### Current developer and maintaner
- [Mikhail Grigorev](https://github.com/cherts)

### Authors of original version
- [Alexey Lesovsky](https://github.com/lesovsky)

### License
BSD-3. See [LICENSE](./LICENSE) for more details.
Loading