Skip to content

Latest commit

 

History

History
304 lines (207 loc) · 7.9 KB

CONTRIBUTING.md

File metadata and controls

304 lines (207 loc) · 7.9 KB
Raw

Contributing

PostgreSQL Partition Manager uses GitHub to manage reviews of pull requests.

Steps to Contribute

Should you wish to work on an issue, please claim it first by commenting on the GitHub issue that you want to work on it. This is to prevent duplicated efforts from contributors on the same issue.

All our issues are regularly tagged so you can filter down the issues involving the components you want to work on.

For quickly compiling and testing your changes do:

# For building.
make build
./postgresql-partition-manager

# For testing.
make test         # Make sure all the tests pass before you commit and push :)

We use:

  • pre-commit to make right first time changes. Enable it for this repository with pre-commit install.

  • golangci-lint for linting the code. If it reports an issue and you think that the warning needs to be disregarded or is a false-positive, you can add a special comment //nolint:linter1[,linter2,...] before the offending line. Use this sparingly though, fixing the code to comply with the linter's recommendation is in general the preferred course of action.

  • markdownlint-cli2 for linting the Markdown documents.

  • yamllint for linting the YAML documents.

Pull Request Checklist

  • Branch from the main branch and, if needed, rebase to the current main branch before submitting your pull request. If it doesn't merge cleanly with main you may be asked to rebase your changes.

  • Commits should be as small as possible while ensuring each commit is correct independently (i.e., each commit should compile and pass tests).

  • Add tests relevant to the fixed bug or new feature.

Install pre-commit

  1. Install pre-commit

  2. Install markdownlint-cli2

  3. Enable pre-commit for the repository

    pre-commit install

Local development

Docker

  1. Install requirements

    Optionals:

  2. Setup PostgreSQL

    Via docker containers:

    cd scripts/localdev/
export POSTGRESQL_VERSION=16 # Optional. Override PostgreSQL version
docker compose up -d postgres

    Or manually:

    \i scripts/localdev/configuration/postgresql/seeds/00_database.sql
\i scripts/localdev/configuration/postgresql/seeds/10_by_date.sql
\i scripts/localdev/configuration/postgresql/seeds/10_by_timestamp.sql
\i scripts/localdev/configuration/postgresql/seeds/10_by_uuidv7.sql

  3. Build application from the root directory

    make build

  4. Optional. Create configuration file

    cat > postgresql-partition-manager.yaml << EOF
---
debug: true

log-format: text

connection-url: postgres://postgres:hackme@localhost/partitions

partitions:
  by_date:
    schema: public
    table: by_date
    partitionKey: created_at
    interval: yearly
    retention: 7
    preProvisioned: 7
    cleanupPolicy: drop
  by_timestamp:
    schema: public
    table: by_timestamp
    partitionKey: created_at
    interval: daily
    retention: 7
    preProvisioned: 7
    cleanupPolicy: drop
  by_uuidv7:
    schema: public
    table: by_uuidv7
    partitionKey: id
    interval: monthly
    retention: 3
    preProvisioned: 1
    cleanupPolicy: drop
EOF

    Run provisioning script to perform provisioning, clean up, and check operations

    ./postgresql-partition-manager run all
Kubernetes

The Kubernetes local development environment located in scripts/kubernetesdev is designed to facilitate Helm chart development and QA in containerized environment.

Requirements:

Steps:

  1. Build application (from repository root directory)

    docker build . -t postgresql-partition-manager:dev

  2. Build Helm chart dependencies

    cd scripts/kubernetesdev/
helm dependency build --skip-refresh

  3. Set deployment parameters

    KUBERNETES_NAMESPACE=default # Replace with your namespace
HELM_RELEASE_NAME=main # Replace with an helm release

  4. Optional. Adjust deployment settings in values.yaml.

    vim values.yaml

  5. Trigger PostgreSQL and Postgresql Partition Manager deployments

    helm upgrade ${HELM_RELEASE_NAME} . --install --values values.yaml

  6. Trigger the PostgreSQL Partition Manager job manually

    Set a Kubernetes job name:

    MANUAL_JOB=ppm-manually-triggered

    Trigger job manually:

    kubectl create job --namespace ${KUBERNETES_NAMESPACE} --from=cronjob/${HELM_RELEASE_NAME}-postgresql-partition-manager ${MANUAL_JOB}

    Check cronjob execution:

    kubectl describe job --namespace ${KUBERNETES_NAMESPACE} ${MANUAL_JOB}

    Check application logs

    # PostgreSQL logs
kubectl logs --namespace ${KUBERNETES_NAMESPACE} deployments/postgres

# PostgreSQL partition manager
kubectl logs --namespace ${KUBERNETES_NAMESPACE} --selector=job-name=${MANUAL_JOB}

    Clean up manual job

    kubectl delete job --namespace ${KUBERNETES_NAMESPACE} ${MANUAL_JOB}

  7. Cleanup, delete PostgreSQL deployment

    helm uninstall ${HELM_RELEASE_NAME}

Useful commands:

Connect to PostgreSQL

export PGHOST=localhost
export PGPORT=$(kubectl get svc postgres-nodeport -o jsonpath='{.spec.ports[0].nodePort}')
export PGUSER=$(kubectl get secret postgres-credentials --template={{.data.user}} | base64 -D)
export PGPASSWORD=$(kubectl get secret postgres-credentials --template={{.data.password}} | base64 -D)
export PGDATABASE=$(kubectl get configmap postgres-configuration --template={{.data.database}})
psql

Tests

Bats

  1. Install dependencies

    brew install bats-core
brew tap bats-core/bats-core
brew install bats-support
brew install bats-assert
brew install yq
brew install libpq # or postgresql

  2. Start PostgreSQL

  3. Export environment variables

    export PGHOST=localhost
export PGDATABASE=unittest
export PGUSER=postgres
export PGPASSWORD=hackme

  4. Launch tests

    make bats-test

Dependency management

Project uses Go modules to manage dependencies on external packages.

To add or update a new dependency, use the go get command:

# Pick the latest tagged release.
go get example.com/some/module/pkg@latest

# Pick a specific version.
go get example.com/some/module/[email protected]

Tidy up the go.mod and go.sum files:

# The GO111MODULE variable can be omitted when the code isn't located in GOPATH.
GO111MODULE=on go mod tidy

You have to commit the changes to go.mod and go.sum before submitting the pull request.