Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse postgres service in risedev for tests #18858

Open
kwannoel opened this issue Oct 11, 2024 · 2 comments
Open

Reuse postgres service in risedev for tests #18858

kwannoel opened this issue Oct 11, 2024 · 2 comments
Assignees
Milestone

Comments

@kwannoel
Copy link
Contributor

kwannoel commented Oct 11, 2024

We can reuse the one we use for meta backend in our tests for pg cdc / pg tvf.

Wait for source tests to migrate to sqlbackend in #18099, before working on this, so we have the pg service managed by risedev.

@github-actions github-actions bot added this to the release-2.1 milestone Oct 11, 2024
@kwannoel kwannoel self-assigned this Oct 11, 2024
@kwannoel kwannoel reopened this Oct 11, 2024
@kwannoel kwannoel modified the milestones: release-2.1, release-2.2 Oct 17, 2024
@kwannoel
Copy link
Contributor Author

Looks tricker than expected:

cargo-make] INFO - Execute Command: "/usr/bin/env" "bash" "/tmp/fsio_xGxyFqq67G.sh" "ci-inline-source-test"
[cargo-make] INFO - Execute Command: "target/debug/risedev-dev" "ci-inline-source-test"
ERROR - Failed to start: service requires docker to be installed
Caused by:
  No such file or directory (os error 2)

I attempted it in https://buildkite.com/risingwavelabs/pull-request/builds/60468#01929ed1-b87a-45f2-a19c-a5ee894db098.

But our ci environment does not have docker, which this depends on. I suppose we have to stick to ci docker compose. Although we can still make the postgres tests inlined.

@xxchan
Copy link
Member

xxchan commented Oct 18, 2024

In CI we use user-managed mode. The services are started in ci/docker-compose.yml, instead of RiseDev. We just tell RiseDev the ports

  • risingwave/risedev.yml

    Lines 846 to 872 in 0226bd5

    ci-inline-source-test:
    config-path: src/config/ci-recovery.toml
    steps:
    - use: minio
    - use: sqlite
    - use: meta-node
    meta-backend: sqlite
    - use: compute-node
    enable-tiered-cache: true
    - use: frontend
    - use: compactor
    - use: pubsub
    persist-data: true
    - use: kafka
    user-managed: true
    address: message_queue
    port: 29092
    - use: schema-registry
    user-managed: true
    address: schemaregistry
    port: 8082
    - use: mysql
    port: 3306
    address: mysql
    user: root
    password: 123456
    user-managed: true

  • https://risingwavelabs.github.io/risingwave/connector/intro.html#escape-hatch-user-managed-mode

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants