Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: iceberg sink failed to create with sql/jdbc catalog #14841

Closed
chenzl25 opened this issue Jan 29, 2024 · 2 comments
Closed

bug: iceberg sink failed to create with sql/jdbc catalog #14841

chenzl25 opened this issue Jan 29, 2024 · 2 comments
Labels
type/bug Something isn't working
Milestone
release-1.7

Comments

@chenzl25
Copy link
Contributor

chenzl25 commented Jan 29, 2024 •
edited
Loading

Describe the bug

CREATE SINK s1_sink FROM s1 WITH (connector = 'iceberg', type = 'append-only', force_append_only = 'true', warehouse.path = 's3a://hummock001/', s3.endpoint = 'http://127.0.0.1:9301', s3.access.key = 'hummockadmin', s3.secret.key = 'hummockadmin', s3.region = 'us-east-1', catalog.type = 'jdbc', catalog.uri = 'jdbc:mysql://127.0.0.1:3306/test_iceberg', database.name = 'my_db', table.name = 'my_table'
2024-01-29T14:50:47.384879+08:00  INFO icelake::catalog: Loading base catalog config from configs: {"iceberg.table.io.root": "", "iceberg.table.io.disable_config_load": "true", "iceberg.catalog.name": "risingwave", "iceberg.catalog.type": "sql", "iceberg.table.io.secret_access_key": "hummockadmin", "iceberg.table.io.endpoint": "http://127.0.0.1:9301", "iceberg.table.io.access_key_id": "hummockadmin", "iceberg.table.io.bucket": "hummock001", "iceberg.table.io.region": "us-east-1"}    
2024-01-29T14:50:47.385108+08:00  INFO icelake::catalog: Parsed base catalog config: BaseCatalogConfig { name: "risingwave", table_io_configs: {"bucket": "hummock001", "access_key_id": "hummockadmin", "disable_config_load": "true", "endpoint": "http://127.0.0.1:9301", "region": "us-east-1", "secret_access_key": "hummockadmin", "root": ""}, table_config: TableConfig { parquet_writer: ParquetWriterConfig { enable_bloom_filter: false, created_by: None, compression: SNAPPY, max_row_group_size: 1048576, write_batch_size: 1024, data_page_size: 1048576 }, rolling_writer: RollingWriterConfig { rows_per_file: 1000, target_file_size_in_bytes: 1048576 }, sorted_delete_position_writer: SortedDeletePositionWriterConfig { max_record_num: 2000 } } }    
2024-01-29T14:50:47.385913+08:00 DEBUG jni::wrapper::java_vm::vm: Attached thread rw-main (ThreadId(10)). 1 threads attached    
thread 'rw-main' panicked at /Users/dylan/Desktop/workspace/risingwave/src/connector/src/sink/iceberg/jni_catalog.rs:175:18:
called `Result::unwrap()` on an `Err` value: JavaException
stack backtrace:
   0: rust_begin_unwind
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/core/src/result.rs:1649:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/core/src/result.rs:1073:23
   4: risingwave_connector::sink::iceberg::jni_catalog::JniCatalog::build::{{closure}}
             at ./src/connector/src/sink/iceberg/jni_catalog.rs:164:39
   5: risingwave_jni_core::jvm_runtime::execute_with_jni_env
             at ./src/jni_core/src/jvm_runtime.rs:216:15
   6: risingwave_connector::sink::iceberg::jni_catalog::JniCatalog::build
             at ./src/connector/src/sink/iceberg/jni_catalog.rs:150:9
   7: risingwave_connector::sink::iceberg::IcebergSink::create_catalog::{{closure}}
             at ./src/connector/src/sink/iceberg/mod.rs:394:17
   8: risingwave_connector::sink::iceberg::IcebergSink::create_table::{{closure}}
             at ./src/connector/src/sink/iceberg/mod.rs:408:14
   9: <risingwave_connector::sink::iceberg::IcebergSink as risingwave_connector::sink::Sink>::validate::{{closure}}
             at ./src/connector/src/sink/iceberg/mod.rs:478:37
  10: risingwave_meta::stream::sink::validate_sink::{{closure}}
             at ./src/meta/src/stream/sink.rs:32:28
  11: risingwave_meta::rpc::ddl_controller::DdlController::create_streaming_job::{{closure}}
             at ./src/meta/src/rpc/ddl_controller.rs:735:41
  12: risingwave_meta::rpc::ddl_controller::DdlController::run_command::{{closure}}::{{closure}}
             at ./src/meta/src/rpc/ddl_controller.rs:304:22
  13: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.40/src/instrument.rs:321:9
  14: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/core/src/future/future.rs:124:9
  15: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.40/src/instrument.rs:321:9
  16: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/core.rs:334:17
  17: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/loom/std/unsafe_cell.rs:16:9
  18: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/core.rs:323:13
  19: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:485:19
  20: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/core/src/panic/unwind_safe.rs:272:9
  21: std::panicking::try::do_call
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panicking.rs:552:40
  22: ___rust_try
  23: std::panicking::try
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panicking.rs:516:19
  24: std::panic::catch_unwind
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panic.rs:142:14
  25: tokio::runtime::task::harness::poll_future
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:473:18
  26: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:208:27
  27: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:153:15
  28: tokio::runtime::task::raw::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/raw.rs:276:5
  29: tokio::runtime::task::raw::RawTask::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/raw.rs:200:18
  30: tokio::runtime::task::LocalNotified<S>::run
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/mod.rs:400:9
  31: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:639:17
  32: tokio::runtime::coop::with_budget
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/coop.rs:107:5
  33: tokio::runtime::coop::budget
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/coop.rs:73:5
  34: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:575:9
  35: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:526:24
  36: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:491:21
  37: tokio::runtime::context::scoped::Scoped<T>::set
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/context/scoped.rs:40:9
  38: tokio::runtime::context::set_scheduler::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/context.rs:176:26
  39: std::thread::local::LocalKey<T>::try_with
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/thread/local.rs:270:16
  40: std::thread::local::LocalKey<T>::with
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/thread/local.rs:246:9
  41: tokio::runtime::context::set_scheduler
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/context.rs:176:9
  42: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:486:9
  43: tokio::runtime::context::runtime::enter_runtime
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/context/runtime.rs:65:16
  44: tokio::runtime::scheduler::multi_thread::worker::run
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:478:5
  45: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:447:45
  46: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/blocking/task.rs:42:21
  47: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tracing-0.1.40/src/instrument.rs:321:9
  48: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/core.rs:334:17
  49: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/loom/std/unsafe_cell.rs:16:9
  50: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/core.rs:323:13
  51: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:485:19
  52: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/core/src/panic/unwind_safe.rs:272:9
  53: std::panicking::try::do_call
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panicking.rs:552:40
  54: ___rust_try
  55: std::panicking::try
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panicking.rs:516:19
  56: std::panic::catch_unwind
             at /rustc/e4c626dd9a17a23270bf8e7158e59cf2b9c04840/library/std/src/panic.rs:142:14
  57: tokio::runtime::task::harness::poll_future
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:473:18
  58: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:208:27
  59: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/harness.rs:153:15
  60: tokio::runtime::task::raw::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/raw.rs:276:5
  61: tokio::runtime::task::raw::RawTask::poll
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/raw.rs:200:18
  62: tokio::runtime::task::UnownedTask<S>::run
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/task/mod.rs:437:9
  63: tokio::runtime::blocking::pool::Task::run
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/blocking/pool.rs:159:9
  64: tokio::runtime::blocking::pool::Inner::run
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/blocking/pool.rs:513:17
  65: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/dylan/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.32.0/src/runtime/blocking/pool.rs:471:13
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Mon Jan 29 06:50:49 UTC 2024 [risedev]: Program exited with 134

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response
@chenzl25 chenzl25 added the type/bug Something isn't working label Jan 29, 2024
@github-actions github-actions bot added this to the release-1.7 milestone Jan 29, 2024
@chenzl25
Copy link
Contributor Author

cc @liurenjie1024

@liurenjie1024
Copy link
Contributor

The feature is not totally complete, and I'm working on the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/bug Something isn't working
Projects
None yet
Milestone
release-1.7
Development

No branches or pull requests
2 participants
@liurenjie1024 @chenzl25