[pull] main from datafuselabs:main #43
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
|
Pull request description must contain CLA like the following: |
|
This pull request's title is not fulfill the requirements. @pull[bot] please update it 🙏. Valid format: Valid types:
|
* add basic sort support for agg index rewrite * add more tests --------- Co-authored-by: sundyli <[email protected]>
* feat(query): grant ownership need modify role privilege * skip built in roles * get_ownership part inside the revoke_ownership() * merge grant_ownership and transfer_ownership
* feat: return latest leader addr if request is forwarded New feature, compatible change. When a meta-service follower receives a request, it forwards it to the leader. This mechanism introduces additional latency. In this commit: - Meta-service responds the leader-address in gRPC header(response metadata): "x-databend-meta-leader-grpc-endpoint", if the meta-service node is a follower. - Meta-client that receives such a reponse metadate then updates the address for further connection to this perceived leader. - Leader endpoint response is added to two API: `transaction()` and `kv_read_v1()`, which are the most used by `SchemaApi`, and very latency sensitive. `upsert()` and `kv_read()`(v0 API) are not changed: `upsert()` will be replaced with `transaction()`, and `kv_read()` is only used by former versions databend-query, which are not capable of parsing leader endpoint response. Detailed Changes - Add `Endpoints` as a container that stores all of the meta-service endpoints for a meta-client. - Updating leader endpoint is done in `EstablishedClient`, which is a wrapper for underlying gRPC client. `HandleRPCResult::update_client()` is responsible to update leader endpoint and mark a client as **error**. - The `endpoints` now is shared between `MetaChannelManager` and `EstablishedClient`, `MetaChannelManager` uses `endpoints` for connecting, `EstablishedClient` uses it for updating. - The old health based node choosing algorithm is discarded, because the responded leader must be in good healthy state, otherwise it will step down. - Add timing and logging to meta-service API. --- - Fix: #14224 * refactor: Endpoints does not have to store index * chore: update tests according to Endpoints change
* add select tests for agg index * add select tests for agg index
* fix(query): remove topk optimization in parquet2 * fix(query): remove topk optimization in parquet2
* support adaptive filter reorder * chore: add more comments * feat: add Expr::Cast for build_select_expr * chore: add more comments
* chore: enable runtime filter for right join * add no_shuffle to group by shuffle mode * x * add network cost * x * fix join order * fix join order * update * update * add semi to inner join rule * rebase * fix explain * fix explain native * fix tests
* refactor: use binary for non-utf8 string type * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix bendsql * fix * fix * fix * fix * fix * fix * fix * fix * fix * try again * fix * update test case * fix test cases * fmt * fix test cases * dbg * workarond #14296 * update test * update test
* Add huggingface config Signed-off-by: Xuanwo <[email protected]> * Add meta api Signed-off-by: Xuanwo <[email protected]> * Add test for huggingfaceconfig Signed-off-by: Xuanwo <[email protected]> * Allow parse huggingface Signed-off-by: Xuanwo <[email protected]> * Add create huggingface stage test Signed-off-by: Xuanwo <[email protected]> * Allow querying from huggingface Signed-off-by: Xuanwo <[email protected]> * Add commento for fields Signed-off-by: Xuanwo <[email protected]> --------- Signed-off-by: Xuanwo <[email protected]>
* chore: update testing parquet assets for stage test * fix
chore: add cardinality to CteScan explain
* Update 02_0063_function_vector.test * feat(query): add float64 version of distance function overload * feat(query): add float64 version of distance function overload * fix the verctor.rs use style --------- Co-authored-by: Bohu <[email protected]>
* chore: update testing parquet assets for stage test * fix * refactor: add BinaryColumn, BinaryColumnBuilder * fixgp * fix * fix * --wip-- [skip ci] * fix
* delete owner in table/db meta get owner info in meta * add meta test --------- Co-authored-by: Bohu <[email protected]>
* refactor unload. * feat: CopyIntoLocation support output. * update tests. * fix test. * fix test.
* feat: query level data cache metrics * show data cache metrics is query_log table * update ut golden files * Update src/query/catalog/src/statistics/data_cache_statistics.rs Co-authored-by: Bohu <[email protected]> * rename fields * stop gather data from disk cache if in memory cache hits * rename query_log table fields * update test golden file * cleanup * remove logging --------- Co-authored-by: Bohu <[email protected]>
fix mark scan
* fix(query): add comment in table meta * fix(query): add comment in table meta
* feat: CSV format add option `binary_format` and `output_header`. * ci: add tests. * fix clippy. * fix tests.
* chore(query): add log prefix filter config * chore(query): add log prefix filter config
* refactor: Add KVPbApi::list_pb_values() returns values without keys * refactor: add KVPbApi::upsert_pb() to update or insert * refactor: Simplify UdfMgr::add_udf()
* improve hash join code * fix: from left single join to inner join * chore: improve left join * fix: right single to inner
…e) (#14637) * reduce small block writes, improve s3 latency * add random test again
* refacot: extract ensure_non_builtin() * chore: move KVPbApi impl to seperate files
…urning ErrorCode (#14640) This way the caller can distinguish between a logic error and a backend meta-service error.
…pprox_count_distinct` function (#14609) * refactor(query): support Additional param to set the error rate in approx_count_distinct function * update * update * update * fix
`UdfApi` has only one implementation `UdfMgr` and the abstraction layer is the underlying `KVApi`. There is no need to introduce another abstraction.
* refactor(query): disallowed thread::spawn * refactor(query): disallowed tokio::Handle::spawn * refactor(query): disallowed tokio::Handle::spawn_blocking * refactor(query): disallowed tokio block_on * refactor(query): disallowed tokio block_on * refactor(query): disallowed tokio block_on * refactor(query): location futrue * refactor(query): location futrue --------- Co-authored-by: Bohu <[email protected]>
refine plan id generation Co-authored-by: Winter Zhang <[email protected]>
* chore(executor): remove catalog ident name for profile * chore(meta): remove tenant for table info * chore(meta): remove tenant for table info * chore(meta): remove tenant for table info
* feat: add create or replace masking policy support * feat: add create or replace masking policy support
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )