Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ColumnReader.loadVector should initiate CometDictionary after re-import arrays #474

Closed
viirya opened this issue May 25, 2024 · 0 comments · Fixed by #473
Closed

ColumnReader.loadVector should initiate CometDictionary after re-import arrays #474

viirya opened this issue May 25, 2024 · 0 comments · Fixed by #473
Labels
bug Something isn't working

Comments

@viirya
Copy link
Member

viirya commented May 25, 2024

Describe the bug

During debugging some test failures in #437, I found that CometScanExec returns empty dictionary values after first batch in failed queries. It is because in Data.importVector API call, it will release existing dictionary vector in the given dictionaryProvider and re-import the dictionary vector.

Currently we only initiate CometDictionary at the first batch and reuse it for later batches. Because of above issue, we should already initiate CometDictionary. It is simply re-importing existing dictionary vector allocated by native side, so it doesn't incur re-allocation of memory.

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant