Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trino error querying column of type 'array of structs' #721

Open
tooptoop4 opened this issue Jan 6, 2025 · 0 comments
Open

trino error querying column of type 'array of structs' #721

tooptoop4 opened this issue Jan 6, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@tooptoop4
Copy link

What language are you using?

python

What version are you using?

connectorx-0.4.1a1

What database are you using?

trino with hive catalog

Can you describe your bug?

import connectorx as cx
conn = 'trino+https://redact:redact@redact:443/hive'
cx.read_sql(conn, "select * from hive.redact.redact limit 2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/connectorx/__init__.py", line 367, in read_sql
    result = _read_sql(
RuntimeError: Trino unknown value at position: (0, 86): Array [Array [String("redact"), String("redact"), String("sstring")], Array [String("..........redacted

the table has some columns that are ARRAY of STRUCT types

it works when i just select simple columns like varchar, int

i guess this error is from

macro_rules! impl_produce_text {
($($t: ty,)+) => {
$(
impl<'r, 'a> Produce<'r, $t> for TrinoSourcePartitionParser<'a> {
type Error = TrinoSourceError;
#[throws(TrinoSourceError)]
fn produce(&'r mut self) -> $t {
let (ridx, cidx) = self.next_loc()?;
let value = &self.rows[ridx].value()[cidx];
match value {
Value::String(x) => {
x.parse().map_err(|_| anyhow!("Trino cannot parse String at position: ({}, {}): {:?}", ridx, cidx, value))?
}
_ => throw!(anyhow!("Trino unknown value at position: ({}, {}): {:?}", ridx, cidx, value))
}
}
}
impl<'r, 'a> Produce<'r, Option<$t>> for TrinoSourcePartitionParser<'a> {
type Error = TrinoSourceError;
#[throws(TrinoSourceError)]
fn produce(&'r mut self) -> Option<$t> {
let (ridx, cidx) = self.next_loc()?;
let value = &self.rows[ridx].value()[cidx];
match value {
Value::Null => None,
Value::String(x) => {
Some(x.parse().map_err(|_| anyhow!("Trino cannot parse String at position: ({}, {}): {:?}", ridx, cidx, value))?)
}
_ => throw!(anyhow!("Trino unknown value at position: ({}, {}): {:?}", ridx, cidx, value))
}
}
}
)+
};
}

@tooptoop4 tooptoop4 added the bug Something isn't working label Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant