insertArrowTable does not create table?

[nshiab]

Hi!

I am trying to create a new table from an array of objects in an Observable Notebook.

The notebook is here: https://observablehq.com/d/9b67ff4f405dd5d8

And here's the relevant cell:

{
  const someData = [
    { key1: "a", key2: 1 },
    { key1: "b", key2: 2 }
  ];
  const arrowTable = arrow.tableFromJSON(someData);
  await c.insertArrowTable(arrowTable, {
    name: "someData"
  });
  return await c.query("select * from someData");
}

But I get this error:

Error: Catalog Error: Table with name someData does not exist!
Did you mean "temp.information_schema.schemata"?
LINE 1: select * from someData

What am I doing wrong? :'(

[carlopi]

Hi! I have to review wether this could be made simpler for users, but for now you can create the table explicitly like:

{
  const someData = [
    { key1: "a", key2: 1 },
    { key1: "b", key2: 2 }
  ];
  await c.query("CREATE TABLE someData(key1 VARCHAR, key2 integer)");              // <----- create table via SQL
  const arrowTable = arrow.tableFromJSON(someData);
  await c.insertArrowTable(arrowTable, {
    name: "someData"
  });
  return await c.query("select * from someData");
}

[nshiab]

Ahhhh! Understood. I thought that insertArrowTable would create the table. Thanks @carlopi !

[dioptre]

Hi guys, did you ever get this working? I've tried all manner of options and still don't get anything out of insertArrowTable #1640

I've tried multiple versions too, and if you have a version that worked for you would def appreciate letting us know what worked. Thanks

[jonathanswenson]

Edit: I managed to get it to work by matching the version of apache arrow that is used by duckdb-wasm.

in 1.28.1-dev106.0 that meant specifying "^14.0.1" (which seems to resolve to 14.0.2).
I also tried 1.28.1-dev132.0 and found that I needed to specify "15.0.0"

I do not need to add the additional EOS insert.

---

This is no longer working for me as well. We're currently using 1.28.0, but are trying more recent versions (1.28.1-dev106.0) in preparation for a release that is compatible with 0.10.0 and cannot load data as before.

In previous versions (namely 1.28.0 and older) there was no need to create the table manually, and the table would be created with insertArrowData but that behavior seems maybe to have changed.

const arrowResultData: Table<any> = ...
await connection.insertArrowTable(arrowResultData, {
  name: 'table_name',
})

Attempting to follow the new docs here: https://duckdb.org/docs/api/wasm/data_ingestion.html#apache-arrow and adding a final connection.insertArrowTable with EOS (see below) also does not work.

const arrowResultData: Table<any> = ...
const tableName = 'table_name',
await connection.insertArrowTable(arrowResultData, {
  name: tableName
})
await connection.insertArrowTable(EOS, { name: tableName })

The promises seem to resolve, but there don't appear to be any new tables added. So the behavior appears definitely to have changed here (at least in that dev version)

[dioptre]

Thanks for the update!