Support nested object columns

[buzzware]

Have you tried latest version of polars?

yes

What version of polars are you using?

0.16.0

What operating system are you using polars on?

MacOS 14.7.1

What node version are you using

18.19.1

Describe your bug.

Fails to read javascript object columns

What are the steps to reproduce the behavior?

using jest

import pl from 'nodejs-polars';

const eventData = [
	{id: 1, name: 'one', attributes: {x: 700, colour: 'black'}},
	{id: 2, name: 'two', attributes: {x: 800, colour: 'blue'}},
	{id: 3, name: 'three', attributes: {x: 100, colour: 'red'}}
]
const eventDf = pl.DataFrame(eventData);

test('get nested object column', async () => {
  expect(eventDf.getColumn('name').toArray()).toEqual(eventData.map(e=>e['name']));
  expect(eventDf.getColumn('attributes').toArray()).toMatchObject(eventData.map(e=>e['attributes']));
});

What is the actual behavior?

Error: expect(received).toMatchObject(expected)

- Expected  - 6
+ Received  + 3

  Array [
    Object {
-     "colour": "black",
-     "x": 700,
+     "": null,
    },
    Object {
-     "colour": "blue",
-     "x": 800,
+     "": null,
    },
    Object {
-     "colour": "red",
-     "x": 100,
+     "": null,
    },
  ]

What is the expected behavior?

polars has no trouble extracting the name column as an array, but fails on the attributes column, returning a strange array like [{"": null},{"": null},{"": null}]

[Bidek56]

This line is the issue, but not sure how to fix it ATM. Sorry.

[Bidek56]

@coastalwhite Would you be able to help here?
I was able to convert the nested Struct object to Fields, but infer_schema does not recognize the nested Struct object.
Thanks

let inner_val: napi::JsObject = unsafe { val.cast() };
let inner_keys = Object::keys(&inner_val).unwrap();
let mut fldvec: Vec<Field> = Vec::with_capacity(inner_keys.len() as usize);

inner_keys.iter().for_each(|key| {
    let inner_val = &inner_val.get::<_, napi::JsUnknown>(&key).unwrap();
    let dtype = match inner_val.as_ref().unwrap().get_type().unwrap() {
        ValueType::Boolean => DataType::Boolean,
        ValueType::Number => DataType::Float64,
        ValueType::BigInt => DataType::UInt64,
        ValueType::String => DataType::String,
        ValueType::Object => DataType::Struct(vec![]),
        _ => DataType::Null
    };

    let fld = Field::new(key.into(), dtype);
    fldvec.push(fld);
});