Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting partial Schema to find_arrow_all and find_pandas_all #243

Open
frbelotto opened this issue Oct 14, 2024 · 3 comments
Open

Setting partial Schema to find_arrow_all and find_pandas_all #243

frbelotto opened this issue Oct 14, 2024 · 3 comments
Labels
enhancement New feature or request linked-to-jira

Comments

@frbelotto
Copy link

Hello guys,
I would like to discuss about setting the Schema for find_arrow_all or find_pandas_all.
I have a database with several columns, two of them are ObjetctIds that are crashing my code (I´ve reported here ), so, I am trying to import all my table columns but just setting such columns to be imported as strings

schema = Schema({'_id': pa.string(), 'referenciaConversao': pa.string()})
pd_confirmacao_conversao = find_pandas_all(pd_confirmacao_conversao, {'estadoContabilizacaoEvento': {'$lt': 100}}, schema=schema)

My issue here is that, as I only set the schema for those two columns, only those columns are being imported from the dataset! Is there any way to improve it?

@aclark4life
Copy link
Contributor

Hi @frbelotto Is this related to #242 or a separate question?

@frbelotto
Copy link
Author

Hi @aclark4life
There I am reporting a issue/bug that I am facing.
Here I propose an improvement/discussion about partial schema definition (that, in my case, is a workaround I´ve found for my issue!)

@keanamo keanamo changed the title Setting parcial Schema to find_arrow_all and find_pandas_all Setting partial Schema to find_arrow_all and find_pandas_all Oct 18, 2024
@aclark4life
Copy link
Contributor

@frbelotto Ah! OK thanks, we'll track both in INTPYTHON-256

@aclark4life aclark4life added the enhancement New feature or request label Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request linked-to-jira
Projects
None yet
Development

No branches or pull requests

3 participants