Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: Load iris and types data in sql tests as needed #55265

Merged
merged 22 commits into from
Oct 25, 2023

Conversation

mroeschke
Copy link
Member

Also factors out timezone data in a fixture to only apply to the single test where it's used

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite IO SQL to_sql, read_sql, read_sql_query labels Sep 24, 2023
},
]
stmt = insert(datetz).values(datetz_data)
if isinstance(conn, Engine):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need the isinstance here - both Engine and Connectable has a begin contextmanager which does the right thing

https://docs.sqlalchemy.org/en/20/core/connections.html#connect-and-begin-once-from-the-engine

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh cool! I didn't know this; I'll give it a go

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we can't fully generalize these two branches due to use of execute, so will just keep the old invocation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm strange. I think before when you had

with conn.begin():
    ...
    conn.execute()

if that instead was

with conn.begin() as con:
    ...
    con.execute()

it would work

@WillAyd
Copy link
Member

WillAyd commented Oct 23, 2023

This generally looks good - thanks for cleaning these up

@@ -245,6 +243,43 @@ def create_and_load_types(conn, types_data: list[dict], dialect: str):
conn.execute(stmt)


def create_and_load_postgres_datetz(conn):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing I've been kind of careful in while setting up the ADBC connectors is an unecessary reliance on SQLAlchemy, since it would be nice to run the test suite without that dependency when not required.

Not a blocker for this PR just a larger consideration; with the ADBC change in particular I think I'd fall back to writing postgres-specific SQL for this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I can do another pass after this PR to try to refactor places where data is loaded to just use raw SQL

@mroeschke mroeschke added this to the 2.2 milestone Oct 25, 2023
@mroeschke
Copy link
Member Author

Thanks for the review @WillAyd. Going to merge this in and look at the data loading cleanups in a follow up PR

@mroeschke mroeschke merged commit 074ab2f into pandas-dev:main Oct 25, 2023
32 of 33 checks passed
@mroeschke mroeschke deleted the tst/ref/test_sql branch October 25, 2023 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO SQL to_sql, read_sql, read_sql_query Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants