Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Loading studies is slow #630

Open
IvoTod opened this issue May 17, 2024 · 0 comments
Open

[BUG] Loading studies is slow #630

IvoTod opened this issue May 17, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@IvoTod
Copy link
Collaborator

IvoTod commented May 17, 2024

What happened:
When loading studies in duckdb2, loading the family data is incredibly slow. The 3 slowest parts are the pandas.DataFrame.to_dict call, the FamiliesData redefine method and the Person redefine method, which gets called on every Person object created. Each of these 3 parts takes 15-20 seconds, adding up to an entire minute of load time.

What you expected to happen:
It should be possible to reduce the loading time drastically, so that starting the GPF system is much faster.

Attached are results from profiling, sorted by cumulative time. The second file is with the to_dict call replaced with manual iteration.

profiling1.txt
profiling2.txt

@IvoTod IvoTod added the bug Something isn't working label May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant