You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened:
When loading studies in duckdb2, loading the family data is incredibly slow. The 3 slowest parts are the pandas.DataFrame.to_dict call, the FamiliesDataredefine method and the Personredefine method, which gets called on every Person object created. Each of these 3 parts takes 15-20 seconds, adding up to an entire minute of load time.
What you expected to happen:
It should be possible to reduce the loading time drastically, so that starting the GPF system is much faster.
Attached are results from profiling, sorted by cumulative time. The second file is with the to_dict call replaced with manual iteration.
What happened:
When loading studies in duckdb2, loading the family data is incredibly slow. The 3 slowest parts are the
pandas.DataFrame.to_dict
call, theFamiliesData
redefine
method and thePerson
redefine
method, which gets called on everyPerson
object created. Each of these 3 parts takes 15-20 seconds, adding up to an entire minute of load time.What you expected to happen:
It should be possible to reduce the loading time drastically, so that starting the GPF system is much faster.
Attached are results from profiling, sorted by cumulative time. The second file is with the
to_dict
call replaced with manual iteration.profiling1.txt
profiling2.txt
The text was updated successfully, but these errors were encountered: