CoreNrn: support for rebalanced files.dat #3147
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3147 +/- ##
==========================================
+ Coverage 67.06% 67.07% +0.01%
==========================================
Files 571 569 -2
Lines 111319 111172 -147
==========================================
- Hits 74658 74573 -85
+ Misses 36661 36599 -62 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
|
✔️ d0f0cfe -> Azure artifacts URL |
This comment has been minimized.
This comment has been minimized.
ferdonline
changed the title
ferdonline
changed the title
ferdonline
force-pushed
the
leite/balanced_files2
branch
from
c72d108
to
d0f0cfe
Compare
ferdonline
force-pushed
the
leite/balanced_files2
branch
from
1d5fab0
to
005577e
Compare
|
✔️ 005577e -> Azure artifacts URL |
This comment has been minimized.
This comment has been minimized.
|
@ferdonline : I am converting this to a draft assuming this is specific to our custom manipulation of |
|
|
✔️ 5a9d014 -> Azure artifacts URL |
|
@pramodk PR updated with a more informative comment |
| // A "-1" entry means that this rank should not be assigned further gid groups. | ||
| // It is a way to create files.dat files which deterministically assign gid groups to | ||
| // ranks, particularly useful for very large simulations which required load balancing. | ||
| if (iFile == -1) { |
There was a problem hiding this comment.
@nrnhines : this is a bit of BBP specific change where we rebalance the files.dat with some extra sauce (i.e. we modify files.dat from external python program). I can explain details (offline) but this doesn't change anything for normal neuron executuon.
Could you just confirm if files.dat could have an entry with -1 as a gid? I don't remember very well and hence want to double check with you.
For MMB (BBPP134-917) we need to be able to load a set of .dat files distributed in given way.
However for the moment CoreNeuron only reads it in plain RoundRobin, which is not suitable to load ranks a different number of files.
This PR
We introduce a mark value (-1 in this case) indicating no-data so that, even with round-robin, we can define precisely which rank will load a given data file. All ranks will be able to jump strides of size N_RANKS (at their respective offset) and find entries targeted at them. Upon seeing a
-1the rank knows it is done and can stop reading.This is a minor addition which enabled us to load balance and run the MMB simulation