a map of indices

[shravanchipli]

Hello,

Have been wondering if there is a function available that directly maps the indices of the voltage vector V that is calculated by the Newton-Raphson algorithm in AC power flow to the indices of buses in the pandapower network?

The source code indicates that ppci is modified by many functions along the way before it reaches the pandapower network with power flow results. Creating such a map would require going through all those functions and understanding how they work, which seems to be tedious. Is there a better way?

Thanks

[ascheidl]

indeed there is,

have a look at "mapping the buses" in https://github.com/e2nIEE/pandapower/blob/develop/tutorials/internal_datastructure.ipynb

[shravanchipli]

Thanks for the link. However, in my grid model, the internal indices for certain buses are higher than the order of Ybus (or V). Am I missing something here? Wouldn't it be very convenient to have a direct map between the internal indices and the pandapower indices for all buses? Like a dictionary which directly spits out the internal index given the pandapower index?

The screenshot below shows the problem. This is by following the code from the link.

[rbolgaryn]

I do not understand why in the tutorial we are taking the index for the Ybus matrix again from the pd2ppc_lookups.

For the Ybus matrix, there is a second different mapping necessary from ppc indices.

ref = net._ppc['internal']['ref']
pvpq = net._ppc['internal']['pvpq']
refpvpq = np.r_[ref, pvpq]
pvpq_lookup[refpvpq] = np.arange(len(refpvpq))

pvpq_lookup is the mapping you use to get from the ppc_index to the internal Ybus index.

I agree that this is very confusing and that an explicit dictionary of some sort must be implemented instead of such mapping arrays, for the Ybus matrix but also for the J matrix.
You can make a suggestion how such a dictionary would look from a design perspective.

Best regards, Roman

[shravanchipli]

Clearly, there is confusion in mapping indices, which has also been reflected in a tutorial. For a real-world grid model, a small mistake in creating these maps could lead to unrealistic results. I suggest that we create the following maps so that the user doesn't have to go through the internal layers (ppc and ppci) all by oneself.

1. pd to internal - A dictionary with pandapower bus indices as keys and corresponding internal indices (of V or Sbus) as values.
2. internal to pd - A dictionary with internal indices as keys and tuples of pandapower bus indices as values. The tuples would also have the redundant buses in them (the ones that are connected by a closed zero-impedance switch for example).

These two dictionaries form maps between the two end layers involved in pandapower load flow calculations and can be used not just by the user but also inside pandapower if required. Furthermore, as @rbolgaryn also says, it would be very convenient to have two more similar dictionaries that map the pandapower indices to the indices of the Jacobian and vice versa.

One could argue that a cleaner way of doing this would be to just have lists instead of dictionaries but that would lead to confusion again considering that the pandapower bus indices need not be from 0 to (N-1).

[lthurner]

There used to be such a dictionary, but the reason why it was changed to an array is performance. For nets with a large number of buses, the dictionary solutions significantly slowed down the loadflow, because we aloways needed to iterate over every bus in a python loop. So we changed it to an array, which allows to vectorize the lookup and is therefore much, much faster.

We could implement a helper function that creates such a mapping dictionary for "external" usage. But for the loadflow, we should stick with the array solution to keep the loadflow lightning fast :)

[shravanchipli]

Very true. In that case, how about functions that only involve arrays instead of dictionaries? The only concern is that it is very confusing for users to trace the changes made to the internal matrices or vectors back to the pandapower net and vice versa with having to go through all the internal layers. Even for external usage, the maps will then be faster. In short, we need an explicit and efficient (ideally vectorized) lookup between the two end layers of the library.

[Joost-van-Dijk]

Hello,

You might also use a class for this: in the init phase it saves net, ppc, ppci, used for the conversions.

Say an object of the class is called conv. Suppose also that we have (say) a numpy array x_ppci with the rows indexed the same as the ppci bus matrix. Then you would like to be able to just do
• x_ppc = conv.ppc2ppci(x_ppci, row='bus'), creating zeros for the dropped buses.
Or something like
• x_bus = conv.ppc2net(x_ppc, row='bus', to='bus', include_lumped=True), getting the net.bus values of x_ppc, or
• x_trafo3w = conv.ppc2net(x_ppci, row='bus', to='trafo3w'), getting the net.trafo3w values of x_ppc.
• etcetera

As shravanchiply says, the advantage is that you can use the speed of only using arrays, but that the logic behind it always stays hidden.

[lthurner]

I like this approach. So there would be a conversion object that holds the different ppc structures and has methods for mapping the indices. Instead of passing the ppc through different functions during the power flow we would always pass the conv object, and functions could get what they need through certain methods from the conv object. Instead of saving the Ybus matrix and other internal matrices, we would just attach the conv object to the net, which contains all internal parameters and mappings.

That might work, but would need a refactor of almost all internal functions. We have a good test suite, so stability should still be guaranteed, but I suppose it is a significant effort.

[lthurner]

Another idea is to look at the system used in pandapipes. pandapipes and pandapower should use the same implementation since they are equivalent tools, but for historic and technical reasons, pandapipes uses a different system. We had some discussions with the pandapipes team about their approach with component models, but I don't remember exactly the details. Maybe its worth having a look there and see if this would also solve the problem we are having here.

[shravanchipli]

This is great @Joost-van-Dijk. Using a class makes it much cleaner and efficient. The maps would also be easy to understand and trace as all the conversions can happen within one object.

@lthurner If we can also use this for powerflow, that would be cool. All maps and case files in one bundle!

How do we execute this?