Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Probe Analyzer function/S-matrix reusability to PyPrismatic #72

Open
lerandc opened this issue Feb 12, 2020 · 3 comments
Open

Add Probe Analyzer function/S-matrix reusability to PyPrismatic #72

lerandc opened this issue Feb 12, 2020 · 3 comments

Comments

@lerandc
Copy link
Collaborator

lerandc commented Feb 12, 2020

As discussed on the gitter chat by @erh3cq, it would be useful to run the probe analyzer and to reuse the S-matrix calculation in PyPrismatic. Implementation could be similar to the Qt slots used on the GUI as mentioned by @apryor6

@thomasaarholt
Copy link
Contributor

Has anyone taken a look at this yet? It would be nice to have implemented, even if it just means saving png's of the approach taken by the prismatic-gui!

@lerandc
Copy link
Collaborator Author

lerandc commented Aug 27, 2020

so I've been recently looking into some options for the implementation of this (and generalized modularity of pyprismatic, where we could ideally run the subroutines of prismatic freely through python). one of the challenges is minimizing memory costs when we transfer data from the python interface to the c++ compiled backend. we could just copy the data over manually from the numpy memory addresses and have a lot of data passing and call it a day, this wouldn't be that tricky, though it would require the re-initialization of the parameter object in both python and the backend libraries every single time an action is taken.

instead, ideally, we could use the same physical memory for everything and never actually move data around. for example, to my understanding the numpy libraries operate directly on C-arrays, and retrieve pointers to the numpy array data to do so. in Prismatic, however, we have an array class abstraction (defined in ArrayND.h) which wraps around C-style arrays and provides an interface. all of the constructors of the class create copies of data currently, like so:

test code:

    std::vector<PRISMATIC_FLOAT_PRECISION> data({0,1,2,3,4,5,6,7});
    std::array<size_t,2> dims = {2,4};
    ArrayND<2, std::vector<PRISMATIC_FLOAT_PRECISION>> testArray(data, dims);

    std::cout << &data[0] << std::endl;
    std::cout << &testArray[0] << std::endl;

sample output (memory addresses for the start of arrays):

0x556eb330d470
0x556eb3303730

the best, sustainable way to accomplish S-matrix (generally, all data) reusabiltiy and things like the probe analyzer would be to do the following:

  1. define a new constructor for the ArrayND class which accepts pointers
  2. define a parameter object/class with all the same fields as the parameter object in the backend, and setters/getters that control the rank and data type of the arrays (i.e., forcing rank 4 and single float)

if done right, this should be safe, but of course, would require a bit of testing- especially when working off of HPC systems over several nodes.

for small sims, the data copying isn't necesarilly bad-- of course it would be much slower than optimal-- but for large sims, you would quickly run into speed and memory issues if we just copy the arrays.

@ericpre
Copy link
Contributor

ericpre commented Aug 28, 2020

Considering that this kind of things would be useful mostly for case where the S-matrix is large, copying to numpy array is not very attractive (worth it?) even if this is the easiest route. Having a python wrapper to the ArrayND object would be definitely better!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants