Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e-Callisto NG - Rest API for e-Callisto data #98

Open
kenfus opened this issue Jul 17, 2023 · 3 comments
Open

e-Callisto NG - Rest API for e-Callisto data #98

kenfus opened this issue Jul 17, 2023 · 3 comments

Comments

@kenfus
Copy link

kenfus commented Jul 17, 2023

Provide a general description of the issue or problem.

Hello, I have developed a Rest API for E-Callisto and wrapped it as a python package:

https://pypi.org/project/ecallisto-ng/

it also contains some basic preprocessing and plotting. However, I would rather develop Radiospectra instead of inventing everything twice.

There are still some missing features, such as:

  • no metadata from the fits file is in the database (what is exactly needed?).
  • better error handling, for requests which return no data.
  • error handling for crazy request (e.g. give me three years of full resolution data).
  • instruments have different capitalization because of SQL limitations.

I am unsure on how to wrap this into Radiospectra, especially with the metadata. Any support would be of great help!

@samaloney
Copy link
Contributor

Hi @kenfus that's really nice work, would be great to integrate as much as possible and I am keen to do so.

I think it should be possible to write a Fido client to query the API much like the current clients work and then transform the data and store the Spectrogram container. We already have some issues open to add in some features that were lost in recent rewrite but I haven't had the time to get around to them yet (e.g. background subtraction).

For the metadata try to pass the entire fits header to the Spectrogram container but currently only rely on having the instrument/detector, time info, frequency info and the data array.

One area that is a bit unclear is that so far radiospecta (and sunpy) has mainly file based for various reasons (reproducibly, long term stability, lack of alternatives, ...). The benefits of an API powered approach are clear but the file based approach also has it's place it so question of how both can be supported in a sensible way.

We could discuss this at the sunpy community meeting held on Wednesday's at 16:00 UTC or if that doesn't work we could arrange another meeting time? Might be easier to discuss over a call than on here.

I'd also be really interested in the backend code, looks like FastAPI from what I could see? In particular if the API could return a list of urls of files which match the query that would be really useful.

@kenfus
Copy link
Author

kenfus commented Jul 18, 2023

Yes, Wednesday at 16:00 UTC works! See you tomorrow, then.

Thank you for your suggestions and feedback.

@samaloney
Copy link
Contributor

The link for the meeting is http://sunpy.org/jitsi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants