Add harmony-py as a dependency
#627
Conversation
|
I will automatically update this comment whenever this PR is modified
|
trey-stafford
changed the title
harmony-py as a dependencyharmony-py as a dependency and remove UAT-specific code
trey-stafford
changed the title
harmony-py as a dependency and remove UAT-specific codeharmony-py as a dependency
trey-stafford
force-pushed
the
use-harmony-py
branch
from
b49389a
to
8349a61
Compare
trey-stafford
force-pushed
the
use-harmony-py
branch
from
8349a61
to
77ad92c
Compare
icepyx/core/harmony.py
Outdated
| # TODO: This will work if the user has a .netrc file available but the other | ||
| # auth options might fail. We might need to add harmony client auth to the | ||
| # icepyx auth package. |
There was a problem hiding this comment.
icepyx.auth is a mixin for earthaccess, so it is easily made available wherever credentials are needed. I poked around harmony-py a bit and it looks like they don't allow you to just pass in a valid session, but I think earthaccess lets you just grab the username and password, so if we have the existing auth object we should be able to authenticate with harmony.Client that way.
trey-stafford
force-pushed
the
use-harmony-py
branch
from
6d9a660
to
192cee2
Compare
trey-stafford
force-pushed
the
use-harmony-py
branch
3 times, most recently
from
4d6430b
to
b5024dd
Compare
Subset orders can now be submitted and the job completes. Some next steps: * Support downloads from harmony * Support non-subset downloads. This might be a little more complicated, as it does not appear that the harmony API can support non-subset orders (it always uses spatial/temporal constraints to do subsetting, not just filtering). So we may need to use `earthaccess` to download our granule list instead. * Remove code supporting variable subsetting. This is not supported by Harmony, but could be in the future.
trey-stafford
force-pushed
the
use-harmony-py
branch
from
b5024dd
to
d80a4ce
Compare
The handling of various parameters is VERY confusing. What is necessary for CMR vs subsetting/ordering is not clear and conifg is mutated by various parts of the code, making it difficult to track down where values come from/get set/overridden
Makes it more clear that non-subset orders are not working right now.
There can be more than one job submitted by a user. Make it easy to track all of the submitted jobs by initializing to a empty list and then appending to it as necessary
trey-stafford
force-pushed
the
use-harmony-py
branch
from
bc4afc0
to
5ad82cf
Compare
| TODO/Question: it looks like this code will always use the provided | ||
| parameters to do subsetting. Are there cases where we just want the data | ||
| downloaded as whole granules? If so, we may need to use another API to | ||
| do so? |
There was a problem hiding this comment.
Interesting. The current icepyx default is to subset automatically for spatial and temporal (and then users could add any other available subsetting/reformatting options - it's probably best explained in the example notebooks for access and subsetting). In ipx.query, the order/download functions accept subset=False (would want to verify I have the kwarg right), so that would probably be the best way to trigger a non-subsetted download. I'm looking at transitioning that functionality over to earthaccess (with the ultimate goal of retiring ipx.Query entirely in favor of earthaccess), so we can work this case in there.
There was a problem hiding this comment.
Thanks for the feedback! I'm planning on using earthaccess to fill the gaps that harmony can't handle (non-subset orders).
LOTS more changes incoming :) Note that you don't need to keep up with this PR. I'm really just "going at it" and learning/breaking a lot as I go. My primary goal right now is assessing how much work it is going to be for us to implement harmony support, and we may need to revisit (or completely re-write) large chunks later.
…subset_order` separate concerns
Email is not an option with harmony
trey-stafford
force-pushed
the
use-harmony-py
branch
from
efbc411
to
6e6fa92
Compare
Separate out the subset and non-subset options
WIP to add harmony-py and remove references to UAT. IS2 collections are now enabled for harmony in prod - we just haven't started advertising this widely yet.