Thalassa latest version

[saeed-moghimi-noaa]

Hi @brey @pmav99

Would you please let me know the code location of the latest stable code that you are using? Folks in our side are getting interested to look into Thalassa.

Would it be possible to update the main at JRC repo? Does pip install grabs the final version?

Best,
-Saeed

[brey]

Hi @saeed-moghimi-noaa. Indeed we are a bit late with the latest update. We will clean up the repo and push the latest version by the end of next week.

[pmav99]

@saeed-moghimi-noaa I updated the master. I still need to update README etc. Nevertheless, the "thalassa" name on pypi is used by a different project. So you can't install with pip install. Not sure how we will resolve this yet.

Anyway, for now, I would suggest to (activate a virtualenv with python 3.8-10 and) use :

pip install "git+https://github.com/ec-jrc/Thalassa.git"

[saeed-moghimi-noaa]

Great! I will try. Please also see my note in the other issue on searvey+Thalassa example/HOWTO. Thanks

[saeed-moghimi-noaa]

Here is my report:

>mamba   create  -n Thalassa  python=3.10
> conda activate Thalassa
> pip install "git+https://github.com/ec-jrc/Thalassa.git"

and I got this:
`Collecting git+https://github.com/ec-jrc/Thalassa.git
Cloning https://github.com/ec-jrc/Thalassa.git to /tmp/pip-req-build-x37djnj8
Running command git clone --filter=blob:none --quiet https://github.com/ec-jrc/Thalassa.git /tmp/pip-req-build-x37djnj8
Resolved https://github.com/ec-jrc/Thalassa.git to commit 06e6ec8
Installing build dependencies ... error
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [7 lines of output]
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f706ef60cd0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/poetry-core/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f706ef2fbe0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/poetry-core/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f706ef2fdf0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/poetry-core/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f706ef61030>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/poetry-core/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f706ef611e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/poetry-core/
ERROR: Could not find a version that satisfies the requirement poetry-core>=1.0.0 (from versions: none)
ERROR: No matching distribution found for poetry-core>=1.0.0
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. `

Any idea what went wrong?

[pmav99]

Strange. I tried something similar before posting and it worked without errors. I tried the exact commands that you posted just now and it also works.

Can you retry with pip install -vvv "git+https://github.com/ec-jrc/Thalassa.git" and paste the output. Also, on what OS are you on?

[saeed-moghimi-noaa]

see here:
log.txt

[saeed-moghimi-noaa]

Hi @pmav99

I installed poetry:
mamba install poetry

Then tried again:
Here is what I get:

log.txt

[saeed-moghimi-noaa]

I am on Linux Subsystem for Windows.

[pmav99]

Can you please try to reinstall poetry with mamba install -c conda-forge poetry? mamba install poetry pulls the package from the anaconda/default repo, which is outdated (1.1.7 vs 1.4.0)

That being said, on my system I completely uninstalled poetry and it still worked without issues, but there might be some difference on WSL.

PS. Practically speaking, you should almost always be using conda-forge instead of the default conda channel. You can specify that globally in your ~/.condarc with

channel_priority: strict
channels:
  - conda-forge

[saeed-moghimi-noaa]

here is what I did:

 3170  conda env remove  -n Thalassa
 3171  mamba   create  -n Thalassa  python=3.9
 3172  conda activate Thalassa
 3173  mamba install -c conda-forge poetry
 3174  pip install -vvv  "git+https://github.com/ec-jrc/Thalassa.git"

Here is the log:
log2.txt

Cartopy is not getting installed.

[pmav99]

yeah, cartopy has additional binary dependencies. The specific error is: lib/cartopy/trace.cpp:767:10: fatal error: geos_c.h: No such file or directory so you also need to install geos, too:

mamba install -yq --channel conda-forge geos

[saeed-moghimi-noaa]

Seems working this time:
log3.txt

[pmav99]

Yeap. So this should be in the instructions:

mamba create \
    --channel conda-forge \
    --name Thalassa \
    python=3.10 \
    geos \
    poetry

[saeed-moghimi-noaa]

Hi @pmav99

For this adcirc files:

aws s3 cp  s3://noaa-gestofs-pds/stofs_2d_glo.20230316/  .  --no-sign-request   --recursive   --exclude "*"  --include "*fields*cwl.nc"

after renaming to test.nc

I am getting this:

Then I tried to investigate using ipynb file and I got this:

I am working on an EC2 instance on AWS on a linux VS.

Thanks,
-Saeed

[saeed-moghimi-noaa]

I also see this in log. may be file formats are not ok:

[pmav99]

@saeed-moghimi-noaa please ignore the notebooks for now, they have not been updated. I will have a look at this adcirc file. Chances are that its format is somewhat different compared to the one you provided in #57

[pmav99]

I am downloading the file but the bucket is 45GB so it will take some time

[pmav99]

@saeed-moghimi-noaa
I had a look. The ADCIRC file that is in the S3 bucket is somewhat different than the in #57. I updated the code to handle both of them. Now, when you put the file in the data directory you should see something like this in the logs:

12:21:53; DEBUG   ; thalassa.normalization             ; can_be_inferred     ; 115: Trying to open: ./data/stofs_2d_glo.t18z.fields.cwl.nc
12:21:53; DEBUG   ; thalassa.normalization             ; infer_format        ; 110: Inferred format: THALASSA_FORMATS.ADCIRC

and the UI should allow you to select the netcdf files.

That being said the file from the S3 bucket cannot be properly visualized... but for that let's continue on #58

To close this ticket, please try to visualize the netcdf file you uploaded in #57. That one should work without problems.

[saeed-moghimi-noaa]

@pmav99

Would you please share with me the notebook that you showed the other time to interact with data directly? I have issue to get even the original data sets to show up on my side. I am running the Thalassa on an EC2 instance.

Would be happy to share screen and show you what happens if you have time.

I am also looking into Kerchunk and dask to see if I can speed up reading files with out downloading them.

Thanks,
-Saeed

[pmav99]

@saeed-moghimi-noaa I assume we can close this now. If not feel free to re-open or open a new ticket.