-xsl flag does not work as intended #21

zdefne-usgs · 2022-01-21T05:48:54Z

I am trying to do this on my desktop with the -custom and -xsl the flags:

 java -Xms1024m -Xmx1024m -jar ncISO-2.3.5.jar -ts https://geoport.usgs.esipfed.org/thredds/sand/usgs/Projects/BBLEH/run071tRX/catalog.xml -num 1 -depth 1 -iso true -custom true -xslt UnidataDD2MI_dev.xsl

but I get this error:

ERROR [main] (DatasetTreeService.java:229) - _id=1
thredds.server.metadata.exception.ThreddsUtilitiesException: Configuration problem: xsl/null TransformerConfigurationException. D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO\xsl\null (The system cannot find the file specified)
        at thredds.server.metadata.util.ThreddsTranslatorUtil.doTransform(ThreddsTranslatorUtil.java:307)
        at thredds.server.metadata.util.ThreddsTranslatorUtil.transform(ThreddsTranslatorUtil.java:199)
        at gov.noaa.eds.service.DatasetTreeService.getNodes(DatasetTreeService.java:213)
...

ncISO is trying to locate the style sheet here:

D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO\xsl\null

I tried with placing the XSL in the same folder and with/without the folder name in the command line but no luck.

This works without the flags and the custom XSL. Also ncISO-2.3.5 throws this error but ncISO-2.3.7 completely ignores the flag and uses the default style sheet therefore no errors generated...

The text was updated successfully, but these errors were encountered:

zdefne-usgs · 2022-01-24T20:44:00Z

@noaaroland @karlmsmith is it possible to verify that the ncISO standalone is working as expected?

noaaroland · 2022-01-24T22:59:15Z

Yes, it works, but the instructions were wrong. I updated the README.md and verified the behavior using the latest jar file release (2.3.7).

zdefne-usgs · 2022-01-25T03:42:49Z

Thanks for checking @noaaroland but in our case with v.2.3.7 -xsl tag does not actually read the custom XSL.

It doesn't generate any error messages but also does not seem read the custom style sheet. I can point to an empty file or a bogus filename it doesn't matter. No errors generated, custom XSL not applied....

EDIT: it was v2.3.7 not v2.3. Corrected above.

noaaroland · 2022-01-25T14:20:00Z

What is stopping you from using the latest version?

noaaroland · 2022-01-25T14:36:44Z

Verify the files are in place...

(base) schweitzer@GREEN:~/nciso$ ls -l *
-rw-r--r-- 1 schweitzer schweitzer 43368880 Dec  7 21:53 ncISO-2.3.7.jar

xsl:
total 256
-rw-r--r-- 1 schweitzer schweitzer 143130 Jan 24 14:40 MYXSL.xsl

Try running with a non-existent file name...

(base) schweitzer@GREEN:~/nciso$ java -Xms1024m -Xmx1024m -jar ncISO-2.3.7.jar -ts https://ferret.pmel.noaa.gov/pmel/thredds/carbontracker.xml -num 1 -depth 20 -iso true -custom true -xsl FAKE_FILE.xml
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ROOT
 INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():Carbontracker CT2009 Fluxes; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 0
 INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
 INFO [main] (CatalogCrawlerImpl.java:99) - adding mdc
 INFO [main] (ThreddsDatasetTree.java:77) - returning: ROOT; url: /ROOT
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ferret.pmel.noaa.gov
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=pmel
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=thredds
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=dodsC
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ct_flux
ERROR:  'Could not compile stylesheet'
FATAL ERROR:  '/home/schweitzer/nciso/xsl/FAKE_FILE.xml (No such file or directory)'
           :/home/schweitzer/nciso/xsl/FAKE_FILE.xml (No such file or directory)

Look at the XSL to see what was changed from the default...

(base) schweitzer@GREEN:~/nciso$ more xsl/MYXSL.xsl
                    <gmd:title>
                          <xsl:copy>
                                  <junk>Hey, boss I added some junk.</junk>
                          </xsl:copy>
                      <xsl:call-template name="writeCharacterString">
                        <xsl:with-param name="testValue" select="$stdNameVocabCnt"/>
                        <xsl:with-param name="stringToWrite"
                          select="/nc:netcdf/nc:attribute[@name='standard_name_vocabulary']/@value"
                        />
                      </xsl:call-template>
                    </gmd:title>
                    <gmd:date gco:nilReason="unknown"/>
                  </gmd:CI_Citation>
                </gmd:thesaurusName>
              </gmd:MD_Keywords>
            </gmd:descriptiveKeywords>
          </xsl:if>
          <xsl:if test="$licenseCnt">
            <gmd:resourceConstraints>
              <gmd:MD_LegalConstraints>
                <gmd:useLimitation>
                  <gco:CharacterString>
                          ADDED SOME JUNK
                    <xsl:choose>
                      <xsl:when test="count(/nc:netcdf/nc:attribute[@name='license']) > 0">
                        <xsl:value-of select="/nc:netcdf/nc:attribute[@name='license']/@value"/>
                      </xsl:when>

Verify the _META.xml file contains the modifications per the new XLS...

(base) schweitzer@GREEN:~/nciso$ java -Xms1024m -Xmx1024m -jar ncISO-2.3.7.jar -ts https://ferret.pmel.noaa.gov/pmel/thredds/carbontracker.xml -num 1 -depth 20 -iso true -custom true -xsl MYXSL.xsl
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ROOT
 INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():Carbontracker CT2009 Fluxes; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 0
 INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
 INFO [main] (CatalogCrawlerImpl.java:99) - adding mdc
 INFO [main] (ThreddsDatasetTree.java:77) - returning: ROOT; url: /ROOT
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ferret.pmel.noaa.gov
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=pmel
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=thredds
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=dodsC
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ct_flux
(base) schweitzer@GREEN:~/nciso$ grep boss ferret.pmel.noaa.gov/pmel/thredds/dodsC/ct_flux_
ct_flux_ISO.xml      ct_flux_META.xml     ct_flux_NCML.xml     ct_flux_REPORT.html
(base) schweitzer@GREEN:~/nciso$ grep boss ferret.pmel.noaa.gov/pmel/thredds/dodsC/ct_flux_META.xml
<junk>Hey, boss I added some junk.</junk>
<junk>Hey, boss I added some junk.</junk>
<junk>Hey, boss I added some junk.</junk>
<junk>Hey, boss I added some junk.</junk>
(base) schweitzer@GREEN:~/nciso$

noaaroland · 2022-01-25T14:37:56Z

I reopened this until we get it sorted out, but I'm not going to investigate the behavior of other versions if I know the current release works.

zdefne-usgs · 2022-01-25T15:35:28Z

Thank you so much for looking into this @noaaroland. I was using the most recent version v2.3.7, corrected the typo in my previous post.

I do not get this part when I run the same command with you:

ERROR:  'Could not compile stylesheet'
FATAL ERROR:  '/home/schweitzer/nciso/xsl/FAKE_FILE.xml (No such file or directory)'
           :/home/schweitzer/nciso/xsl/FAKE_FILE.xml (No such file or directory)

Here's my screenshot:

PS D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO> java -Xms1024m -Xmx1024m -jar ncISO-2.3.7.jar -ts https://ferret.pmel.noaa.gov/pmel/thredds/carbontracker.xml -num 1 -depth 20 -iso true -custom true -xsl FAKE_FILE.xml
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ROOT
 INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():Carbontracker CT2009 Fluxes; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 0
 INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
 INFO [main] (CatalogCrawlerImpl.java:99) - adding mdc
 INFO [main] (ThreddsDatasetTree.java:77) - returning: ROOT; url: /ROOT
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ferret.pmel.noaa.gov
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=pmel
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=thredds
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=dodsC
 INFO [main] (ThreddsDatasetTree.java:25) - displayName=ct_flux

META.html is empty:

Could something (like a file or a path to a library) be hardcoded in the script? Would Java software or version matter?

noaaroland · 2022-01-25T16:37:02Z

Do you have a directory called 'xsl' in D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO?

zdefne-usgs · 2022-01-25T18:05:23Z

Yes

noaaroland · 2022-01-25T19:22:35Z

I have successfully run it under Linux with:

(base) schweitzer@GREEN:~/nciso$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.242-b08, mixed mode)

and Windows PowerShell with:

PS C:\Users\schweitzer\Documents\nciso> java -version
java version "1.8.0_311"
Java(TM) SE Runtime Environment (build 1.8.0_311-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.311-b11, mixed mode)

So if you don't have at least Java 8 installed that is the last suggestion I can offer, because I cannot reproduce the error.

zdefne-usgs · 2022-01-26T05:19:23Z

I am on Windows PowerShell with OpenJDK Temurin. Could that be the problem? I will look into it...

PS D:\Documents\DATA_RELEASE\ISOcompare\standalone_ISO> java -version
openjdk version "16.0.2" 2021-07-20
OpenJDK Runtime Environment Temurin-16.0.2+7 (build 16.0.2+7)
OpenJDK 64-Bit Server VM Temurin-16.0.2+7 (build 16.0.2+7, mixed mode, sharing)

zdefne-usgs · 2022-01-26T06:03:38Z

...and interesting! that seems to make a change!!!
¯\_(ツ)_/¯

noaaroland · 2022-01-26T15:56:19Z

Fixed in 2.3.8.

jcwarner-usgs · 2022-01-26T16:22:03Z

thanks for your help on this. I am working with zdefne at usgs. i am finding that the java version is critical.
If i use exactly the same xls sheet, it does not work on Linux but it does work on Windows.

**********> on our Linux system we have
[jwarner@poseidon-l1 nciso]$ java -version
openjdk version "1.8.0_312"
OpenJDK Runtime Environment (build 1.8.0_312-b07)
OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)

But i get an error
[jwarner@poseidon-l1 nciso]$ java -Xms1024m -Xmx1024m -jar ncISO-2.3.7.jar -custom true -xsl UnidataDD2MI_copyZ.xsl -ts https://geoport.usgs.esipfed.org/thredds/vortexfs1/usgs/Projects/breach/breach_matanzas/catalog.xml -num 1 -depth 20 -iso true
INFO [main] (ThreddsDatasetTree.java:25) - displayName=ROOT
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():breach_matanzas/00_dir_roms.ncml; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 0
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:99) - adding mdc
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():breach_matanzas/00_dir_roms_ISO.xml; dataset.hasAccess(): true maxLeaves: 20 leafcnt: 1
....
FATAL ERROR: 'Syntax error in '(/nc:netcdf/nc:attribute[@name='license']/@value, /nc:netcdf/nc:group[@name='THREDDSMetadata']/nc:group[@name='documentation']/nc:group[@name='document']/nc:attribute[@type='rights']/@value)'.'
Compiler warnings:
line 38: Illegal attribute 'as'.
...

**********> on my windows system i have
E:\data3\Projects\Matanzas\Christie_JGR_paper_metadata>java -version
openjdk version "1.8.0_322"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_322-b06)
OpenJDK 64-Bit Server VM (Temurin)(build 25.322-b06, mixed mode)
E:\data3\Projects\Matanzas\Christie_JGR_paper_metadata>java -Xms1024m -Xmx1024m -jar ncISO-2.3.7.jar -custom true -xsl Unidata2MI_copyZ.xsl -ts https://geoport.usgs.esipfed.org/thredds/vortexfs1/usgs/Projects/breach/breach_matanzas/catalog.xml -num 1 -dpeth 20 -iso true
INFO [main] (ThreddsDatasetTree.java:25) - displayName=ROOT
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():breach_matanzas/00_dir_roms.ncml; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 0
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():breach_matanzas/00_dir_roms_ISO.xml; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 1
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 0 dataset.getFullName():breach_matanzas/Build; dataset.hasAccess(): false maxLeaves: -1 leafcnt: 2
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/GTM_inwave_grd8.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 2
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/GTM_inwave_grd8_bry_R7.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 3
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/GTM_inwave_grd8_ini_R7.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 4
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/Output; dataset.hasAccess(): false maxLeaves: -1 leafcnt: 5
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/breach_inwave_grd8_ini.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 5
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/breach_matanzas.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 6
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/coamps_air_7318.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 7
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/coamps_rad_7318.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 8
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/coamps_wind_7318.nc; dataset.hasAccess(): true maxLeaves: -1 leafcnt: 9
INFO [main] (CatalogCrawlerImpl.java:82) - allowable service
INFO [main] (CatalogCrawlerImpl.java:78) - maxDepth: 1 depth: 1 dataset.getFullName():breach_matanzas/geoport.usgs.esipfed.org; dataset.hasAccess(): false maxLeaves: -1 leafcnt: 10

So i works on the WINDOWS but not on Linux.
they have different java.
-john

zdefne-usgs · 2022-01-28T17:21:15Z

Hi @noaaroland, Now I get errors that complain about "as=string" tags in the XSL. It looks like this is an issue with XSLT1.0 but not with XSLT2.0 (see here for more info). Was the XSLT version downgraded from 2.0 to 1.0 sometime in the past? I see a similar discussion here: Unidata/threddsIso#19 (comment)
When I replace the UnidataDD2MI.xsl on our Thredds server the customizations are applied correctly. How can I compare the standalone ncISO to what I have on the server (apple to apple comparison)? Thanks...

noaaroland · 2022-01-28T20:41:11Z

Yes, the XSLT processor did change. I'm trying to move away from old unsupported packages. I put up a new jar as a beta which uses Saxon which supports XSLT 2 and 3. Try it and let me know please.

zdefne-usgs · 2022-01-28T23:32:40Z

Thank you @noaaroland! This seems to work. It creates an _ISO.xml which is based on the original XSL, and a _META.xml which is based on the modified XSL (as expected). In the previous versions _META.xml always came out empty. I can now go back to throwing wrenches at it!!! 🐵🛠

What is the source of original ISO style sheet used to create the _ISO.xml by the way?
Thanks a lot again!

noaaroland · 2022-01-31T15:23:53Z

Thanks for letting me know. All of the XSL and the Java code originates from work done by @dneufeldcu and colleagues. I am just the caretaker. I made 2.3.9 official.

zdefne-usgs changed the title ~~-xsl flag does not work as intended #59~~ -xsl flag does not work as intended Jan 21, 2022

noaaroland closed this as completed Jan 24, 2022

noaaroland reopened this Jan 25, 2022

noaaroland closed this as completed Jan 26, 2022

zdefne-usgs mentioned this issue Jan 28, 2022

-xsl flag does not work as intended Unidata/threddsIso#59

Open

salme146 mentioned this issue Jun 8, 2022

README.md latest version is 2.3.9 zdefne-usgs/ocean-iso-metadata#19

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

-xsl flag does not work as intended #21

-xsl flag does not work as intended #21

zdefne-usgs commented Jan 21, 2022 •

edited

Loading

zdefne-usgs commented Jan 24, 2022

noaaroland commented Jan 24, 2022

zdefne-usgs commented Jan 25, 2022 •

edited

Loading

noaaroland commented Jan 25, 2022

noaaroland commented Jan 25, 2022 •

edited

Loading

noaaroland commented Jan 25, 2022

zdefne-usgs commented Jan 25, 2022

noaaroland commented Jan 25, 2022

zdefne-usgs commented Jan 25, 2022

noaaroland commented Jan 25, 2022

zdefne-usgs commented Jan 26, 2022

zdefne-usgs commented Jan 26, 2022 •

edited

Loading

noaaroland commented Jan 26, 2022

jcwarner-usgs commented Jan 26, 2022

zdefne-usgs commented Jan 28, 2022 •

edited

Loading

noaaroland commented Jan 28, 2022

zdefne-usgs commented Jan 28, 2022 •

edited

Loading

noaaroland commented Jan 31, 2022

-xsl flag does not work as intended #21

-xsl flag does not work as intended #21

Comments

zdefne-usgs commented Jan 21, 2022 • edited Loading

zdefne-usgs commented Jan 24, 2022

noaaroland commented Jan 24, 2022

zdefne-usgs commented Jan 25, 2022 • edited Loading

noaaroland commented Jan 25, 2022

noaaroland commented Jan 25, 2022 • edited Loading

noaaroland commented Jan 25, 2022

zdefne-usgs commented Jan 25, 2022

noaaroland commented Jan 25, 2022

zdefne-usgs commented Jan 25, 2022

noaaroland commented Jan 25, 2022

zdefne-usgs commented Jan 26, 2022

zdefne-usgs commented Jan 26, 2022 • edited Loading

noaaroland commented Jan 26, 2022

jcwarner-usgs commented Jan 26, 2022

zdefne-usgs commented Jan 28, 2022 • edited Loading

noaaroland commented Jan 28, 2022

zdefne-usgs commented Jan 28, 2022 • edited Loading

noaaroland commented Jan 31, 2022

zdefne-usgs commented Jan 21, 2022 •

edited

Loading

zdefne-usgs commented Jan 25, 2022 •

edited

Loading

noaaroland commented Jan 25, 2022 •

edited

Loading

zdefne-usgs commented Jan 26, 2022 •

edited

Loading

zdefne-usgs commented Jan 28, 2022 •

edited

Loading

zdefne-usgs commented Jan 28, 2022 •

edited

Loading