allow selection of (alternate) data from maps

[teixeirak]

We want to be able to load additional data maps, which will provide additional data source options to the user. In the short term, we'd like to get the maps listed below into the calculator. In the longer term, if possible, I'd like to have instructions on how to do this so that someone without web programming skills can load them. If that could already be accomplished, perhaps the best would be to teach @ValentineHerr how to do this, and she can help load the maps we have ready.

The basic idea is that, when the user goes to edit an ecosystem, they have the option of selecting an alternate data layer. For example, as was already implemented in the previous version, a user could go to edit the aboveground biomass attribute (under "initial storage of organic matter") and choose the Saatchi map from the drop-down menu. We would like to retain that setup with the Saatchi map, and add a similar setup with additional maps.

The following maps are ready to be loaded (in Box: gcs>maps to be added to calculator; listed by folder name):

to appear as options for aboveground biomass when selected vegtype = trees:
-IPCC 2006 biomass defaults for forests (@ValentineHerr , I don't see this map in box. Could you please add?)
-"NACP_AGB and C baseline..."
-"NACP-LiDAR AGB...."
-"US forest biomass"
-"Lidar AGB boreal Eurasia"

to appear as option for soil organic carbon (any ecosystem, following same rules as HWSD):
-"Soil organic carbon density"

All units must be converted to Mg/ha. I will provide units conversion.

[ValentineHerr]

The IPCC map is added in the box now. https://app.box.com/files/0/f/11067008935/IPCC

[dlebauer]

@potterzot What are the steps?

First, I assume we need to make the file and variable information available to the generic R function. In #58 (comment) it looks like there is an R list called data_list that specifies the data sources. For each data source, there are three elements, the directory name, the file name, and the variable name, for example:

    "saatchi_agb_num" = list(
      ncdir = "",
      ncfile = "saatchi.nc",
      variable = "agb_1km"

• where is this list stored?
• once it is edited, what else do we need to do to add a new data set?

[potterzot]

I think the below is about right:

Step 1
The list is defined in biome.R. At the end of the data_list list there
are several of the new map files. They are all stored in "GCS/Maps"
subdirectory, so easiest would be to copy something like the synmap entry
and change the map name and the variable name (the variable is the name of
the variable in the netcdf file).

Step 2
That is all you need to do to have the new map "added". The easiest option
from there would probably be to write the biomass options as a new key in
the biome json, probably best inserted on line 393 of biome.R, something
like:

biome_default$biomass_ipcc <- res$

Step 3
Then in the rails app you'd have to add the ui logic to be able to select
different options. I have no real sense of how that should work.

Nicholas Potter
http://potterzot.com
9C73 3AAA 6D99 86F1 F8F4 601D 298F 8713 1BE6 5941

On Fri, Aug 26, 2016 at 10:29 AM, David LeBauer [email protected]
wrote:

@potterzot https://github.com/potterzot What are the steps?

First, I assume we need to make the file and variable information
available to the generic R function. In #58 (comment)
#58 (comment)
it looks like there is an R list called data_list that specifies the data
sources. For each data source, there are three elements, the directory
name, the file name, and the variable name, for example:

"saatchi_agb_num" = list(
  ncdir = "",
  ncfile = "saatchi.nc",
  variable = "agb_1km"

• where is this list stored?
• once it is edited, what else do we need to do to add a new data set?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#86 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAdIbil9_jr-XiANjVzlVv4TaNDRpMaXks5qjyKVgaJpZM4Jr9On
.

[teixeirak]

I have added a file to Box giving the unit conversions for the maps listed above. https://uofi.box.com/s/y8fsl0obdtzb56wyksgvjin7sc282ice

[dlebauer]

@ValentineHerr what are the next steps? Is it just to download the files and add them to the server running the calculator?

@potterzot can you help with this?

[teixeirak]

@ValentineHerrhttps://github.com/ValentineHerr has loaded the files to the server running the calculator. She is out until next Monday, so I’ll try to help if you have any questions.

On Oct 17, 2016, at 3:01 PM, David LeBauer <[email protected]mailto:[email protected]> wrote:

@ValentineHerrhttps://github.com/ValentineHerr what are the next steps? Is it just to download the files and add them to the server running the calculator?

@potterzothttps://github.com/potterzot can you help with this?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com//issues/86#issuecomment-254300910, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGD7jlACquB_JrmSsleRia7BRGDJ76b4ks5q08X3gaJpZM4Jr9On.

[potterzot]

A few things on this:

• I can't find the SOC.nc file listed on line 293 of ghgvcR/biome.R
• I don't think I have edit permissions on box, but I wonder if we can rename the *.nc files to have a standard naming convention? The US Forest biomass file has spaces in it that cause problems, and the IPCC file has two underscores in a row.

[dlebauer]

@potterzot

• SOC.nc and other files are here https://uofi.box.com/s/uyfcy756jvhgs7qf6l84cufor9cvzi5a
• I updated your permissions to edit on Box

sorry for the many mails - I edited then combined all of the previous comments

[dlebauer]

@potterzot I just renamed the US Forest biomass file to remove spaces. Others look ok re spaces ...

It would be great but low priority to standardize the naming, might help (but be tedious) if all file names indicated source, variables and temporal / spatial domain. On the other hand, where the data are just raw dumps it may make sense to keep the original name just to make it clear that this is straight from the source ...

@ValentineHerr could you put a text README in the Box folder that describes the contents of each file? Much of this could probably be copy-pasted from Krista's spreadsheet.

[teixeirak]

How urgent is the README file? Is it required to make progress, or just for proper documentation?

[dlebauer]

For proper documentation ... not essential but would make development easier. A readme with a link to the spreadsheet if it exists woul suffice.

[teixeirak]

@ValentineHerr prepared a README file, which is now in Box. I've also updated the map units file.

@potterzot, do you need anything else to finish this issue?

[potterzot]

Reopening this because while the alternative maps are being read in ghgvcR, the user is not currently able to select any alternative maps AFAIK.

[potterzot]

Wanted to recap and get some feedback on this. I've now spent ~2.5 hours figuring out where things stand, what needs to happen, etc...

First, it would be easy enough to just add "Anderson-Teixeira and DeLucia (2011)" to the app for sources, which would at least fix the "0" issue, but would not allow dropdown source selection.

For implementing source selection:

• default biome values come from here based on the biome code that we get from processing logic that happens in biome.R. How do we want to assign sources for each of these values? If the source is the same across all rows or all columns we can add a "source" column or row. If not, we'll need some kind of corresponding matrix that, for a given biome code and variable name, provides the default source used.

• R reads in these values based on the biome code, and then makes adjustments according to "Overview of biomes mapping & assignment of default values.docx", then writes the final biome data to "tmp/run/biome.json", which is read in by the ruby app.

• If we can link the specific biome default values to a specific source, I can add that source to "tmp/run/biome.json", but that won't allow selection. An alternative would be to add all of the values and all of the sources. This isn't necessarily super tough, but as of right now I am not aware of any way of linking the biome_default values to a specific source.

[teixeirak]

It is not necessary to link the biome default values (i.e., those found here: https://github.com/ebimodeling/ghgvc/blob/master/public/data/maps/biome_defaults.csv) to the original source. This is too complex, and will be thoroughly documented in our current in-prep publication. Please cite this-- "Anderson-Teixeira et al. (in prep)"-- for all of the values default biome values matrix (https://github.com/ebimodeling/ghgvc/blob/master/public/data/maps/biome_defaults.csv) and for the values extracted from the biophysical maps (lines 55-57 in that matrix).

For soil carbon, the label should be "Harmonized World Soil Database", and the citation is:

Hiederer R, Köchy M (2011) Global soil organic carbon estimates and the harmonized world soil database, Vol. EUR 25225 EN (eds European Commission, Institute for Environment and Sustainability). Publications Office of the European Union, Luxembourg.

---

What we do want is to allow the user to select an alternate value when maps are available. Say the user goes to edit a forest site in the Amazon (T1). The default value should be the one from the default values matrix, so we want them to see "Anderson-Teixeira et al. (in prep)" with a value of 315. The drop-down menu to allow them to select the Saatchi biomass map, and if selected the displayed value should change to whatever the map gives for that location. And so forth with all the maps listed at the top of this issue.

[potterzot]

A few more thoughts that might be helpful:

The order of operations for the app is:

1. The user selects a lat/lon point. That point gets sent to R, which sends the available ecosystems back to Rails
2. The user clicks the edit box and has a list of map options for each value (i.e. a "Source"), or can input their own custom value.
3. The user clicks "run" and the parameters are sent to R.
4. The results are returned as a json string and a graph as a png string.

I believe the list of "Source" options should be returned from R in step 1 to Rails, and Rails should use those values to populate the the Source and value parameters. Then selecting the Source should change the parameter value.

Or if we want to allow selecting maps/sources globally then the edit box should allow a new set of selection boxes that are not parameter specific.

[teixeirak]

We will no longer attempt to have this ready for the publication.

I'd definitely like to get to it someday.