r.report: add JSON support #3935
Conversation
github-actions
bot
added
raster
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
|
Considering the following: {
"mask": "none",
"description": "no data"
}We should not output special strings for "null" values. Python uses >>> import json
>>> json.dumps({"max": None})
'{"max": null}' |
|
@wenzeslaus I agree that we should not output special strings for "null" values. |
|
Hi @wenzeslaus and @cwhite911, I have made the requested changes but the code segfaults when Thread 1 "r.report" received signal SIGSEGV, Segmentation fault.
Rast_quant_get_ith_rule (q=0x555555594808, i=-2147483648, dLow=0x7fffffffd100, dHigh=0x7fffffffd108, cLow=0x7fffffffd0d4, cHigh=0x7fffffffd0d4) at quant.c:330
330 *dLow = q->table[i].dLow;
(gdb) bt
#0 Rast_quant_get_ith_rule (q=0x555555594808, i=-2147483648, dLow=0x7fffffffd100, dHigh=0x7fffffffd108, cLow=0x7fffffffd0d4, cHigh=0x7fffffffd0d4) at quant.c:330
#1 0x00007ffff7f9041e in Rast_get_ith_d_cat (pcats=0x5555555947e0, i=-2147483648, rast1=0x7fffffffd100, rast2=0x7fffffffd108) at cats.c:1039
#2 0x000055555555a30c in make_category (ns=211, nl=0, sub_categories=0x0) at prt_json.c:75
#3 0x000055555555a68a in make_categories (start=0, end=212, level=0) at prt_json.c:110
#4 0x000055555555adfb in print_json () at prt_json.c:215
#5 0x000055555555c9bd in report () at report.c:9
#6 0x0000555555558b1e in main (argc=4, argv=0x7fffffffd598) at main.c:77But I am not sure how to fix this issue. Can you please help with it? |
Next time, please, provide the command you used to ease reproduction of the issue. I ran the code but it did not crash. Looking at the gdb bt output, value of |
|
Hi @marisn! Sorry I forgot to mention the command. It was |
There was a problem hiding this comment.
@kritibirda26 Nice job on the revision. We still have a couple of items to clean up, but it's almost there.
|
The tests aren't passing, I think because it assumes a custom dataset to be present. Is there a reason the dataset (location) is different from the other tests? |
|
@echoix I have no idea. The test passes locally for me. I'll try to debug and fix it. |
|
@cwhite911 and @echoix the tests are now fixed. |
|
If you would've downloaded the same dataset as the one available in CI, downloaded in https://github.com/OSGeo/grass/blob/d7b13e0a63612cb98d3205eacde3acc922954533/.github/workflows/test_thorough.sh, I don't think that your last two commits were needed (maybe in part). (The name is nc_spm_full_v2alpha2) The thing is, it's not true that the grass makes the fields vary as mentioned in the test comment, it only depends on the data used. Well knowing what data was sent as a test input is kinda important to have a well defined test. After the last two commits to the test, if I understand correctly, it doesn't check if the location field is correctly filled out. (Only the presence) And with the test with extra options, is there something related to the created field that should be done (I'm not sure it is touched just by reading). I know our test infrastructure doesn't handle mocking date/time info properly to have a fully deterministic test, so the approach of "golden master" isn't possible here. Probably checking existence, and the format of it should be used. |
|
I'll download the CI dataset locally and update the test to check both of those fields. As for dates, yes that was an oversight. Can you please let me know if an ISO date formatter is included in grass, otherwise I'll write up the utility manually? |
|
I'm not the best person to answer on what the C-based code contains. Lately I'm getting better at knowing about the Python files in the repo, but even with over 391 000 lines of Python code, it's only 31% of the repo. So even if there were, I wouldn't know about it. |
Might this code help? Ah, you wanted C code: so probably |
|
I would expect that on Linux at least, it would be a single format code, as it's so standard. If the datetimes are only in UTC, it's even easier. Something should exist in a standard library. |
|
I saw that the new tests pass now! You are getting close! |
Co-authored-by: Edouard Choinière <[email protected]>
Co-authored-by: Nicklas Larsson <[email protected]>
* r.report: add json support * address PR feedback * fix segfault * address PR feedback * update tests to use CI dataset * change created field to iso8601 datetime format * fix created field test * fix map description field * Apply suggestions from code review Co-authored-by: Edouard Choinière <[email protected]> * update documentation * Update raster/r.report/format.c Co-authored-by: Nicklas Larsson <[email protected]> --------- Co-authored-by: Edouard Choinière <[email protected]> Co-authored-by: Nicklas Larsson <[email protected]>
* r.report: add json support * address PR feedback * fix segfault * address PR feedback * update tests to use CI dataset * change created field to iso8601 datetime format * fix created field test * fix map description field * Apply suggestions from code review Co-authored-by: Edouard Choinière <[email protected]> * update documentation * Update raster/r.report/format.c Co-authored-by: Nicklas Larsson <[email protected]> --------- Co-authored-by: Edouard Choinière <[email protected]> Co-authored-by: Nicklas Larsson <[email protected]>
Using parson, add JSON output support to r.report module.
Sample JSON output is according to the discussion in #3033, example:
{ "location": "nc_spm_08_grass7", "created": "Fri Jun 28 18:41:29 2024", "region": { "north": 228500, "south": 215000, "east": 645000, "west": 630000, "ew_res": 10, "ns_res": 10 }, "mask": "none", "maps": [ { "name": "zipcodes@PERMANENT", "description": "PERMANENT", "layer": "zipcodes@PERMANENT", "type": "raster" }, { "name": "landclass96@PERMANENT", "description": "PERMANENT", "layer": "landclass96@PERMANENT", "type": "raster" } ], "categories": [ { "category": 27511, "description": "CARY", "units": [ { "unit": "hectares", "value": 1058 }, { "unit": "% cover", "value": 5.2246913580246916 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 197.94 }, { "unit": "% cover", "value": 18.708884688090738 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 24.440000000000001 }, { "unit": "% cover", "value": 2.3100189035916823 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 58.080000000000005 }, { "unit": "% cover", "value": 5.4896030245746692 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 775.91000000000008 }, { "unit": "% cover", "value": 73.337429111531193 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 1.54 }, { "unit": "% cover", "value": 0.14555765595463138 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 0.090000000000000011 }, { "unit": "% cover", "value": 0.0085066162570888466 } ] } ] }, { "category": 27513, "description": "CARY", "units": [ { "unit": "hectares", "value": 205.30000000000001 }, { "unit": "% cover", "value": 1.0138271604938272 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 32.579999999999998 }, { "unit": "% cover", "value": 15.869459327812956 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 5.9300000000000006 }, { "unit": "% cover", "value": 2.8884559181685336 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 52.030000000000001 }, { "unit": "% cover", "value": 25.343399902581588 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 114.76000000000001 }, { "unit": "% cover", "value": 55.898684851436919 } ] } ] }, { "category": 27518, "description": "CARY", "units": [ { "unit": "hectares", "value": 1345.1900000000001 }, { "unit": "% cover", "value": 6.6429135802469137 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 106.46000000000001 }, { "unit": "% cover", "value": 7.9141236553944054 } ] }, { "category": 2, "description": "agriculture", "units": [ { "unit": "hectares", "value": 7.5 }, { "unit": "% cover", "value": 0.55754205725585237 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 84.620000000000005 }, { "unit": "% cover", "value": 6.2905611846653633 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 179.63 }, { "unit": "% cover", "value": 13.353503965982501 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 919.35000000000002 }, { "unit": "% cover", "value": 68.343505378422378 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 47.630000000000003 }, { "unit": "% cover", "value": 3.5407637582794997 } ] } ] }, { "category": 27529, "description": "GARNER", "units": [ { "unit": "hectares", "value": 1415.72 }, { "unit": "% cover", "value": 6.99120987654321 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 614.97000000000003 }, { "unit": "% cover", "value": 43.438674314129912 } ] }, { "category": 2, "description": "agriculture", "units": [ { "unit": "hectares", "value": 7.25 }, { "unit": "% cover", "value": 0.51210691379651341 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 35.960000000000001 }, { "unit": "% cover", "value": 2.5400502924307067 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 12.890000000000001 }, { "unit": "% cover", "value": 0.91049077501200804 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 719.13 }, { "unit": "% cover", "value": 50.796061368067129 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 4.1699999999999999 }, { "unit": "% cover", "value": 0.29454976972847741 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 21.350000000000001 }, { "unit": "% cover", "value": 1.50806656683525 } ] } ] }, { "category": 27539, "description": "APEX", "units": [ { "unit": "hectares", "value": 254.44000000000003 }, { "unit": "% cover", "value": 1.2564938271604937 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 2.46 }, { "unit": "% cover", "value": 0.96682911491903789 } ] }, { "category": 2, "description": "agriculture", "units": [ { "unit": "hectares", "value": 28.720000000000002 }, { "unit": "% cover", "value": 11.28753340669706 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 4.04 }, { "unit": "% cover", "value": 1.5878006602735419 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 18.560000000000002 }, { "unit": "% cover", "value": 7.294450558088351 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 192.37 }, { "unit": "% cover", "value": 75.605250746737937 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 0.87 }, { "unit": "% cover", "value": 0.34192736991039147 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 7.4199999999999999 }, { "unit": "% cover", "value": 2.9162081433736833 } ] } ] }, { "category": 27601, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 454.68000000000001 }, { "unit": "% cover", "value": 2.2453333333333334 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 436.65000000000003 }, { "unit": "% cover", "value": 96.034573766165209 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 11.610000000000001 }, { "unit": "% cover", "value": 2.5534441805225652 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 6.4199999999999999 }, { "unit": "% cover", "value": 1.4119820533122196 } ] } ] }, { "category": 27603, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 4291.79 }, { "unit": "% cover", "value": 21.194024691358024 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 1183.24 }, { "unit": "% cover", "value": 27.569848478140823 } ] }, { "category": 2, "description": "agriculture", "units": [ { "unit": "hectares", "value": 14.770000000000001 }, { "unit": "% cover", "value": 0.34414544980066591 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 808.62 }, { "unit": "% cover", "value": 18.841089615288727 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 216.73000000000002 }, { "unit": "% cover", "value": 5.0498742948746331 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 2036.2900000000002 }, { "unit": "% cover", "value": 47.446170478984293 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 16.100000000000001 }, { "unit": "% cover", "value": 0.37513485049361689 } ] }, { "category": 7, "description": "sediment", "units": [ { "unit": "hectares", "value": 10.66 }, { "unit": "% cover", "value": 0.24838121156906559 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 5.3799999999999999 }, { "unit": "% cover", "value": 0.12535562084817756 } ] } ] }, { "category": 27604, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 473.89000000000004 }, { "unit": "% cover", "value": 2.3401975308641974 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 338.04000000000002 }, { "unit": "% cover", "value": 71.333009770199837 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 49.710000000000001 }, { "unit": "% cover", "value": 10.489776108379582 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 83.77000000000001 }, { "unit": "% cover", "value": 17.677098060731392 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 2.3700000000000001 }, { "unit": "% cover", "value": 0.50011606068918946 } ] } ] }, { "category": 27605, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 236.77000000000001 }, { "unit": "% cover", "value": 1.1692345679012346 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 187.45000000000002 }, { "unit": "% cover", "value": 79.169658318199097 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 5.71 }, { "unit": "% cover", "value": 2.4116230941419943 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 6.6000000000000005 }, { "unit": "% cover", "value": 2.7875153102166661 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 37.010000000000005 }, { "unit": "% cover", "value": 15.631203277442243 } ] } ] }, { "category": 27606, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 6626.4200000000001 }, { "unit": "% cover", "value": 32.723061728395059 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 1030.6200000000001 }, { "unit": "% cover", "value": 15.553194636017638 } ] }, { "category": 2, "description": "agriculture", "units": [ { "unit": "hectares", "value": 115.23 }, { "unit": "% cover", "value": 1.7389480292525978 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 631.49000000000001 }, { "unit": "% cover", "value": 9.5298818970122632 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 682.66000000000008 }, { "unit": "% cover", "value": 10.302093739907823 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 3807.6200000000003 }, { "unit": "% cover", "value": 57.461193223490213 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 349.17000000000002 }, { "unit": "% cover", "value": 5.2693611331608921 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 9.6300000000000008 }, { "unit": "% cover", "value": 0.14532734115857432 } ] } ] }, { "category": 27607, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 2089.8900000000003 }, { "unit": "% cover", "value": 10.320444444444444 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 727.16000000000008 }, { "unit": "% cover", "value": 34.794175770016601 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 329.22000000000003 }, { "unit": "% cover", "value": 15.752982214374919 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 126.89 }, { "unit": "% cover", "value": 6.0716114245247361 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 894.04000000000008 }, { "unit": "% cover", "value": 42.77928503414055 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 7.1400000000000006 }, { "unit": "% cover", "value": 0.34164477556235018 } ] }, { "category": 7, "description": "sediment", "units": [ { "unit": "hectares", "value": 5.4400000000000004 }, { "unit": "% cover", "value": 0.26030078138083823 } ] } ] }, { "category": 27608, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 439.80000000000001 }, { "unit": "% cover", "value": 2.1718518518518519 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 307.54000000000002 }, { "unit": "% cover", "value": 69.927239654388359 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 1.72 }, { "unit": "% cover", "value": 0.39108685766257389 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 130.54000000000002 }, { "unit": "% cover", "value": 29.681673487949066 } ] } ] }, { "category": 27610, "description": "RALEIGH", "units": [ { "unit": "hectares", "value": 1358.1100000000001 }, { "unit": "% cover", "value": 6.7067160493827158 } ], "categories": [ { "category": 1, "description": "developed", "units": [ { "unit": "hectares", "value": 757.76999999999998 }, { "unit": "% cover", "value": 55.79592227433713 } ] }, { "category": 3, "description": "herbaceous", "units": [ { "unit": "hectares", "value": 68.219999999999999 }, { "unit": "% cover", "value": 5.0231571816715874 } ] }, { "category": 4, "description": "shrubland", "units": [ { "unit": "hectares", "value": 5.8700000000000001 }, { "unit": "% cover", "value": 0.43221830337748784 } ] }, { "category": 5, "description": "forest", "units": [ { "unit": "hectares", "value": 500.04000000000002 }, { "unit": "% cover", "value": 36.81881438175111 } ] }, { "category": 6, "description": "water", "units": [ { "unit": "hectares", "value": 1.9200000000000002 }, { "unit": "% cover", "value": 0.14137293739093298 } ] }, { "category": -2147483648, "description": "no data", "units": [ { "unit": "hectares", "value": 24.290000000000003 }, { "unit": "% cover", "value": 1.7885149214717513 } ] } ] } ], "totals": [ { "unit": "hectares", "value": 20250 }, { "unit": "% cover", "value": 100 } ] }