Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix of aws-full.csv path in csv2bufr session. #91

Merged
merged 1 commit into from
Oct 5, 2023
Merged

Fix of aws-full.csv path in csv2bufr session. #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions documentation/docs/practical-sessions/bufr-command-line-tools.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -559,15 +559,24 @@ python3 wis2box-ctl.py restart
```

Once the containers have restarted log in to the management container and verify that the mappings and templates have
been updated:
been updated.

!!! hint
Before entering the last command make sure you are listening to your broker with MQTT Explorer. You should see
the notification being sent for the data you have just ingested.

``` {.copy}
First login to the container and re-download the test data, this was modified during the last exercise.

```
cd ~/wis2box-1.0b5/
python3 wis2box-ctl.py login
cd /data/wis2box/working/bufr-cli
curl https://training.wis2box.wis.wmo.int/sample-data/csv2bufr-ex1.csv --output csv2bufr-ex1.csv
```

Now ingest the data (note, you will need to update the topic in the code below):

```
wis2box data ingest \
--topic-hierarchy <your-topic> \
--path /data/wis2box/working/bufr-cli/
Expand All @@ -583,6 +592,8 @@ that the originating center has been updated.
<center><img alt="Image showing notification in MQTT Explorer, with href highlighted"
src="../../assets/img/bufr-cli-mqtt-ex.png"/></center>

Finally, navigate to the notifications page on the web-application and inspect the data.

## Housekeeping

Clean up your working directory by removing files you no longer need and clean up your station list to remove any
Expand Down
14 changes: 9 additions & 5 deletions documentation/docs/practical-sessions/converting-csv-data-to-bufr.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ included in the format:

{{ read_csv('docs/assets/tables/aws-minimal.csv') }}

The full file can be downloaded from: [aws-full.csv](/assets/tables/aws.full.csv)
The full file can be downloaded from: [aws-full.csv](/assets/tables/aws-full.csv)

Now download the example for this exercise from the link below:

Expand Down Expand Up @@ -163,14 +163,14 @@ You should also be able to see these notifications in MQTT Explorer.
### Exercise 3 - debugging the input data

In this exercise we will examine what happens with invalid input data. Download the next example file by clicking the
link below. This contains the same data as the first file but with the columns containing missing data removed.
Follow the same process to convert the data to BUFR.
link below. This contains the same data as the first file but with the empty columns removed.
Examine the file and confirm which columns have been removed and then follow the same process to convert the data to BUFR.

[csv2bufr-ex3a.csv](/sample-data/csv2bufr-ex3a.csv)

!!! question
With the columns missing from the file were you able to convert the data to BUFR?
Did you notice anything different on the validation page?
Did you notice any change to the warnings on the validation page?

??? success "Click to reveal answer"
You should have still been able to convert the data to BUFR but the warning messages will have been updated
Expand All @@ -183,18 +183,22 @@ In this next example an additional column has been added to the CSV file.
!!! question
Without uploading or submitting the file can you predict what will happen when you do?

Now upload and confirm whether your prediction was correct.

??? success "Click to reveal answer"
When the file is validated you should now receive a warning that the column ``index``
is not found in the schema and that the data will be skipped. You should be able to click
through and convert to BUFR as with the previous example.

In the final example in the exercise the data has been modified. Examine the contents of the CSV file.
In the final example in this exercise the data has been modified. Examine the contents of the CSV file.

[csv2bufr-ex3c.csv](/sample-data/csv2bufr-ex3c.csv)

!!! question
What has changed in the file and what do you think will happen?

Now upload the file and confirm whether you were correct.

??? warning "Click to real answer"
The pressure fields have been converted from Pa to hPa in the input data. However, the CSV to BUFR
converter expects the same units as BUFR (Pa) and, as a result, these fields fail the validation due to being
Expand Down
Loading