Custom output encoding and debug info

[matteo-cristino]

When setting a custom output encoding, like base58, along with the configuration logfmt=json, it can happen that if at any time an error is thrown and in the memory there is an element that uses the default encoding but it is smaller than 3 bytes, then the trace and heap will not be printed. This seems due to the fact the when using the json logformat all data in the heap is encoded (with default encoding) before printing it.

I have not found a real example of that but it can be reproduced with

Rule output encoding base58

Given nothing
When I set 'str' to 'ab' as 'base58'
Then print the data

with config

logfmt=json

will results in

[ "ZENROOM JSON LOG START",
" Release version: v4.36.0",
" Build commit hash: 760ca7bb",
" Memory manager: libc",
" ECDH curve is SECP256K1",
" ECP curve is BLS381",
" Rule output encoding base58",
"[W] Zencode is missing version check, please add: rule check version N.N.N",
"[!] fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[!] fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[!] Zencode runtime error",
"[!] /zencode_data.lua:367: fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[!] Execution aborted with errors.",
"[*] Zenroom teardown.",
" Memory used: 399 KB",
"ZENROOM JSON LOG END" ]

hiding heap and trace.

This issue is important since it happened in out didroom admin dashboard where the error log was

[ "ZENROOM JSON LOG START",
" .  Release version: v4.45.2",
" .  Build commit hash: ad00236a",
" .  Memory manager: libc",
" .  ECDH curve is SECP256K1",
" .  ECP curve is BLS381",
" .  rule input encoding base58",
" .  rule output encoding base58",
"[W] Zencode is missing version check, please add: rule check version N.N.N",
"[W] identifier not found in DATA or KEYS",
"[!] fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[!] Zencode runtime error",
"[!] [source 'zencode_data']:367: fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[*] Zenroom teardown.",
" .  Memory used: 867 KB",
"ZENROOM JSON LOG END" ]

giving us no clues about what was the real error.

The only difference about my error and this one is that in mine the "[!] fatal to_base58: base58 cannot encode octets smaller than 3 bytes" is present only one, I think that is because the real error was not a conversion to base58, but something else and when zenroom tried to print the error there was something with default encoding not printable as base58.

Maybe the default encoding for heap and trace in case of error should be set to a specific encoding like base64 or hex and not modifiable by rules?

[matteo-cristino]

More precise example

Rule output encoding base58

Given nothing

When I set 'str' to 'ab' as 'base58'
When I copy 'mimmo' to 'pippo'

Then print the data

with a generic error (more similar to didroom case), will result in

[ "ZENROOM JSON LOG START",
" .  Release version: v4.46.1",
" .  Build commit hash: a7e9ac35",
" .  ECDH curve is SECP256K1",
" .  ECP curve is BLS381",
" .  Rule output encoding base58",
"[W] Zencode is missing version check, please add: rule check version N.N.N",
"[!] fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[!] Zencode runtime error",
"[!] [source 'zencode_data']:368: fatal to_base58: base58 cannot encode octets smaller than 3 bytes",
"[*] Zenroom teardown.",
" .  Memory used: 601 KB",
"ZENROOM JSON LOG END" ]

As it can be seen the error in reality here is that I'm tryting to copy an element that does not exist, but I will never know it 😢