lang_cpp_stl.adoc

Kaitai Struct: C++/STL notes

Primitive type mapping

Mapping KS types to C++ is pretty straight-forward:

type	C++ type
no type	std::string
u1	uint8_t
u2	uint16_t
u4	uint32_t
u8	uint64_t
s1	int8_t
s2	int16_t
s4	int32_t
s8	int64_t
str, strz	std::string

Note that both byte arrays and strings are mapped to std::string — that’s because when we store byte array, we need something that would be able to both hold the byte buffer and store it’s length (or at least able to derive it).

String encoding

There’s no universal agreement on dealing with encodings in C++, so KS allows you to choose one of the few popular approaches. You can choose how to deal with string encoding using a compile-time define.

• Ignore encodings at all. In this mode, all string parsing operations just ignore any encoding specifications and pass raw bytes as a string to application. Note that in some cases it might break some .ksy files that actually depend on string being properly decoded / converted.

• Convert all incoming byte streams into strings in a single, one-size-fits-all encoding (for example, UTF8, as suggested by UTF8 Everywhere Manifesto). Since there’s no universal way to do it, KS would use one of platform-dependent ways (which can be also enforced by specifying specific defines):

  • Use POSIX iconv library — usually preinstalled (or included in libc) on all POSIX systems, can be linked as external library on most other systems (i.e. Windows)

  • Use Windows API functions MultiByteToWideChar and WideCharToMultiByte — obviously, available only on Windows platform

  • Use ICU library

Null values

In certain cases, namely when using if with an expression that will be false, a certain attribute won’t be parsed. For example:

seq:
  - id: foo
    type: u1
  - id: bar
    type: u1
    if: foo == 42

If foo is not 42, then an unsigned 1-byte integer bar won’t be parsed. By general convention, Kaitai Struct makes sure that bar is equal to a null value, to be able to distinguish such a situation (as opposed to having some value). However, it’s not possible to do so for many primitive (non-reference) types in C++. In the example above, bar will have type uint8_t, and assigning null to it would just set it to 0, thus we won’t be able to distinguish a situation when bar was read and we’ve got 0, and bar wasn’t read.

To work around this situation, Kaitai Struct generates special null checking methods for every attribute that can be null: _is_null_ATTRIBUTE, where ATTRIBUTE is the name of the attribute. Thus, the proper way to use such nullable values is something like:

if (!r->_is_null_bar()) {
    uint8_t bar = r->bar();
    // `bar` is defined, use `bar` here
} else {
    // `bar` is null because of failed `if` comparison
    // note that accessing r->bar() will return an uninitialized value
    // (i.e. random garbage)
}