Chunk stream and versioned serialization

PIChunkStream is a binary stream where data is stored as chunks: each chunk has an integer id and a value. Reading is id-based, so you can add or reorder fields over time and stay backward compatible: old readers ignore unknown ids, new readers can skip optional ids.

Two format versions exist (PIChunkStream::Version_1 and Version_2); the writer chooses the version, the reader detects it automatically. By default new data is written in Version_2.

When to use

Use chunk streams when:

• You need to extend structures without breaking existing stored data (add new fields with new ids).
• You want optional or reordered fields in a single stream.
• You use Code generation to generate serialization: with default (chunk) mode, pip_cmg emits operators that read/write via PIChunkStream and field ids (see code_model for PIMETA id and simple-stream / no-stream).

For fixed, non-extensible layouts, plain PIBinaryStream operators (see Input/Output stream) are enough.

Usage

Build a PIChunkStream from a PIByteArray (read or read-write). Write with cs << cs.chunk(id, value) or add(id, value); read with read() to get the next id, then get(value). Call data() to get the byte buffer for storing or sending. Example (conceptually as in code_model output):

PIByteArray buf;
PIChunkStream cs(&buf);
cs << cs.chunk(1, i) << cs.chunk(2, s);
// later:
PIChunkStream reader(buf);
while (!reader.atEnd()) {
    switch (reader.read()) {
    case 1: reader.get(i); break;
    case 2: reader.get(s); break;
    }
}

Generated operators for structs use the same pattern; see Code generation.