TechOverflow Logo
TechOverflow

NanoPB: Custom array converters with offset and size in C++

Deutsch Français 简体中文 English

See also: C version: How to handle custom array converters in C

NanoPB is a code-size optimized Protocol Buffers implementation for embedded systems. This post shows how to handle custom array converters in C++ with NanoPB, based on the KKS-Firmware implementation.

Proto definition

First, create a .proto file with repeated fields:

custom_array.proto
syntax = "proto3";

package example;

message CustomArrayMessage {
  repeated uint32 values = 1;
}

Generate NanoPB code

Generate the NanoPB code with a .options file to specify max count:

Create custom_array.options:

custom_array.options
example.CustomArrayMessage.values max_count:20

Then generate:

generate_nanopb_custom_array.sh
protoc --nanopb_out=. custom_array.proto

This will generate custom_array.pb.h and custom_array.pb.c.

C++ example with custom array converter

Here’s a complete C++ example implementing a custom array converter with offset and size:

custom_array_example.cpp
#include <stdio.h>
#include <algorithm>
#include <array>
#include <limits>
#include "custom_array.pb.h"
#include "pb_encode.h"
#include "pb_decode.h"

// Custom array converter with offset and size template parameters
template<class ITEM_CONVERTER, class CONTAINER, size_t OFFSET=0, size_t SIZE=std::numeric_limits<size_t>::max()/2>
class ArrayConverterWithOffsetAndSize {
public:
    static bool encodeCallback(pb_ostream_t *stream, const pb_field_t *field, const CONTAINER &container) {
        size_t i = 0;
        // Use for loop to iterate through container
        for (const auto &item : container) {
            i++; // First index we check is 1
            if(i <= OFFSET) { // <=, not "<", since we ++ before this line
                continue;
            }
            if(i > OFFSET + SIZE) {
                break;
            }
            // Encode the item
            if (!ITEM_CONVERTER::encodeCallback(stream, field, item)) {
                return false;
            }
        }
        return true;
    }

    static bool decodeCallback(pb_istream_t *stream, const pb_field_t *field, CONTAINER &container) {
        // Add item to container
        // Note: This is a simplified example - real implementation depends on container type
        return false; // Not implemented in this example
    }
};

// Simple uint32 converter
class UInt32Converter {
public:
    static bool encodeCallback(pb_ostream_t *stream, const pb_field_t *field, uint32_t value) {
        if (!pb_encode_tag_for_field(stream, field))
            return false;
        return pb_encode_varint(stream, value);
    }
};

int main() {
    // Buffer for encoded message
    uint8_t buffer[256];
    size_t message_length;
    
    // Create array with 10 elements
    std::array<uint32_t, 10> values = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
    
    // --- ENCODING ---
    example_CustomArrayMessage message = example_CustomArrayMessage_init_zero;
    
    // Use custom converter: encode elements 3-7 (OFFSET=2, SIZE=5)
    using CustomConverter = ArrayConverterWithOffsetAndSize<UInt32Converter, std::array<uint32_t, 10>, 2, 5>;
    
    // Set up callback
    message.values.funcs.encode = [](pb_ostream_t *stream, const pb_field_t *field, void * const *arg) {
        const std::array<uint32_t, 10>* container = (const std::array<uint32_t, 10>*)*arg;
        return CustomConverter::encodeCallback(stream, field, *container);
    };
    message.values.arg = &values;
    
    // Create stream for encoding
    pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
    
    // Encode the message
    if (!pb_encode(&ostream, example_CustomArrayMessage_fields, &message)) {
        printf("Encoding failed: %s\n", PB_GET_ERROR(&ostream));
        return 1;
    }
    
    message_length = ostream.bytes_written;
    printf("Encoded %zu bytes (elements 3-7)\n", message_length);
    
    // Print hex dump
    printf("Encoded data: ");
    for (size_t i = 0; i < message_length; i++) {
        printf("%02x ", buffer[i]);
    }
    printf("\n");
    
    // Print what was encoded
    printf("Encoded values: ");
    for (size_t i = 2; i < 2 + 5 && i < values.size(); i++) {
        printf("%u ", values[i]);
    }
    printf("\n");
    
    return 0;
}

Compile command

Compile the example with nanopb. NanoPB is typically used by including the source files directly in your project:

compile_custom_array_example.sh
g++ -o custom_array_example custom_array_example.cpp custom_array.pb.c pb_common.c pb_encode.c pb_decode.c -I.

Note: NanoPB source files (pb_common.c, pb_encode.c, pb_decode.c) need to be compiled directly with your project. You can obtain these from the NanoPB GitHub repository.

Key points

When to use custom array converters

Expected output

custom_array_expected_output.txt
Encoded 10 bytes (elements 3-7)
Encoded data: 08 03 08 04 08 05 08 06 08 07 
Encoded values: 3 4 5 6 7

Note that only elements 3, 4, 5, 6, 7 are encoded (5 elements starting at offset 2).

Advanced: Decoding support

To support decoding, extend the converter:

decode_callback_snippet.cpp
static bool decodeCallback(pb_istream_t *stream, const pb_field_t *field, CONTAINER &container) {
    uint64_t value;
    if (!pb_decode_varint(stream, &value))
        return false;
    
    // Add to container (implementation depends on container type)
    // For std::array, you might track a separate index
    return true;
}

Real-world usage

This pattern is used in KKS-Firmware for:

More NanoPB posts

Check out similar posts by category: Embedded, C/C++, Protocol Buffers