TechOverflow Logo
TechOverflow

NanoPB: Custom array converters with offset and size in C

Deutsch Français 简体中文 English

See also: C++ version: How to handle custom array converters in C++

NanoPB is a code-size optimized Protocol Buffers implementation for embedded systems. This post shows how to handle custom array converters in C with NanoPB.

Proto definition

First, create a .proto file with repeated fields:

custom_array.proto
syntax = "proto3";

package example;

message CustomArrayMessage {
  repeated uint32 values = 1;
}

Generate NanoPB code

Generate the NanoPB code with a .options file to specify max count:

Create custom_array.options:

custom_array.options
example.CustomArrayMessage.values max_count:20

Then generate:

generate_nanopb_custom_array.sh
protoc --nanopb_out=. custom_array.proto

This will generate custom_array.pb.h and custom_array.pb.c.

C example with custom array converter

Here’s a complete C example implementing a custom array converter with offset and size:

custom_array_example.c
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>
#include <limits.h>
#include "custom_array.pb.h"
#include "pb_encode.h"
#include "pb_decode.h"

// Custom array converter context
typedef struct {
    const uint32_t* data;
    size_t size;
    size_t offset;
    size_t count;
} array_context_t;

// Encode callback for uint32 array with offset and size
bool uint32_array_encode_callback(pb_ostream_t *stream, const pb_field_t *field, void * const *arg) {
    const array_context_t* ctx = (const array_context_t*)*arg;
    
    size_t start = ctx->offset;
    size_t end = (ctx->offset + ctx->count < ctx->size) ? 
                 ctx->offset + ctx->count : ctx->size;
    
    for (size_t i = start; i < end; i++) {
        if (!pb_encode_tag_for_field(stream, field))
            return false;
        
        if (!pb_encode_varint(stream, ctx->data[i]))
            return false;
    }
    
    return true;
}

int main() {
    // Buffer for encoded message
    uint8_t buffer[256];
    size_t message_length;
    
    // Create array with 10 elements
    uint32_t values[10] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
    
    // --- ENCODING ---
    example_CustomArrayMessage message = example_CustomArrayMessage_init_zero;
    
    // Set up context: encode elements 3-7 (offset=2, count=5)
    array_context_t ctx = {
        .data = values,
        .size = 10,
        .offset = 2,
        .count = 5
    };
    
    // Set up callback
    message.values.funcs.encode = uint32_array_encode_callback;
    message.values.arg = &ctx;
    
    // Create stream for encoding
    pb_ostream_t ostream = pb_ostream_from_buffer(buffer, sizeof(buffer));
    
    // Encode the message
    if (!pb_encode(&ostream, example_CustomArrayMessage_fields, &message)) {
        printf("Encoding failed: %s\n", PB_GET_ERROR(&ostream));
        return 1;
    }
    
    message_length = ostream.bytes_written;
    printf("Encoded %zu bytes (elements 3-7)\n", message_length);
    
    // Print hex dump
    printf("Encoded data: ");
    for (size_t i = 0; i < message_length; i++) {
        printf("%02x ", buffer[i]);
    }
    printf("\n");
    
    // Print what was encoded
    printf("Encoded values: ");
    for (size_t i = 2; i < 2 + 5 && i < 10; i++) {
        printf("%u ", values[i]);
    }
    printf("\n");
    
    return 0;
}

Compile command

Compile the example with nanopb. NanoPB is typically used by including the source files directly in your project:

compile_custom_array_example.sh
gcc -o custom_array_example custom_array_example.c custom_array.pb.c pb_common.c pb_encode.c pb_decode.c -I.

Note: NanoPB source files (pb_common.c, pb_encode.c, pb_decode.c) need to be compiled directly with your project. You can obtain these from the NanoPB GitHub repository.

Key points

When to use custom array converters

Expected output

custom_array_expected_output.txt
Encoded 10 bytes (elements 3-7)
Encoded data: 08 03 08 04 08 05 08 06 08 07 
Encoded values: 3 4 5 6 7

Note that only elements 3, 4, 5, 6, 7 are encoded (5 elements starting at offset 2).

Advanced: Generic template-like implementation

For a more generic C implementation (simulating templates):

custom_array_generic.c
#include <stddef.h>
#include <limits.h>

#define MAX_OFFSET_SIZE (SIZE_MAX / 2)

typedef struct {
    const void* data;
    size_t element_size;
    size_t size;
    size_t offset;
    size_t count;
    bool (*encode_element)(pb_ostream_t*, const pb_field_t*, const void*);
} generic_array_context_t;

bool generic_array_encode_callback(pb_ostream_t *stream, const pb_field_t *field, void * const *arg) {
    const generic_array_context_t* ctx = (const generic_array_context_t*)*arg;
    
    size_t start = ctx->offset;
    size_t end = (ctx->offset + ctx->count < ctx->size) ? 
                 ctx->offset + ctx->count : ctx->size;
    
    const uint8_t* data = (const uint8_t*)ctx->data;
    
    for (size_t i = start; i < end; i++) {
        const void* element = data + (i * ctx->element_size);
        if (!ctx->encode_element(stream, field, element)) {
            return false;
        }
    }
    
    return true;
}

Real-world usage

This pattern is used in KKS-Firmware for:

Differences from C++

The C version differs from C++ in several ways:

More NanoPB posts

Check out similar posts by category: Embedded, C/C++, Protocol Buffers