ProtoBuf Schematics

Convert ProtoBuf proto file to cute Schematics classes file.

Google Protobuf is great when it comes to high performance schema aware APIs, but when Google designed Protobuf, it didn't tried to make the generated code idiomatic in Python, which brings a problem when exporting messages outside interface modules or having nice IDE auto-completions. Schematics is a cute and Pythonic schema library that goes well with most applications. Why not join both?

Currently this package does not support the Protobuf binary format and will work with a any other textual representation which is easily generated with the original Protobuf API for any language. Ease of use was prioritized while writing this package rather than mere performance.

• Free software: Apache Software License 2.0
• Documentation: https://protobuf-schematics.readthedocs.io.

Usage

1. Convert the proto file to python Schematics classes:

   protobuf_schematics <path-to-file.proto> generated_schematics_proto.py # or any output filename
   

2. Convert your ProtoBuf message to Json:

In Java:

import com.google.protobuf.util.JsonFormat;

FileWriter file = new FileWriter("protoBufMessage.json")
JsonFormat.Printer printer = JsonFormat.printer().preservingProtoFieldNames();
String message = printer.print(someProtoBufMessage);
file.write(message)

or from Python:

import json
from google.protobuf.json_format import MessageToJson

json = MessageToJson(org, preserving_proto_field_name=True, including_default_value_fields=True)
with open("protoBufMessage.json", 'w') as output:
    json.dump(json, output)

3. In your project, load the message in python as Schematics object:

import json
from generated_schematics_proto import SomeMessage # import the schematics message class

schematics_root_message = SomeMessage(json.load(open('protoBufMessage.json')))

Or use a message loaded in python by Protobuf:

import json
from generated_schematics_proto import SomeMessage # import the schematics message class

# ... get your protobuf message as the pb class representation
schematics_root_message = SomeMessage.import_from_protobuf_message(protobuf_message)

Example

This proto file:

syntax = "proto3";

enum IPAddressFamily {
    INVALID = 0;
    IPv4 = 1;
    IPv6 = 2;
};

message ProtocolAndPorts {
    repeated uint32 ports = 3;
}

message FlowFilter {
    enum SomeEnum {
        VALUE = 0;
    };
    string id = 1 [deprecated = true];
    SomeEnum consumer_filter_id = 2;
    map<string, ProtocolAndPorts> ports = 3;
    repeated ProtocolAndPorts protocol_and_ports = 4;
}

Will be converted to:

class IPAddressFamily(Enum):
    INVALID = 0
    IPv4 = 1
    IPv6 = 2


class ProtocolAndPorts(ProtobufMessageModel):
    ports = ListType(IntType())


class FlowFilter(ProtobufMessageModel):
    class InnerEnum(Enum):
        VALUE = 0

    id = StringType()
    consumer_filter_id = EnumType(SomeEnum)
    ports = DictType(ModelType(ProtocolAndPorts), str)
    protocol_and_ports = ListType(ModelType(ProtocolAndPorts))

Features

• Support both Protobuf syntax 2 and 3.
• Support builtin types such as StringType, IntType.
• Support proto map fields as Schematics DictType.
• Support repeated modifier as convert to ListType.
• Support Enum class generation and custom Schematics EnumType.
• Support custom schematics ByteArrayType base64 encoded byte arrays converted from Java.

Development

First, install the Pipfile and create the proper virtual environment:

pipenv install --dev

To check linting with flake8, run:

make lint

To run the unittests against your working python version:

py.test

To see coverage report:

make coverage

To run tests against all supported python versions:

tox

To make the docs (which will be automatically published to readthedocs on commits to the master branch):

make docs

Credits

The parsing work of .proto files is provided thanks to the awesome guys at PyroBuf.

This package was created with Cookiecutter and the elgertam/cookiecutter-pipenv project template, based on audreyr/cookiecutter-pypackage.