HN
Today

Protobuf-py: Protobuf for Python, without compromises

Buf introduces protobuf-py, a new, ground-up Python Protocol Buffers library designed to be both complete and idiomatic, directly addressing long-standing developer frustrations. It aims to overcome the C++ and Java API constraints of Google's official package while maintaining full spec compliance, unlike other Pythonic alternatives. This promises a much-needed, high-performance, and Python-native Protobuf experience, exciting for developers frequently dealing with data serialization in Python.

25
Score
1
Comments
#4
Highest Rank
9h
on Front Page
First Seen
Jul 12, 8:00 AM
Last Seen
Jul 12, 4:00 PM
Rank Over Time
444488111318

The Lowdown

Buf has announced protobuf-py, a new Python library for Protocol Buffers built completely from scratch. This project aims to provide Python developers with a complete, spec-compliant Protobuf implementation that also feels native to Python, unlike previous options.

  • The existing official Google protobuf package for Python is criticized for its C++ and Java-influenced API, relying on a C engine (upb), making generated code and interactions feel un-Pythonic.
  • Other attempts like betterproto offered a more pleasant Python experience but compromised on Protobuf spec completeness, omitting support for features like proto2, editions, or extensions.
  • protobuf-py keeps message data in plain Python objects with __slots__ and ordinary Python values, allowing for readable, typed generated code that integrates seamlessly with Python's type system and features like pattern matching for Oneof.
  • It achieves high performance through an optional Rust accelerator for critical operations like parsing and serialization, which writes results directly into Python objects. Benchmarks show it can outperform upb in real-world, end-to-end scenarios by reducing Python translation costs for field access.
  • The library fully supports the entire Protobuf spec, including proto2, proto3, editions, extensions, custom options, and passes Google's official conformance suite for both binary and JSON encoding.
  • Developed by Buf, a team with a strong track record in Protobuf tooling and contributions to the Protobuf spec itself, protobuf-py offers an installation process integrated with uv and buf CLI.

protobuf-py represents a significant effort to reconcile the often-conflicting goals of Protobuf spec completeness, high performance, and idiomatic Python usage, offering a robust solution for developers.