A new Go API for JSON

Go's encoding/json gets a long-awaited upgrade with v2 and jsontext, addressing a decade of behavioral flaws and performance bottlenecks. After six years of careful development and community feedback, this new API delivers stricter compliance, improved streaming, and up to 10x faster unmarshaling, while maintaining backward compatibility through an ingenious v1-on-v2 implementation. It's a deep dive into JSON's nuances, sparking debate among Gophers about Go's development pace, the definition of "simple," and even the future of LLM-assisted coding.

308

Score

142

Comments

Highest Rank

18h

on Front Page

First Seen

Nov 4, 6:00 PM

Last Seen

Nov 5, 11:00 AM

Rank Over Time

The Lowdown

The Go team has officially released the encoding/json/v2 and encoding/json/jsontext packages in Go 1.27, marking the culmination of a six-year effort to modernize JSON handling in the language. This significant update tackles numerous long-standing issues with the original encoding/json (now referred to as v1), aiming for greater robustness, efficiency, and flexibility.

Problems with v1: The original package suffered from several behavioral flaws, including imprecise handling of JSON syntax (accepting invalid UTF-8, duplicate object member names), leaking nilness of slices/maps (marshaling them as null instead of empty), case-insensitive unmarshaling, and inconsistent MarshalJSON method calls. API deficiencies included difficulty with io.Reader for unmarshaling, limited options propagation, and inflexible utility functions. Performance was also an issue due to mandatory []byte allocations in MarshalJSON, full value parsing in UnmarshalJSON, and a lack of true streaming capabilities.
Introducing jsontext: The new architecture separates JSON processing into syntactic (jsontext) and semantic (v2) components. The jsontext package provides low-level, streaming-oriented encoding and decoding of JSON values and tokens, without relying on Go reflection, serving as the performant foundation for v2.
v2 API Improvements: encoding/json/v2 builds on jsontext to offer a familiar yet enhanced API. Key features include functions that accept options for fine-grained control, direct operations on io.Writer and io.Reader for true streaming, and new MarshalerTo/UnmarshalerFrom interfaces for custom type serialization that leverage jsontext's efficiency. It also allows caller-specified customization functions to override default marshaling/unmarshaling behavior.
Behavioral Differences: By default, v2 adopts stricter behavior: it errors on invalid UTF-8 and duplicate object names, marshals nil slices/maps as empty structures, and uses case-sensitive matching for unmarshaling into Go structs. The omitempty tag is also redefined to omit fields that encode to "empty" JSON values (null, "", [], {}).
Performance & Compatibility: While Marshal performance is roughly on par with v1, Unmarshal sees significant improvements, up to 10x faster in some benchmarks. The v2 design ensures compatibility with v1 by implementing v1 on top of v2 using specific compatibility options, enabling gradual migration and interoperability within the ecosystem.
A Long Journey: The project spanned about six years, involving extensive community input, public prototypes, and a formal proposal process, demonstrating the Go team's commitment to careful, community-driven evolution.

This release marks a significant step forward for Go developers, offering a more robust, performant, and standards-compliant way to handle JSON, all while ensuring a smooth transition for existing applications.

The Gossip

JSON's 'Simplicity' Skewed

Many commenters expressed skepticism or outright disagreement with the blog post's opening line calling JSON a "simple data interchange format." They argue that the persistent complexities in implementing robust JSON parsers across languages, the nuances of the RFC specification (like duplicate keys or number precision), and the need for a `v2` itself, demonstrate that JSON is far from simple. While acknowledging JSON's undeniable interoperability, some suggest that its lax specification leads to divergent implementations and potential security vulnerabilities, preferring stricter protocols like gRPC for typed data, or highlighting languages like Haskell that prioritize strict spec adherence. Others counter that JSON's widespread adoption proves its practical simplicity, regardless of underlying implementation complexities.

Go's Gradual Grinds: Generics to JSON

The six-year development timeline for `json/v2` was a prominent point of discussion, drawing comparisons to the similarly lengthy process for Go generics and modules. Some users appreciated the Go team's careful, deliberate approach to ensure stability and compatibility, likening it to C++'s rollout of features. Others, however, questioned why such a fundamental and performance-critical component like JSON processing took so long, suggesting it reflects a slow velocity in Go's standard library or even a perceived incompleteness of the language. A defense was made that Go's 'good enough' standard library often leads to third-party solutions for niche high-performance needs, and the focus is on stability over rapid change.

Nil Niggles: The Null vs. Empty Array Debate

A specific point of contention was `v2`'s default behavior to marshal `nil` Go slices and maps as empty JSON arrays or objects, respectively, instead of `null`. Proponents argue this is a more intuitive and useful default, aligning with how most Go developers expect to handle empty collections in JSON, and resolving frequent `v1` bugs related to `nil` leakage. They highlight that an empty array/object is often preferred in JSON schemas over `null`. Conversely, critics argued that this breaks round-tripping for applications that intentionally distinguish between `nil` (absence) and empty (presence of nothing), potentially causing issues for `v3` if not carefully managed. They assert that `nil` vs. empty is a Go-specific quirk, not a JSON problem.

LLMs and the Go-ing Concern

A significant tangent explored the impact of AI/LLMs on coding, particularly with Go. Several users noted they now use LLMs to generate Go code, viewing Go as an ideal "target language" for AI due to its performance, simplicity, and fast compilation times, especially compared to Rust. This led to a broader philosophical discussion: some embrace AI's role in boosting productivity for backend services, while others voiced concerns about atrophied human coding skills and the potential loss of enjoyment from hand-coding. The debate also touched on the quality of LLM-generated code, with some finding it "shit" and requiring extensive human review, contrasting with others who believe LLMs are already producing sufficiently good code, shifting the developer's role more towards design and architecture.

Marshalling Mayhem: Error Returns and Type Troubles

The discussion delved into why `json.Marshal` returns an error, with some questioning whether it should be a compile-time error or a panic for unmarshalable types, citing it as a "code bug." Others defended the error return, explaining it's necessary because `Marshal` accepts `any` types whose marshalability cannot be determined at compile time (e.g., channels, functions, non-finite floats like `NaN` or `infinity` which JSON doesn't support), and custom `MarshalJSON` methods can introduce runtime errors. The broader conversation touched upon Go's type system, with some arguing a more powerful type system could offer compile-time guarantees for serialization, while others emphasized that the current API's flexibility (like interfaces for `io.Writer`) is a strength despite its type-safety trade-offs.