Skip to content

Conversation

@chaokunyang
Copy link
Collaborator

@chaokunyang chaokunyang commented Nov 10, 2025

Why?

This PR addresses the gap in Fory C++ by implementing a complete xlang serialization framework with full polymorphism support, bringing C++ to feature parity with Rust, Java, and Python implementations. Currently, C++ only supports row format serialization, which limits its ability to:

  1. Serialize arbitrary object graphs with shared/circular references
  2. Handle polymorphic types (virtual classes, smart pointers to derived types)
  3. Perform cross-language object serialization compatible with other Fory implementations
  4. Support schema evolution with field name compression

This implementation enables C++ developers to leverage Fory's full cross-language serialization capabilities in high-performance applications like microservices, game engines, and distributed systems.

What does this PR do?

This PR implements a complete xlang serialization framework for C++ consisting of 26 new files (~12K lines):

Core Framework

  • fory.h: Main entry point with builder pattern API for Fory instance configuration
  • type_resolver.h/cc: Comprehensive type system with registration and resolution for all supported types
  • context.h/cc: Read/write contexts for serialization with buffer management and depth tracking
  • ref_resolver.h: Reference tracking for handling shared and circular references
  • config.h: Configuration options (compatible mode, ref tracking, etc.)

Polymorphism Support

  • serializer_traits.h: Type traits system including:
    • is_polymorphic<T> trait for compile-time polymorphism detection
    • get_concrete_type_id() for runtime type identification using RTTI
    • Container type detection (maps, sets, vectors, optionals)
  • context.h/cc: Type info encoding/decoding:
    • write_any_typeinfo(): Writes polymorphic type information (type_id, meta_index, or namespace+typename)
    • read_any_typeinfo(): Reads and resolves polymorphic type information
  • Smart pointer polymorphism: Full support in smart_ptr_serializers.h for std::unique_ptr, std::shared_ptr
  • Map polymorphism: Polymorphic key/value serialization in map_serializer.h

Serializer Implementations

  • basic_serializer.h: Primitives (integers, floats, bool, char), strings
  • collection_serializer.h: std::vector, std::set, std::unordered_set
  • map_serializer.h: std::map, std::unordered_map with polymorphic key/value support
  • array_serializer.h: std::array
  • smart_ptr_serializers.h: Smart pointers with full polymorphism
  • struct_serializer.h: User-defined structs with reflection, compatible mode with field compression
  • enum_serializer.h: Scoped and unscoped enums
  • temporal_serializers.h: std::chrono::duration and time points
  • skip.h/cc: Utilities for skipping unknown types during deserialization

Testing

  • tests across 5 test suites, all passing:
    • serialization_test.cc: Basic types and collections (26 tests)
    • struct_test.cc: Struct serialization (27 tests)
    • struct_compatible_test.cc: Schema evolution (12 tests)
    • map_serializer_test.cc: Map polymorphism (17 tests)
    • smart_ptr_serializer_test.cc: Smart pointer polymorphism (6 tests)

Key Features Implemented

  1. Complete Type System: Support for all C++ standard types with wire format compatibility
  2. Polymorphism: Runtime type identification, polymorphic serialization for smart pointers and collections
  3. Schema Evolution: Compatible struct mode with meta string compression
  4. Reference Tracking: Handles shared references and circular object graphs
  5. Thread-Safe: Immutable type resolver design eliminates locking overhead
  6. Zero-Overhead: Uses C++17 constexpr if for compile-time polymorphism checks
  7. Error Handling: Comprehensive error handling using Result<T, Error> pattern with FORY_TRY macro

Implementation Highlights

Polymorphic Type Encoding (mirrors Rust implementation):

// Write polymorphic type info
if (key_is_polymorphic) {
    auto concrete_type_id = get_concrete_type_id(key);
    FORY_TRY(ctx.write_any_typeinfo(
        static_cast<uint32_t>(Serializer<K>::type_id), concrete_type_id));
} else {
    FORY_TRY(write_type_info<K>(ctx));
}

Thread-Safe Type Resolver:

  • Immutable design: All type registration done during initialization
  • Lock-free reads after construction
  • Shared across serialization operations

Field Ordering Optimization:

  • Struct fields sorted by fory xlang fields sort spec
  • Snake_case conversion for field names
  • Compatible with schema evolution

Related issues

Closes #2907

#2906

Related to discussion on C++ xlang serialization support. This implementation follows the same architecture and protocol as the Rust implementation to ensure cross-language compatibility.

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change?

    • Yes: Adds new fory::serialization namespace with complete xlang serialization API
    • New public classes: Fory, ForyBuilder, WriteContext, ReadContext, TypeResolver, RefResolver
    • New serializer traits: Serializer<T>, is_polymorphic<T>, SerializationMeta<T>
    • Public configuration types: Config, reference tracking modes, compatible modes
  • Does this PR introduce any binary protocol compatibility change?

    • No: This implementation follows the existing xlang serialization specification and maintains wire format compatibility with Rust, Java, and Python implementations

API Example

#include "fory/serialization/fory.h"

// Configure and build Fory instance
auto fory = Fory::builder().xlang(true).track_ref(true)
    .compatible(true).build();

// Serialize object
FORY_TRY(buffer, fory->serialize(buffer, my_object));

// Deserialize object
FORY_TRY(result, fory->deserialize<MyType>(buffer));

Build and Test

cd cpp

# Build all targets
bazel build //fory/serialization/...

# Run all serialization tests
bazel test //fory/serialization/...

# Run specific test suite
bazel test //fory/serialization:struct_test
bazel test //fory/serialization:smart_ptr_serializer_test
bazel test //fory/serialization:map_serializer_test

Documentation

Will be provided in following-up PR

Performance Considerations

  • Zero-overhead polymorphism checks: Compile-time constexpr if eliminates runtime overhead for non-polymorphic types
  • Immutable type resolver: Lock-free concurrent reads, no synchronization cost
  • Buffer pooling: Reusable read/write context buffers reduce allocations
  • Meta string compression: Efficient field name encoding in compatible mode (shares infrastructure, full compression pending)

Cross-Language Compatibility

Wire format should to be compatible with Rust implementation (primary reference)

This enables seamless data exchange between C++ and other Fory language implementations in polyglot systems.

@pandalee99
Copy link
Contributor

It's amazing!

Copy link
Contributor

@pandalee99 pandalee99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chaokunyang chaokunyang force-pushed the cpp_serializer_framework branch from 86aef65 to f32df58 Compare November 10, 2025 15:34
@chaokunyang chaokunyang changed the title feat(c++): implement fory cpp serializer framework feat(c++): implement fory cpp object graph serializer framework Nov 10, 2025
@chaokunyang chaokunyang changed the title feat(c++): implement fory cpp object graph serializer framework feat(c++): implement fory cpp object graph serialization framework Nov 10, 2025
@chaokunyang chaokunyang force-pushed the cpp_serializer_framework branch 3 times, most recently from 802f2e6 to d965618 Compare November 10, 2025 16:42
@chaokunyang chaokunyang force-pushed the cpp_serializer_framework branch from d965618 to 72ae168 Compare November 10, 2025 16:51
@chaokunyang chaokunyang merged commit 3f7fff5 into apache:main Nov 10, 2025
59 checks passed
chaokunyang added a commit that referenced this pull request Nov 26, 2025
## Why?

#2908 implemented c++ serialization framework, but don't fully implement
xlang spec, and don't align with java/rust.

## What does this PR do?

- implement xlang serialization for c++ 
- align with  java/rust type system
- implement meta string encoding for c++
- add write_type_info/read_type_info
- align collection/map serialization with java/rust
- add xlang serialization tests

## Related issues

#2906

Closes #2909
Closes #1543

## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/fory/issues/new/choose) describing the
need to do so and update the document if necessary.

Delete section if not applicable.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?

## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.

Delete section if not applicable.
-->
@chaokunyang chaokunyang mentioned this pull request Nov 26, 2025
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++] Fory C++ Object Graph Serialization Framework

2 participants