-
Notifications
You must be signed in to change notification settings - Fork 358
refactor(go): redesign serialization implementation with performance and usability improvements #2998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
chaokunyang
merged 36 commits into
apache:main
from
chaokunyang:refactor_golang_serializaiton
Dec 7, 2025
Merged
refactor(go): redesign serialization implementation with performance and usability improvements #2998
chaokunyang
merged 36 commits into
apache:main
from
chaokunyang:refactor_golang_serializaiton
Dec 7, 2025
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Go struct values are value types, not reference types, so they should not participate in reference tracking. Previously, Go incorrectly used JSON marshaling to deduplicate struct values by content, which was both semantically wrong and slow. Changes: - Go: Struct values now write NotNullValueFlag (-1) instead of RefValueFlag (0), correctly treating them as non-referenceable - Go: Removed unused basicValueCache and json import - Python: Fixed reference() to handle empty read_ref_ids when NotNullValueFlag is received (prevents segfault) This aligns Go behavior with the xlang spec and C++/Rust implementations where value types don't use reference tracking.
03617c1 to
abd762e
Compare
Resolved go.mod conflict by using newer yaml.v3 version (v3.0.1).
- Use snake_case field names in hash computation for reflection - Add FORY_STRING_ARRAY and other array types to isPrimitiveArrayType - Use generic sliceSerializer for struct fields for cross-language compatibility - Update codegen to use context-based API with WriteContext/ReadContext - Add ptrToCodegenSerializer wrapper for pointer type serialization - Fix Unmarshal to handle pointer dereference cases
Update golang.org/x/tools from v0.1.12 to v0.39.0 for compatibility with Go 1.23+. The old version caused a panic during code generation.
abd762e to
8eb36d3
Compare
ffe32c1 to
64ed87f
Compare
…hod names conflict
pandalee99
approved these changes
Dec 7, 2025
22 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why?
The previous Go serialization implementation had several limitations that affected both performance and developer experience:
This refactoring addresses these fundamental issues to make Fory Go more performant, safer, and easier to use.
What does this PR do?
This PR comprehensively refactors the Fory Go serialization implementation:
1. New Generic API Design (Closes #2992 )
Serialize[T]andDeserialize[T]SerializeAny/DeserializeAnyfor polymorphic use cases2. Performance Optimizations (Closes #3016 )
ReadContextandWriteContexteliminate mixed state overhead[]int,[]string,[]bool, etc.3. Thread-Safe Package (Closes #3017 )
fory/threadsafepackage usingsync.Poolfor zero-contention concurrency4. Code Organization (Closes #3018 )
reader.go,writer.gofor clear read/write separationarray.go,primitive.go,pointer.go,time.go, etc.resolver.go→meta_string_resolver.go,type.go→type_resolver.goAdditional Improvements
Related issues
Closes #2992 - API redesign with generics
Closes #3016 - Performance optimizations
Closes #3017 - Thread-safe package
Closes #3018 - Code refactoring
Closes #3010
Closes #3011
Closes #2611
Closes #2661
Closes #2639
#2982
#1017
Does this PR introduce any user-facing change?
Yes, this PR introduces breaking API changes:
Breaking Changes:
Main API changes:
fory.Serialize(value)→ New:fory.Serialize(foryInstance, &value)orfory.Marshal(&value)fory.Deserialize(data, &target)→ New:fory.Deserialize(foryInstance, data, &target)orfory.Unmarshal(data, &target)fory.NewFory(refTracking)→fory.New(fory.WithTrackRef(enabled))Type-safe generic API (recommended):
Thread-safe usage:
Migration Guide:
Users need to update their code to:
threadsafepackage for concurrent applicationsDoes this PR introduce any public API change?
Yes, extensive public API changes as detailed above. The new API is more performant and type-safe, but requires migration from the old interface{}-based API.
Does this PR introduce any binary protocol compatibility change?
No, the binary protocol remains fully compatible. This is purely an implementation and API refactoring. Data serialized with the old implementation can be deserialized with the new one and vice versa.