feat: support uniqueItems validation on arrays of complex objects#1564
Merged
ernado merged 21 commits intoogen-go:mainfrom Nov 24, 2025
Merged
feat: support uniqueItems validation on arrays of complex objects#1564ernado merged 21 commits intoogen-go:mainfrom
ernado merged 21 commits intoogen-go:mainfrom
Conversation
Add foundational infrastructure for supporting uniqueItems validation on arrays of complex objects (issue ogen-go#1563). This includes IR structures, error types, test schemas, and comprehensive planning documentation. Changes: - Add IR structures in gen/ir/equality.go for equality method generation - Add DuplicateItemsError and DepthLimitError to validate/errors.go - Create test schemas for simple, deep, and wide object validation - Add comprehensive spec, plan, and implementation guide documentation - Update .gitignore with Go test artifacts and temp files - Add CLAUDE.md project documentation This is Phase 1 & 2 (foundational) of the implementation. Next steps will add code generation for Equal() and Hash() methods. Refs: ogen-go#1563
Implement core code generation for complex uniqueItems validation:
Code Generation:
- Remove ErrNotImplemented check for complex types (arrays/objects)
- Add detection logic to identify types needing equality methods
- Implement Equal() method generation with depth tracking
- Implement Hash() method generation using FNV-1a
- Support all 7 field type categories (primitive, optional, nullable,
pointer, nested, array, map)
Detection & IR:
- Add collectEqualitySpecs() to scan types after generation
- Create EqualityMethodSpec for each complex uniqueItems type
- Track in Generator.equalitySpecs field
Integration:
- Wire into WriteSource() generation pipeline
- Generate separate files: oas_{type}_equal_gen.go, oas_{type}_hash_gen.go
- Use goimports for proper formatting
Files Modified:
- gen/schema_gen.go: Remove complex uniqueItems block
- gen/generator.go: Add equalitySpecs field, call collectEqualitySpecs()
- gen/write.go: Call generateEqualityMethodsWithFS() in pipeline
Files Added:
- gen/gen_equality.go: Equal/Hash method generation logic
- gen/gen_equality_detect.go: Type detection and spec creation
Next: Test code generation with test schemas
Enhanced Equal() and Hash() generation to handle optional and nullable wrappers around nested objects. Changes: - Updated comparison logic to detect nested objects inside Opt/Nil wrappers - Generate Equal() calls for nested objects instead of using != operator - Updated hash generation to call Hash() on nested objects - Enhanced isNestedObject() to look inside Optional/Nullable wrappers - Added collectNestedTypes() to recursively collect all nested types Status: - ✅ Simple schemas (workflow-status.yaml) compile successfully -⚠️ Deep nested schemas need additional work on type collection Known Issue: The collectNestedTypes() function doesn't yet fully traverse all nested types within Optional wrappers. This causes some nested types to not get Equal/Hash methods generated, leading to compilation errors for deeply nested schemas. Next: Fix type collection to ensure all transitive nested types are found
…eration - Add KindGeneric case to collectNestedTypes() to unwrap OptT/NilT types - Add KindGeneric case to isNestedObject() for proper nesting detection - Add KindGeneric case to categorizeFieldType() for correct field categorization - Simplify hasNestedObjects() to treat all structs uniformly for consistent Equal() signatures - Add unwrapOptional() helper to handle both Generic and Struct-based optional wrappers - Add IsMap field to FieldEqualitySpec to track map types within Optional wrappers - Implement writeMapComparison() for proper map equality checking - Fix Optional+Map field comparison to use map iteration instead of != This fixes deep nested type collection (workflow-deep.yaml now generates 7 Equal/Hash files) and resolves map comparison errors for types like OptParameterValueMetadata.
- Add IsArrayOfStructs field to FieldEqualitySpec to track array element types - Update array comparison to call Equal() on struct elements instead of using != - Detect arrays of structs during spec creation by checking item type - Handle arrays with depth parameter when elements are structs This fixes compilation for schemas with array fields containing struct elements like workflow-wide.yaml (Actions []TransitionAction).
Add hash-based O(n) duplicate detection for arrays of complex objects: - Create gen_validators_unique.go to generate validateUnique[TypeName]() functions - Implement hash bucket structure (map[uint64][]entry) for efficient lookups - Add Equal() verification for hash collision handling per requirement ogen-go#24 - Include depth limit error recovery with defer/recover mechanism - Wire into generation pipeline in write.go alongside Equal/Hash generation - Update validators template to call validateUnique[Type]() for struct arrays - Fall back to validate.UniqueItems() for primitive arrays Generated validators now correctly detect duplicates in complex object arrays with proper error reporting including duplicate indices. Completes T034-T039 (Phase 4 runtime validation).
…ation Add test suite for validateUnique[TypeName]() functions covering: T040-T045 Test Coverage: - Empty array validation (passes with no error) - Single element validation (passes with no error) - Duplicate detection with correct indices reporting - All unique items validation (passes with no error) - Nested object equality verification - Hash collision false positive prevention Performance Benchmarks: - 1,000 unique items: ~0.25ms per operation (40x faster than 10ms target) - 1,000 items with duplicate: ~0.25ms per operation - Validates O(n) hash-based algorithm performance Test Results: All tests PASS - Correct error types (DuplicateItemsError) - Accurate duplicate indices (e.g., [0, 3]) - Clear error messages with index information - No false positives from hash collisions Completes Phase 4 testing tasks (T040-T045).
Member
|
Hey @lanej, thank you for contribution! Pleas make pipelines green. |
Member
|
Let's ignore commitlint, but golangci-lint issues should be fixed to merge this PR |
Add comprehensive tests for hash collision scenarios: - TestValidateUniqueWorkflowStatus_HashCollisionHandling: Verifies Equal() correctly distinguishes different items - TestValidateUniqueWorkflowStatus_CollisionResolution: Tests bucket mechanism with 100 items to ensure hash patterns are handled correctly and duplicates are still detected
Add integration test schema and comprehensive test suite combining all field type categories: - Primitives (string, number, integer, boolean) - Optional fields - Nullable fields - Enums (required and optional) - Arrays (primitives and nested objects) - Maps (additionalProperties) - Nested objects (simple and complex) Test coverage includes: - All field types populated and validated together - Duplicate detection with complex objects - Optional field differences - Nested object differences - Array differences - Map differences - Hash/Equal consistency - Unset optional fields - Performance benchmark: ~0.14ms for 100 complex items Verified correct generation of Equal() and Hash() methods for: - ComprehensiveItem (18 fields, all types) - User (nested object) - Configuration (nested with maps and arrays) - Feature (nested with maps) All 8 integration tests pass, confirming the implementation correctly handles all OpenAPI field type combinations.
Add regression tests to ensure Equal() and Hash() generation remains stable: - Created golden-test.yaml reference schema with diverse field types - Generated golden reference files for GoldenItem and Metadata types - Added 4 golden file tests that compare generated code against references - Tests verify stable code generation for: - Equal() with primitives, optionals, arrays, maps, nested objects - Hash() with FNV-1a hashing of all field types - Depth limit enforcement in Equal() - Nested object recursion with depth+1 All 4 golden file tests pass, ensuring code generation stability across changes.
Add focused unit tests for each field type category: - T046: Primitive fields (string, boolean) - equality and hashing - T047: Optional fields - Set/Value pattern, unset handling - T048: Nullable fields - null vs non-null comparison - T049: Array fields - length, element comparison, different arrays - T050: Map fields - key-value comparison, missing keys - T051: Nested object fields - depth parameter passing, nested equality - T052: Enum fields - enum value comparison in structs Additional test: - Hash consistency across all field types - Verification that hash changes when any field changes All 8 unit tests pass, providing comprehensive coverage of: - Equal() behavior for each field type - Hash() consistency for each field type - Edge cases like unset optionals, empty arrays, nested objects
Add minimal JIRA API subset showcasing operations that were previously blocked by complex uniqueItems validation (GitHub issue ogen-go#1563). Schema includes: - updateWorkflowTransitionRules: 3 uniqueItems arrays (postFunctions, conditions, validators) - updateWorkflowMapping: uniqueItems array of IssueTypesWorkflowMapping Types demonstrate real-world patterns: - WorkflowTransitionRule: ruleKey + configuration map + optional ID - IssueTypesWorkflowMapping: workflowId + array of issue types Tests verify: - Unique JIRA rules pass validation - Duplicate rules are detected with correct indices - Complete JIRA request structures validate properly - Multiple uniqueItems arrays in single request body - Duplicate detection across all arrays Performance: ~7μs for 50 workflow rules This proves the implementation handles Atlassian JIRA's production API patterns that previously required ErrNotImplemented workarounds.
Add detailed documentation covering: - Feature overview and motivation (GitHub ogen-go#1563, JIRA API) - Implementation details: - Equal() method generation with depth tracking - Hash() method generation with FNV-1a - Runtime validation with O(n) hash buckets - Examples for all test schemas - Test coverage summary (38 tests across 5 suites) - Performance benchmarks (7μs - 0.24ms) - Error types (DuplicateItemsError, DepthLimitError) - Implementation file references - Limitations and future improvements This provides complete documentation for users and maintainers of the complex uniqueItems validation feature.
- Remove unused ValidationFunctionSpec from gen/ir/equality.go - Simplify gen_validators_unique.go (121 -> 91 lines) - Extract writeValidateUnique as standalone function - Use shared writeFormattedCode helper
Improve Equal() and Hash() generation to handle additional patterns encountered in real-world usage (Terraform provider): Field detection enhancements: - Add IsArray flag to detect arrays within Optional/Nullable wrappers - Add IsArrayOfNullable flag for arrays of nullable elements ([]NilT[T]) - Add IsByteSlice flag for proper []byte/jx.Raw handling Code generation improvements: - Use bytes.Equal() for byte slices instead of != operator - Handle Optional/Nullable arrays with proper iteration (can't use !=) - Support arrays of nullable elements (compare Null flag, then Value) - Extract writeArrayComparison() helper for reusable array logic - Extract writeArrayComparisonWithNullable() for nullable element arrays Testing: - Add gen_equality_test.go with 14 unit tests - Cover optional arrays, nullable arrays, arrays of structs - Test proper code generation for all edge cases - All tests passing gitignore: - Add 'ogen' binary to .gitignore (should not be committed)
The test framework walks _testdata/examples and tries to parse all files as OpenAPI specs. Moving generated code and test files to internal/integration/test_complex_uniqueitems/ follows the pattern used by other integration tests and prevents CI failures. Changes: - Keep only .yaml spec files in _testdata/examples/complex-uniqueitems/ - Move generated code to internal/integration/test_complex_uniqueitems/ - Move README and test files to internal/integration/test_complex_uniqueitems/ This matches the pattern used by autorest/ and redoc/ directories which contain only spec files, not generated code.
6790e43 to
b1e2f79
Compare
Fix compilation errors introduced by changing collectEqualitySpecs() and writeValidateUnique() to return void. Remove error handling from call site and remove unused errors import.
Fix three linting issues identified in CI: - tools/sgcollector: remove sparse array initialization to prevent potential slice index out of range - gen/gen_equality_detect.go: format single import without parentheses - gen/gen_equality.go: refactor if-else chain to switch statement for better readability All tests pass after these changes.
goimports requires const values to be aligned without extra spacing between the equal sign and value.
ernado
approved these changes
Nov 24, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements support for
uniqueItems: trueon arrays of complex objects (structs), enabling code generation for previously blocked OpenAPI specifications.Closes #1563
✅ All CI checks passing (tests, linting, commit messages)
Motivation
The JIRA API v3 specification contains 11 operations that use
uniqueItemson arrays of complex objects. Previously, ogen would reject these schemas withErrNotImplemented: complex uniqueItems, preventing code generation for the entire API.This implementation removes that limitation by generating
Equal()andHash()methods for types used in uniqueItems arrays, then using hash-based O(n) duplicate detection at runtime.Implementation
Core Changes
IR Structures (
gen/ir/equality.go): DefinesEqualityMethodSpecandFieldEqualitySpecfor describing generated equality methodsType Detection (
gen/gen_equality_detect.go): Traverses the IR type graph to identify types needing equality methods, handling ogen's type system (KindGeneric, OptT, NilT wrappers)Code Generation (
gen/gen_equality.go): GeneratesEqual()andHash()methods:Runtime Validation (
gen/gen_validators_unique.go): GeneratesvalidateUnique[TypeName]()functions using hash buckets for O(n) duplicate detectionError Types (
validate/errors.go): AddsDuplicateItemsErrorandDepthLimitErrorfor actionable error messagesSchema Parsing (
gen/schema_gen.go): Removes block preventing complex uniqueItemsEdge Case Handling
Recent improvements from real-world usage (Terraform provider):
bytes.Equal()instead of!=operator for[]byteandjx.Raw!=)Nullflag first, then.Valuefor non-null elementsExample Generated Code
For a type
WorkflowStatus, generates:Testing
52 tests passing across comprehensive test suites:
Unit Tests (14 tests)
Integration Tests (38 tests)
Test schemas in
_testdata/examples/complex-uniqueitems/, generated code and tests ininternal/integration/test_complex_uniqueitems/.Performance Benchmarks
Compatibility
validate.UniqueItems()Documentation
See
internal/integration/test_complex_uniqueitems/README.mdfor comprehensive documentation including:Files Changed
Core implementation (~1,500 lines):
gen/ir/equality.go- IR structures with edge case flagsgen/gen_equality_detect.go- Type detection and traversalgen/gen_equality.go- Equal() and Hash() generation with edge case handlinggen/gen_validators_unique.go- validateUnique() generationgen/gen_equality_test.go- Unit tests for code generationvalidate/errors.go- New error typesTest infrastructure (~26K lines generated):