fix: [#818] Cursor cannot scan json column by hwbrzzl · Pull Request #1328 · goravel/framework

hwbrzzl · 2025-12-30T03:08:51Z

📑 Description

This pull request significantly enhances the database row scanning and decoding capabilities, especially for handling JSON data and custom types. It introduces new decode hooks for slices, maps, and types implementing the Scan method, and updates the test suite to validate these improvements. Additionally, it simplifies the codebase by delegating scanning logic and updates dependencies.

Enhancements to Row Scanning and Decoding:

Added new decode hooks in database/db/row.go to support automatic conversion of JSON strings to Go slices and maps, and to handle types implementing the Scan(any) error interface (such as custom and Carbon types). This improves flexibility and correctness when scanning database rows into Go structs. [1] [2]
Improved field name matching logic in the decoder to support both snake_case and StudlyCase mappings, increasing compatibility with various struct field naming conventions.

Refactoring and Codebase Simplification:

Refactored database/gorm/row.go to delegate the Scan method to the new, centralized logic in database/db/row.go, removing duplicated code and reducing maintenance overhead. [1] [2]

Testing Improvements:

Added comprehensive tests in tests/query_test.go for scanning JSON columns into map[string]any, slices ([]string, []int), and custom structs, ensuring robust support for a variety of data types and structures.
Updated the test query builder to only enable SQLite tests, likely for focused or faster testing.

Dependency Updates:

Upgraded the github.com/goravel/framework dependency from v1.15.9 to v1.16.5 and added a new indirect dependency on github.com/urfave/cli/v3 in the test module. [1] [2]

✅ Checks

Added test cases for my code

codecov · 2025-12-30T03:10:37Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.07%. Comparing base (6355289) to head (1ed58e9).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1328   +/-   ##
=======================================
  Coverage   70.07%   70.07%           
=======================================
  Files         282      282           
  Lines       16752    16752           
=======================================
  Hits        11739    11739           
  Misses       4522     4522           
  Partials      491      491

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

This pull request fixes issue #818 by enhancing the database row scanning capabilities to properly handle JSON columns when using cursors. The implementation introduces a unified approach for handling custom types with Scan methods (including Carbon types) and adds dedicated hooks for JSON deserialization to slices and maps.

Key Changes:

Replaced the specialized ToCarbonHookFunc with a generic ToScannerHookFunc that handles all types implementing Scan(any) error, including Carbon types and custom types
Added ToSliceHookFunc and ToMapHookFunc to automatically deserialize JSON strings into Go slices and maps
Refactored database/gorm/row.go to delegate scanning logic to the centralized database/db/row.go, eliminating code duplication
Enhanced field name matching to support both snake_case and StudlyCase mappings

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
database/db/row.go	Added new decode hooks (ToScannerHookFunc, ToSliceHookFunc, ToMapHookFunc) for JSON and custom type handling; improved field name matching logic
database/gorm/row.go	Simplified by delegating Scan implementation to centralized db.Row logic
database/db/row_test.go	Added comprehensive unit tests for new decode hooks covering time, carbon, deleted_at, scanner, and basic type conversions
tests/query_test.go	Added integration tests for scanning JSON columns into maps, string slices, int slices, and custom structs with Scan methods

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

database/db/row.go

tests/query_test.go

database/db/row.go

hwbrzzl · 2025-12-30T08:06:40Z

/gemini summary

gemini-code-assist · 2025-12-30T08:07:03Z

Summary of Changes

This pull request resolves an issue where the database cursor could not properly scan JSON columns by introducing robust enhancements to the row scanning and decoding mechanisms. It centralizes and extends the decoding logic to seamlessly handle JSON data into Go slices and maps, as well as custom types that implement the sql.Scanner interface. The changes also improve field name matching and are thoroughly tested to ensure reliability and correctness across diverse data structures.

Highlights

Enhanced JSON and Custom Type Scanning: Introduced new decode hooks (ToScannerHookFunc, ToSliceHookFunc, ToMapHookFunc) to automatically convert JSON strings from database columns into Go slices, maps, and custom types implementing the sql.Scanner interface. This significantly improves flexibility when scanning database rows into Go structs.
Improved Field Name Matching: The Scan method's field name matching logic has been enhanced to support snake_case in addition to StudlyCase and EqualFold, making it more compatible with various database column naming conventions.
Centralized Scanning Logic: Refactored database/gorm/row.go to delegate its Scan method to the more general and enhanced logic in database/db/row.go, reducing code duplication and simplifying maintenance.
Comprehensive Test Coverage: Added extensive unit tests in database/db/row_test.go and integration tests in tests/query_test.go to validate the new scanning capabilities, including various JSON structures, empty strings, and nil values.
Dependency Update: The github.com/goravel/framework dependency has been upgraded from v1.15.9 to v1.16.5.

Changelog

database/db/row.go
- Added encoding/json import for JSON processing.
- Modified DecodeHook composition in Scan to include ToScannerHookFunc, ToSliceHookFunc, and ToMapHookFunc, replacing the direct ToCarbonHookFunc.
- Enhanced field name matching logic to support snake_case conversions.
- Introduced ToScannerHookFunc to handle types implementing sql.Scanner (including Carbon types).
- Added ToSliceHookFunc for automatic JSON string to Go slice conversion.
- Added ToMapHookFunc for automatic JSON string to Go map conversion.
database/db/row_test.go
- New file added containing comprehensive unit tests for the Row.Scan method.
- Includes tests for basic type scanning, carbon types, gorm.DeletedAt, JSON to map, JSON to slice, string conversion, and time conversion.
- Features specific test cases for handling empty strings and nil values in JSON map and slice conversions.
- Utilizes a custom Json struct to test the ToScannerHookFunc.
database/gorm/row.go
- Removed strings, mapstructure/v2, and support/str imports.
- Refactored the Scan method to delegate its logic to db.NewRow(r.row, r.err).Scan(value), centralizing the decoding process.
tests/query_test.go
- Added encoding/json import.
- Updated TestCursor to use Where("name", "cursor_user") for consistency.
- Introduced new integration tests (TestCursor_WithJson_ScanMap, TestCursor_WithJson_ScanStringSlice, TestCursor_WithJson_ScanIntSlice, TestCursor_WithJson_ScanStruct) to verify JSON column scanning into various Go types.
- Defined a ResultData struct with a Scan method for testing custom type scanning.

Activity

Codecov bot reported 100% coverage for modified lines and 70.07% overall project coverage.
Copilot provided review comments suggesting improvements for handling empty JSON strings in ToSliceHookFunc and ToMapHookFunc.
Copilot suggested enhancing the ResultData.Scan method in the test file to properly handle nil or non-[]byte input values.
Copilot recommended handling []uint8 data in ToScannerHookFunc for consistency, converting it to []byte before scanning.
The author hwbrzzl requested a summary using the /gemini summary command.

fix: [#818] Cursor cannot scan json column

0158fc2

hwbrzzl requested a review from a team as a code owner December 30, 2025 03:08

Copilot AI review requested due to automatic review settings December 30, 2025 03:08

Copilot started reviewing on behalf of hwbrzzl December 30, 2025 03:09 View session

Copilot AI reviewed Dec 30, 2025

View reviewed changes

database/db/row.go Show resolved Hide resolved

database/db/row.go Show resolved Hide resolved

tests/query_test.go Show resolved Hide resolved

database/db/row.go Show resolved Hide resolved

database/db/row.go Show resolved Hide resolved

database/db/row.go Show resolved Hide resolved

hwbrzzl added 4 commits December 30, 2025 11:31

optimize

f0285b7

optimize

ec2dd8f

optimize

9f7fe4c

optimize

98912ab

hwbrzzl added the 🚀 Review Ready label Dec 30, 2025

optimize

1ed58e9

hwbrzzl merged commit 77af594 into master Dec 30, 2025
13 checks passed

hwbrzzl deleted the bowen/#818-1 branch December 30, 2025 09:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: [#818] Cursor cannot scan json column#1328

fix: [#818] Cursor cannot scan json column#1328
hwbrzzl merged 6 commits intomasterfrom
bowen/#818-1

hwbrzzl commented Dec 30, 2025

Uh oh!

codecov bot commented Dec 30, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hwbrzzl commented Dec 30, 2025

Uh oh!

gemini-code-assist bot commented Dec 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

hwbrzzl commented Dec 30, 2025

📑 Description

✅ Checks

Uh oh!

codecov bot commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hwbrzzl commented Dec 30, 2025

Uh oh!

gemini-code-assist bot commented Dec 30, 2025

Summary of Changes

Highlights

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Dec 30, 2025 •

edited

Loading