Skip to content

feature: add validate method to space and tuple.format objects#10965

Merged
locker merged 3 commits intotarantool:masterfrom
rorororom:9979_add_function_validate_tuple
May 27, 2025
Merged

feature: add validate method to space and tuple.format objects#10965
locker merged 3 commits intotarantool:masterfrom
rorororom:9979_add_function_validate_tuple

Conversation

@rorororom
Copy link
Contributor

@rorororom rorororom commented Dec 20, 2024

Added field format_object to spaces and method validate to tuple.format

Implementation note

The format_object field is not available for spaces accessed over net.box,
because creating it would trigger C-level validation of field types and collation
IDs on the client. If the format references a field type unknown to the client or
a collation ID that isn’t present locally, an error will be raised during schema
installation.

Test adjustments:

The format_object field creates references to tuple_format for space objects
via tuple_format_ref. These references can be retained by spaces remaining in
memory (e.g., in the global space table), especially if other tests create spaces.

  • In gh_4693_formats_for_standalone_tuples_test, added an initial call to
    collectgarbage() to clear the tuple format pool before creating new tuple_format
    objects. This ensures a clean environment for the test.

  • In net.box_reuse_tuple_formats_gh-6217.test, increased the
    ERRINJ_TUPLE_FORMAT_COUNT limit from 100 to 113. Initially, more than 100 tuple
    formats are created in the session due to references held by the format_object method.
    Increasing the limit accounts for these existing formats.

  • In wal_off/alter.test, added explicit deletion of references to allow the Lua garbage
    collector to free space objects and their format_object instances, preventing format
    pool exhaustion.

Closes #9979

@rorororom rorororom requested a review from a team as a code owner December 20, 2024 10:31
@rorororom rorororom force-pushed the 9979_add_function_validate_tuple branch 3 times, most recently from 06595ef to 58ec88d Compare December 20, 2024 11:37
@rorororom rorororom force-pushed the 9979_add_function_validate_tuple branch 10 times, most recently from 3e47e89 to 91891bd Compare January 20, 2025 13:17
@coveralls
Copy link

coveralls commented Jan 20, 2025

Coverage Status

coverage: 87.512% (-0.01%) from 87.525%
when pulling ceb3dcb on rorororom:9979_add_function_validate_tuple
into 53fd97d
on tarantool:master
.

@rorororom rorororom force-pushed the 9979_add_function_validate_tuple branch 14 times, most recently from 781e25f to de484d1 Compare January 21, 2025 08:24
@rorororom rorororom changed the title Added validate methods to space.format and tuple.format feature: add validate method to space.format and tuple.format objects Mar 31, 2025
@rorororom rorororom force-pushed the 9979_add_function_validate_tuple branch 2 times, most recently from 709fcfd to 358ee8d Compare April 4, 2025 00:28
@rorororom rorororom assigned rorororom and unassigned drewdzzz Apr 4, 2025
@rorororom rorororom force-pushed the 9979_add_function_validate_tuple branch 2 times, most recently from ad2a474 to b90fe9b Compare April 4, 2025 08:21
Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your work, I think we are close to the finish line!

Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your work! I left a bunch of comments, also:

  1. Please, use Closes #xxx only in the last commit. You can write Part of #xxx in other commits.
  2. Please, update all the doc requests: Since: 3.4 -> Since: 3.5 - Tarantool 3.4 was released recently, so this feature will appear only in Tarantool 3.5.
  3. We forgot about net.box! It also has its space objects that imitate behavior of box.space. Please, populate them with format_object field too (and don't forget about tests). It is gonna be very easy since they are written in pure Lua (src/box/lua/net_box.lua).

Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's about net.box? (See this comment, pt. 3)

Starting from commit 5072227 ("sysview: set format for
spaces with sysview engine"), each space is guaranteed
to have a valid tuple format. Added assert(format != NULL)
in space_create and the redundant NULL check has been removed.

Part of tarantool#9979

NO_DOC=no behavior changes
NO_CHANGELOG=no behavior changes
NO_TEST=no behavior changes
Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes, we are almost done!

@rorororom
Copy link
Contributor Author

The implementation of format_object for space in net_box has been removed, the reason is written in the pr comment.

Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Let's just add one more test case and fix some wordings, and I'll approve the patch!

Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nitpicks.
Please, do not forget to answer my comments and re-request the review.

Copy link
Contributor

@drewdzzz drewdzzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for working on this patch! LGTM!
I'm passing it to the second reviewer. Also, I'm going to re-request doc-team review since the changelogs were significantly changed.

rorororom added 2 commits May 26, 2025 15:10
Added a `validate` method to `box.tuple.format` objects to
simplify tuple validation against a format without requiring
tuple creation or insertion.

Part of tarantool#9979

@TarantoolBot document
Title: Add method validate to box.tuple.format
Since: 3.5

To simplify validation of tuples without needing to create or insert them,
a new `validate` method was added to `tuple.format` objects.

This method verifies that a tuple (or a Lua table representing a tuple)
conforms to the format. If the tuple is invalid, an error is raised.

Example:
```lua
format = box.tuple.format.new({
    {name = 'id', type = 'unsigned'},
    {name = 'name', type = 'string'},
})
format:validate({1, 'A'}) -- ok
format:validate({true, 'A'}) -- error
```
Add a `format_object` field  to spaces that returns the associated
`box.tuple.format` object. This enables users to access tuple format
metadata and invoke methods like `validate()` without creating or
inserting a tuple.

The `format_object` field is not available for spaces accessed over
net.box, because creating it would trigger C-level validation of
field types and collation IDs on the client. If the format references
a field type unknown to the client or a collation ID that isn’t
present locally, an error will be raised during schema installation.

Editing tests:

The `format_object` method  creates references to `tuple_format` for
space object via `tuple_format_ref`. These references can be held by
spaces remaining in memory (for example, in the global space table),
especially if other tests create spaces.

In `gh_4693_formats_for_standalone_tuples_test` added an initial call to
`collectgarbage()` to clear the tuple format pool before creating new
`tuple_formats`.

In `net.box_reuse_tuple_formats_gh-6217.test` increased the
`ERRINJ_TUPLE_FORMAT_COUNT` limit from 100 to 113. Sine initially more
than 100 tuple formats are created in the session due to the links help
by the `format_object` method. Increasing the limit allows you to take
into account existing formats.

In `war_off/alter.test` added explicit deletion of references, which
allows the Lua garbage collector to free object of spaces and their
format_object.

Closes tarantool#9979

@TarantoolBot document
Title: Add format_object field to spaces
Since: 3.5

The `format_object` field returns the `box.tuple.format` object associated
with the space’s format. It provides the same API as `box.tuple.format`,
including `:validate()` and other.

Example:
```lua
space = box.schema.space.create('test', {
    format = {
        {name = 'id', type = 'unsigned'},
        {name = 'name', type = 'string'}
    }
})

space.format_object:validate({123, 'example'})
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci Enables all tests for a pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add function to validate tuple against space format in Lua

5 participants