Project

General

Profile

Actions

Bug #71111

closed

rgw: test_schema_definition (squid)

Added by J. Eric Ivancich about 1 year ago. Updated 10 months ago.

Status:
Duplicate
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
Backport:
tentacle
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

Example run: https://qa-proxy.ceph.com/teuthology/hyelloji-2025-03-12_09:28:17-rgw-wip-hemanth-testing-2025-03-03-1505-squid-distro-default-smithi/8183687/teuthology.log
Reproduced frequently: https://pulpito.ceph.com/hyelloji-2025-03-12_09:28:17-rgw-wip-hemanth-testing-2025-03-03-1505-squid-distro-default-smithi/

=================================== FAILURES ===================================
____________________ test_schema_definition ____________________

    @pytest.mark.s3select
    def test_schema_definition():

        number_of_rows = 10000

        # purpose of test is to validate functionality using csv header info
        csv_obj = create_random_csv_object(number_of_rows,10,csv_schema="c1,c2,c3,c4,c5,c6,c7,c8,c9,c10")

        csv_obj_name = get_random_string()
        bucket_name = get_new_bucket_name()

        upload_object(bucket_name,csv_obj_name,csv_obj)

        # ignoring the schema on first line and retrieve using generic column number
        res_ignore = remove_xml_tags_from_result( run_s3select(bucket_name,csv_obj_name,"select _1,_3 from s3object;",csv_header_info="IGNORE") ).replace("\n","")

        # using the scheme on first line, query is using the attach schema
        res_use = remove_xml_tags_from_result( run_s3select(bucket_name,csv_obj_name,"select c1,c3 from s3object;",csv_header_info="USE") ).replace("\n","")
        # result of both queries should be the same
        s3select_assert_result( res_ignore, res_use)

        # using column-name not exist in schema
        res_multiple_defintion = remove_xml_tags_from_result( run_s3select(bucket_name,csv_obj_name,"select c1,c10,int(c11) from s3object;",csv_header_info="USE") ).replace("\n","")

        assert ((res_multiple_defintion.find("alias {c11} or column not exist in schema")) >= 0)

        #find_processing_error = res_multiple_defintion.find("ProcessingTimeError")
>       assert ((res_multiple_defintion.find("ProcessingTimeError")) >= 0)
E       AssertionError: assert -1 >= 0
E        +  where -1 = <built-in method find of str object at 0x7fad80b2f840>('ProcessingTimeError')
E        +    where <built-in method find of str object at 0x7fad80b2f840> = 'An error occurred (s3select-ProcessingTime-Error) when calling the SelectObjectContent operation: [int failed : alias {c11} or column not exist in schema]'.find

s3tests_boto3/functional/test_s3select.py:1371: AssertionError

Related issues 2 (1 open1 closed)

Related to rgw - Bug #67634: FAILED s3tests_boto3/functional/test_s3select.py::test_schema_definitionCan't reproduceGal Salomon

Actions
Is duplicate of rgw - Bug #65468: rgw: set correct requestId and hostId on s3select errorPending BackportSeena Fallah

Actions
Actions #1

Updated by J. Eric Ivancich about 1 year ago

  • Related to Bug #67634: FAILED s3tests_boto3/functional/test_s3select.py::test_schema_definition added
Actions #2

Updated by J. Eric Ivancich about 1 year ago

  • Description updated (diff)
Actions #3

Updated by J. Eric Ivancich about 1 year ago

  • Backport set to tentacle
Actions #4

Updated by J. Eric Ivancich 11 months ago

This is causing tons of failures on squid, which makes approving backports more challenging.

See: test_schema_definition

@Gal Salomon , can we get this addressed?

Actions #5

Updated by J. Eric Ivancich 11 months ago

  • Priority changed from Normal to High
Actions #7

Updated by Gal Salomon 11 months ago

J. Eric Ivancich wrote in #note-6:

@Gal Salomon Almost all the failures here are due to test_schema_definition:

https://pulpito.ceph.com/hyelloji-2025-03-12_09:28:17-rgw-wip-hemanth-testing-2025-03-03-1505-squid-distro-default-smithi/

this failures relate to missing change in s3-tests (it should be there)

we can comment out that test.

Actions #8

Updated by Gal Salomon 11 months ago

Actions #9

Updated by Gal Salomon 11 months ago

J. Eric Ivancich wrote in #note-6:

@Gal Salomon Almost all the failures here are due to test_schema_definition:

https://pulpito.ceph.com/hyelloji-2025-03-12_09:28:17-rgw-wip-hemanth-testing-2025-03-03-1505-squid-distro-default-smithi/

this teuthology run is too old
it is from 2025-03-12.

Actions #10

Updated by J. Eric Ivancich 10 months ago

@Gal Salomon What do you recommend for this bug? Are we waiting to hit it again during QA? Do you think it's resolved?

Actions #11

Updated by Gal Salomon 10 months ago

https://github.com/ceph/ceph/pull/62959
https://github.com/ceph/s3-tests/pull/642

the fixes above(backport from upstream) merged after the 2025-03-12

did it hit again since(the fixes)?

Actions #12

Updated by J. Eric Ivancich 10 months ago

  • Is duplicate of Bug #65468: rgw: set correct requestId and hostId on s3select error added
Actions #13

Updated by J. Eric Ivancich 10 months ago

  • Status changed from New to Duplicate
Actions

Also available in: Atom PDF