Bug #71111
closedrgw: test_schema_definition (squid)
0%
Description
Example run: https://qa-proxy.ceph.com/teuthology/hyelloji-2025-03-12_09:28:17-rgw-wip-hemanth-testing-2025-03-03-1505-squid-distro-default-smithi/8183687/teuthology.log
Reproduced frequently: https://pulpito.ceph.com/hyelloji-2025-03-12_09:28:17-rgw-wip-hemanth-testing-2025-03-03-1505-squid-distro-default-smithi/
=================================== FAILURES ===================================
____________________ test_schema_definition ____________________
@pytest.mark.s3select
def test_schema_definition():
number_of_rows = 10000
# purpose of test is to validate functionality using csv header info
csv_obj = create_random_csv_object(number_of_rows,10,csv_schema="c1,c2,c3,c4,c5,c6,c7,c8,c9,c10")
csv_obj_name = get_random_string()
bucket_name = get_new_bucket_name()
upload_object(bucket_name,csv_obj_name,csv_obj)
# ignoring the schema on first line and retrieve using generic column number
res_ignore = remove_xml_tags_from_result( run_s3select(bucket_name,csv_obj_name,"select _1,_3 from s3object;",csv_header_info="IGNORE") ).replace("\n","")
# using the scheme on first line, query is using the attach schema
res_use = remove_xml_tags_from_result( run_s3select(bucket_name,csv_obj_name,"select c1,c3 from s3object;",csv_header_info="USE") ).replace("\n","")
# result of both queries should be the same
s3select_assert_result( res_ignore, res_use)
# using column-name not exist in schema
res_multiple_defintion = remove_xml_tags_from_result( run_s3select(bucket_name,csv_obj_name,"select c1,c10,int(c11) from s3object;",csv_header_info="USE") ).replace("\n","")
assert ((res_multiple_defintion.find("alias {c11} or column not exist in schema")) >= 0)
#find_processing_error = res_multiple_defintion.find("ProcessingTimeError")
> assert ((res_multiple_defintion.find("ProcessingTimeError")) >= 0)
E AssertionError: assert -1 >= 0
E + where -1 = <built-in method find of str object at 0x7fad80b2f840>('ProcessingTimeError')
E + where <built-in method find of str object at 0x7fad80b2f840> = 'An error occurred (s3select-ProcessingTime-Error) when calling the SelectObjectContent operation: [int failed : alias {c11} or column not exist in schema]'.find
s3tests_boto3/functional/test_s3select.py:1371: AssertionError
Updated by J. Eric Ivancich about 1 year ago
- Related to Bug #67634: FAILED s3tests_boto3/functional/test_s3select.py::test_schema_definition added
Updated by J. Eric Ivancich 11 months ago
This is causing tons of failures on squid, which makes approving backports more challenging.
See: test_schema_definition
@Gal Salomon , can we get this addressed?
Updated by J. Eric Ivancich 11 months ago
@Gal Salomon Almost all the failures here are due to test_schema_definition:
Updated by Gal Salomon 11 months ago
J. Eric Ivancich wrote in #note-6:
@Gal Salomon Almost all the failures here are due to test_schema_definition:
this failures relate to missing change in s3-tests (it should be there)
we can comment out that test.
Updated by Gal Salomon 11 months ago
the fix is for squid
https://github.com/ceph/s3-tests/pull/642
Updated by Gal Salomon 11 months ago
J. Eric Ivancich wrote in #note-6:
@Gal Salomon Almost all the failures here are due to test_schema_definition:
this teuthology run is too old
it is from 2025-03-12.
Updated by J. Eric Ivancich 10 months ago
@Gal Salomon What do you recommend for this bug? Are we waiting to hit it again during QA? Do you think it's resolved?
Updated by Gal Salomon 10 months ago
https://github.com/ceph/ceph/pull/62959
https://github.com/ceph/s3-tests/pull/642
the fixes above(backport from upstream) merged after the 2025-03-12
did it hit again since(the fixes)?
Updated by J. Eric Ivancich 10 months ago
- Is duplicate of Bug #65468: rgw: set correct requestId and hostId on s3select error added