-
Notifications
You must be signed in to change notification settings - Fork 173
Description
What happened?
I've tried changing adbc.snowflake.statement options, but none of the settings appear to have any effect with adbc_ingest().
I've tried the following:
"adbc.snowflake.statement.ingest_copy_concurrency": "0",
"adbc.snowflake.statement.ingest_copy_concurrency": "1",
"adbc.snowflake.statement.ingest_target_file_size": "100mb",
"adbc.snowflake.statement.ingest_target_file_size": "100",
"adbc.snowflake.statement.ingest_target_file_size": "100000000",
But the result always ends up with ~120 PUTs with concurrent overlapping COPY INTO(s) running in parallel..
Changing any of the above to int(s) like:
"adbc.snowflake.statement.ingest_copy_concurrency": 0,
results in an error which is expected, so the parameters are being checked, but they just don't impact anything..
File "../miniconda3/lib/python3.9/site-packages/adbc_driver_snowflake/__init__.py", line 156, in connect
return adbc_driver_manager.AdbcDatabase(driver=_driver_path(), **kwargs)
File "adbc_driver_manager/_lib.pyx", line 482, in adbc_driver_manager._lib.AdbcDatabase.__init__
File "adbc_driver_manager/_lib.pyx", line 276, in adbc_driver_manager._lib._to_bytes
ValueError: value must be str or bytes
I'm also passing a recordbatchreader into adbc_ingest()
Stack Trace
No response
How can we reproduce the bug?
No response
Environment/Setup
No response