Skip to content

Stop changing the case for COPY TO option values #10853

New issue
New issue
Closed
#11330
Closed
Stop changing the case for COPY TO option values#10853
#11330
Labels
bugSomething isn't working
@tinfoil-knight

Description

@tinfoil-knight
tinfoil-knight
opened on Jun 10, 2024

Describe the bug

Currently, we're changing the case of every option value passed with the COPY statement:

datafusion/datafusion/sql/src/statement.rs

Lines 891 to 900 in 3773fb7

if !(&key.contains('.')) {
// If config does not belong to any namespace, assume it is
// a format option and apply the format prefix for backwards
// compatibility.
let renamed_key = format!("format.{}", key);
options.insert(renamed_key.to_lowercase(), value_string.to_lowercase());
} else {
options.insert(key.to_lowercase(), value_string.to_lowercase());
}
}

This causes an issue in cases where the option values for some external integrations might be case sensitive (eg. like access keys).

@xinlifoobar found this while working on adding an integration to huggingface in #10792.

See previous discussion here: #9723 (comment)

To Reproduce

No response

Expected behavior

There should be some way to pass option values as is for certain integrations.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions