Skip to content

Parquet format options are ignored during BQ table creation #2146

@tomasherman

Description

@tomasherman

It seems to me that during table creation bigquery client will ignore Parquet FormatOptions.

The issue is in the following code where ExternalTableDefinition class is being turned into proto message ... as you can see, there is no code that checks for parquet format options (as opposed to AVRO, CSV and BIGTABLE format options).

On the other hand, when the code is parsing a proto message to ExternalTableDefinition, the ParquetOptions are being correctly parsed - see here

My expectation to solve this issue is following:
Something similar to the following should be added to ExternalTableDefinition#toExternalDataConfigurationPb

    if (getFormatOptions() != null && FormatOptions.PARQUET.equals(getFormatOptions().getType())) {
      externalConfigurationPb.setParquetOptions(((ParquetOptions) getFormatOptions()).toPb());
    }

Do you think my assessment is correct and this is indeed a bug? Or have i missed something? Thank you

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/java-bigquery API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions