Skip to content

[Task][Go SDK]: Remove reflect.Type parameter in parquetio.Write #28490

@johannaojeling

Description

@johannaojeling

What needs to happen?

The parquetio.Write transform takes a t reflect.Type parameter, which is redundant since the type can be derived from the incoming PCollection.

It will be a backward incompatible change, however will be more consistent with the other IOs (bigqueryio, bigtableio and so on) and will remove the need for developers to write wrapper functions such as:

func WriteParquet(s beam.Scope, filename string, col beam.PCollection) {
    t := col.Type().Type()
    parquetio.Write(s, filename, t, col)
}

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions