Skip to content

[Task]: Planned removal of Spark 2 runner support #25259

@mosche

Description

@mosche

What needs to happen?

The runner for Spark 2 was deprecated quite a while back in August 2022 with the release of Beam 2.41.0. We’re planning to move ahead with this and finally remove support for Spark 2 (beam-runners-spark) to only maintain support for Spark 3 (beam-runners-spark-3) going forward.

Note, Spark 2.4.8 is the latest release of Spark 2. It was released in May 2021 and no further releases of 2.4.x should be expected, even for bug fixes.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions