Skip to content

Commit a3067dc

Browse files
committed
add documentation
1 parent 4f8160b commit a3067dc

1 file changed

Lines changed: 59 additions & 0 deletions

File tree

sdks/python/apache_beam/transforms/managed.py

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,62 @@
1+
#
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with
4+
# this work for additional information regarding copyright ownership.
5+
# The ASF licenses this file to You under the Apache License, Version 2.0
6+
# (the "License"); you may not use this file except in compliance with
7+
# the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
"""Managed Transforms.
19+
20+
This module builds and instantiates turnkey transforms that can be managed by
21+
the underlying runner.
22+
23+
Using Managed Transforms
24+
=================
25+
Managed transforms have a defined configuration and can be built using an
26+
inline :class:`dict` like so::
27+
28+
results = p | beam.managed.Read(
29+
beam.managed.ICEBERG,
30+
config={"param_1": "foo",
31+
"param_2": "bar"})
32+
33+
A YAML configuration file can also be used to build a Managed transform. Say we
34+
have the following `config.yaml` file::
35+
36+
param_1: "foo"
37+
param_2: "bar"
38+
39+
Simply provide the location to the file like so::
40+
41+
input_rows = p | beam.Create(...)
42+
input_rows | beam.managed.Write(
43+
beam.managed.KAFKA,
44+
config_url="path/to/config.yaml")
45+
46+
Available transforms
47+
=============
48+
Available transforms are:
49+
50+
- **Kafka**
51+
- **Iceberg**
52+
- **BigQuery**
53+
54+
**Note:** inputs and outputs need to be PCollections of Beam
55+
:py:class:`apache_beam.pvalue.Row` elements.
56+
57+
**Note:** This Managed API uses Java's ManagedSchemaTransform under the hood.
58+
"""
59+
160
from typing import Any
261
from typing import Dict
362
from typing import Optional

0 commit comments

Comments
 (0)