[Feature Request]: Iceberg Support for Python

[asheeshgarg]

What would you like to happen?

As of pyiceberg release of 0.6.0 pyiceberg gives the read write capability to read and write data to iceberg table format in Python. It will be good if we have a Source/Sink support for Iceberg in Python SDK.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[liferoad]

This should be available through Managed IO for Python later. @ahmedabu98

[liferoad]

#31495 FYI

[ahmedabu98]

@asheeshgarg just merged ##31495. See this example for how to use (note that input PCollection elements are Beam Rows)

import apache_beam as beam

with beam.Pipeline() as p:
    (p 
     | beam.Create([beam.Row(...), beam.Row(...)]) 
     | beam.managed.Write(
        "iceberg",
        config={
            "table": "namespace.table",
            "catalog_name": "test-catalog",
            "catalog_properties": {
                "catalog-impl": "org.apache.iceberg.hadoop.HadoopCatalog",
                "warehouse": "path/to/warehouse". # local file system and GCS are supported
            }}))