[Feature Request]: Add library stubs or py.typed marker

[davidcavazos]

What would you like to happen?

Support running mypy on my own Beam pipeline code by adding type stubs.

Simply importing apache_beam fails with mypy.

#-- main.py
import apache_beam as beam

Install and run mypy.

pip install mypy apache-beam

mypy main.py

Gives the following output:

main.py:1: error: Skipping analyzing "apache_beam": module is installed, but missing library stubs or py.typed marker  [import]
main.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

They can be automatically generated with stubgen, it would be nice to make that part of the release process on each new version. Since Beam already has type hints, I don't think there is too much tweaking needed.

I did try to generate them, but it failed trying to parse some complex type hint.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[davidcavazos]

I'm not too familiar with this, but it looks like other libraries with inline type annotations like Beam simply place an "empty" py.typed marker file.

• https://github.com/googleapis/python-aiplatform/blob/main/google/cloud/aiplatform/py.typed
• https://github.com/googleapis/python-bigquery/blob/main/google/cloud/bigquery/py.typed

[davidcavazos]

I just tested adding a file like that and it works.

[gariepyalex]

Can we re-open this issue? I installed version 2.52 I still have the error

main.py:1: error: Skipping analyzing "apache_beam": module is installed, but missing library stubs or py.typed marker  [import]
main.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

I looked into the installed package and py.typed is not distributed.

It should be added as a new file inpackage_data:
https://github.com/apache/beam/blob/master/sdks/python/setup.py#L254

I could find for info there: https://peps.python.org/pep-0561/#packaging-type-information