[Task]: Remove version restriction on requests in python SDK

[axelmagn]

What needs to happen?

It looks like requests==2.32.* was explicitly blocked at some point as a workaround for a docker bug:

beam/sdks/python/setup.py

Line 388 in e4f2322

'requests>=2.24.0,<3.0.0,!=2.32.*',

Unfortunately, this leads to dependency version conflicts in practice whenever a pipeline depends on other packages that make use of requests. In my case it is huggingface/datasets. Example:

$ poetry add datasets
Using version ^2.21.0 for datasets

Updating dependencies
Resolving dependencies... (0.9s)

Because no versions of apache-beam match >2.58,<2.58.1 || >2.58.1,<3.0
 and apache-beam (2.58.0) depends on requests (>=2.24.0,<2.32.dev0 || >=2.33.dev0,<3.0.0), apache-beam (>=2.58,<2.58.1 || >2.58.1,<3.0) requires requests (>=2.24.0,<2.32.dev0 || >=2.33.dev0,<3.0.0).
And because apache-beam (2.58.1) depends on requests (>=2.24.0,<2.32.dev0 || >=2.33.dev0,<3.0.0)
 and no versions of requests match >=2.33.dev0,<3.0.0, apache-beam (>=2.58,<3.0) requires requests (>=2.24.0,<2.32.dev0).
Because no versions of datasets match >2.21.0,<3.0.0
 and datasets (2.21.0) depends on requests (>=2.32.2), datasets (>=2.21.0,<3.0.0) requires requests (>=2.32.2).
Thus, datasets (>=2.21.0,<3.0.0) is incompatible with apache-beam (>=2.58,<3.0).
So, because XXX depends on both apache-beam (^2.58) and datasets (^2.21.0), version solving failed.

It also looks like the PR mentioned in the workaround has been resolved: docker/docker-py#3257

I propose that this restriction is removed, but I am not familiar with the original issue that was worked around, so I'm not sure how to confirm that it's resolved.

@tvalentyn FYI

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[axelmagn]

Oops, looks like this may be a dupe: #32080