[Bug]: Pickling error when running pipelines with TFModelHandlerTensor on Python 3.9.2

[riteshghorse]

What happened?

Running python pipelines with TFModelHandlerTensor when used with load_model_args got some pickling issues on Direct Runner on GCP cloud shell. Haven't verified on other runners

  File "/home/student_00_40bc0e514d9c/devrel-demos/data-analytics/beam_ml_toxicity_in_gaming/part1.py", line 141, in <module>
    run(known_args.project_id, known_args.gaming_loc, known_args.movie_loc, pipeline_args)
  File "/home/student_00_40bc0e514d9c/devrel-demos/data-analytics/beam_ml_toxicity_in_gaming/part1.py", line 117, in run
    _ = (not_filter
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/apache_beam/pipeline.py", line 600, in __exit__
    self.result = self.run()
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/apache_beam/pipeline.py", line 574, in run
    pickler.dump_session(os.path.join(tmpdir, 'main_session.pickle'))
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/apache_beam/internal/pickler.py", line 61, in dump_session
    return desired_pickle_lib.dump_session(file_path)
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/apache_beam/internal/dill_pickler.py", line 439, in dump_session
    dill.dump_session(file_path)
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/dill/_dill.py", line 351, in dump_session
    pickler.dump(main)
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/dill/_dill.py", line 445, in dump
    StockPickler.dump(self, obj)
  File "/usr/lib/python3.9/pickle.py", line 487, in dump
    self.save(obj)
  File "/usr/lib/python3.9/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/apache_beam/internal/dill_pickler.py", line 301, in save_module
    return old_save_module(pickler, obj)
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/dill/_dill.py", line 1294, in save_module
    pickler.save_reduce(_import_module, (obj.__name__,), obj=obj,
  File "/usr/lib/python3.9/pickle.py", line 717, in save_reduce
    save(state)
  File "/usr/lib/python3.9/pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/apache_beam/internal/dill_pickler.py", line 349, in new_save_module_dict
    return old_save_module_dict(pickler, obj)
  File "/home/student_00_40bc0e514d9c/.local/lib/python3.9/site-packages/dill/_dill.py", line 912, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/usr/lib/python3.9/pickle.py", line 971, in save_dict
    self._batch_setitems(obj.items())
  File "/usr/lib/python3.9/pickle.py", line 997, in _batch_setitems
    save(v)
  File "/usr/lib/python3.9/pickle.py", line 578, in save
    rv = reduce(self.proto)
TypeError: cannot pickle 'LazyLoader' object

Issue Priority

Priority: 3 (minor)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[riteshghorse]

Upgrading the python's minor version to 3.9.13 solved it.