add support for concurrent.futures

[mmckerns]

the pathos interface is nearly a match to concurrent.futures, and it seems to be preferred over multiprocessing as the standard.

[PawelTroka]

@mmckerns Any updates on this?

[mmckerns]

No, I haven't had time for it. In theory, it should be an easy PR -- however, I'm short on time at the moment.

[evanbiederstedt]

+1

This would be very very useful

[colibri17]

+1 Agree

[h3jia]

+1

[matthewyangcs]

+1 please!

[ddelange]

fwiw, in case you're in the asyncio world and you're in need of dill, this gist might be useful to you (particularly the AioPool part):
https://gist.github.com/ddelange/643fbb791b398783c04d1ceb90102163

See also the PR I liked above in 2021. It's not a drop in replacement of ProcessPoolExecutor, but the gist mostly gets the job done for me.

[mmckerns]

I just had this come up in a funded project yesterday, so this may get prioritized...

[ddelange]

@mmckerns that's cool to hear! the unresolved comment might be relevant for the implementation

[rachtsingh]

One important difference between multiprocessing.Pool and concurrent.futures.ProcessPoolExecutor is that the former hangs if a worker process is killed. Does the pathos/multiprocess project have a plan to handle that behavior (or is it already handled in multiprocess.Pool)?

As an alternative for those looking, it looks like the joblib/loky project does try to handle this? I'll give it a shot.

[ddelange]

I think that's #265