Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA]: Support for Async Operators #386

Open
2 tasks done
cwharris opened this issue Sep 17, 2023 · 0 comments · May be fixed by #392
Open
2 tasks done

[FEA]: Support for Async Operators #386

cwharris opened this issue Sep 17, 2023 · 0 comments · May be fixed by #392
Labels
feature request New feature or request

Comments

@cwharris
Copy link
Contributor

cwharris commented Sep 17, 2023

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

High

Please provide a clear description of problem this feature solves

MRC supports 1 of the following 4 mapping operations. Supporting all 4 would open up a multitude of opportunities.

# mrc.operators.map
def on_data(self, value: int):
    return value
# mrc.operators.flatmap
def on_data_iterator(self, value: int):
    for i in range(value):
        yield i
# mrc.operators.map_async
async def on_data_async(self, value: int):
    await asyncio.sleep(0.25)
    return value
# mrc.operators.flatmap_async
async def on_data_async_gen(self, value: int):
    for i in range(value):
        await asyncio.sleep(0.25)
        yield i

Describe your ideal solution

Write a custom fiber-enabled asyncio Event Loop and create one-per-thread to be used from within MRC operators.

Describe any alternatives you have considered

No response

Additional context

No response

Code of Conduct

  • I agree to follow MRC's Code of Conduct
  • I have searched the open feature requests and have found no duplicates for this feature request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
Status: Todo
Development

Successfully merging a pull request may close this issue.

2 participants