Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement systematic retry logic especially for sa.Table(autoload_with=) #161

Open
windiana42 opened this issue Mar 20, 2024 · 1 comment

Comments

@windiana42
Copy link
Member

DB2 is pretty quick in killing transactions as "deadlock victim". We see this often as test failures. This even happens on the table reflection table where operations working on completely different tables may collide. One thread might reflect a table while another just creates another one.

One solution to this problem might be to play with transaction isolution modes (so far we were not successful to tackle this problem that way).

Thus we already implemented some hacky retry mechanims within pipedag code (see hook.py:138). However, this could be done much nicer with a decorator or some helper functions. The retry behavior should also be centrally configurable. And to date, not all table reflections are covered by retry code. So we need to search for sa.Table\([^)]*autoload_with= in a multiline fashion. (see hook.py:404)

@windiana42
Copy link
Member Author

Problem is partially addressed by 93b5adf

@windiana42 windiana42 added partially done and removed good first issue Good for newcomers labels Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant